IA generativa em sala de aula: um experimento controlado para medir o que ela ensina — e o que apaga

Abstract

The growing adoption of Generative Artificial Intelligence (AI) tools in software development raises relevant questions about their impacts on code quality, developer productivity, and student learning, especially given the absence of consolidated guidelines on their use in academic contexts. This work presents the planning of a controlled laboratory study aimed at investigating these impacts on the development process carried out by students and graduates of the Software Engineering program at the University of Brasília (UnB). The methodology is structured in three stages: recruitment, programming exercise, and final questionnaire. Participants are randomly divided into two experimental groups, one required to use AI tools and another without access to such resources, and both perform the same activity using the React library. Data collection combines objective code quality metrics, such as lines of code (LOC), code health, lint errors, and execution time, with subjective psychological metrics, including satisfaction, perceived usefulness, self-efficacy, positive emotions, and autonomous learning disposition, measured through Likert scale questionnaires. The planning is structured based on the Goal-Question-Metric (GQM) model and inspired by the guidelines of the ACM SIGSOFT Empirical Standard for Experiments with Human Participants. The results are expected to contribute to the understanding of the benefits and limitations of Generative AI in educational contexts, providing evidence to support researchers, educators, and academic institutions in the conscious adoption of these technologies.