Izvorni znanstveni članak
https://doi.org/10.36978/cte.7.2.1
Challenges in using the language models of the OpenAI platform in the context of evaluating student solutions to project tasks from the field of computer programming
Danijel Kučak
orcid.org/0000-0003-2947-2389
; Visoko učilište Algebra, Zagreb
Sažetak
The authors of this study have noted for years that students' motivation for continuous work is relatively low. To increase student motivation to practice, University College Algebra faculty have decided to change their approach in recent years. Instead of forcing students to study for an exam on specific dates, they were given projects (assignments) to complete. The research results showed that the results obtained later improved compared to traditional exams. However, with the advent of powerful and free or very low-cost language models such as those of the OpenAI platform, teachers are faced with the challenge that students may use such models to solve a project task, which is against the code of ethics. This research aims to determine what kind of results the OpenAI Platform language model can achieve when it is assigned a project task from a specific course. 2 research questions were defined. What result does the language model on the OpenAI platform achieve when it is assigned a project task from the Java Web Programming course and is there a difference in the results compared to the results achieved by the students (Q1)? Is there a difference in the results of the language model compared to the natural language in which it receives the project task (Croatian or English) (Q2)? The analysis of the results has shown that in certain cases the language model can solve the test without additional user intervention to obtain a good grade, although the results obtained are significantly weaker than those of the students. The language model achieves better results when the exam is given in English than in Croatian.
Ključne riječi
evaluation of knowledge; project-based learning; OpenAI; academic integrity; Artificial Intelligence
Hrčak ID:
311602
URI
Datum izdavanja:
18.12.2023.
Posjeta: 198 *