Original scientific paper

https://doi.org/10.29162/jez.2024.6

Evaluation of AI-based Grammar Correction for Portuguese

Vedran Juričić ; Faculty of Humanities and Social Sciences, Zagreb *
Daliborka Sarić ; Faculty of Humanities and Social Sciences, Zagreb

* Corresponding author.

Abstract

Correction of grammatical errors is today integrated into the most widely used text processing tools and is accessible online. However, these tools are primarily half-automatic, merely suggesting possible corrections and variations, and require interaction with a user, which can be a tedious task when used on lengthy texts. Recent advancements in the field of artificial intelligence and natural language processing offer a more efficient strategy. This paper analyzes a possibility of using ChatGPT for correcting grammar in Portuguese texts written by native speakers of Croatian. The texts were corrected by a native speaker of European Portuguese and by ChatGPT. The authors analyzed error detection and correction at various linguistic levels and accompanied it with examples. Due to class imbalance, the system’s performance was evaluated using the F-measure. The calculation of false positives and true negatives was adjusted because of special cases of improper correction. Taking that into consideration, the F0.5 score was 0.805. Nevertheless, it should be noted that the results would likely vary if the input corpus had different structure and proficiency level.

Keywords

Grammar correction; ChatGPT; Evaluation; System’s performance

Hrčak ID:

324931

URI

https://hrcak.srce.hr/324931

Publication date:

23.12.2024.

Article data in other languages: croatian

Visits: 0 *