Preliminary communication

https://doi.org/10.17559/TV-20240906001971

Harnessing Advanced Transfer Learning Techniques in GPT-2 for Real-World Multilingual Applications

Dejan Dodić ; The Academy of Applied Technical and Preschool Studies, Beogradska 18, Niš, Serbia *
Dušan Regodić ; MB University, Faculty of Business and Law, Department of Advanced information technologies, Teodora Drajzera 27, Belgrade, Serbia
Ana Vukić ; MB University, Faculty of Business and Law, Teodora Drajzera 27, Belgrade, Serbia
Vuk Vujović ; MB University, Faculty of Business and Law, Department of Advanced information technologies, Teodora Drajzera 27, Belgrade, Serbia
Nikola Milutinović ; The Academy of Applied Technical and Preschool Studies, Beogradska 18, Niš, Serbia

* Corresponding author.

Abstract

In an era of increasing demand for robust multilingual natural language processing, leveraging advanced transfer learning techniques has become essential. This paper explores the application of the GPT-2 model using a comprehensive Serbian dataset of 750 million tokens. By employing meticulous data preprocessing, effective tokenization, and precise hyperparameter optimization with Optuna, the model's performance in language tasks is significantly improved. These findings underscore the model's adaptability to diverse linguistic contexts, facilitating deployment in real-world applications. The significant performance improvements highlight broader applicability in multilingual AI environments. The paper addresses key challenges such as data heterogeneity and computational efficiency, providing insights and proposing strategies for future research. By overcoming these challenges, the research demonstrates the transformative potential of refined GPT-2 models in multilingual AI. The advancements made lay a solid foundation for further exploration and refinement of multilingual language models, paving the way for more inclusive and accurate AI-driven communication tools.

Keywords

GPT-2; hyperparameter optimization; multilingual applications; natural language processing; Serbian dataset; transfer learning

Hrčak ID:

348719

URI

https://hrcak.srce.hr/348719

Publication date:

30.6.2026.

Visits: 0 *