Efficiency of Code Generation by Artificial Intelligence Gpt-4o for Ukrainian-language Tasks of Various Complexity on the Eolymp Platform
Year: 2024
Type: conference-paper
Abstract: In this study, 45 programming tasks from the Eolymp automated system (eolymp.com) were selected and categorized into three groups (A, B, C) based on difficulty (easy, medium, hard). The problem statements, constraints, and input/output examples were provided to the GPT-4o model in Ukrainian. Using a predefined prompt, the model generated C++ solutions, which were tested on Eolymp. Results showed that GPT-4o achieved the highest success rate for easy problems (54.8±44.6%), significantly outperforming medium (16.73±23.8%) and hard problems (21.4±24.7%) (p<0.017). However, no significant difference was found between medium and hard problem success rates (p>0.05). The model fully solved 46.7% of easy problems but failed all test cases for several tasks across all groups. These findings suggest that while GPT-4o performs well on simple tasks, it struggles with more complex programming problems.
Authors: Mykhalko, Andrii, Mitsa, Oleksandr
Institution: Uzhhorod National University
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W6930812465
FWCI:
Cites:
Cited by:
Related to: