Work

Evaluating the Performance of Chatbots in Literary Criticism on Lima Barreto

Year: 2026
Type: article
Abstract: In the early 2020s, with the popularization of generative artificial intelligences (GenAIs), discussions intensified regarding the utility of chatbot technology — generative text intelligences — in teaching-learning dynamics. Aiming to verify the capacity of these systems to produce utterances suitable for literary criticism, and to understand their practical limitations. This research compared qualitative and quantitative evaluations of interactions conducted by a human model and by artificial models (ChatGPT, Gemini, and GPT4ALL). The analysis of their performances demonstrated that ChatGPT presented the best results, followed by Gemini and GPT4ALL. Among the discrepant aspects observed in the responses, the following stood out: originality, response length, continuity between responses, vocabulary variability, objectivity of enunciation and the complexity of adapting terms to context. It was also observed that the models' tendency to achieve persuasion through discursive simplification and rhetorical standardization represents a problem for pedagogical dynamics, which should stimulate academic erudition and deep critical thinking.
Language: English
Subfield: Education
Open Access status: diamond
OpenAlex ID: W7225449029
FWCI:
Cites: 2
Cited by:
Related to: 10