Year: 2026
Type: article
Abstract: In this research, we assessed four publicly available large-language models(LLM): ChatGPT-4o, Claude 3.5 Sonnet, DeepSeek V2, and Gemini 1.5 Pro, to study a curated set of 10 math questions, varying in fields such as algebra, arithmetic, trigonometry, geometry, and statistics, categorised by topic and different levels of complexity. We used Text analysis, Cosine similarity, Jaccard similarity and a Column chart of word-count to compare the results given by each model. To analyse the performance thoroughly, we applied Analysis Of Variance (ANOVA) for word count, Fleschscore, Polarity and Sensitivity analysis on the results to test accuracy, clarity of explanation, range of problems supported, ease of platform navigation and possible errors made by the chatbots. Our findings show slight variation in the performance among the chatbots. This research also highlights the limitations of each model providing insights to make optimum use of these chatbots in solving mathematical problems.
Source: Discover Education
Institution: Rochester Institute of Technology - Dubai
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: gold
Estimated APC paid: $1,390
OpenAlex ID: W7165029149