Work

A Study of the Effects of Language on Large Language Model Responses and Human Perception

Year: 2026
Type: article
Abstract: People often feel an idea more strongly in their mother tongue, and they soften a hard message so that the listener is not hurt. This study asks whether chat-based large language models (LLMs) handle hard messages with the same care in English, Hindi and Marathi. Ten industry situations, such as a rejected build, a loan denial and the removal of an engineer from duty, were given to ChatGPT, Claude and Gemini in the three languages. The 90 collected messages were scored with a concept-matched word list that has five harshness categories and four softening categories. The scores were Harshness H, Softening S and Net tone N = H − S, with content coverage and length as extra checks. Differences between models were large (Kendall’s W of 0.30 for N and 0.56 for S). Differences between languages were small and not significant after correction. Gemini wrote short messages with almost no softening (S = 0.06), whereas ChatGPT and Claude used courtesy, regret and offers of help. Gemini also left out more key facts in Marathi (82% coverage) than in English (94%). Each cell holds one response, the word list was built after reading the data, and no human rater was used, so the results are exploratory.
Language: English
Subfield: Marketing
Open Access status: green
OpenAlex ID: W7236684504
FWCI:
Cites:
Cited by:
Related to: 10