Most confident where least informative: Limits of AI-augmented thematic analysis for language teacher identity research
Year: 2026
Type: article
Abstract: The use of generative artificial intelligence (AI) as a tool for qualitative analysis is becoming increasingly popular, and its application to (reflexive) thematic analysis is a topic of open controversy. Rather than asking whether large language models (LLMs) can reproduce language teacher identity (LTI), this article examines what LLMs actually do when applied to a sensitive, context-embedded construct. Using the Generative AI-Augmented Thematic Analysis (GAATA) framework, three LLMs (ChatGPT, Claude, and Gemini) performed thematic analyses of interviews with eight pre-service English teachers in a German Master of Education program. Instead of regarding the agreement between models as validation, the analysis compared their outputs to a situated human counter-reading of the same transcripts. The models converged on five dimensions of teacher identity, only one of which was specific to being a language teacher. The remaining four dimensions reiterated well-established and literature-congruent patterns. The counter-reading revealed that this convergent account retained the generic arc of each interview while eliminating what was most language-specific, contradictory, and locally situated. The main finding is that the models converge on what is typical and lose what is distinctive because they are trained on overlapping data. Therefore, agreement between them reflects shared assumptions, not analytic robustness. The study argues that the trustworthiness of human analysis supported by LLM-augmented interpretation of sensitive constructs should be the basis for judgment, not agreement between models. Furthermore, model output is most useful as a foil that makes situated interpretation visible.
Author: David Gerlach
Institution: University of Wuppertal
Language: English
Subfield: General Social Sciences
Field: Social Sciences
Domain: Social Sciences
SDG: Quality education
Open Access status: hybrid
Estimated APC paid: $2,000
OpenAlex ID: W7212305790