What Makes Learning with GenAI Effective? Text Support and Deeper Engagement are Associated with Higher Performance in an Ecology Context
Year: 2026
Type: article
Study designs: Randomized Controlled Trial, Clinical Trial
Abstract: Generative artificial intelligence (GenAI) tools such as ChatGPT are increasingly used for learning, yet evidence remains limited regarding their effectiveness in science learning contexts and how learner–AI interaction relates to performance. In a randomized experiment, 185 university students worked on task sheets about lake eutrophication and restoration. Depending on condition, they could use two informational texts (text-only), ChatGPT configured to answer only on the basis of these texts (ChatGPT-only), or both (text + ChatGPT). Afterwards, all students completed a knowledge test without access to any materials. Test performance was highest in the text-only and text + ChatGPT conditions and lowest in the ChatGPT-only condition, yielding a robust overall condition effect. Chat logs from the ChatGPT conditions were analyzed to examine interaction patterns associated with performance. Although ChatGPT-only students produced substantially more interaction turns than students with access to both texts and ChatGPT, interaction frequency was unrelated to test performance. Instead, specific interaction patterns were differentially associated with performance: deepening questions were positively associated, whereas direct acceptance of AI-generated responses was negatively associated with test performance. Output-steering and questioning additionally emerged as positively associated with performance in individual analyses. Prompting behavior varied with task difficulty, with deepening questions increasing for more demanding ecology tasks. The findings suggest that, in science learning contexts, the educational value of GenAI may depend less on access to the technology than on learners’ epistemic engagement with it.
Institutions: Justus-Liebig-Universität Gießen, University of Passau
Language: English
Subfield: Education
Field: Social Sciences
Domain: Social Sciences
SDG: Quality education
Open Access status: hybrid
Estimated APC paid: $3,490
Funder: Universität Passau
OpenAlex ID: W7225870967
