Enhancing Quantitative Analysis in Social Sciences With Large Language Models (LLMs): A Methodological Case Study in Educational Research
Year: 2025
Type: article
Abstract: This paper explores the potential of Large Language Models (LLMs) for assisting with quantitative data analysis in social science research. Specifically, it introduces key concepts to help researchers effectively integrate LLMs into their workflows. For this purpose, we replicate a research paper in educational leadership on the relationship between school program coherence and student achievement. By leveraging LLMs to generate code for statistical tools like Mplus and R, researchers can streamline their data analysis, potentially saving time and effort. The quality of analytical code generated by LLMs can be influenced by the researcher’s understanding and application of concepts like context windows, LLM training data and training cut-off, model parameter settings like temperature, zero- and few-shot learning, and Retrieval-Augmented Generation (RAG). By describing and demonstrating the applications of these concepts, we aim to equip researchers with a basic toolset to leverage LLMs effectively to assist with coding for quantitative analysis.
Source: AERA Open
Authors: James Sebastian, Jeong-Mi Moon, Eric Camburn
Institutions: University of Missouri, University of Missouri–Kansas City
Language: English
Subfield: General Social Sciences
Field: Social Sciences
Domain: Social Sciences
SDG: Quality education
Open Access status: gold
Estimated APC paid: $1,000
OpenAlex ID: W4417216147