Work

Exploring Query-Driven Centroiding and Embedding Strategies for Automated Thematic Analysis in Physics Education

Year: 2025

Type: conference-paper

Abstract: Understanding how Physics Education Research (PER) has changed and matured can help researchers and materially improve physics teaching and learning.The advent of large language models (LLMs) and embedding-based NLP techniques opens new avenues for this work by enabling the analysis of large corpora with minimal manual coding.This preliminary study builds off the work of [T.O. B. Odden et al., Physical Review Physics Education Research 20, 020151 (2024)].We explore the use of text embeddings and retrieval-augmented generation (RAG)-like methods without generative AI to analyze themes across a random sample of 94 articles from major PER journals.Specifically, we examine how two methodological choices affect topic modeling performance: (1) representing articles using a single embedding versus multiple sentence-level embeddings, and (2) deriving topic centroids from representative texts versus researcher-defined queries.All results are evaluated against a human-coded dataset focused on four overarching thematic categories: teacher-centered, student-centered, physics content, and journal business.Our findings inform best practices for researchers seeking scalable, interpretable, and low-barrier approaches to literature analysis in PER.

Source: Physics Education Research Conference (PERC)

DOI: 10.1119/perc.2025.pr.mingyar

Authors: Michael Mingyar, Tor Ole B. Odden, Shannon D. Willoughby

Institution: Montana State University

Language: English

Topic: Computational and Text Analysis Methods

Subfield: General Social Sciences

Field: Social Sciences

Domain: Social Sciences

SDG: Quality education

Open Access status: hybrid

Funder: Montana State University