Work

The Complexity of Young Learners: Rethinking Assessment Methodologies to Model Cognitive, Metacognitive, and Psychological Traits

Year: 2020
Type: dissertation
Abstract: Rapid changes to student demographics and learning environments call for reconceptualization of what and how we assess. There is growing consensus on the importance of assessing students’ growth in multidimensional traits such cognitive, affective, metacognitive, and psychological characteristics. However, traditional assessment methods rely on selected-response formats, which are poorly equipped to support valid interpretations about complex learner traits. This dissertation seeks to address methodological challenges with multimodal data sources by probing ways to profile students’ reading competencies with fine-grained processing skills, track changes in self-regulated learning behaviours over time, and model students’ writing competence as a function of reading comprehension and self-efficacy. These issues were addressed through three distinct empirical studies. The first study explored the potential of cognitive diagnostic modelling for profiling students’ reading achievement focusing on cognitive reading processes instead of aggregated total scores. Five core reading processing skills were identified through the content analysis of the curriculum expectations and subsequently were mapped onto assessment item specifications called a Q-matrix. Results demonstrated the prospective of profiling using processing skills. This calls for further research on the appropriate granularity of processing skills to maximize their interpretability and pedagogical usefulness for teachers and students. The second study examined the dynamic nature of self-regulated learning in a technology-rich learning environment. Latent transition analysis was employed to investigate how self-regulated learning progressed across two time points. Findings showed that learners displayed different combinations of judgment of learning and content evaluation to regulate learning at different times, revealing the transitory characteristic of self-regulated learning. The third study approached a long-standing challenge of utilizing written responses for assessment by investigating the potential of machine learning applications for automated scoring. Multinomial naïve Bayes was applied to investigate whether this method can accurately evaluate written responses. Results demonstrated that reading ability and self-efficacy could be accurate classification labels of written responses. Collectively, this dissertation explored the utility of multimodal data including responses to reading items, process data of self-regulated learning strategies, and written responses of reading comprehension. By applying measurement-modelling approaches on multimodal data, this dissertation demonstrated a broader capacity to understand multivariate learning characteristics.
Author: Clarissa Lau
Language: English
Subfield: Education
Open Access status: green
OpenAlex ID: W3147901497
Cites: 262
Cited by:
Related to: 10