Large Language Model (LLM) Powered
Year: 2026
Type: dissertation
Abstract: Scholarship evaluation is a labor-intensive process in which human reviewers are required to assess a large amount of applicant documents. This study presents the design,implementation, and empirical evaluation of a scholarship system that integrates large language model (LLM) based scoring within a full web application. The system supports the full scholarship pipeline across three user roles including applicant, reviewer,and admin. The experimental component investigates the capability of LLMs to score scholarship personal statements and resumes against a structured rubric in zeroshot and fine-tuned settings. Evaluation of LLaMA, Gemma, Qwen, and Mistral were done in zero-shot settings, with LLaMA subsequently fine-tuned using QLoRA on A100 GPU. Results were assessed using Quadratic Weighted Kappa, Intraclass Correlation Coefficient, Pearson, Spearman, RMSE, and MAE. The fine-tuned model achieved a QWK of 0.751 for resume scoring and 0.602 for personal statement scoring, with two criteria exceeding the 0.80 strong agreement threshold on resume evaluation. Findings were compared againsted published benchmarks in the automated essay scoring. The study contributes to the domain of LLM-based scoring with a proof-of-concept deployment architecture that bridges the gap between commercial scholarship management platformed and automated scoring research.
Source: American University of Bahrain
Author: Ruqaya AlAsoomi
Language: English
Topic: AI and HR Technologies
Domain: Social Sciences
SDG: Quality education
Open Access status: green
OpenAlex ID: W7213395773