Efektivitas Pembelajaran Analisa Sistem Informasi: Analisis N-gain, Uji Wilcoxon, Evaluasi Butir Soal, Dan Deteksi Anomali Akademik
Year: 2026
Type: article
Study design: Clinical Trial
Abstract: This study integrates two data sources to evaluate the effectiveness of learning in the Information Systems Analysis and Design (ISAD) course among 2023 Industrial Engineering students while simultaneously examining academic anomalies identified through a Google Forms–based assessment instrument. The study employed a pre-experimental one-group pretest–posttest design conducted on March 6–7, 2026. Of the 79 students who completed the pretest and the 93 students who completed the posttest, 75 participated in both assessments and met the requirements for paired data analysis. The Wilcoxon Signed-Rank Test revealed no statistically significant difference between the pretest and posttest scores (p = 0.5508; Cohen's d = −0.0734). This result was primarily attributed to a substantial ceiling effect, as 62.7% of the participants (47 out of 75 students) achieved perfect scores on the pretest. N-Gain analysis conducted on the 28 valid participants produced an average normalized gain of 0.22, indicating an overall low level of learning improvement. Nevertheless, 42.9% of the valid participants achieved a high N-Gain category, with Class 2023 C demonstrating the highest average N-Gain (0.649). Item analysis indicated that all 20 posttest questions were classified as easy. However, the question requiring the identification of an Expert System in a fraud detection case scenario was the most challenging, with a correct response rate of 71%. Furthermore, the Kruskal–Wallis test found no statistically significant differences in learning outcomes among the different class groups. A comprehensive analysis identified several indicators of potential academic misconduct. These included four major anomalies: widespread identical response patterns (62% in the pretest and 54% in the posttest), unusually rapid submission times (ranging from fractions of a second to less than one second), identity irregularities involving one invalid student identification number, and a decline in posttest scores for 36% of the participants. These findings suggest the need to redesign the assessment instrument by incorporating higher-order thinking skills (HOTS)-oriented questions and to strengthen monitoring mechanisms for online examinations in order to improve the validity and integrity of learning assessment.
Institution: Universitas PGRI Ronggolawe
Language: English
Subfield: Information Systems
Field: Computer Science
Domain: Physical Sciences
SDG: Quality education
Open Access status: hybrid
OpenAlex ID: W7202371137