AI-Augmented Testing solution of LLM-Integrated Mobile Applications: Architecture, Test Oracle Strategies, and Empirical Evaluation
Year: 2025
Type: article
Abstract: The integration of large language models (LLMs) into mobile applications introduces adaptive tutoring, conversational question answering, and automated feedback generation, but it also breaks the deterministic input-output assumptions on which conventional mobile test automation relies. This paper proposes an AI-augmented testing framework for LLM-integrated e-learning mobile applications that combines automated test-case generation, a hybrid test oracle built from semantic similarity scoring and LLM-as-a-judge rubric evaluation, and cross-platform mobile UI execution using Appium. We present a layered system architecture that situates the AI test harness as a first-class component alongside the prompt orchestrator, retrieval-augmented knowledge base, and application microservices. We further describe an implementation using Python, pytest, sentence-transformer embeddings, and Appium WebDriver, and report an empirical evaluation comparing manual/scripted testing against the proposed approach across authoring effort, regression-cycle duration, defect-escape rate, and oracle false-negative rate. Results indicate substantial reductions in authoring time and regression duration alongside improved defect detection, while highlighting open challenges in oracle calibration and non-determinism of LLM outputs. Keywords: e-learning; large language models; AI-augmented testing; test oracle problem; mobile application testing; Appium; software quality assurance.
Author: Urvish Gajjar
Institution: Health Care Service Corporation
Language: English
Subfield: Software
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4415942425
FWCI:
Cites:
Cited by:
Related to: