An interpretable machine learning framework for automated screening of patient-facing LLM consultations: a comparative evaluation in medical aesthetic consultation
Year: 2026
Type: article
Abstract: The integration of large language models (LLMs) into patient-facing clinical decision support systems (CDSS) has outpaced the development of automated quality assurance mechanisms. In medical aesthetic consultation—where patient safety and informed decision-making are paramount—no validated framework exists for pre-deployment screening of LLM-generated content. To develop and evaluate an interpretable machine-learning framework that screens LLM-generated patient-facing consultations using automatically computable generation metadata and readability features—as a triage aid for prioritising expert review—and to comparatively evaluate DeepSeek-V3 and Kimi in medical aesthetic consultation. We generated 600 responses from 25 standardized queries across five aesthetic topics using DeepSeek-V3 (DeepSeek-AI, Hangzhou, China) and Kimi (Moonshot AI, Beijing, China) under six prompt strategies and two conversational modes. Two blinded dermatologists rated quality on six dimensions (accuracy, relevance, comprehensiveness, empathy, falsehood, safety). Readability was quantified using Flesch Reading Ease, Flesch-Kincaid Grade Level, Gunning Fog Index, and SMOG Index. Four machine learning models were trained to predict high-quality responses (composite score ≥ 4.0, pre-specified threshold) and evaluated with question-level grouped 5-fold cross-validation—in which all 24 responses of each seed question were confined to a single fold to prevent information leakage—with SHAP analysis to identify key predictive features. Between-model quality comparisons used paired tests reflecting the matched design. To align evaluation with the triage purpose, performance was additionally assessed with low-quality responses as the positive class (precision–recall analysis and review-burden operating points), and robustness was examined using alternative quality thresholds and an applicability-adjusted composite score. DeepSeek-V3 achieved higher composite quality than Kimi (3.89 ± 0.45 vs. 3.70 ± 0.38, paired Wilcoxon P < 0.001; robust under applicability-adjusted scoring). Prompt strategy was the dominant quality determinant (Friedman test, Kendall’s W = 0.376), with sixth-grade reading level prompts paradoxically yielding the lowest scores (3.25 ± 0.40). All ML models achieved moderate discrimination under question-level grouped cross-validation (AUC 0.714–0.732; response-level sensitivity analysis: 0.758–0.773), with findings robust to alternative quality thresholds (AUC 0.749–0.797 at cutoff 3.5 and 0.725–0.838 at cutoff 4.5) and to applicability-adjusted labelling (AUC 0.679–0.702); prompt strategy, sentence length, and model identity were the strongest SHAP-identified predictors. Aligned with the triage purpose, average precision for low-quality detection reached 0.802 (Logistic Regression), and capturing 90% of low-quality responses required flagging 83.2% of responses for expert review. The proposed framework demonstrates feasibility as an explainable triage aid that flags potentially low-quality LLM consultations for expert review using generation metadata and readability features; it does not assess semantic or clinical content, and expert review remains indispensable. Findings underscore that prompt engineering—not model selection alone—drives output quality, and that readability optimization must be decoupled from clinical completeness to ensure safe deployment in aesthetic medicine information systems.
Source: Scientific Reports
Language: English
Subfield: Health Informatics
Field: Medicine
Domain: Health Sciences
Open Access status: gold
Estimated APC paid: $2,690
OpenAlex ID: W7236343994
