Work

1299-GRM Decoding Health Prompts in AI: A Comparative Machine Learning Study

Year: 2026
Type: article
Abstract: Automated detection of health-related content in conversational AI datasets is essential for population health surveillance and clinical intelligence. This study evaluates six machine learning classifiers: Support Vector Machine (SVM), Multi-Layer Perceptron (MLP), Random Forest (RF), Logistic Regression (LR), Decision Tree (DT), and Naïve Bayes (NB) on a corpus of 16,523 annotated chat prompts. Features include TF-IDF n-grams (unigrams through trigrams), Latent Dirichlet Allocation (LDA) topics, and BERTopic clusters. Chi-square feature selection (SelectKBest) is applied to optimize the feature space. Using 3-fold stratified cross-validation, the optimal configuration — Uni+Bi+Tri+LDA with chi-square selection (k=3,000, N=4,000) yields F1=0.775 and AUC=0.941, demonstrating that principled feature selection substantially elevates classification performance.
Language: English
Open Access status: green
OpenAlex ID: W7237397139
FWCI:
Cites: 5
Cited by:
Related to: