Attention Pooling-Based IndoBERT with High-Confidence Pseudo-Labeling for Indonesian Mental Health Classification
Year: 2026
Type: article
Abstract: Mental health disorders are a significant global concern, while social media provides user-generated text reflecting emotional and psychological conditions. However, Indonesian mental health classification remains challenging due to limited high-quality annotated datasets and limitations of conventional document representation methods. This study develops an Attention Pooling-Based IndoBERT framework integrated with high-confidence pseudo-labeling for multiclass Indonesian mental health classification. Unlabeled YouTube comments were automatically labeled using a BERT teacher model evaluated on an independent test set and filtered using a confidence threshold of 0.90, retaining 1,213 high-confidence samples (22.20%). The Attention Pooling mechanism assigns adaptive weights to contextual token representations, while Focal Loss, label smoothing, and learning-rate scheduling address class imbalance. The proposed model achieved 97.80% Accuracy and 79.63% Macro F1-score on an independent test set of 182 samples. Ablation analysis showed higher test Accuracy for Attention Pooling than CLS, Mean, and Max Pooling. McNemar’s test indicated significant differences between Attention Pooling and RoBERTa, standard IndoBERT, CLS, Mean, and Max Pooling, whereas the difference from BERT was not significant. These findings support Attention Pooling as an effective representation strategy for Indonesian mental health classification in low-resource settings.
Source: JUITA Jurnal Informatika
Institution: Universitas Ahmad Dahlan
Language: English
Topic: Mental Health via Writing
Subfield: Social Psychology
Field: Psychology
Domain: Social Sciences
Open Access status: diamond
OpenAlex ID: W7225745821
