Beyond Single Models: Unsupervised Ensemble Selection for Small Language Models in Medical QA
Year: 2026
Type: article
Abstract: Small Language Models (SLMs) provide efficient alternatives to large models for clinical open-ended question answering (QA) but often show variable performance. We propose two unsupervised answer selection strategies for SLM ensembles: a confidence-based method using normalized perplexity and a consensus-based medoid method capturing semantic similarity among model outputs. Evaluations on three clinical QA benchmarks show that both strategies outperform single-model and random selection baselines. The results show that unsupervised confidence and consensus mechanisms can enhance the performance of SLM ensembles for medical QA without requiring additional training or increasing model size.
Source: OJS
Authors: Ventulett, Nicolas, Nicklas, Fabian, Gaida, Eric, Wallach, Dieter, Conrad, Jan
Institution: University of Applied Sciences Kaiserslautern
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7141655966
FWCI:
Cites:
Cited by:
Related to: