Work

Response Content Units: Evaluating Completeness and Proactiveness in Medical Open-Response Question Answering

Year: 2026

Type: conference-paper

Abstract: Remote clinical care has significantly increased the workload for healthcare professionals managing digital inquiries.While automated systems aim to alleviate this burden, consumer health questions present unique challenges due to their linguistic complexity and the need for proactive clinical guidance, which traditional question-answering models often overlook.We introduce the medical Response Content Units (RCU) schema, a framework that facilitates automatic analysis to identify questionanswer completeness and critical answer subparts, which can then be used as tools for supporting clinician response or for automatic metric evaluation.Our analysis using this schema reveals a 16.4% gap in response completeness in professional replies and demonstrates that essential medical directives are provided 2.4 to 12.1 times as frequently as direct answers.We provide baseline results and publicly release our annotations and source code to offer an evaluation framework that is more closely aligned with real-world clinical requirements.

Source: Workshop on Generation, Evaluation and Metrics (GEM)

DOI: 10.18653/v1/2026.gem-main.25

Authors: Yongsin Park, Wen-wai Yim, Emma McKibbin, Asma Ben Abacha, Fei Xia

Institutions: University of Washington, Microsoft (United States)

Language: English

Topic: Artificial Intelligence in Healthcare and Education

Subfield: Health Informatics

Field: Medicine

Domain: Health Sciences

SDG: Good health and well-being

Open Access status: hybrid

OpenAlex ID: W7166824839