Synthesizing Physiological Stress Facial Expressions for Medical Simulation Mannequins Using AI-Driven Facial Action Units
Year: 2026
Type: preprint
Abstract: Realistic facial expressions of physiological stress, such as pain, choking, and coughing, are important for high-fidelity medical simulation, yet existing simulators provide limited dynamic facial animation. This paper presents a technical pipeline for synthesizing stress-related facial expressions on 3D medical mannequins using the Facial Action Coding System (FACS) as a unifying framework. The model-based component generates graded pain expressions from force sensor input using the Prkachin-Solomon formulation, in which the intensities of the core action units (AUs) are derived by uniformly distributing the pain intensity value. The video-based component captures choking and coughing expressions from video recordings of a performer demonstrating these expressions, estimates AU intensity time series with OpenFace 2.0, and drives shape-key deformations of a 3D head model in Blender. A validation study with 16 nursing students assessed the recognizability of the synthesized expressions across four presentation conditions. The highest recognition accuracy (11 of 16 participants, 68.8%) was achieved when the facial animation was presented without audio, while the lowest accuracy (4 of 16, 25.0%) was observed when audio accompanied the dynamic expression, indicating that poorly synchronized audio cues can reduce recognition performance. The pipeline offers an open-source, low-cost approach for integrating dynamic patient affect into simulation-based healthcare training.
Source: Research Square
Authors: Ahmad Ridwan Fauzi, Trini Handayani
Institution: Telkom University
Language: English
Subfield: Education
Field: Social Sciences
Domain: Social Sciences
Open Access status: green
OpenAlex ID: W7204166387
