Work

Controllable Facial Expression Synthesis via implicit keypoints and continuous labels

Year: 2026

Type: article

Abstract: Facial Expression Synthesis (FES) is essential for applications in virtual reality, humancomputer interaction, and digital media. Existing FES methods either rely on emotion labels, which lead to low image quality, or adopt image-driven approaches that reduce controllability. Thus, we propose VAPortraits, a novel framework that synthesizes diverse and high-quality facial expressions by manipulating implicit keypoints using an Expression Adjustment Network (EAN). Our method is conditioned on lowdimensional continuous labels, including valence-arousal (VA) and a more intuitive valence-eye-lip (VEL) representation, which replaces arousal with explicit eye and lip openness ratios. To balance expression flexibility and identity preservation, we also introduce a selective regularization loss, constraining only keypoints with minimal impact on expression. Extensive experiments demonstrate that VAPortraits surpasses state-of-the-art FES models in expression accuracy and identity retention, offering a controllable and scalable solution for realistic facial expression synthesis. Keywords: Image-to-image translation, Facial expression synthesis, Manipulating implicit keypoints, Valence-arousal

Source: Pattern Recognition

DOI: 10.1016/j.patcog.2026.113844

Authors: Yeong Min Lee, Byung Cheol Song

Institution: Inha University

Language: English

Topic: Generative Adversarial Networks and Image Synthesis

Subfield: Computer Vision and Pattern Recognition

Field: Computer Science

Domain: Physical Sciences

Open Access status: hybrid

Estimated APC paid: $2,590

Funders: National Research Foundation of Korea, Ministry of Science and ICT, South Korea, Institute for Information and Communications Technology Promotion