Exploring Correlated Facial Attributes in Text-to-Image Models: Unintended Consequences in Synthetic Face Generation
Year: 2025
Type: conference-paper
Abstract: Text-based face manipulation is a powerful tool for generating and editing facial attributes in synthetic data, particularly for addressing imbalances in biometric datasets and helping to build fair and privacy-aware systems. However, controlling specific attributes remains challenging, as modifying one attribute often leads to unintended changes in others, potentially introducing new biases. In this paper, we investigate the cause of these unintended changes, hypothesizing that they stem from attribute correlations in the training data. By analyzing three face attribute datasets (CelebA, MAAD-Face, FFText-HQ), we identify common patterns of attribute co-occurrence and assess the behavior of three state-of-the-art manipulation models (ManiCLIP, FFCLIP, and DeltaEdit) using attribute classifiers to detect unwanted modifications. We found two types of errors: one where more attributes were altered than intended, and another where changes failed to occur due to dependencies on the original attributes of the face. While we hypothesized that these manipulation issues might correlate with dataset attribute correlations, our analysis using the Matthews Correlation Coefficient found no statistically significant relationships, suggesting that other factors may contribute to these unintended changes. This work can serve as a starting point for further research into the underlying causes of attribute entanglement in face manipulation models, helping to improve understanding and address related challenges.
Authors: Sander De Coninck, Sam Leroux, Pieter Simoens
Institutions: Ghent University, IMEC
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
Award: G2869903268
OpenAlex ID: W4409916833