Work

Semantic unit guided image understanding for human AI collaborative image editing via multimodal deep learning

Year: 2026
Type: article
Abstract: This paper addresses natural-language-feedback-assisted image editing by examining how semantic role decision-making can reduce unintended changes to unspecified subjects, backgrounds, styles, and contextual relations. Existing instruction-based methods commonly represent editing intent or spatial scope without explicitly distinguishing content to be edited, preserved, or confirmed. This paper therefore proposes a semantic-unit-guided framework that assigns target, protected, and uncertain roles before editing. Target units guide the requested modification, protected units impose spatial or feature-level consistency constraints, and uncertain units support confirmation-based role updating. Experiments on InstructPix2Pix and MagicBrush, with zero-shot evaluation on a 500-sample RealEdit subset, compare the framework with nine representative baselines and binary target/non-target masking. The three-role model improves Protected Semantic Preservation over binary masking by 2.3% points on InstructPix2Pix and 2.9 points on MagicBrush. Its preservation advantage remains 4.5 and 6.5 points under matched-Edit-Success conditions. Protected Semantic Preservation correlates with human preservation ratings at 0.718. Alternative annotation criteria preserve the main performance pattern, while failure and sensitivity analyses identify ambiguous instructions, grounding quality, confidence thresholds, and preservation weight as influential factors. Feedback results are obtained under a benchmark-based uncertainty-confirmation protocol using subsequent instructions and annotation-derived responses; direct user comprehension, usability, and engagement are not evaluated.
Language: English
Open Access status: gold
Estimated APC paid: $1,590
OpenAlex ID: W7237344167
FWCI:
Cites: 34
Cited by:
Related to: