Semantic unit guided image understanding for human AI collaborative image editing via multimodal deep learning
Year: 2026
Type: article
Abstract: This paper addresses natural-language-feedback-assisted image editing by examining how semantic role decision-making can reduce unintended changes to unspecified subjects, backgrounds, styles, and contextual relations. Existing instruction-based methods commonly represent editing intent or spatial scope without explicitly distinguishing content to be edited, preserved, or confirmed. This paper therefore proposes a semantic-unit-guided framework that assigns target, protected, and uncertain roles before editing. Target units guide the requested modification, protected units impose spatial or feature-level consistency constraints, and uncertain units support confirmation-based role updating. Experiments on InstructPix2Pix and MagicBrush, with zero-shot evaluation on a 500-sample RealEdit subset, compare the framework with nine representative baselines and binary target/non-target masking. The three-role model improves Protected Semantic Preservation over binary masking by 2.3% points on InstructPix2Pix and 2.9 points on MagicBrush. Its preservation advantage remains 4.5 and 6.5 points under matched-Edit-Success conditions. Protected Semantic Preservation correlates with human preservation ratings at 0.718. Alternative annotation criteria preserve the main performance pattern, while failure and sensitivity analyses identify ambiguous instructions, grounding quality, confidence thresholds, and preservation weight as influential factors. Feedback results are obtained under a benchmark-based uncertainty-confirmation protocol using subsequent instructions and annotation-derived responses; direct user comprehension, usability, and engagement are not evaluated.
Source: Discover Artificial Intelligence
Authors: Xiaoqian Yang, Xiaoping Gu
Institution: Zhixing College of Hubei University
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
Estimated APC paid: $1,590
OpenAlex ID: W7237344167
