A multi stage coarse-to-fine network with semantic similarity for infrared and visible image registration
Year: 2026
Type: article
Abstract: Multimodal image registration aligns images in a common geometric space and is essential for tasks like image fusion and segmentation. Existing methods often focus only on rigid or non-rigid transformations, which limits their effectiveness when both are present. We propose a coarse-to-fine image registration framework, termed SA-CFNet. It performs rigid image registration in the coarse stage and non-rigid image registration in the fine stage. It consists of two key components: The Rigid Transformation Network (RNet) and Semantic-Guided Deformation Network (SGDNet). The role of RNet is to estimate a 6D transformation matrix that enables rotation and translation of the moving image, resulting in a coarse registered image. We use a strategy based on computing the similarity between image semantic to determine correspondences. The role of SGDNet is to determine a deformable field, through which the coarse-registered image can be deformed to achieve fine registration. The deformation field is also calculated using semantic similarity. We employ a coarse-to-fine estimation strategy to enhance the accuracy of the deformation field. We conducted experiments on image rigid transformation and non-rigid transformation using datasets such as RoadScene, TNO, and M3FD. To validate the effectiveness for downstream tasks, we designed an experiment on multimodal image fusion. The results show that our method remains robust in handling different rotations and translations for rigid image registration. Additionally, it achieves the best performance in other experiments.
Source: Optics & Laser Technology
Language: English
Subfield: Media Technology
Field: Engineering
Domain: Physical Sciences
Open Access status: hybrid
Estimated APC paid: $3,230
OpenAlex ID: W7125004703