Year: 2025
Type: book-chapter
Abstract: Multimodal AI systems have the ability to represent and relate information in multiple data modalities. This enables a wide range of new applications when deployed in real world. They also face major challenges particularly due to domain shift arising from differences in the data they are trained on and the environment they encounter. This chapter focuses on domain adaptation for these systems. It involves assisting models to adjust to new conditions across different input modalities, such as text, images, or sensor data. Unlike single-modality systems, multimodal models must deal with unique challenges. This includes mismatch in data types, data quality, and conflicts during training across multiple modalities. We introduce these unique challenges with the multimodal systems. We also provide theoretical foundations using risk minimization and measures of divergence under domain shift across different modalities. From there, we also explore practical approaches such as adversarial training to align features, contrastive learning, and flexible fusion techniques that adjust when some inputs are unreliable. We also explore advanced techniques such as deployment-stage adaptation and fine-tuning large foundation models without complete retraining. We provide a summary of different datasets developed for multimodal domain adaptation. The chapter ends with summarizing key insights discussed throughout the chapter along with highlighting emerging opportunities in the domain.
Source: Artificial intelligence
Author: Raghavendran Ramakrishnan
Institution: IQVIA (United States)
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: hybrid
OpenAlex ID: W4416129205