Multimodal GIP-TTA: A Somatic-Psychic-Noetic Framework for Cross-Modal Test-Time Adaptation
Year: 2025
Type: preprint
Abstract: Multimodal systems (vision-language-audio) excel in-distribution but often fail under domain shift, missing modalities, or adversarial perturbation. We propose Multimodal GIP-TTA, a principled framework for test-time adaptation (TTA) grounded in the Genesis-Integration-Optimization (GIP) triad. Each modality is modeled across three strata: SOMA (signal reliability), PSYCHE (representation agreement), and NOUS (goal alignment). Our contributions are threefold: (i) a tri-layer functional map linking multimodal processing to SOMA/PSYCHE/NOUS; (ii) the Systemic Coherence Score (SCS-mm), a metric integrating intra-modal stability, cross-modal agreement, and goal fidelity; and (iii) a lightweight TTA procedure combining entropy minimization, agreement maximization, and goal-prior regularization. Preliminary benchmarks on CLIP, BLIP-2/LLaVA, and Whisper/AudioSet demonstrate improved robustness under synthetic and natural shifts. This work reframes test-time adaptation from a narrow technical fix into a core capacity for adaptive intelligence, establishing a foundation for more resilient multimodal AI.
Author: Takashi Kubo
Institution: Nihon University
Language: English
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: gold
OpenAlex ID: W4414799045
Cites:
Cited by:
Related to: