Year: 2024
Type: article
Abstract: Multi-modal vector search systems extend state-of-the-art information retrieval and enable unified search across data modalities that are as varied as text, image, audio and video.The contribution of this work is to present in detail the architecture needed for efficiently implementing a Multi-modal vector search system.In this paper, we discuss challenges in each of these different aspects: from embedding generation and storage optimization to cross-modal retrieval.The framework introduces the layered design approach that deals with main challenges such as vector space alignment, efficient indexing strategy and dynamic query processing.It uses advanced neural models in generating embeddings, uses optimized storage solutions with HNSW graphs and superior cross-modal fusion techniques.This paper provides practical guidelines to an organization that is implementing a Multi-modal vector search solution on how to choose an appropriate strategy regarding embeddings, optimizing the index and cross-modal fusion techniques.We also present some methodologies and metrics to evaluate such systems in a real-world deployment scenario.
Author: Prabu Arjunan
Language: English
Subfield: Computer Vision and Pattern Recognition
Field: Computer Science
Domain: Physical Sciences
Open Access status: bronze
OpenAlex ID: W4407704939