Work

An Empirical Analysis of Embedded versus Dedicated Client-Server Architectures for High-Throughput Vector Retrieval-Augmented Generation Systems

Year: 2026
Type: preprint
Abstract: (1) Background: The emergence of the Retrieval-Augmented Generation (RAG) family of approaches in enterprise AI has made vector similarity search one of the key infrastructure pillars, requiring a choice between an in-process storage engine and dedicated client-server vector database nodes. (2) Methods: We present a thorough empirical comparison between representative vector storage architectures including embedded storage (Chroma), lightweight client-server frameworks based on Rust (Qdrant), and cloud-native distributed databases (Milvus) across vector dimensions (384 to 1536), scale of datasets (up to 50k vectors), and multi-client concurrency queries (up to 100 virtual users). (3) Results: We found that embedded storage architectures perform well as a single-tenant solution, having low latency and memory footprint during startup, yet they experience significant throughput decrease in the presence of concurrent write/read activity. Client-server architectures provide better tail latency stability (\texorpdfstring{$P_{99}$}{P99}) and scalable throughput but introduce greater complexity and higher memory cost. (4) Conclusions: We summarize our results in a formal framework for selecting architecture by software engineers developing RAG backend solutions.
Language: English
Open Access status: green
OpenAlex ID: W7234875663
Cites:
Cited by:
Related to: 10