Optimizing Chunk Size in Retrieval-Augmented Generation (RAG) Pipelines for Enhanced API Latency
Year: 2026
Type: article
Abstract: : Retrieval-Augmented Generation (RAG) has become the standard architecture for integrating large language models (LLMs) with custom knowledge bases. However, in production environments, API latency remains a critical bottleneck. This study investigates the impact of document chunk size on the overall response time of a Naive RAG pipeline. By evaluating three distinct token configurations (250, 500, and 1000) using FAISS and Google's Gemini models, we identify an optimal "sweet spot" that balances vector search efficiency with LLM context processing overhead.
Author: AKBAR, SHAHZAD
Language: English
Topic: Topic Modeling
Subfield: Artificial Intelligence
Field: Computer Science
Domain: Physical Sciences
Open Access status: green
OpenAlex ID: W7204116876
FWCI:
Cites:
Cited by:
Related to: