Chunking

Data
About 1 min read

The process of splitting long documents into smaller segments suitable for retrieval and embedding, serving as the foundation of RAG quality.

Also known as
ChunkingDocument segmentationText splitting

Detailed explanation

Chunking is the process of splitting a long document into smaller segments (chunks) suitable for retrieval and embedding. Because RAG systems embed, store, and retrieve documents in chunk units rather than as a whole, how you split them significantly affects search accuracy and response quality. If chunks are too large, multiple topics mix into a single chunk, degrading search precision; if they are too small, context is lost. Key strategies include paragraph or sentence-based splitting, fixed token-length splitting, using overlaps to bridge chunks, or semantic splitting that preserves document structure (like headers and tables). Choosing the right method for the nature of your data is critical.

Why It Matters in Tool Selection

The answer quality of RAG and document search tools is often determined by the chunking strategy. Coarse splitting can mix irrelevant content, muddling search results, while ignoring tables or heading structures breaks the context. Whether a tool allows adjusting the chunking method, chunk size, and overlap, and whether it preserves structures like tables and code, dictates search accuracy.

Key Considerations

  • Can you adjust the chunk size and overlap?
  • Does it support splitting that preserves document structure like paragraphs, headers, and tables?
  • Does it accurately recognize Korean sentence boundaries for splitting?
  • Can you preview and debug the chunking results?

Real-World Examples

When building a RAG chatbot using a product manual, splitting the manual arbitrarily by every 1,000 characters can split a single procedure across two chunks, leading to inaccurate retrieval. Splitting by preserving headers and sequential structures while introducing a slight overlap ensures a procedure is contained entirely within a single chunk, improving response accuracy.

Related terms

RAGEmbeddingVector DatabaseContext Window