Guava.RAG (Guava v0.34.0)

Copy Markdown View Source

Retrieval-augmented-generation helpers.

Summary

Functions

Split a document into overlapping chunks on paragraph boundaries.

Functions

chunk_document(document, chunk_size \\ 5000, overlap \\ 200)

@spec chunk_document(String.t(), pos_integer(), non_neg_integer()) :: [String.t()]

Split a document into overlapping chunks on paragraph boundaries.

Paragraphs (separated by blank lines) are grouped until chunk_size characters, then a new chunk begins. With overlap > 0, the last paragraph of a chunk is carried into the next to preserve cross-boundary context.