AgentSea.Ingest.Chunker (agentsea_ingest v0.1.0)

Splits text into overlapping word windows. Overlap preserves context across chunk boundaries (so a fact split between two chunks still embeds coherently in at least one).

Summary

Functions

chunk(text, opts \\ [])

Chunk text into word windows.

Functions

chunk(text, opts \\ [])

@spec chunk(
  String.t(),
  keyword()
) :: [String.t()]

Chunk text into word windows.

Options: :size (words per chunk, default 120) and :overlap (words shared with the previous chunk, default 20).