Extracts structured data from text with source grounding. Maps extraction strings back to exact byte positions in source text.
This module is the main entry point: new/2 builds a client,
template/2 builds a validated task definition, and run/4 /
stream/4 execute the full pipeline (chunk → LLM → parse → align).
For replaying stored model output without another API call, see
extract/3.
Beyond the facade:
LangExtract.Prompt.Validator— pre-flight check that few-shot examples align against their own source textLangExtract.Serializer— convert results to plain maps and JSONL for storage or interopLangExtract.Extraction— the extraction struct used in template examples and parsed LLM outputLangExtract.Alignment.Aligner— the grounding engine the pipeline already runs for you (public but best-effort; see Stability in the README)
Summary
Functions
Parses raw LLM output, aligns extractions against source text, and returns spans with class and attributes.
Creates a configured LLM client for extraction.
Runs the full extraction pipeline: prompt → LLM → parse → align.
Streams per-chunk extraction results as each chunk completes.
Builds a validated extraction template.
Types
Functions
@spec extract(String.t(), String.t(), keyword()) :: {:ok, [LangExtract.Span.t()]} | {:error, {:invalid_format, String.t()} | :missing_extractions}
Parses raw LLM output, aligns extractions against source text, and returns spans with class and attributes.
Use this to replay or re-ground a stored model response without calling the
provider again. For live extraction from a document, use run/4.
Accepts both canonical and dynamic-key format (where each entry uses the class name as the key). JSON only (since 0.7.0). Strips markdown fences and think tags before parsing.
Options
:fuzzy_threshold- minimum overlap ratio for fuzzy match (default0.75):min_density- minimum matched-token density for fuzzy (default1/3):accept_lesser- accept prefix partial matches (defaulttrue):exact_algorithm-:dp(default) or:first_occurrence
Examples
iex> raw = ~s({"extractions": [{"class": "word", "text": "fox"}]})
iex> {:ok, [span]} = LangExtract.extract("the quick brown fox", raw)
iex> span.status
:exact
@spec new( provider(), keyword() ) :: LangExtract.Client.t()
Creates a configured LLM client for extraction.
Raises ArgumentError on an unknown provider or unbuildable HTTP client
(e.g. missing API key). The raise is deliberate: misconfiguration here is
a programmer error caught at client construction, while runtime failures
during extraction stay data — per-chunk errors in run/4's Result,
tagged tuples from extract/3.
Examples
client = LangExtract.new(:claude, api_key: "sk-...")
client = LangExtract.new(:openai, api_key: "sk-...", model: "gpt-4o")
client = LangExtract.new(:gemini, api_key: "gm-...")
@spec run(LangExtract.Client.t(), String.t(), LangExtract.Template.t(), keyword()) :: LangExtract.Result.t()
Runs the full extraction pipeline: prompt → LLM → parse → align.
Returns a %Result{} — document-ordered spans plus per-chunk errors.
Handled failures (parse, HTTP, task timeout) stay per-chunk in
Result.errors with the chunk's byte range; surviving chunks' spans
are still returned. This path does not isolate bug-level crashes:
chunk tasks run linked, so a raise inside one exits the caller (no
Result). Prefer LangExtract.Runner.run/4 in production for retries,
a shared request budget, and crash isolation (supervised tasks report
crashes as ChunkErrors too). See the failure-semantics table in the
"Running in Production" guide.
Options
:max_chunk_chars- chunk size in characters (default1000):max_concurrency- parallel chunk requests (default10):task_timeout- per-chunk task timeout (default:infinity). Standalonerun/4/stream/4only —LangExtract.Runnerignores it; its requests are bounded by HTTP timeouts and the retry policy:fuzzy_threshold- minimum LCS coverage for fuzzy match (default0.75):min_density- minimum matched-token density of a fuzzy span (default1/3):accept_lesser- allow prefix-fragment grounding as:lesserspans (defaulttrue):exact_algorithm-:dp(occurrence DP, default) or:first_occurrence
The chunk budget is measured in characters because it mirrors upstream
langextract's max_char_buffer: counting the same way keeps chunk
boundaries identical across the two libraries, which the cross-library
benchmarks depend on. Every output offset is bytes, and since chunking
is sentence-aware, boundaries can't be computed from the budget in
either unit — the byte ranges on results are the boundary source of
truth. See the "Alignment and Spans" guide for what the alignment
options tune.
Examples
client = LangExtract.new(:claude, api_key: "sk-...")
template = LangExtract.template("Extract entities.")
%LangExtract.Result{spans: spans, errors: errors} =
LangExtract.run(client, "the quick brown fox", template)
@spec stream(LangExtract.Client.t(), String.t(), LangExtract.Template.t(), keyword()) :: Enumerable.t()
Streams per-chunk extraction results as each chunk completes.
Returns a lazy stream of {:ok, %ChunkResult{}} and
{:error, %ChunkError{}} events in completion order, not
document order — consumers who need latency don't wait for slow chunks;
consumers who need order sort by the byte ranges every event carries.
Nothing runs until the stream is consumed, and a slow consumer naturally
limits how many chunk requests are in flight.
Failure semantics match run/4: every failure stays per-chunk. A chunk
whose task times out is reported as
{:error, %ChunkError{reason: {:task_exit, :timeout}}} with its byte
range, and the remaining chunks keep flowing — run/4 is exactly this
stream, collected and restored to document order.
Takes the same options as run/4.
Examples
client
|> LangExtract.stream(document, template)
|> Enum.each(fn
{:ok, chunk_result} -> handle_spans(chunk_result.spans)
{:error, chunk_error} -> log_failure(chunk_error)
end)
@spec template( String.t(), keyword() ) :: LangExtract.Template.t()
Builds a validated extraction template.
Examples are given as plain maps (string or atom keys, so JSON-loaded
task definitions work verbatim) or as ready-made structs. Map-authored
attributes are normalized to string keys, matching the wire format's
decoded shape; ready-made structs pass through unchanged. Each example's
extraction texts are validated against the example text using the
production aligner; misaligned examples raise
LangExtract.Prompt.Validator.ValidationError — a template that
constructs is a template whose examples align, unconditionally.
Malformed example maps raise ArgumentError naming the offending field.
Raising without a bang follows the convention for single-variant
functions (new/2 is the same shape): a malformed template is a
programmer error, caught closest to the typo.
Examples
iex> template =
...> LangExtract.template("Extract conditions.",
...> examples: [
...> %{text: "Patient has diabetes.",
...> extractions: [%{class: "condition", text: "diabetes"}]}
...> ]
...> )
iex> [example] = template.examples
iex> example.extractions
[%LangExtract.Extraction{class: "condition", text: "diabetes", attributes: %{}}]