LangExtract (LangExtract v0.4.0)

Extracts structured data from text with source grounding. Maps extraction strings back to exact byte positions in source text.

This module is the main entry point: new/2 builds a client, run/4 executes the full pipeline, and align/3 / extract/3 expose the lower-level steps. Beyond the facade:

LangExtract.Prompt.Validator — pre-flight check that few-shot examples align against their own source text
LangExtract.Serializer — convert results to plain maps and JSONL for storage or interop
LangExtract.Extraction — the extraction struct used in template examples and parsed LLM output

Summary

Functions

align(source, extractions, opts \\ [])

Aligns extraction strings to byte spans in source text.

extract(source, raw_llm_output, opts \\ [])

Parses LLM output, aligns extractions against source text, and returns enriched spans with class and attributes.

new(provider, opts \\ [])

Creates a configured LLM client for extraction.

run(client, source, template, opts \\ [])

Runs the full extraction pipeline: prompt → LLM → parse → align.

Types

provider()

@type provider() :: :claude | :openai | :gemini

Functions

align(source, extractions, opts \\ [])

@spec align(String.t(), [String.t()], keyword()) :: [LangExtract.Alignment.Span.t()]

Aligns extraction strings to byte spans in source text.

Returns a list of %LangExtract.Alignment.Span{} structs, one per extraction.

Options

:fuzzy_threshold - minimum overlap ratio for fuzzy match (default 0.75)

Examples

iex> LangExtract.align("the quick brown fox", ["quick brown"])
[%LangExtract.Alignment.Span{text: "quick brown", byte_start: 4, byte_end: 15, status: :exact}]

extract(source, raw_llm_output, opts \\ [])

@spec extract(String.t(), String.t(), keyword()) ::
  {:ok, [LangExtract.Alignment.Span.t()]}
  | {:error, {:invalid_format, String.t()} | :missing_extractions}

Parses LLM output, aligns extractions against source text, and returns enriched spans with class and attributes.

Accepts both canonical and dynamic-key format (where each entry uses the class name as the key). Strips markdown fences and think tags before parsing YAML.

Options

:fuzzy_threshold - minimum overlap ratio for fuzzy match (default 0.75)

Examples

iex> yaml = "extractions:\n- class: word\n  text: fox"
iex> {:ok, [span]} = LangExtract.extract("the quick brown fox", yaml)
iex> span.status
:exact

new(provider, opts \\ [])

@spec new(
  provider(),
  keyword()
) :: LangExtract.Client.t()

Creates a configured LLM client for extraction.

Examples

client = LangExtract.new(:claude, api_key: "sk-...")
client = LangExtract.new(:openai, api_key: "sk-...", model: "gpt-4o")
client = LangExtract.new(:gemini, api_key: "gm-...")

run(client, source, template, opts \\ [])

@spec run(
  LangExtract.Client.t(),
  String.t(),
  LangExtract.Prompt.Template.t(),
  keyword()
) ::
  {:ok,
   {[LangExtract.Alignment.Span.t()], [LangExtract.Pipeline.ChunkError.t()]}}
  | {:error, term()}

Runs the full extraction pipeline: prompt → LLM → parse → align.

Returns {:ok, {spans, chunk_errors}} on success. When some chunks fail to parse, the successful spans are still returned alongside the errors. Returns {:error, reason} only for infrastructure failures (task exits, timeouts).

Options

:fuzzy_threshold - minimum overlap ratio for fuzzy match (default 0.75)

Examples

client = LangExtract.new(:claude, api_key: "sk-...")
template = %LangExtract.Prompt.Template{description: "Extract entities."}
{:ok, {spans, errors}} = LangExtract.run(client, "the quick brown fox", template)