LangExtract.Serializer (LangExtract v0.9.0)

Copy Markdown View Source

Serialization and deserialization of extraction results.

Converts between LangExtract structs and plain maps/JSON for storage, debugging, and interop with external systems. result_to_map/2 / result_from_map/1 cover the full LangExtract.Result (spans, errors, usage); to_map/2 / from_map/1 cover bare span lists (e.g. from LangExtract.align/3).

Error reasons are open terms, so they serialize as their inspect/1 rendering — JSON-safe, but one-way: a loaded ChunkError carries the rendered string, not the original term.

Summary

Functions

Converts a LangExtract.ChunkError to a plain map with string keys.

Converts a plain map back to extraction results.

Loads extraction results from a JSONL file.

Converts a plain map back to {source, %LangExtract.Result{}}.

Converts a full LangExtract.Result and its source to a plain map.

Saves a list of extraction results to a JSONL file.

Converts a single span to a plain map with string keys.

Converts extraction results to a plain map.

Functions

chunk_error_to_map(error)

@spec chunk_error_to_map(LangExtract.ChunkError.t()) :: map()

Converts a LangExtract.ChunkError to a plain map with string keys.

The open reason term is rendered with inspect/1 so the map is always JSON-encodable.

from_map(arg1)

@spec from_map(map()) ::
  {:ok, {String.t(), [LangExtract.Span.t()]}} | {:error, :invalid_data}

Converts a plain map back to extraction results.

Returns {:error, :invalid_data} if the shape is wrong, an extraction has an unknown "status", or field types don't match the Span invariants (located spans carry integer offsets, not_found spans carry nil).

load_jsonl(path)

@spec load_jsonl(Path.t()) ::
  {:ok, [{String.t(), [LangExtract.Span.t()]}]}
  | {:error, File.posix() | :invalid_data}

Loads extraction results from a JSONL file.

result_from_map(map)

@spec result_from_map(term()) ::
  {:ok, {String.t(), LangExtract.Result.t()}} | {:error, :invalid_data}

Converts a plain map back to {source, %LangExtract.Result{}}.

Returns {:error, :invalid_data} if the shape or field types are wrong — validation is strict, so a decoded struct upholds the same invariants as a pipeline-produced one. Error reasons come back as the inspect/1 strings result_to_map/2 wrote.

result_to_map(source, result)

@spec result_to_map(String.t(), LangExtract.Result.t()) :: map()

Converts a full LangExtract.Result and its source to a plain map.

The map extends to_map/2's shape with "errors" (see chunk_error_to_map/1) and "usage" (string-keyed token counts, or nil when the run reported none).

save_jsonl(results, path)

@spec save_jsonl([{String.t(), [LangExtract.Span.t()]}], Path.t()) ::
  :ok | {:error, File.posix()}

Saves a list of extraction results to a JSONL file.

Each element is a {source, spans} tuple.

span_to_map(span)

@spec span_to_map(LangExtract.Span.t()) :: map()

Converts a single span to a plain map with string keys.

to_map(source, spans)

@spec to_map(String.t(), [LangExtract.Span.t()]) :: map()

Converts extraction results to a plain map.