Obscura.NLP.Artifacts (obscura v0.1.0)

Dependency-light NLP artifacts for analyzer scoring.

Presidio runs NLP once and passes token, lemma, keyword, and offset artifacts through the analyzer. Obscura keeps the default implementation deterministic and dependency-free while exposing the same kind of data to context scoring and future recognizers.

Summary

Types

t()

token_offset()

Functions

build(text)

Builds deterministic token artifacts from raw text.

normalize_token(token)

Normalizes a token for whole-word context matching.

put_model_outputs(artifacts, outputs)

Attaches precomputed model outputs to artifacts.

surrounding_terms(artifacts, start, end_offset, opts \\ [])

Returns normalized surrounding token text around a byte span.

surrounding_token_indexes(artifacts, start, end_offset, opts \\ [])

Returns token indexes around a byte span.

Types

t()

@type t() :: %Obscura.NLP.Artifacts{
  keywords: [String.t()],
  lemmas: [String.t()],
  model_outputs: [map()],
  model_outputs_ready: boolean(),
  normalized_tokens: [String.t()],
  text: String.t(),
  token_offsets: [token_offset()],
  tokens: [String.t()]
}

token_offset()

@type token_offset() :: %{byte_start: non_neg_integer(), byte_end: non_neg_integer()}

Functions

build(text)

@spec build(String.t()) :: t()

Builds deterministic token artifacts from raw text.

normalize_token(token)

@spec normalize_token(String.t()) :: String.t()

Normalizes a token for whole-word context matching.

put_model_outputs(artifacts, outputs)

@spec put_model_outputs(t(), [map()]) :: {:ok, t()} | {:error, term()}

Attaches precomputed model outputs to artifacts.

This mirrors Presidio's pattern where a model/NLP engine can run once and downstream recognizers consume the resulting artifacts instead of invoking model serving independently.

surrounding_terms(artifacts, start, end_offset, opts \\ [])

@spec surrounding_terms(t(), non_neg_integer(), non_neg_integer(), keyword()) :: [
  String.t()
]

Returns normalized surrounding token text around a byte span.

surrounding_token_indexes(artifacts, start, end_offset, opts \\ [])

@spec surrounding_token_indexes(t(), non_neg_integer(), non_neg_integer(), keyword()) ::
  [
    non_neg_integer()
  ]

Returns token indexes around a byte span.