LemonAi.PromptDiagnostics (lemon_ai v0.1.0)

View Source

Lightweight, opt-in diagnostics for prompt size + token usage.

This module is intentionally conservative about what it records: it captures sizes, counts, and hashes (no raw prompt text).

Enable with:

export LEMON_AI_PROMPT_DIAGNOSTICS=1

Optionally set:

export LEMON_AI_PROMPT_DIAGNOSTICS_LOG_LEVEL=info
export LEMON_AI_PROMPT_DIAGNOSTICS_TOP_N=5

Recorded introspection event type:

  • :ai_llm_call

Summary

Functions

enabled?()

Return true if diagnostics are enabled via env var.

record_complete_call(model, ctx, opts, msg)

Record a combined prompt+usage snapshot for non-streaming calls.

record_llm_call(model, ctx, opts, msg)

Record a combined prompt+usage snapshot for a completed LLM call.

stats(ctx)

Compute a conservative size breakdown for an LLM context.

Functions

enabled?()

@spec enabled?() :: boolean()

Return true if diagnostics are enabled via env var.

record_complete_call(model, ctx, opts, msg)

@spec record_complete_call(
  LemonAi.Types.Model.t(),
  LemonAi.Types.Context.t(),
  LemonAi.Types.StreamOptions.t() | map(),
  LemonAi.Types.AssistantMessage.t()
) :: :ok

Record a combined prompt+usage snapshot for non-streaming calls.

record_llm_call(model, ctx, opts, msg)

@spec record_llm_call(
  LemonAi.Types.Model.t(),
  LemonAi.Types.Context.t(),
  LemonAi.Types.StreamOptions.t(),
  LemonAi.Types.AssistantMessage.t()
) :: :ok

Record a combined prompt+usage snapshot for a completed LLM call.

This records a single introspection event (:ai_llm_call) containing:

  • request sizing (bytes + hashes)
  • response usage (tokens, including cache read/write when available)

stats(ctx)

@spec stats(LemonAi.Types.Context.t()) :: map()

Compute a conservative size breakdown for an LLM context.