Toolnexus.ContentPart (toolnexus v0.17.0)

Copy Markdown View Source

A multimodal content part (SPEC §1B) — text | image | file | audio.

A non-text part carries a mime_type (wire key mimeType) plus exactly one of data (standard base64, padded, unwrapped) or url. Both, or neither, is a typed construction error. A part never holds a filesystem path — a path does not survive a persisted-and-replayed transcript nor the MCP / A2A process boundary, so the edge constructors read and base64-encode the bytes at construction time.

ContentPart.image!("shot.png")                        # path  → data
ContentPart.image!("data:image/png;base64,iVBOR…")    # data: → {mimeType, data}
ContentPart.image!("https://example.com/a.png")       # https → url
ContentPart.image!({:bytes, bin}, mime_type: "image/png")
ContentPart.image!(["chunk", [?a | "bc"]], mime_type: "image/png")   # iodata
ContentPart.image!(File.stream!("shot.png", 2048))                   # File.Stream
ContentPart.image!(chunks, mime_type: "image/png")                   # any Enumerable

Accept broadly, store narrowly. A caller who already holds an iolist or a File.Stream should not have to flatten it by hand; whatever comes in, what lands in the part is bytes and a mime_type. A stream is consumed eagerly at construction — a part holding a half-read stream would not survive the transcript boundary any better than a path does.

A bare binary is always read as a path (or a data:/https: URL), never as raw content: a binary is ambiguous — "abc" is both a plausible filename and plausible bytes — and guessing would silently turn a caller's file into its own name. Raw bytes therefore stay explicitly tagged as {:bytes, bin}. An iolist has no such ambiguity (a path is never a list), so it is accepted unwrapped.

Every constructor has a {:ok, part} | {:error, exception} form and a raising ! form, mirroring Toolnexus.create_toolkit/1 / create_toolkit!/1.

type is a String, never an atom, so from_map/1 round-trips an unknown wire type without ever calling String.to_atom/1 on untrusted input.

Provider emission (to_block/2, encode/3) lives here rather than in Toolnexus.Adapters — the adapter module is tool-schema only (SPEC §8A) — and is public precisely so the (style × part-type) matrix is unit-testable without an LLM.

Summary

Functions

The positive allowlist of provider block types, per client style (SPEC §8A).

Assert an encoded block against the style's positive allowlist (SPEC §8A). A block that is not on the list never reaches the wire: an unknown block type upstream returns HTTP 200 with the content silently discarded, not an error.

Build an audio part from any accepted source (see new/3).

Like audio/2 but raises on failure and returns the part.

Decoded byte length of a part's payload — 0 for a URL-backed or text part.

Log/event rendering of a part (SPEC §9): {type, mimeType, bytes} — never data.

Encode a list of parts into provider blocks for style, applying the SPEC §8A provenance rule to any part the style cannot represent

Byte-derived token estimate for a part (SPEC §9). Never the length of the mimeType string — that would score a 5 MB image at ~3 tokens and make it uncompactable.

Build a file part from any accepted source (see new/3).

Like file/2 but raises on failure and returns the part.

Decode from a JSON-shaped map (string keys, mimeType spelling).

Build an image part from any accepted source (see new/3).

Like image/2 but raises on failure and returns the part.

{mimeType, type} for a path's extension, or :error when it is not in the table.

The fixed extension → {mimeType, part type} media table (SPEC §6).

Build a non-text part of type from source, normalising at construction (SPEC §1B)

Like new/3 but raises on failure and returns the part.

True when m is a ContentPart (or its wire map) rather than a provider block.

A one-line, data-free description of a non-text part, used as output when a tool or an MCP server returned parts and no text at all (so output is never a bare empty string).

A text part.

Map one part onto a provider block for style ("openai" | "anthropic").

Encode to a JSON-shaped map (string keys, nil fields omitted).

Validate a part built by hand: exactly one of data/url on a non-text part, and a mime_type unless the source is a bare URL.

Types

t()

@type t() :: %Toolnexus.ContentPart{
  data: String.t() | nil,
  mime_type: String.t() | nil,
  name: String.t() | nil,
  text: String.t() | nil,
  type: String.t(),
  url: String.t() | nil
}

Functions

allowlist(style)

@spec allowlist(String.t()) :: [String.t()]

The positive allowlist of provider block types, per client style (SPEC §8A).

assert_allowed!(blk, style, source \\ nil)

@spec assert_allowed!(map(), String.t(), term()) :: map()

Assert an encoded block against the style's positive allowlist (SPEC §8A). A block that is not on the list never reaches the wire: an unknown block type upstream returns HTTP 200 with the content silently discarded, not an error.

audio(source, opts \\ [])

@spec audio(
  term(),
  keyword()
) :: {:ok, t()} | {:error, Exception.t()}

Build an audio part from any accepted source (see new/3).

audio!(source, opts \\ [])

@spec audio!(
  term(),
  keyword()
) :: t()

Like audio/2 but raises on failure and returns the part.

byte_size_of(p)

@spec byte_size_of(t() | map()) :: non_neg_integer()

Decoded byte length of a part's payload — 0 for a URL-backed or text part.

describe(p)

@spec describe(t() | map()) :: map()

Log/event rendering of a part (SPEC §9): {type, mimeType, bytes} — never data.

encode(parts, style, opts \\ [])

@spec encode([t() | map()], String.t(), keyword()) :: [map()]

Encode a list of parts into provider blocks for style, applying the SPEC §8A provenance rule to any part the style cannot represent:

  • provenance: :attached (the caller attached it) ⇒ a typed error, raised at assembly, before any HTTP call;
  • provenance: :derived (it came off a tool / MCP result) ⇒ a text placeholder naming the type and mime type, warned at most once, the run continues.

on_unsupported ("error" | "text") overrides both uniformly. Every produced block is asserted against the style's positive allowlist before it is returned.

:max_part_bytes is enforced HERE, at assembly, over every part whatever its provenance — not only in the edge constructors. A part that arrived from an MCP server never passed through a constructor, so a limit it can walk around is not a limit. Going over routes through the same provenance rule as an unsupported part, so a server volunteering a huge image still cannot fail the caller's run.

estimated_tokens(p)

@spec estimated_tokens(t() | map()) :: non_neg_integer()

Byte-derived token estimate for a part (SPEC §9). Never the length of the mimeType string — that would score a 5 MB image at ~3 tokens and make it uncompactable.

file(source, opts \\ [])

@spec file(
  term(),
  keyword()
) :: {:ok, t()} | {:error, Exception.t()}

Build a file part from any accepted source (see new/3).

file!(source, opts \\ [])

@spec file!(
  term(),
  keyword()
) :: t()

Like file/2 but raises on failure and returns the part.

from_map(p)

@spec from_map(map()) :: t()

Decode from a JSON-shaped map (string keys, mimeType spelling).

image(source, opts \\ [])

@spec image(
  term(),
  keyword()
) :: {:ok, t()} | {:error, Exception.t()}

Build an image part from any accepted source (see new/3).

image!(source, opts \\ [])

@spec image!(
  term(),
  keyword()
) :: t()

Like image/2 but raises on failure and returns the part.

media_for_path(path)

@spec media_for_path(String.t()) :: {:ok, {String.t(), String.t()}} | :error

{mimeType, type} for a path's extension, or :error when it is not in the table.

media_table()

@spec media_table() :: %{required(String.t()) => {String.t(), String.t()}}

The fixed extension → {mimeType, part type} media table (SPEC §6).

new(type, source, opts \\ [])

@spec new(String.t(), term(), keyword()) :: {:ok, t()} | {:error, Exception.t()}

Build a non-text part of type from source, normalising at construction (SPEC §1B):

  • a data:<mime>;base64,<b64> URL → {mime_type, data}, never stored as url
  • an http:/https: URL → kept as url
  • {:bytes, binary} → base64 now; :mime_type required
  • iodata (any iolist, improper lists included) → IO.iodata_to_binary/1 now, base64 now; :mime_type required
  • a File.Stream → read to bytes now, base64 now; mime from the stream's .path via the fixed table (§6) unless :mime_type says otherwise
  • any other Enumerable of binary chunks → consumed eagerly now; :mime_type required
  • any other binary → a filesystem path: read now, base64 now, mime from the fixed table

A bare binary is never treated as raw content — see the module doc for why {:bytes, bin} stays explicitly tagged.

Options: :mime_type (required for bytes/iodata/enumerables, overrides the table for a path or a File.Stream), :name, :max_part_bytes (a cap on decoded bytes).

new!(type, source, opts \\ [])

@spec new!(String.t(), term(), keyword()) :: t()

Like new/3 but raises on failure and returns the part.

part?(m)

@spec part?(term()) :: boolean()

True when m is a ContentPart (or its wire map) rather than a provider block.

summary(p)

@spec summary(t() | map()) :: String.t()

A one-line, data-free description of a non-text part, used as output when a tool or an MCP server returned parts and no text at all (so output is never a bare empty string).

text(s)

@spec text(String.t()) :: t()

A text part.

to_block(part, style)

@spec to_block(t() | map(), String.t()) :: {:ok, map()} | {:unsupported, String.t()}

Map one part onto a provider block for style ("openai" | "anthropic").

Returns {:ok, block}, or {:unsupported, reason} when the style defines no shape for that part — anthropic × audio and openai × file+url are the named refusals.

to_map(p)

@spec to_map(t() | map()) :: map()

Encode to a JSON-shaped map (string keys, nil fields omitted).

validate(p)

@spec validate(t()) :: {:ok, t()} | {:error, Exception.t()}

Validate a part built by hand: exactly one of data/url on a non-text part, and a mime_type unless the source is a bare URL.