Changelog

Copy Markdown View Source

0.5.0 — 2026-08-30

Specs are now stored decomposed and read lazily. Ingest no longer dereferences a whole document into one artifact the registry holds: it takes the document apart — canonical evidence, one row per operation, one per component, every payload $ref left where the document wrote it — and a registration binds the {store, ref} those pieces live at, reading one operation's refs per call. The same change reaches the sandbox: instead of a dereferenced document spliced into a search run, search_apis ships a slim operation index (id, method, path, summary, tags, parameter names) and the model calls specs.<api>.describe(id | [ids]) for the schemas of the few operations it picked — a metered host round trip into the store. A large spec therefore costs a run its index rather than its document, and a host that keeps specs in a database of its own implements one behaviour instead of caching artifacts.

Added

  • OapiCodemode.SpecStore: the behaviour between the library and wherever specs actually live. A store is {module, handle}, a ref is {spec_id, decomposer_version} naming one projection of one document, and the six callbacks are put/2, meta/2, index/2, operation/3, components/3 (bulk, one call per breadth-first level of a resolve) and pointer/4 (the slow path, byte-bounded). Module-level functions mirror the callbacks so callers hold a store instead of unpacking tuples. %SpecStore.Meta{} is what a registration reads once; %SpecStore.OperationSummary{} is one slim-index entry.
  • OapiCodemode.SpecStore.ETS: the implementation the library ships with — one :set table, new/0 for the handle, rows gated on a {:meta, ref} commit marker written last so a projection mid-write is invisible rather than half-visible, and delete_projection/2 to free one.
  • OapiCodemode.SpecStoreCase: the conformance suite, shipped in lib/ so a host's own store is held to the same contract — use OapiCodemode.SpecStoreCase and supply store_fixture/0. It covers the decompose → put → meta/index/operation/components round trip, put of a known hash, two decomposer versions coexisting, concurrent put, pointer/4's byte budget and array walk, absent component names, :not_found for an unknown ref or id, and index order.
  • OapiCodemode.Resolve: read-time $ref resolution — one stored operation row plus the store in, one usable operation out. A breadth-first fetch pass (a whole layer of refs per components/3 call) followed by a depth-first substitute pass, so output is byte-identical to the whole-document expansion 0.4.0 shipped; for: :request resolves parameters and the request body, for: :describe also responses; max_bytes: bounds the resolved closure. Unfollowable refs stay readable as data: %{"$circular" => name}, %{"$unresolved" => ref}, and now %{"$truncated" => true} for a subtree that did not fit the budget.
  • OapiCodemode.Pointer: same-document JSON pointer resolution in one place (Ingest, Deref and a store's pointer/4 all need the same answer), with escape/1 and unescape/1. resolve/3 takes arrays: true to walk numeric array-index tokens per RFC 6901 — off by default, so every existing caller keeps the maps-only 0.4.0 walk.
  • OapiCodemode.Ingest.decompose/1 and %OapiCodemode.Decomposed{}: take a document apart into canonical evidence, the document's own fields, this decomposer's projection, one entry per operation and one per component, inlining nothing. Read-time resolution is the caller's job.
  • OapiCodemode.Ingest.decomposer_version/0 and OapiCodemode.Ingest.component_kinds/0: the version a projection is stored under, and the component kinds a store therefore holds (everything standard except examples).
  • OapiCodemode.Ingest.index_entry/1: the single definition of a slim-index entry. The projection's index_bytes is the encoded size of exactly these entries, so an index builder must go through it rather than rebuild the shape.
  • OapiCodemode.Ingest.Normalize.extract_body/1 and parse_segments/1 are now public. The first means read-time resolution of a $ref'd requestBody reaches the same %{"required", "content_type", "schema"} shape as an inline body by construction; the second lets an %OperationSummary{} derive its match segments without persisting a second spelling of path.
  • OapiCodemode.Registry.private_store/1: the library-private SpecStore.ETS store ingest_and_register/4 puts into, owned by the registry process. Exposed so a host can see what the convenience path did; see the warning under Changed before holding a ref into it.
  • OapiCodemode.Registry.collect_unreferenced/2: frees a projection in that private store which no live binding names. ingest_and_register/4 calls it on the one path that can strand one — the put landed and register/4 then refused (a name that is not a JS identifier, an index too large, a bad idempotency header) — so a refusal leaves the store exactly as it found it.
  • OapiCodemode.Proxy.Matcher.nearest_ids/2: the five operation ids nearest a given one by Jaro distance, so describe can name the near miss an unknown id usually is (the same self-correction the matcher already offers a request that matches no route).
  • OapiCodemode.Tools.describe_limits/0: the budgets a search run's describe is held to — 10 ids per call, 512 KB of resolved JSON per run. Public so the tool description states the numbers the callback enforces rather than a second copy of them.
  • ApiConfig.max_resolve_bytes (default 5_000_000): a cap on the spec bytes one request may resolve. Sibling of max_response_bytes, which bounds what comes back from the API; this bounds what comes back from the spec store. Each stored row is capped at 1 MB, but a request resolves an operation's whole transitive $ref closure and nothing else capped that aggregate. Over-budget subtrees arrive as {"$truncated": true}, which validation reads as "no schema here" and lets through — a guardrail against a pathological spec, not a reason to refuse the model's request.
  • A :resolve phase for [:oapi_codemode, :request, :error] telemetry: the spec store failing to hand back an operation the index lists, or handing back one that disagrees with it. Our storage, not the model's request, so the message crossing back to the sandbox is fixed and the store's own reason goes to Logger.

Changed

  • Breaking: OapiCodemode.register/4 binds a store, not an artifact. It takes {store, ref} — a OapiCodemode.SpecStore module/handle pair and the {spec_id, decomposer_version} a put/2 returned — and the registry holds that binding plus the runtime ApiConfig and a small cache (the slim index, pre-encoded for the sandbox, plus title, tags and security schemes). No spec bytes live in the registry, with one exception: ingest_and_register/4 puts into the library-private ETS store the registry process owns, and those bytes are a store's, not a registration's — re-registering an api name through it frees the projection that name superseded. Registration reads the store exactly twice (meta/2, index/2) and refuses, rather than degrading, when: neither the config nor the document names a base URL (:no_base_url), the slim index exceeds 8 MB (:index_too_large), the ref names a decomposer_version this library did not derive ({:unknown_decomposer_version, version}), or the store's index/2 does not encode to the index_bytes its meta/2 reported ({:index_bytes_mismatch, spliced, reported}).
  • Breaking: OapiCodemode.ingest/1 returns %OapiCodemode.Decomposed{}. It delegates to Ingest.decompose/1. ingest_and_register/4 is unchanged from the outside: it decomposes and puts into the library-private ETS store, then registers the resulting binding.
  • Breaking: the executor callback contract is named host callbacks. env.callbacks is a map of name to an arity-1 function over the JSON-decoded argument list (fn [api_name, opts] -> ... end, was fn api_name, opts -> ... end), and an executor exposes each one to the guest as host.<name>(...args) — :request for execute runs, :describe for search runs. The apis.<name>.request(opts) sugar the model is taught is unchanged; it now forwards through host.request. A custom executor must build the host object and switch its callback invocation to the list form; a host testing with Executor.Mock calls env.callbacks.request.([name, opts]).
  • Breaking: search_apis's specs.<api> global is {operations, describe}, not the dereferenced document. specs.<api>.operations is the slim index — an array of {id, method, path, summary, tags, params}, no schemas — and await specs.<api>.describe(id | [ids]) resolves the operations the model picked in full (parameter schemas, request body, responses, security), reading the spec store per call rather than shipping a document into the sandbox. Sandbox code that walked specs.<api>.paths must iterate specs.<api>.operations instead. describe is metered: at most 10 ids per call, one call charged per id against :max_calls, and 512 KB of resolved JSON per search run, after which it answers {id, error: "describe budget exhausted"}. Refusals — an unknown id (which names the nearest ids), a batch too big, a spent budget, an API outside the call's :api_allowlist — are data in the result, never raised errors, and an array of ids always answers an array of the same length in the same order, one entry per id: a batch over the cap refuses per id rather than handing back a bare error object a .map() would trip over. The numbers are OapiCodemode.Tools.describe_limits/0, which the tool description reads so it states what the callback enforces (the worked example included, which slices its batch to the cap). Metering comes first: a run that has spent its calls or its bytes refuses further describes before touching the registry, so a loop of them costs no lookups.
  • Operation gained description, responses, deprecated and param_index. request_body is now a union: either a $ref map kept verbatim for lazy resolution, or the extracted media-type shape.
  • param_index is filtered where the row is built, not where the index is projected: a parameter whose identity could not be resolved — a dangling $ref, a chain past the hop limit — has no entry, so param_index is exactly what the slim index publishes and Ingest.index_entry/1 is a pure projection of the operation row. A store persists param_index and hands it back; it decides nothing about it. It is therefore no longer positionally aligned with parameters, which still carries every $ref verbatim. OapiCodemode.SpecStoreCase pins both ends.
  • Behaviour change on malformed input. Decomposition applies Normalize's scalar coercions throughout, which slightly changes what ingesting does with a document that lies about its types: a non-string operationId is treated as absent (the id is derived instead of being carried through), a non-string summary or description becomes nil rather than the raw value, non-string elements of tags are dropped, and a non-list security is coerced rather than passed along. Well-formed documents are unaffected.
  • The proxy's per-request store read is bounded, ordered and cross-checked. Both policy checks now run before the read — the read-only refusal needs only the method, which the index entry carries, so a request this tool may not make costs zero store reads — the resolved closure is bounded by max_resolve_bytes, and the resolved operation's path is checked against the index entry's. The matcher bound path params against the index template, so a store that disagreed would have those bindings substituted into a different one: a credentialed request to a URL the model never named. That is a :resolve-phase error, with the disagreement logged.
  • The proxy passes Resolve's plain atom-keyed map downstream rather than struct!-ing it into an %Operation{}: a stored operation holds $refs, a resolved one holds markers and, under for: :request, no responses key at all, where the struct would have defaulted it to nil and made "we did not look" indistinguishable from "the document declared none". Proxy.Validator and Proxy.Matcher take either shape.
  • OapiCodemode.Registry.private_store/1's handle is documented as collectible. The registry frees projections in that store which no live binding names, so a ref held where the registry cannot see it — put directly and kept, registered into a second registry, held across a re-registration of the name it was bound to — can start answering {:error, :not_found}. A stale read, never a wrong one. Hosts that want a retention policy of their own make their own SpecStore.ETS.
  • OapiCodemode.SpecStore.OperationSummary carries segments, derived from path (it is not part of the JSON entry), so the proxy can match against the registry's cached index and resolve only the one operation it is about to send.

Removed

  • Breaking: OapiCodemode.Artifact and the whole-document ingest/1 pipeline. Nothing dereferences a whole document at ingest any more: decomposition stores the pieces and OapiCodemode.Resolve follows the refs of the one operation being read. OapiCodemode.Ingest.Deref survives as the parity oracle the resolve suite checks lazy resolution against — tests only, not library code.