# Kinda Kinda is a Zig-first binding framework for Elixir. It is designed for projects that need to bind a large C API to BEAM without hand-writing every NIF, resource wrapper, and dispatch stub. The shortest useful description is: - `kinda` treats a wrapper header as the binding surface - extracts functions into a framework-owned manifest - extracts named C records and field types into that same framework-owned manifest - preserves Clang-extracted function docs when comments are available - preserves raw C parameter/return type facts in that manifest - generates generic Zig/Elixir wrapper outputs - lets consumer policy project those raw C facts into public Elixir typespecs - lets the same typed projection describe public record/field types in a machine-readable signature manifest - lets the consumer keep product-specific policy outside the framework This makes `kinda` a better fit for projects like `beaver`, where the problem is not “write one ergonomic NIF”, but “keep a large drifting native API coherent over time”. ## Run One Example If you want the shortest stable repo-local proof of the wrapper/reporting surface, run: ```sh mix kinda.wrapper.example ``` Useful modes: ```sh mix kinda.wrapper.example --json mix kinda.wrapper.example --report-only ``` To verify the whole repo from the root, including: - root tests - wrapper-reporting JSON smoke - bundled `kinda_example` - the `kinda_sqlite` driver and multi-NIF coexistence suite run: ```sh mix kinda.verify ``` This is the command CI should prefer for repo-level verification. If you only want the bundled application example from the repo root, run: ```sh mix kinda.example.verify ``` The SQLite integration can be verified independently with: ```sh mix kinda.sqlite.verify ``` The DuckDB packages live under `packages/kinda_duckdb` and `packages/ecto_kinda_duckdb`. The former provides a columnar direct API, Appender ingestion, pending queries, and DBConnection; the latter is an explicitly experimental query/transaction Ecto adapter and does not promise full migration or OLTP semantics. The backend-neutral sandbox lifecycle API lives in the independent `packages/kinda_sandbox` Mix project. Its first backend, `LocalNative`, provides isolated native build identities and directories without making the sandbox package depend on Kinda's Zig code generator. The old `Kinda.Testing.Sandbox` API has been removed rather than retained as a compatibility shim. If you want the same thing without compiling the project first, run: ```sh elixir examples/wrapper_reporting.exs ``` It prints: - the generated Elixir wrapper manifest - the machine-readable typed signature manifest - the human-readable callback-bridge backlog report - the machine-readable callback-bridge manifest - including dirty scheduler metadata when the example policy emits it ## What Kinda Ships Today Core runtime/building blocks: - `Kinda.ResourceKind` - `Kinda.CodeGen` - `Kinda.Precompiler` - `Kinda.Codec` - `Kinda.CallbackRuntime` for the common BEAM callback/reply boundary - `kinda.callback_runtime` for Zig-side native-thread callbacks into BEAM - `beam.ResourceRef(T)` for retaining parent resources from child resources Wrapper extraction/generation blocks: - `Kinda.Wrapper.Function` - `Kinda.Wrapper.Manifest` - `Kinda.Wrapper.Extract` - `Kinda.Wrapper.Generate` - `Kinda.Wrapper.Policy` - `Kinda.Wrapper.CallbackBridge` That means `kinda` now has an explicit split between: 1. generic wrapper extraction/generation 2. consumer-owned policy 3. callback-bridge backlog and runtime substrate ## Mental Model Kinda is not primarily a “write NIFs directly in Zig” library. It is a framework for binding a C library through three layers: 1. wrapper surface - a `.h` file says what native functions and types you want to expose 2. framework surface - `kinda` extracts that surface into a normalized manifest - `kinda` generates generic wrapper outputs from the manifest 3. consumer surface - your project decides naming, scheduler routing, diagnostics variants, unsupported APIs, and callback-heavy backlog For a large binding, this is the important compression: - `kinda` owns the generic mechanics - your library owns the semantics ## Quick Example Define a boundary codec, generate the raw NIF surface, and bind a resource kind directly to it: ```elixir defmodule Foo.Native do use Kinda.Codec end defmodule Foo.CAPI.Raw do use Kinda.CodeGen, with: Foo.Generated, root: Foo.CAPI, codec: Foo.Native, surface: :raw @on_load :load_nif def load_nif do :erlang.load_nif(~c"path/to/foo_nif", 0) end end defmodule Foo.CAPI do use Kinda.CodeGen, with: Foo.Generated, root: __MODULE__, raw_module: __MODULE__.Raw, codec: Foo.Native, surface: :public end defmodule Foo.Handle do use Kinda.ResourceKind, raw_module: Foo.CAPI.Raw, codec: Foo.Native end ``` Generated public wrappers call concrete functions in `Foo.CAPI.Raw`, unwrap resource arguments, and pass only the returned value through `Foo.Native`. The codec never selects a native function. Keeping `:public` wrappers and `:raw` NIF stubs in separate modules lets large bindings compile both surfaces independently and avoids a generated proxy layer. The raw module owns NIF loading, so the native entry module must be `Elixir.Foo.CAPI.Raw`. Generated `use Kinda.CodeGen` modules now expose one formalized declaration surface from the same typed source: - the resolved declaration IR through `__kinda_declaration_surfaces__/0` When a generator module implements `declaration_manifest/0`, `Kinda.CodeGen` can now source generated function/type declarations directly from that unified manifest instead of reconstructing them from parallel callbacks. That callback is now the canonical declaration interface in `kinda`: it may return a loaded manifest value or a checked-in `.ex` / `.json` sidecar path. Kind-derived helper NIFs still come from `kinds()/0`. `kinda` now also exposes a top-level downstream declaration facade through `Kinda.Declaration`. That facade sits over the same underlying `Kinda.CodeGen.DeclarationSurfaces` IR and formalizes the distinction between the declaration source and the generated module surface: - `Kinda.Declaration.load_source/1` reads the canonical source contract from a generator module - `Kinda.Declaration.from_generator/2` resolves that source contract into the final generated surfaces for a specific root module, including normalized `nif_name`s, kind-derived helper entries, guaranteed/materialized generated `TypeDecl`s, and the derived signature view - that resolved surface lands as the formal framework-owned IR: `Kinda.CodeGen.DeclarationSurfaces` Downstreams should treat `Kinda.Declaration.from_generator/2` as the formalized public resolution interface. Generated modules expose that same resolved IR directly through `__kinda_declaration_surfaces__/0`. Inside that IR, the canonical resolved payload is the declaration manifest itself; `nif_decls`, `type_decls`, and `signature_manifest` are no longer stored a second time on the resolved surface. That lets downstream repos such as Beaver unify their own public rewrite/pass DSL names without re-owning the declaration contract. When that manifest includes projected records, `Kinda.CodeGen` also emits deterministic public type aliases such as `foo_handle_record()/0` from the same single source. These generated aliases use atom field keys derived from extracted C field names, while the machine-readable manifest keeps the original string names. ## Structured NIF Call Errors Generated NIF failures raise `Kinda.CallError`. The legacy `:message` remains available, while stable fields identify the failed function and boundary phase. Argument decoding failures also include the one-based argument index, manifest parameter name, expected C type and the category of the original Elixir value when those facts are available. ```elixir try do Foo.CAPI.add(1, "2") rescue error in Kinda.CallError -> error.phase #=> :argument_decode Exception.message(error) #=> "add/2 rejected argument #2 (rhs): expected int, got binary" end ``` Internal native failures retain the native Zig error name and suggest `KINDA_DUMP_STACK_TRACE=1`. Caller-correctable argument errors omit that hint. Other framework boundaries use the same structured-error approach: - `Kinda.GenerationError` identifies declaration and code-generation failures with stable `:stage` and `:reason` fields plus the source, expected and actual values when relevant. - `Kinda.CommandError` retains the command, arguments, working directory, exit status and captured output from failed `mix`, `zig` or other tool invocations. - `Kinda.NIFLoadError` retains the attempted library path and the original `:erlang.load_nif/2` reason instead of printing the failure and continuing. This keeps terminal messages useful while allowing Mix tasks and downstream projects to handle failures without parsing those messages. For larger generated bindings, the newer wrapper pipeline is usually more important. ## Wrapper Pipeline The wrapper pipeline is the part of `kinda` that scales to large native APIs. ### 1. Extract a manifest ```elixir manifest = Kinda.Wrapper.Extract.from_clang_ast(ast_json) ``` The result is a framework-owned shape: ```elixir %Kinda.Wrapper.Manifest{ records: [ %Kinda.Wrapper.CRecord{ name: "MlirContext", kind: :struct, fields: [ %Kinda.Wrapper.CField{ name: "ptr", ctype: %Kinda.Wrapper.CType{spelling: "const void*", kind: :pointer} } ] } ], functions: [ %Kinda.Wrapper.Function{ name: "mlirContextCreate", params: [], arity: 0, doc: "Creates a new MLIR context.", param_ctypes: [], return_ctype: %Kinda.Wrapper.CType{ spelling: "MlirContext", kind: :unknown } } ] } ``` ### 2. Apply consumer policy ```elixir defmodule MyLib.WrapperPolicy do @behaviour Kinda.Wrapper.Policy def generation_blocker_entries, do: %{} def generation_blocked?(_name), do: false def generation_blocker_reason(_name), do: nil def callback_bridge_entries, do: %{} def callback_bridge?(_name), do: false def callback_bridge(_name), do: nil def variants(name), do: [{:normal, name, name}] def public_name({_kind, public, _base}), do: public def elixir_params({_kind, _public, _base}, params), do: params def dirty({_kind, _public, _base}), do: false def typespec_field(_record, field), do: ... def typespec_params(_variant, function), do: ... def typespec_return(_variant, function), do: ... def zig_entry({_kind, _public, base}), do: ~s{nif("#{base}"),} end ``` ### 3. Generate outputs ```elixir Kinda.Wrapper.Generate.render_elixir_manifest(manifest, MyLib.WrapperPolicy) Kinda.Wrapper.Generate.render_zig_nif_entries(manifest, MyLib.WrapperPolicy) Kinda.Wrapper.Generate.declaration_surfaces_struct(manifest, MyLib.WrapperPolicy) Kinda.Wrapper.Generate.declaration_manifest_struct(manifest, MyLib.WrapperPolicy) Kinda.Wrapper.Generate.declaration_manifest(manifest, MyLib.WrapperPolicy) ``` ## Callback-Bridge Backlog Not every native function should be emitted as a plain generated wrapper. Some functions need: - callbacks back into BEAM - richer decoder logic - lifetime review - scheduler review Kinda now represents that explicitly with `Kinda.Wrapper.CallbackBridge`. Example: ```elixir Kinda.Wrapper.CallbackBridge.required(:some_function, scheduler: :dirty_cpu, facets: [:beam_callback, :scheduler_contract] ) ``` This metadata does not invent a consumer ABI automatically. It is the framework-owned policy layer that lets a consumer say: - this API is not “missing” - this API belongs to the callback-bridge backlog, or is resolved by the dispatcher runtime Pure kind-surface gaps belong to a different path. In a consumer such as `beaver`, handle-like CAPIs are often unblocked by extending the Zig-side kind surface and the matching Elixir `KindDecl` surface; callback-bridge metadata is specifically for the remainder that kind-surface sync cannot solve. ## Callback Runtime Consumers can implement those callback bridges with the generic Zig runtime: ```zig const kinda = @import("kinda"); const Dispatcher = kinda.callback_runtime.Dispatcher(.{ "run", "destruct" }); const dispatcher = try Dispatcher.initWithOptions(handler_pid, .{ .timeout_ms = 30_000, }); dispatcher.setCallback("run", run_callback_term); const response = try dispatcher.invoke("run", message_env, .{argument_term}); ``` On the BEAM side, the consumer supplies its own reply NIF while Kinda normalizes success, expected failure, and exceptions: ```elixir Kinda.CallbackRuntime.invoke( reply_token, fn -> {:ok, run_callback.()} end, &MyNative.my_raw_callback_reply/2 ) ``` Callbacks with scalar, enum, or projected handle results use `invoke_reply/4`. The consumer validates its own resource before completing the shared token: ```elixir Kinda.CallbackRuntime.invoke_reply(reply_token, callback, fn token, outcome -> MyNative.reply_projected_result(token, outcome) end) ``` The consuming NIF library exports and opens the shared reply resource: ```zig const callback_nifs = .{ kinda.callback_runtime.ReplyToken.nif("my_raw_callback_reply"), kinda.callback_runtime.ReplyToken.codeNif("my_raw_callback_reply_code"), kinda.callback_runtime.ReplyToken.cancelNif("my_raw_callback_cancel"), }; export fn nif_load(env: kinda.beam.env, _: [*c]?*anyopaque, _: kinda.beam.term) c_int { kinda.callback_runtime.ReplyToken.open(env); return 0; } ``` `Dispatcher` owns copied callback terms and its persistent environment. Each invocation sends `{callback_name, reply_token, callback_fun, dispatcher_id, ...args}`, then waits on a non-scheduler native thread. The process replying through the consumer-exported NIF becomes the next callback owner. Native callback signatures, argument conversion, domain state, and diagnostics remain the consumer's responsibility. Waits are bounded to 30 seconds by default. A response reports whether it was replied, canceled, dropped, or timed out; completion after any terminal state returns `stale`. Dropping the reply resource or terminating its owner therefore cannot leave a native worker waiting forever. `ReplyToken` and consumer-owned registration resources use NIF resource-type takeover so live resources keep the correct native library generation pinned across reload and unload. `kinda.callback_adapter` provides the reusable projection shapes used by consumer ABI trampolines: one handle, ranges of handles, scalar results, enum results, and validated consumer projections. It never interprets an MLIR- or library-specific handle. Callback-bridge manifest version 2 records `runtime_backed`, `runtime`, `owner`, `destructor`, `lifetime`, `scheduler`, and `timeout_ms`. Entries built with `Kinda.Wrapper.CallbackBridge.runtime_backed/2` no longer carry a `callback_bridge_required` blocker and their consumer-provided declaration variant remains in the normal resolved declaration surface. ## Reporting Surface Kinda now exposes one canonical declaration contract plus one derived signature view around that wrapper surface: 1. resolved declaration surfaces 2. unified declaration manifest 3. callback-bridge manifest Resolved declaration surfaces: ```elixir Kinda.Wrapper.Generate.declaration_surfaces_struct(manifest, policy) ``` This is now the canonical wrapper-side in-memory declaration IR in `kinda`. It carries: - the source declaration-manifest slot, set to `nil` for wrapper-generated surfaces - the canonical resolved declaration manifest Unified declaration manifest: ```elixir Kinda.Wrapper.Generate.declaration_manifest_struct(manifest, policy) Kinda.Wrapper.Generate.declaration_manifest(manifest, policy) ``` The declaration-manifest struct is now the canonical resolved payload stored inside `DeclarationSurfaces`, and the map form is the JSON-friendly serialization. It keeps: - named C records and fields from extraction - raw C param/return type facts from extraction - consumer-projected public params and return typespecs - consumer-projected public record and field typespecs - generated `NIFDecl` entries - generated `TypeDecl` entries - dirty scheduler metadata on emitted variants - generation-blocker reasons when the function is not emitted as a plain generated wrapper Internally, `Kinda.CodeGen.DeclarationManifest.build/2` is now the canonical way to derive declaration metadata from typed wrapper facts, including the generated `TypeDecl` layer. That keeps type declarations sourced from the same declaration contract rather than from a parallel signature-only path. When that declaration contract is consumed by `use Kinda.CodeGen`, the formal resolution path now lives in `Kinda.CodeGen.DeclarationSurfaces.from_generator/2`, so downstreams do not have to reimplement normalization or merge logic just to observe the final generated declaration surface. That resolved surface is now materialized as `Kinda.CodeGen.DeclarationSurfaces`, rather than an ad hoc map. The typed signature manifest remains available as a derived compatibility view: ```elixir Kinda.Wrapper.Generate.signature_manifest(manifest, policy) ``` The callback-bridge backlog remains a separate reporting mode: 1. human-readable report 2. machine-readable manifest Human-readable: ```elixir Kinda.Wrapper.Generate.render_callback_bridge_report(manifest, policy) ``` Machine-readable: ```elixir Kinda.Wrapper.Generate.callback_bridge_manifest(manifest, policy) ``` The manifest is versioned and JSON-friendly: ```json { "version": 1, "entries": [ { "function": { "name": "mlirTypeConverterAddConversion", "arity": 1, "params": ["converter"] }, "callback_bridge": { "function": "mlirTypeConverterAddConversion", "reason": "callback_bridge_required", "unblock_path": "callback_bridge_runtime", "scheduler": "unspecified", "facets": ["beam_callback", "rich_input_decoder"] } } ] } ``` This is the main new “reporting surface” for consumers and CI. The repo-local example above exercises this end to end. ## Build / Prebuilt Surface Kinda ships `Kinda.Precompiler`, which consumers can use with `elixir_make` precompiled builds. Example: ```elixir def project do [ make_precompiler: {:nif, Kinda.Precompiler} ] end ``` Today this is target-selection substrate, not a full RustlerPrecompiled-style product story. ## What Kinda Is Good At - binding large C APIs with repetitive structure - generating resource-centric Elixir/Zig surfaces - separating framework mechanics from consumer policy - keeping callback-heavy APIs visible as backlog instead of hiding them in a flat blacklist - acting as a binding substrate for projects that drift with upstream native APIs ## What Kinda Is Not Yet Kinda is not yet a Rustler-complete framework. What is still missing: - policy-driven generation from callback metadata to runtime adapters - richer scheduler-aware NIF declaration surface - a complete prebuilt/download/checksum story - a more polished one-command reporting UX ## Status Kinda is already useful as a framework substrate. It is now moving from: - “resource kind + codegen helper” toward: - “wrapper-driven Zig/Elixir binding framework with explicit policy and backlog surfaces” That is the right frame to evaluate future work.