Gitility.ODB.Backend behaviour (Gitility v0.3.0)

The behaviour for Elixir-backed object storage.

Implement this to serve Git objects from anywhere — a database, an HTTP API, an object store — and Gitility's full query API runs against it with no repository directory and no filesystem.

Contract

Two deliberate decisions shape the callbacks:

Batch retrieval is required. read_many/2 is the primitive; a single read is a one-element batch. A single-object callback would be an attractive but catastrophically chatty remote interface, so it does not exist. Explicit Gitility.ODB.read_many/3 calls and prefetch hints retain real batches. V1 deliberately does not coalesce independent jobs; adding cross-job coalescing later is compatible with this batch-first contract.

Callbacks are stateless and concurrent. init/1 produces a state term passed to every callback read-only; callbacks do not return updated state. The provider dispatches callbacks to supervised tasks, up to the configured concurrency. A backend that needs mutable state — connection pools, rate limiters, metrics — owns it explicitly (an ETS table, an Agent, its own process) rather than inheriting serialization from a GenServer-shaped contract.

Callbacks must be safe to run concurrently up to the configured concurrency. A callback must not call back into Gitility against the same provider: under pool exhaustion that deadlocks. :not_found is a normal per-object result, distinct from {:error, reason} which fails the whole batch; backend error reasons are sanitized before they reach query results.

Objects returned are verified (verify: :always recomputes each object ID) before anything downstream trusts them; unexpected or duplicate IDs in a reply are rejected.

Example

defmodule MyApp.PostgresObjects do
  @behaviour Gitility.ODB.Backend

  @impl true
  def init(repo_id), do: {:ok, %{repo_id: repo_id}}

  @impl true
  def read_many(oids, %{repo_id: repo_id}) do
    rows = MyApp.Repo.fetch_objects(repo_id, Enum.map(oids, & &1.bytes))

    {:ok,
     Map.new(oids, fn oid ->
       case rows[oid.bytes] do
         nil -> {oid, :not_found}
         {type, data} -> {oid, %Gitility.Object{oid: oid, type: type, data: data}}
       end
     end)}
  end
end

Summary

Types

state()

Opaque backend state, produced by init/1, passed read-only.

Callbacks

init(term)

Builds the backend state from the configuration term given to Gitility.ODB.start_link/1.

prefetch(list, state)

A hint that these OIDs are likely to be read soon. Optional; fire-and- forget — the provider never waits on it.

read_headers(list, state)

Fetches type/size headers without payloads. Optional: when absent, the provider falls back to read_many/2 and discards payloads (correct but wasteful — implement this when your storage can answer cheaply).

read_many(list, state)

Fetches a batch of objects. Every requested OID must appear in the result map, mapped to its object or :not_found.

refresh(state)

Invalidates whatever the backend caches about object availability — called when a query hits a missing object that may since have arrived (shallow or incrementally populated stores). Optional.

terminate(term, state)

Cleanup on provider shutdown. Optional.

Types

state()

@type state() :: term()

Opaque backend state, produced by init/1, passed read-only.

Callbacks

init(term)

@callback init(term()) :: {:ok, state()} | {:error, term()}

Builds the backend state from the configuration term given to Gitility.ODB.start_link/1.

This callback runs in the process calling start_link/1, before the provider tree exists. Processes it starts are linked to that caller unless the backend supervises them itself. Start long-lived resources under your own supervisor and pass their registered name or pid in the init argument.

prefetch(list, state)

(optional)

@callback prefetch([Gitility.OID.t()], state()) :: :ok | {:error, term()}

A hint that these OIDs are likely to be read soon. Optional; fire-and- forget — the provider never waits on it.

read_headers(list, state)

(optional)

@callback read_headers([Gitility.OID.t()], state()) ::
  {:ok, %{required(Gitility.OID.t()) => Gitility.ObjectHeader.t() | :not_found}}
  | {:error, term()}

Header replies cannot be verified (there is no payload to hash). Gitility trusts them for type/size metadata only; they never influence which bytes are served (payload reads always verify). A backend that cannot answer headers truthfully should not export read_headers — the fallback verifies via full reads. Header sizes are bounded by a protocol sanity ceiling of 2^40 bytes, independently of a query's payload limit.

read_many(list, state)

@callback read_many([Gitility.OID.t()], state()) ::
  {:ok, %{required(Gitility.OID.t()) => Gitility.Object.t() | :not_found}}
  | {:error, term()}

Fetches a batch of objects. Every requested OID must appear in the result map, mapped to its object or :not_found.

refresh(state)

(optional)

@callback refresh(state()) :: :ok | {:error, term()}

Invalidates whatever the backend caches about object availability — called when a query hits a missing object that may since have arrived (shallow or incrementally populated stores). Optional.

terminate(term, state)

(optional)

@callback terminate(term(), state()) :: term()

Cleanup on provider shutdown. Optional.