Host App Integration

This document defines the initial integration contract for:

Phoenix applications
OTP applications with an existing Repo
existing installations that already run background jobs

Tested Toolchain

Current CI and onboarding smoke tests run with:

Erlang/OTP 28.4.1
Elixir 1.19.5-otp-28
Jido 2.0+

Installation

Add :squidie to the host application's dependencies and fetch dependencies as usual with Mix.

Preferred Hex dependency:

defp deps do
  [
    {:squidie, "~> 0.1.2"}
  ]
end

If the host app defines custom steps with use Jido.Action, add :jido explicitly to the host app as well rather than relying on a transitive dependency:

defp deps do
  [
    {:jido, "~> 2.0"},
    {:squidie, "~> 0.1.2"}
  ]
end

Then install Squidie's library-owned migrations into the host app:

mix squidie.install
mix ecto.migrate

mix squidie.install creates one current-schema Squidie migration in the host application's priv/repo/migrations directory. It does not install or run migrations for the host application's job backend.

Configuration

Start with three pieces:

Squidie config points at the host repo and runtime boundary.
The journal runtime owns its dispatch queue through Squidie config; the host app only needs a worker process that calls Squidie.execute_next/1.
Journal workers call Squidie.execute_next/1 to claim and execute visible attempts.

The host application configures Squidie under the :squidie application:

config :squidie,
  repo: MyApp.Repo,
  queue: "default"

Required keys:

:repo - the Ecto repo Squidie uses for persisted runtime state

Optional keys:

:runtime - :journal by default; routes public start, execution, and manual-control APIs through the Jido-native journal runtime
:read_model - :read_model by default; routes inspection, graph inspection, and explanation through journal projections
:journal_storage - optional for the default Ecto-backed setup; when omitted, Squidie uses {Squidie.Runtime.Journal.Storage.Ecto, repo: MyApp.Repo}. Set it only to override the storage adapter. Explicit nil is rejected for journal-backed runtime or read-model paths.
:queue - "default" by default; selects the journal dispatch queue used by the configured journal runtime and read model

Stale-worker handling comes from journal claim fencing or the host backend's lease system.

For most host apps, the inferred Ecto storage is the recommended starting point when MyApp.Repo uses Postgres or a Postgres-compatible Ecto adapter. It persists Jido threads and checkpoints in Squidie's installed tables and keeps journal storage in the same transactional database boundary as the host app. The boundary remains adapter-shaped, so other Jido-compatible stores can be used later, but production stores must still provide ordered per-thread appends, durable checkpoint reads, and conflict detection for :expected_rev. See Storage strategy for the full adapter contract and compatibility expectations.

The current journal default covers start, cron start, cancellation, replay, global and workflow-filtered list_runs/2, inspect, explain, graph inspection, manual resume/approval controls, and Squidie.execute_next/1. Journal listing is backed by a durable run catalog fact rather than a storage-adapter scan, and returns redacted summaries; use inspect_run/2 for one run when a caller needs inputs, outputs, attempts, or claim metadata. Dashboards can call list_runs([]) for the index view, then pass the selected summary's run_id and queue to inspect_run(run_id, queue: queue, include_history: true) or inspect_run_graph(run_id, queue: queue) for detail views.

Do not serialize inspection or graph detail directly to untrusted clients. Host apps should authorize the caller, select only the fields the view needs, and redact host-domain inputs, outputs, errors, manual metadata, idempotency keys, and claim identifiers before returning the payload. See Observability.

Runtime Boundaries

Most host apps can use Squidie without writing Jido agents, storage calls, or Bedrock code. The public integration boundary is:

workflow modules declare triggers, payloads, steps, transitions, retries, and manual controls
host code starts runs and exposes inspection through Squidie.start/3, Squidie.list_runs/2, Squidie.inspect_run/2, Squidie.inspect_run_graph/2, and Squidie.explain_run/2
host workers provide execution capacity by calling Squidie.execute_next/1
host schedulers may deliver cron activations with Squidie.Executor.Payload.cron/3 and Squidie.Runtime.Runner.perform/2

Jido is the runtime foundation behind that boundary. Squidie uses Jido journals, storage callbacks, actions, and rebuildable agents internally so run state can be reconstructed from durable facts. Users only need to learn those details when they are contributing to the runtime, replacing the default journal storage adapter, or debugging low-level runtime behavior.

Bedrock is optional. Use the basic execute_next/1 worker loop when a host only needs Squidie to claim visible journal work from the configured storage. Use Bedrock or another lease-capable backend when the host needs backend-owned delivery, delayed visibility, worker leases, heartbeats, retry requeue, dead-letter handling, or stale-worker recovery outside the Squidie journal. Those backend concerns belong in adapter modules, not workflow modules.

Journal Worker Contract

Step execution is pulled by host-owned workers. A minimal worker can be a small GenServer loop under the host supervision tree:

defmodule MyApp.SquidieWorker do
  use GenServer

  def start_link(opts \\ []) do
    GenServer.start_link(__MODULE__, opts, name: __MODULE__)
  end

  def init(opts) do
    {:ok, %{owner_id: Keyword.get(opts, :owner_id, "my-app-squidie")}, {:continue, :drain}}
  end

  def handle_continue(:drain, state), do: {:noreply, drain_once(state)}
  def handle_info(:drain, state), do: {:noreply, drain_once(state)}

  defp drain_once(state) do
    interval =
      case Squidie.execute_next(
             owner_id: state.owner_id,
             lease_for: 30,
             heartbeat_interval_ms: 10_000
           ) do
        {:ok, :none} -> 100
        {:ok, _snapshot} -> 0
        {:error, _reason} -> 1_000
      end

    Process.send_after(self(), :drain, interval)
    state
  end
end

This loop is intentionally small. Production hosts can add capacity limits, back-pressure, node placement, metrics, and shutdown policy around the same public call. Squidie still owns the journaled claim, completion, retry, manual-control, and terminal-state facts.

lease_for and heartbeat_interval_ms are journal executor controls, not an external backend requirement. Hosts without Bedrock or another leased job backend may still pass them when steps can run longer than a claim window. Oban OSS workers fall into this plain-host category for this purpose: keep Oban job delivery concerns separate and let Squidie.execute_next/1 maintain the journal claim lease. Short step workers can omit heartbeat_interval_ms. Hosts that also use a backend lease must maintain that backend lease separately from the journal claim lease. The runtime rejects intervals below 50ms to keep heartbeat write volume bounded.

Cron Payload Contract

Cron starts are the Squidie.Executor payload boundary. Hosts that already have a scheduler can enqueue Squidie.Executor.Payload.cron/3 and deliver the stored payload to Squidie.Runtime.Runner.perform/2:

defmodule MyApp.SquidieCronExecutor do
  @behaviour Squidie.Executor

  alias Squidie.Executor.Payload

  def enqueue_cron(_config, workflow, trigger, opts) do
    workflow
    |> Payload.cron(trigger, Keyword.take(opts, [:signal_id, :intended_window]))
    |> enqueue(opts)
  end

  defp enqueue(payload, opts) do
    job = %{payload: payload, queue: queue(), schedule_in: opts[:schedule_in]}

    case MyApp.JobQueue.enqueue(job) do
      {:ok, job} ->
        {:ok, %{job_id: job.id, queue: job.queue, schedule_in: opts[:schedule_in]}}

      {:error, reason} ->
        {:error, reason}
    end
  end

  defp queue do
    :my_app
    |> Application.get_env(__MODULE__, [])
    |> Keyword.get(:queue, :squidie)
  end
end

The cron callback receives:

workflow and trigger - the cron workflow activation target
opts[:signal_id] - optional stable scheduler signal id for a cron activation
opts[:intended_window] - optional logical schedule window for a cron activation

Return {:ok, metadata} after enqueueing. Metadata is returned to the caller and can be included in host-owned logs or telemetry, so useful values are :job_id, :queue, :worker, and :scheduled_at.

The queued job should deliver the stored payload back to Squidie without knowing workflow details:

defmodule MyApp.SquidieJob do
  def perform(%{payload: payload}) do
    Squidie.Runtime.Runner.perform(payload)
  end
end

MyApp.JobQueue is intentionally a placeholder. In a real host app, replace it with the app's durable job backend. Cron activation is host-owned; the host scheduler should call enqueue_cron/4 or enqueue Squidie.Executor.Payload.cron/3.

When a scheduler can provide deterministic schedule metadata, pass it with the cron payload instead of adding it to workflow input:

Payload.cron(MyApp.Workflows.DailyStandup, :daily_standup,
  signal_id: "daily-standup:2026-05-15T09:00:00Z",
  intended_window: %{
    start_at: "2026-05-15T09:00:00Z",
    end_at: "2026-05-15T10:00:00Z"
  }
)

Squidie persists this under run.context.schedule before workflow processing. Steps can read it from context.state.schedule, and inspection or explanation surfaces can show the intended window separately from actual worker receive time.

If the workflow declares cron ..., idempotency: :return_existing_run or idempotency: :skip_duplicate, the scheduler identity also becomes the start idempotency key. Duplicate delivery of the same workflow, trigger, and key will not insert a second run. Idempotent cron starts must include signal_id or a complete intended_window; otherwise Squidie returns {:error, {:missing_schedule_idempotency_key, trigger_name}}.

With the journal default, cron payload delivery through Squidie.Runtime.Runner.perform/2 starts a journal run and persists the schedule context on the :run_started journal fact. Only cron payloads are accepted because step execution is claimed through Squidie.execute_next/1.

That is the whole execution contract for the journal-backed runtime. Workflow modules, context modules, and controllers should not need to know which job backend the scheduler uses.

Optional Lease Contract

Backends that expose worker leases can also implement Squidie.Executor.Leases. This is separate from the queue delivery adapter: it claims visible work, heartbeats active claims, completes delivered work, and returns failed work to the backend's retry or dead-letter policy.

The journal-backed runtime does not require a lease adapter. The behavior exists so Bedrock or another durable backend can expose lease semantics through a stable Squidie boundary without changing workflow modules.

Bedrock Lease Backend Setup

Squidie stays backend-neutral: workflow modules and runtime state do not depend on Bedrock APIs. For hosts that want backend-owned leasing today, Bedrock is the recommended reference backend because it already owns durable delivery, delayed visibility, leases, heartbeats, retry timing, and recovery. That same ownership model is also a better foundation for distributed workflows, where multiple workers may claim, heartbeat, fail, or recover work across process and node boundaries.

Use examples/bedrock_minimal_host_app as the concrete setup guide. The example keeps the storage and lease boundaries explicit:

BedrockMinimalHostApp.Repo stores Squidie workflow and attempt state.
BedrockMinimalHostApp.JobQueue stores queue items, delayed visibility, leases, retries, and queue metadata.
BedrockMinimalHostApp.SquidieDeliveryAdapter adapts cron activations to Bedrock Job Queue payloads.
BedrockMinimalHostApp.SquidieLeaseAdapter adapts Bedrock claims, heartbeats, completion, and failure to Squidie.Executor.Leases.
BedrockMinimalHostApp.Jobs.SquidiePayload delivers cron payloads and then drains visible journal attempts while the Bedrock lease is held.

There are two independent lease layers in that setup. The Bedrock lease belongs to the host job backend and controls whether the payload job can be redelivered. The Squidie journal claim lease belongs to Squidie.execute_next/1 and controls whether another workflow worker can reclaim a journal attempt. The Bedrock example passes journal_heartbeat_interval_ms into execute_next/1 so long-running journal steps keep their Squidie claim alive while the Bedrock payload job is executing. That option does not renew the Bedrock job lease; the host backend must size and renew its own lease separately.

The payload worker is the executor boundary. It should deliver a Squidie payload, then drain visible journal attempts with Squidie.execute_next/1. Do not enqueue one Bedrock job per workflow step. Do not use Bedrock job retry settings to represent workflow step retry policy. Step retry, terminal failure, pause, approval, and compensation routing are Squidie runtime facts driven by the workflow DSL and persisted by execute_next/1.

Treat {:ok, snapshot} from execute_next/1 as successful job progress even when the snapshot reports a failed workflow run. Return {:error, reason} to Bedrock only when the payload delivery or journal drain itself failed and should be redelivered by the backend.

A host app using the same shape should:

Configure :squidie with the host repo and journal queue.
Configure the cron adapter's Bedrock queue id and topic.
Start the host repo, Bedrock cluster, and Bedrock job queue under supervision.
Keep workflow definitions backend-neutral; only the Bedrock adapter modules should know Bedrock exists.
Configure both lease policies explicitly: Bedrock job lease duration for payload delivery, and journal_heartbeat_interval_ms for long-running Squid Mesh attempts.

The example config shape is:

config :my_app, MyApp.SquidieDeliveryAdapter,
  queue_id: "tenant_a",
  topic: "squidie:payload"

config :squidie,
  repo: MyApp.Repo,
  queue: "tenant_a"

config :my_app, MyApp.Jobs.SquidiePayload,
  journal_heartbeat_interval_ms: 10_000,
  max_journal_attempts: 50

To verify the reference path locally:

cd examples/bedrock_minimal_host_app
mix setup
MIX_ENV=test mix test test/bedrock_job_queue_stress_test.exs test/bedrock_minimal_host_app/squidie_lease_adapter_test.exs

That test path covers Bedrock queue behavior plus the lease adapter contract. It does not make Bedrock a required Squidie dependency; another durable delivery adapter can use the same Squidie boundaries if it provides equivalent delivery, lease, heartbeat, retry, and recovery semantics.

For background on why durable workflow systems often benefit from queueing close to the data and tenancy model they serve, see Apple's QuiCK: A Queuing System in CloudKit paper.

First Run Checklist

For a new integration, the shortest path to a successful first run is:

Add :squidie to the host app's dependencies.
Add or confirm a working Postgres-backed Repo.
Run mix squidie.install.
Run mix ecto.migrate.
Configure :squidie with the host app's Repo.
Start the host app's Repo under supervision.
Start one workflow through the public API, execute visible attempts with Squidie.execute_next/1, and inspect it with history enabled.

Add a host job system only when the app needs one for cron scheduling, backend-owned leases, or other application work.

Existing Application Setup

For an existing Phoenix or OTP application:

Add the :squidie dependency.
Configure :repo to point at the app's existing repo.
Call Squidie.config!/0 during boot or integration setup to verify the required contract is present.
Integrate Squidie from the host application's contexts, services, controllers, or internal APIs.

The host application is responsible for:

database setup and migrations
journal worker lifecycle for Squidie.execute_next/1
any HTTP or internal API endpoints exposed to end users

That means the embedded install path assumes:

the host app already owns its Repo
the host app starts workers that call Squidie.execute_next/1
the host app adds job-backend tables only for its own scheduler or lease backend

Minimal OTP Host Skeleton

For a plain OTP application, the minimum moving pieces are:

a Repo module
Repo in the application supervision tree
a supervised worker that periodically calls Squidie.execute_next/1
:squidie configuration pointing at that Repo
one host-facing module that calls Squidie

Dependency shape:

defp deps do
  [
    {:ecto_sql, "~> 3.13"},
    {:postgrex, "~> 0.20"},
    {:squidie, "~> 0.1.2"}
  ]
end

Add :jido only when the host app defines raw Jido.Action steps directly. Add the host job backend separately.

Application supervision shape:

children = [
  MyApp.Repo,
  MyApp.JobQueue
]

Host-facing boundary:

defmodule MyApp.WorkflowRuns do
  def start_payment_recovery(payload) do
    Squidie.start(MyApp.Workflows.PaymentRecovery, :payment_recovery, payload)
  end

  def inspect_run(run_id) do
    Squidie.inspect_run(run_id, include_history: true)
  end

  def resume(run_id, attrs \\ %{}) do
    Squidie.resume(run_id, attrs)
  end

  def approve(run_id, attrs) do
    Squidie.approve(run_id, attrs)
  end

  def reject(run_id, attrs) do
    Squidie.reject(run_id, attrs)
  end
end

If the host app exposes pause-resume or approval workflows, keep the latest Squidie migrations applied before deploying the feature. Paused step runs now persist internal resume metadata so resume/2, approve/3, and reject/3 can continue with stable output and transition semantics after restarts or code changes.

Operational review shape:

{:ok, paused_run} = MyApp.WorkflowRuns.inspect_run(run_id)

Enum.map(paused_run.audit_events, &{&1.type, &1.step})
#=> [{:paused, :wait_for_review}]

{:ok, _run} =
  MyApp.WorkflowRuns.approve(run_id, %{
    actor: "ops_123",
    comment: "customer verified",
    metadata: %{ticket: "SUP-42"}
  })

{:ok, completed_run} = MyApp.WorkflowRuns.inspect_run(run_id)

Enum.map(completed_run.audit_events, &{&1.type, &1.actor, &1.comment})
#=> [{:paused, nil, nil}, {:approved, "ops_123", "customer verified"}]

include_history: true is the public audit boundary. With history enabled, the run includes chronological step_runs, declared steps state, and durable audit_events for pause, resume, approval, and rejection actions.

Minimal Phoenix Host Skeleton

A Phoenix application uses the same runtime contract. The main difference is that Squidie usually sits behind a context or controller boundary.

Typical shape:

add :squidie to the Phoenix app
keep using the Phoenix app's existing Repo
start a supervised worker that calls Squidie.execute_next/1
configure :squidie to use that Repo
expose workflow operations through a context or controller

Add :jido explicitly only when the Phoenix app defines raw Jido.Action modules as an interop path.

Context boundary:

defmodule MyApp.WorkflowRuns do
  def start_payment_recovery(attrs) do
    Squidie.start(MyApp.Workflows.PaymentRecovery, :payment_recovery, attrs)
  end

  def inspect_run(run_id) do
    Squidie.inspect_run(run_id, include_history: true)
  end

  def resume(run_id, attrs \\ %{}) do
    Squidie.resume(run_id, attrs)
  end

  def approve(run_id, attrs) do
    Squidie.approve(run_id, attrs)
  end

  def reject(run_id, attrs) do
    Squidie.reject(run_id, attrs)
  end
end

Controller shape:

def create(conn, params) do
  with {:ok, run} <- MyApp.WorkflowRuns.start_payment_recovery(params) do
    json(conn, %{id: run.run_id, status: run.status})
  end
end

Development Setup

For local development and examples, a minimal host app can provide:

a local Postgres-backed repo
a local background job setup
direct application code calls into Squidie

This uses the same configuration contract as an existing application setup. In that mode, the example app may also own its job-backend migrations because it is acting as a standalone development harness rather than an embedded install.

Validation

Host applications can validate the contract directly:

{:ok, config} = Squidie.config()

Or raise on missing required keys:

config = Squidie.config!()

Example Development Harness

The example host app smoke-test harness builds on this same contract and is the reference setup for end-to-end development and verification.

Path:

examples/minimal_host_app

Suggested workflow:

Start Postgres for the example app.
Run mix setup inside examples/minimal_host_app.
Run mix example.smoke to exercise the host app boundary.

Fast verification path:

run MIX_ENV=test mix example.smoke inside examples/minimal_host_app

The example app wires:

its own MinimalHostApp.Repo
journal runtime smoke paths that use inferred Ecto storage and Squidie.execute_next/1, including cron activation through the journal runtime
cron activation smoke paths that deliver Squidie.Executor.Payload.cron/3 through Squidie.Runtime.Runner.perform/1
Squidie through MinimalHostApp.WorkflowRuns

Inspecting History

For real host apps, inspect_run/2 is most useful with history enabled:

Squidie.inspect_run(run_id, include_history: true)

That returns the top-level run plus:

steps: logical per-step state in workflow order, including dependency edges
step_runs: persisted execution history
attempts: persisted retry history for each step run

This split gives host apps both declared per-step state and the raw execution timeline from one inspection call.

Use explain_run/2 when an operator surface needs the current reason and safe next actions instead of the full inspection snapshot:

{:ok, explanation} = Squidie.explain_run(run_id)

%{
  status: explanation.status,
  reason: explanation.reason,
  step: explanation.step,
  next_actions: explanation.next_actions
}

inspect_run/2 answers "what persisted state exists?". explain_run/2 answers "why is this run here, what evidence supports that, and what can an operator do next?". The explanation keeps details and evidence structured so Phoenix apps, CLIs, and dashboards can render their own messages.

← Previous Page Workflow Authoring

Next Page → Operations Guide