Getting started

This guide takes you from mix deps.get to a watch you have seen fire.

1. Add the dependency

# mix.exs
def deps do
  [{:kepler, "~> 0.1.0"}]
end

Kepler is a library that happens to start a supervision tree. Adding it to your deps is enough — Kepler.Supervisor boots inside your release when your application starts, and there is nothing to add to your own supervisor.

mix deps.get

2. Declare some watches

Create a module — anywhere in your app; lib/my_app/watches.ex is a reasonable home — and declare the conditions you care about.

defmodule MyApp.Watches do
  use Kepler

  watch :process_crash do
    source crash_report: :any
    enrich [:stacktrace, :process_state, :last_message, :request_context]
    severity :error
    fire immediately, cooldown: :timer.minutes(1)
  end

  watch :checkout_latency do
    source telemetry: [:my_app, :checkout, :stop]
    measure :duration, percentile: 99, unit: {:native, :millisecond}
    severity :warning
    fire when: value > 2_000, sustained: :timer.seconds(30)
  end
end

Start with the crash watch. It is the one that carries information nothing outside the VM can assemble, and it needs no thresholds to be useful — see crash attribution.

Then add two or three more. The failure mode of a reactive system is not "I did not declare enough conditions", it is "there are so many notices that nobody reads them". Writing watches covers the whole language.

Every declaration is checked when you compile. A typo in a measurement name or a condition is a build failure, not a watch that quietly never fires:

** (Kepler.CompileError) lib/my_app/watches.ex:12: watch :export_backlog has an
unknown measurement :queue_len; available for :process: :alive, :heap_size,
:memory, :message_queue_len, :reductions, :stack_size, :total_heap_size

3. Turn on SASL reports

Elixir filters supervisor reports and proc_lib crash reports out before any handler sees them. One line gets them back:

# config/config.exs
config :logger, handle_sasl_reports: true

Without it, crash_report: :any still sees GenServer, gen_statem, and gen_event terminations — the rich ones — but not plain Tasks, and supervisor_report: never fires at all. Kepler warns at boot if you declared a watch this affects.

4. Point Kepler at them, and at somewhere to send events

Put this in config/runtime.exs so the URL and secret come from the environment rather than from your repository.

# config/runtime.exs
import Config

if config_env() == :prod do
  config :kepler,
    watches: MyApp.Watches,
    sinks: [
      investigator: {Kepler.Sink.Webhook,
        url: System.fetch_env!("KEPLER_WEBHOOK_URL"),
        secret: System.fetch_env!("KEPLER_WEBHOOK_SECRET")}
    ]
end

Sinks are named, and a watch can route to specific ones with sink :investigator. A watch that names none goes to all of them.

Kepler validates all of this at boot and refuses to start on anything invalid, because the failure it is preventing — a Kepler that runs happily and delivers nowhere — is the one you would not notice until you needed it. Configuring watches with no sinks logs a warning for the same reason.

Kepler.Config documents every key. The ones worth knowing early:

5. Watch it work in development

Point a Kepler.Sink.Callback at your console in dev and use the introspection functions from iex -S mix.

# config/dev.exs
config :kepler,
  watches: MyApp.Watches,
  sinks: [console: {Kepler.Sink.Callback, handler: &IO.inspect/1}]

iex> Kepler.watches() |> Enum.map(& &1.name)
[:checkout_latency, :export_backlog]

# What is Kepler seeing right now?
iex> Kepler.status()
%{
  ticks: 412,
  tick_ms: 1000,
  budget: %{share: 0.0008, limit: 0.01, level: 0, ...},
  emitter: %{queued: 0, delivered: 3, dropped: 0, failed: 0, ...},
  watches: [
    %{name: :checkout_latency, tier: 1, state: :idle, value: 143, ...},
    %{name: :export_backlog, tier: 2, state: :firing, value: 14_233, ...}
  ]
}

# Evaluate immediately instead of waiting for the next tick.
iex> Kepler.tick()
:ok

# What did it actually send?
iex> Kepler.recent(5)
[%Kepler.Event{watch: :export_backlog, state: :firing, ...}]

Kepler.status/0 is the first thing to look at when a watch is not firing. The common causes are all visible there:

• value is nil — the source produced no data. A process: watch on a process that is not registered, or a telemetry: watch on an event name that is never emitted.
• state is :pending — the condition is true but has not held for sustained: yet.
• state is :firing but nothing arrived — look at emitter.dropped and emitter.failed.
• budget.level is not 0 — Kepler is over budget and has slowed itself down.

6. Prove it before you rely on it

Write a test that makes the condition true and asserts the event. Do this once, for one watch; it catches the whole chain — declaration, installation, evaluation, delivery.

defmodule MyApp.WatchesTest do
  use ExUnit.Case, async: false

  setup do
    start_supervised!(
      {Kepler.Supervisor,
       watches: MyApp.Watches,
       tick: 60_000,
       sinks: [test: {Kepler.Sink.Callback, handler: &send(self(), {:kepler, &1})}]}
    )

    :ok
  end

  test "checkout latency fires when p99 crosses the threshold" do
    for _i <- 1..50 do
      :telemetry.execute(
        [:my_app, :checkout, :stop],
        %{duration: System.convert_time_unit(3_000, :millisecond, :native)},
        %{}
      )
    end

    Kepler.tick()

    assert_receive {:kepler, %Kepler.Event{watch: :checkout_latency, state: :firing}}
  end
end

Note the sustained: in the declaration and the single Kepler.tick() here — the watch above declares sustained: :timer.seconds(30), so a single tick will put it in :pending, not :firing. Testing covers how to handle that without sleeping for thirty seconds.

7. Receive the events

The payload is documented in sinks and payloads, including how to verify the signature. The short version: route on the "watch" field, and check the signature before parsing the body.

Running Kepler yourself

If your configuration is not known until runtime, or you want one Kepler per test, set enabled: false in config and start the supervisor directly:

config :kepler, enabled: false

Kepler.Supervisor.start_link(
  watches: MyApp.Watches,
  sinks: MyApp.Runtime.kepler_sinks()
)

Options passed here override the application environment. The children are named, so only one Kepler runs per node.

What to do next

• Crash attribution — the headline source, and what it can honestly tell you.
• Writing watches — every source, aggregate, and condition form, and what each one costs.
• Performance — the cost model, and how to check Kepler is holding up its end.