defmodule Extatus do @moduledoc """ Extatus is an application that reports metrics to Prometheus via the HTTP endpoint `/metrics` from an instrumented `GenServer`. ## Small Example The following is an uninstrumented `GenServer` that tracks its uptime. ``` defmodule Uninstrumented do use GenServer def start_link do GenServer.start_link(__MODULE__, nil) end def get_uptime(pid) do GenServer.call(pid, :uptime) end def init(_) do start_time = :os.system_time(:seconds) {:ok, seconds} end def handle_call(:uptime, _from, start_time) do uptime = :os.system_time(:seconds) - start_time {:reply, uptime, start_time} end end ``` If we would want to track the uptime of this process in Prometheus with `Extatus`, we would need: 1. `use Extatus.Process` behaviour. 2. Declare a gauge metric to track the uptime e.g. a metric called `:uptime` with a label for the module name. 3. Implement the function `get_name/1` that receives the `GenServer` process state and returns the name of the process. This name must be unique. 4. Implement the function `report/1` that receives the `GenServer` process state. In this function, you can report the metrics. 5. Start the `Extatus` watchdog for your process in the `init/1` function by adding the function `add_extatus_watchdog/0` (included by `use Extatus.Process`). ``` defmodule Instrumented do use GenServer use Extatus.Process # Extatus behaviour def start_link do GenServer.start_link(__MODULE__, nil) end # Metric declaration defmetrics do gauge :uptime do label :module help "Uptime gauge" end end # Name of the process. This must be unique. def get_name(_state) do {:ok, Atom.to_string(__MODULE__)} end # Report function def report(start_time) do uptime = :os.system_time(:seconds) - start_time Gauge.set(:uptime, [module: Atom.to_string(__MODULE__)], uptime) end def init(_) do :ok = add_extatus_watchdog() # Add extatus watchdog {:ok, :os.system_time(:seconds)} end end ``` The HTTP `/metric` endpoint is implemented in `:cowboy` and the output is provided by the library `:prometheus_ex`. If you start this process, you will see the metric `:uptime` being reported. Additionally, for every instrumented `GenServer` process, extatus reports the metric `:extatus_process_activity` (gauge). This metric indicates that a process is up (2), down (0) or idle (1) depending on its value. `Extatus` uses `Yggdrasil` to report the status of the processes in the following channel: ```elixir %Yggdrasil.Channel{name: :extatus} ``` This can be used to get the updates on the current state of the processes in a subscriber e.g: ```elixir iex> chan = %Yggdrasil.Channel{name: :extatus} iex> Yggdrasil.subscribe(chan) iex> flush() {:Y_CONNECTED, (...)} iex> {:ok, _} = Instrumented.start_link() {:ok, #PID<0.603.0>} iex> flush() {:Y_EVENT, _, %Extatus.Message{name: "instrumented_process", state: :up}} ``` ## Configuration The following are the configuration arguments available: - `:timeout` - Frequency the handlers get the metrics from the processes. - `:port` - Port where cowboy is listening to request coming from Prometheus. - `:prometheus_registry` - Prometheus registry. By default is `:default`. e.g: ```elixir config :extatus, timeout: 5000, port: 1337, prometheus_registry: :test ``` """ use Application alias Extatus.Settings alias Extatus.Generator @doc """ Starts a status handler for the provided `module` and `pid`. Links to calling process. This is to avoid having processes without Extatus handlers. """ @spec set(module, pid) :: Supervisor.on_start_child def set(module, pid) do {:ok, watchdog} = Generator.start_handler(Generator, module, pid) true = Process.link(watchdog) {:ok, watchdog} end ################### # Application start @doc false def start(_type, _args) do import Supervisor.Spec, warn: false children = get_children() opts = [strategy: :one_for_one, name: Extatus.Supervisor] Supervisor.start_link(children, opts) end @doc false def get_children, do: run_sandbox() |> get_children() @doc false def run_sandbox do Settings.extatus_counter_mod() == Extatus.Sandbox.Counter or Settings.extatus_gauge_mod() == Extatus.Sandbox.Gauge or Settings.extatus_histogram_mod() == Extatus.Sandbox.Histogram or Settings.extatus_summary_mod() == Extatus.Sandbox.Summary end @doc false def get_children(true) do import Supervisor.Spec, warn: false [ worker(Extatus.Sandbox.Metric, []), worker(Extatus.Server, [[name: Extatus.Server]]), supervisor(Generator, [[name: Generator]]) ] end def get_children(false) do import Supervisor.Spec, warn: false [ worker(Extatus.Server, [[name: Extatus.Server]]), supervisor(Generator, [[name: Generator]]) ] end end