defmodule Provider do @moduledoc """ Retrieval of configuration settings from external sources, such as OS env vars. This module is an alternative to app env for retrieval of configuration settings. It allows you to properly consolidate system settings, define per-env defaults, add strong typing, and compile time guarantees. ## Basic example defmodule MySystem.Config do use Provider, source: Provider.SystemEnv, params: [ {:db_host, dev: "localhost"}, {:db_name, dev: "my_db_dev", test: "my_db_test"}, {:db_pool_size, type: :integer, default: 10}, # ... ] end This will generate the following functions in the module: - `load!` - validates that all parameters are correctly provided and stores them in the cache - `db_host`, `db_name`, `db_pool_size`, ... - getter of each declared parameter ## Describing params Each param is described in the shape of `{param_name, param_spec}`, where `param_name` is an atom, and `param_spec` is a keyword list. Providing only `param_name` (without a tuple), is the same as `{param_name, []}`. The following keys can be used in the `param_spec`: - `:type` - Param type (see `t:type/0`). Defaults to `:string`. - `:default` - Default value used if the param is not provided. Defaults to `nil` (no default). - `:dev` - Default value in `:dev` and `:test` mix env. Defaults to `nil` (no default). - `:test` - Default value in `:test` mix env. Defaults to `nil` (no default). Default options are considered in the following order: 1. `:test` (if mix env is `:test`) 2. `:dev` (if mix env is either `:dev` or `:test`) 3. `:default` For example, if `:test` and `:default` options are given, the `:test` value will be used as a default in `:test` env, while `:default` will be used in all other envs. When you invoke the generated functions, values will be retrieved from the external storage (e.g. OS env). If some value is not available, a default value will be used (if provided). The values are then casted according to the parameter type. Each default can be a constant, but it can also be an expression, which is evaluated at runtime. For example: defmodule MySystem.Config do use Provider, source: Provider.SystemEnv, params: [ # db_name/0 will be invoked when you try to retrieve this parameter (or all parameters) {:db_name, dev: db_name()}, # ... ] defp db_name(), do: #... end It's worth noting that `Provider` performs compile-time purging of needless defaults. When you compile the code in `:prod`, `:dev` and `:test` defaults will not be included in the binaries. Consequently, any private function invoked only in dev/test will also not be invoked, and you'll get a compiler warning when compiling the code in prod. To eliminate such warnings, you can conditionally define the function only in required mix env, by moving the function definition under an `if Mix.env() == ` conditional. ## Generating template The config module will contain the `template/0` function which generates the configuration template. To print the template to stdout, you can invoke: MIX_ENV=prod mix compile MIX_ENV=prod mix run --no-start -e 'IO.puts(MySystem.Config.template())' ## Lower level API The basic retrieval functionality is available via functions of this module, such as `fetch_all/2`, or `fetch_one/2`. These functions are a lower level plumbing API which is less convenient to use, but more flexible. Most of the time the `use`-based interface will serve you better, but if you have more specific needs which are not covered by that, you can reach for these functions. """ use Boundary, exports: [Source, SystemEnv], deps: [Ecto.Changeset, {Mix, :compile}] alias Ecto.Changeset @type source :: module @type params :: %{param_name => param_spec} @type param_name :: atom @type param_spec :: %{type: type, default: value} @type type :: :string | :integer | :float | :boolean @type value :: String.t() | number | boolean | nil @type data :: %{param_name => value} @type ecto_context :: :supervisor | :runtime # ------------------------------------------------------------------------ # API # ------------------------------------------------------------------------ @doc "Retrieves all params according to the given specification." @spec fetch_all(source, params) :: {:ok, data} | {:error, [String.t()]} def fetch_all(source, params) do types = Enum.into(params, %{}, fn {name, spec} -> {name, spec.type} end) data = params |> Stream.zip(source.values(Map.keys(types))) |> Enum.into(%{}, fn {{param, opts}, provided_value} -> value = if is_nil(provided_value), do: opts.default, else: provided_value {param, value} end) {%{}, types} |> Changeset.cast(data, Map.keys(types)) |> Changeset.validate_required(Map.keys(types), message: "is missing") |> case do %Changeset{valid?: true} = changeset -> {:ok, Changeset.apply_changes(changeset)} %Changeset{valid?: false} = changeset -> {:error, changeset_error(source, changeset)} end end @spec ecto_config(module(), ecto_context(), (-> {:ok, Keyword.t()})) :: {:ok, Keyword.t()} def ecto_config(mod, context, fun) do mod.start_link(nil) ret = fun.() if context == :runtime do GenServer.stop(mod) end ret end # ------------------------------------------------------------------------ # Private # ------------------------------------------------------------------------ defp changeset_error(source, changeset) do changeset |> Ecto.Changeset.traverse_errors(fn {msg, opts} -> Enum.reduce( opts, msg, fn {key, value}, acc -> String.replace(acc, "%{#{key}}", to_string(value)) end ) end) |> Enum.flat_map(fn {key, errors} -> Enum.map(errors, &"#{source.display_name(key)} #{&1}") end) |> Enum.sort() end @doc false defmacro __using__(spec) do spec = update_in( spec[:params], fn params -> Enum.map(params, &normalize_param_spec(&1, Mix.env())) end ) quote bind_quoted: [spec: spec] do # Generate typespec mapping for each param typespecs = Enum.map( Keyword.fetch!(spec, :params), fn {param_name, param_spec} -> type = case Keyword.fetch!(param_spec, :type) do :integer -> quote(do: integer()) :float -> quote(do: float()) :boolean -> quote(do: boolean()) :string -> quote(do: String.t()) end {param_name, type} end ) # Convert each param's spec into a quoted map. This is done so we can inject the map # with constants direcly into the function definition. In other words, this ensures that # we converted the input keyword list into a map at compile time, not runtime. quoted_params = spec |> Keyword.fetch!(:params) |> Enum.map(fn {name, spec} -> {name, quote(do: %{unquote_splicing(spec)})} end) use GenServer @spec start_link(term()) :: GenServer.on_start() def start_link(arg) do GenServer.start_link(__MODULE__, arg, name: __MODULE__) end @impl GenServer def init(_arg) do :ets.new(__MODULE__, [:named_table, {:read_concurrency, true}, :public]) load!() {:ok, nil} end @doc "Loads and validates all parameters, raising if some values are missing or invalid." @spec load!() :: :ok def load! do case Provider.fetch_all( unquote(Keyword.fetch!(spec, :source)), %{ unquote_splicing(quoted_params) } ) do {:ok, values} -> :ets.insert(__MODULE__, Map.to_list(values)) :ok {:error, errors} -> raise "Following OS env var errors were found:\n#{Enum.join(Enum.sort(errors), "\n")}" end end # Generate getter for each param. Enum.each( quoted_params, fn {param_name, param_spec} -> @spec unquote(param_name)() :: unquote(Keyword.fetch!(typespecs, param_name)) @doc "Returns the value of the `#{param_name}` param, raising on error." # bug in credo spec check # credo:disable-for-next-line Credo.Check.Readability.Specs def unquote(param_name)() do [{unquote(param_name), value}] = :ets.lookup(__MODULE__, unquote(param_name)) value end end ) @doc "Returns a template configuration file." @spec template :: String.t() def template do unquote(Keyword.fetch!(spec, :source)).template(%{unquote_splicing(quoted_params)}) end end end defp normalize_param_spec(param_name, mix_env) when is_atom(param_name), do: normalize_param_spec({param_name, []}, mix_env) defp normalize_param_spec({param_name, param_spec}, mix_env) do default_keys = case mix_env do :test -> [:test, :dev, :default] :dev -> [:dev, :default] :prod -> [:default] end default_value = default_keys |> Stream.map(&Keyword.get(param_spec, &1)) |> Enum.find(&(not is_nil(&1))) # We need to escape to make sure that default of e.g. `foo()` is correctly passed to # `__using__` quote block and properly resolved as a runtime function call. # # The `unquote: true` option ensures that default of `unquote(foo)` is resolved in the # context of the client module. |> Macro.escape(unquote: true) {param_name, [type: Keyword.get(param_spec, :type, :string), default: default_value]} end defmodule Source do @moduledoc "Contract for storage sources." alias Provider @doc """ Invoked to provide the values for the given parameters. This function should return all values in the requested orders. For each param which is not available, `nil` should be returned. """ @callback values([Provider.param_name()]) :: [Provider.value()] @doc "Invoked to convert the param name to storage specific name." @callback display_name(Provider.param_name()) :: String.t() @doc "Invoked to create operator template." @callback template(Provider.params()) :: String.t() end end