DataFetcher.child_spec

You're seeing just the function child_spec, go back to DataFetcher module for more information.

Specs

child_spec([option()]) :: Supervisor.child_spec()

Build the supervisor child spec to start the data_fetcher.

Supported options

  • :name - The identity of the job, should be unique.
  • :fetcher - Definition of how data is fetched from the source. A fetcher must return {:ok, data} if successful.
  • :interval - A value in ms indicating how long it waits until next fetching. The exuction time will not affect the interval, neither does the failure of a fetching. 10 minutes by default.
  • :cache_storage - What cache storage adapter to store the data. Ets by default. See Cache Storage section below.

Cache Storage

When data is fetched from the source, it will be saved in the cache storage. By default it uses an ETS table so that later reads can directly read from the ETS table. This is adaquate for most cases unless the data size is huge. In such large data cases, copying and messaging between processes can be expensive and have performance penalties.

If the data rarely or never changes, you can improve the performance in such cases by employing the persistent_term adapter, which is faster at reading no matter how large the data is. Be aware that on the other hand, writing into the cache storage is slower for persitent_term because global GC will happen for every write.

You can change the adapter globally with:

config :data_fetcher, :cache_storage, DataFetcher.CacheStorage.PersistentTerm

or for each fetcher with the cache_storage option mentioned above.

Example

# lib/my_app/application.ex
defmodule MyApp.Application do
  use Application

  @impl true
  def start(_type, _args) do
    children = [
      my_fetcher_spec(),
      ...
    ]

    opts = [strategy: :one_for_one, name: MySupervisor]
    Supervisor.start_link(children, opts)
  end

  defp my_fetcher_spec,
    do: {
        DataFetcher,
        name: :my_fetcher,
        fetcher: &MyWorker.fetch/1,
        interval: :timer.minutes(20)
      }
end