View Source Skitter.Component behaviour (Skitter v0.5.0)

Component type definition and utilities.

A component is a reusable data processing step that can be embedded inside of a workflow. It is defined as the combination of a set of callbacks, which implement the logic of the component, and some metadata, which define how the component is embedded inside a workflow and how it is distributed over the cluster at runtime.

A skitter component is defined as an elixir module which implements the Skitter.Component behaviour. This behaviour defines various elixir callbacks which are used to track component information such as the defined callbacks. Instead of implementing a component as an elixir module, it is recommend to use Skitter.DSL.Component.defcomponent/3, which automatically generates the appropriate callbacks.

This module defines the component type and behaviour along with utilities to handle components.

Callbacks

A component defines various callbacks: functions which implement the data processing logic of a component. These callbacks need to have the ability to modify state and emit data when they are called. Callbacks are implemented as elixir functions with a few properties:

  • Callbacks accept state/0 and config/0 as their first and second arguments.
  • Callbacks return a result/0 struct, which wraps the result of the callback call along with the updated state and emitted data.

Besides this, callbacks track additional information about whether they access or modify state and which data they emit. This information is stored inside the behaviour callbacks defined in this module.

Examples

Since components need to be defined in a module the example code shown in this module's documentation assumes the following module is defined:

defmodule ComponentModule do
  @behaviour Skitter.Component
  alias Skitter.Component.Callback.{Info, Result}

  def _sk_component_info(:strategy), do: Strategy
  def _sk_component_info(:in_ports), do: [:input]
  def _sk_component_info(:out_ports), do: [:output]

  def _sk_component_initial_state, do: 42

  def _sk_callbacks, do: MapSet.new(example: 1)

  def _sk_callback_info(:example, 1) do
    %Info{read?: true, write?: false, emit?: true}
  end

  def example(state, config, arg) do
    result = state * config
    %Result{state: state, emit: [arg: arg], result: result}
  end
end

Link to this section Summary

Types

Arguments passed to a callback when it is called.

Configuration passed to the callback when it is called.

Output emitted by a callback.

Additional callback information. Can be retrieved with info/2.

Values returned by a callback when it is called.

State passed to the callback when it is called.

t()

A component is defined as a module.

Callbacks

Return the callback information of callback name, arity.

Return the names and arities of all the callbacks defined in this module.

Returns the meta-information of the component.

Returns the initial state of the component.

Functions

Obtain the arity of component.

Call callback callback_name with an empty state and config and arguments.

Call callback callback_name with an empty state and config and arguments.

Call callback callback_name with an empty state, config and arguments.

Call callback callback_name with state, config and arguments.

Call callback_name defined by component if it exists.

Call callback_name defined by component if it exists.

Call callback_name defined by component if it exists.

Call callback_name defined by component if it exists.

Check if component defines a callback with name and arity.

Obtain the callback information for callback name, arity in component.

Obtain the set of all callbacks defined in component.

Check if a given value refers to a component module.

Get the index of an in port.

Obtain the list of in ports of component.

Convert an index of an in port to a port name.

Convert an index of an out port to a port name.

Create the initial state for component.

Get the index of an out port.

Obtain the list of out ports of component.

Check if a component is a sink.

Check if a component is a source.

Obtain the strategy of component.

Link to this section Types

Specs

args() :: [any()]

Arguments passed to a callback when it is called.

The arguments are wrapped in a list.

Specs

config() :: any()

Configuration passed to the callback when it is called.

The configuration represents the immutable state of a component. It is explicitly separated from the mutable state/0 to enable strategies to explicitly differentiate between handling mutable and immutable data.

Specs

emit() :: [{Skitter.Port.t(), Enumerable.t()}]

Output emitted by a callback.

Emitted data is returned as a keyword list where the output for each out port is specified. When no data is emitted on a port, the port should be omitted from the list. The data emitted by a callback for a port should be wrapped in an Enumerable.t/0. Each element in this enumerable will be sent to downstream components separately.

Specs

info() :: %Skitter.Component.Callback.Info{
  emit?: boolean(),
  read?: boolean(),
  write?: boolean()
}

Additional callback information. Can be retrieved with info/2.

The following information is stored:

  • :read?: Boolean which indicates if the callback reads the component state.
  • :write?: Boolean which indicates if the callback updates the component state.
  • :emit: Boolean which indicates if the callback emits data.

Specs

result() :: %Skitter.Component.Callback.Result{
  emit: emit(),
  result: any(),
  state: state()
}

Values returned by a callback when it is called.

The following information is stored:

  • :result: The actual result of the callback, i.e. the final value returned in its body.
  • :state: The (possibly modified) state after calling the callback.
  • :emit: The list of output emitted by the callback.

Specs

state() :: any()

State passed to the callback when it is called.

Specs

t() :: module()

A component is defined as a module.

This module should implement the Skitter.Component behaviour.

Link to this section Callbacks

Link to this callback

_sk_callback_info(name, arity)

View Source

Specs

_sk_callback_info(name :: atom(), arity()) :: info()

Return the callback information of callback name, arity.

Specs

_sk_callbacks() :: MapSet.t({atom(), arity()})

Return the names and arities of all the callbacks defined in this module.

Link to this callback

_sk_component_info(atom)

View Source

Specs

_sk_component_info(:in_ports) :: [Skitter.Port.t()]
_sk_component_info(:out_ports) :: [Skitter.Port.t()]
_sk_component_info(:strategy) :: Skitter.Strategy.t() | nil

Returns the meta-information of the component.

The following information is stored:

  • :in_ports: A list of port names which represents in ports through which the component receives incoming data.

  • :out_ports: A list of out ports names which represents the out ports the component can use to emit data.

  • :strategy: The Skitter.Strategy of the component. nil may be provided instead, in which case a strategy must be provided when the component is embedded in a workflow.

Link to this callback

_sk_component_initial_state()

View Source

Specs

_sk_component_initial_state() :: any()

Returns the initial state of the component.

This function returns an initial state for the component. The state of a component is component specific: Skitter places no constraints on this state.

Link to this section Functions

Specs

arity(t()) :: arity()

Obtain the arity of component.

The arity is defined as the amount of in ports the component defines.

Examples

iex> arity(ComponentModule)
1
Link to this function

call(component, callback_name)

View Source

Specs

call(t(), atom()) :: result()

Call callback callback_name with an empty state and config and arguments.

This function calls Skitter.Component.call/5 with the state created by initial_state/1. nil is used as the value for config, no arguments are passed.

Link to this function

call(component, callback_name, args)

View Source

Specs

call(t(), atom(), args()) :: result()

Call callback callback_name with an empty state and config and arguments.

This function calls Skitter.Component.call/5 with the state created by initial_state/1. nil is used as the value for config.

Link to this function

call(component, callback_name, config, args)

View Source

Specs

call(t(), atom(), config(), args()) :: result()

Call callback callback_name with an empty state, config and arguments.

This function calls Skitter.Component.call/5 with the state created by initial_state/1.

Examples

iex> call(ComponentModule, :example, 2, [:foo])
%Skitter.Component.Callback.Result{state: 42, result: 84, emit: [arg: :foo]}
Link to this function

call(component, name, state, config, args)

View Source

Specs

call(t(), atom(), state(), config(), args()) :: result()

Call callback callback_name with state, config and arguments.

Examples

iex> call(ComponentModule, :example, 10, 2, [:foo])
%Skitter.Component.Callback.Result{state: 10, result: 20, emit: [arg: :foo]}
Link to this function

call_if_exists(component, callback_name)

View Source

Specs

call_if_exists(t(), atom()) :: result()

Call callback_name defined by component if it exists.

Like call_if_exists/5, but state is replaced by the initial state of the component, config is nil and args is the empty list.

Link to this function

call_if_exists(component, callback_name, args)

View Source

Specs

call_if_exists(t(), atom(), args()) :: result()

Call callback_name defined by component if it exists.

Like call_if_exists/5, but state is replaced by the initial state of the component and config is nil.

Link to this function

call_if_exists(component, callback_name, config, args)

View Source

Specs

call_if_exists(t(), atom(), config(), args()) :: result()

Call callback_name defined by component if it exists.

Like call_if_exists/5, but state is replaced by the initial state of the component.

Link to this function

call_if_exists(component, callback_name, state, config, args)

View Source

Specs

call_if_exists(t(), atom(), state(), config(), args()) :: result()

Call callback_name defined by component if it exists.

Calls the callback with the given name with state, config and args if {name, length(args)} exists. If the callback does not exist, a result with the initial_state/1 of the component, an empty emit list and nil as result is returned.

Examples

iex> call_if_exists(ComponentModule, :example, 10, 2, [:foo])
%Skitter.Component.Callback.Result{state: 10, result: 20, emit: [arg: :foo]}
iex> call_if_exists(ComponentModule, :example, 10, 2, [:foo, :bar])
%Skitter.Component.Callback.Result{state: 42, result: nil, emit: []}
Link to this function

callback_exists?(component, name, arity)

View Source

Specs

callback_exists?(t(), atom(), arity()) :: boolean()

Check if component defines a callback with name and arity.

Examples

iex> callback_exists?(ComponentModule, :example, 1) true

iex> callback_exists?(ComponentModule, :example, 2) false

Link to this function

callback_info(component, name, arity)

View Source

Specs

callback_info(t(), atom(), arity()) :: info()

Obtain the callback information for callback name, arity in component.

Examples

iex> callback_info(ComponentModule, :example, 1)
%Info{read?: true, write?: false, emit?: true}

Specs

callbacks(t()) :: [{atom(), arity()}]

Obtain the set of all callbacks defined in component.

Examples

iex> callbacks(ComponentModule) #MapSet<[example: 1]>

Specs

component?(any()) :: boolean()

Check if a given value refers to a component module.

Examples

iex> component?(5)
false
iex> component?(String)
false
iex> component?(ComponentModule)
true
Link to this function

in_port_to_index(component, port)

View Source

Specs

in_port_to_index(t(), Skitter.Port.t()) :: Skitter.Port.index() | nil

Get the index of an in port.

Examples

iex> in_port_to_index(ComponentModule, :input)
0
iex> in_port_to_index(ComponentModule, :other)
nil

Specs

in_ports(t()) :: [Skitter.Port.t()]

Obtain the list of in ports of component.

Examples

iex> in_ports(ComponentModule)
[:input]
Link to this function

index_to_in_port(component, idx)

View Source

Specs

index_to_in_port(t(), Skitter.Port.index()) :: Skitter.Port.t() | nil

Convert an index of an in port to a port name.

Examples

iex> index_to_in_port(ComponentModule, 0)
:input
iex> index_to_in_port(ComponentModule, 1)
nil
Link to this function

index_to_out_port(component, idx)

View Source

Specs

index_to_out_port(t(), Skitter.Port.index()) :: Skitter.Port.t() | nil

Convert an index of an out port to a port name.

Examples

iex> index_to_out_port(ComponentModule, 0)
:output
iex> index_to_out_port(ComponentModule, 1)
nil
Link to this function

initial_state(component)

View Source

Specs

initial_state(t()) :: state()

Create the initial state for component.

Examples

iex> initial_state(ComponentModule)
42
Link to this function

out_port_to_index(component, port)

View Source

Specs

out_port_to_index(t(), Skitter.Port.t()) :: Skitter.Port.index() | nil

Get the index of an out port.

Examples

iex> out_port_to_index(ComponentModule, :output)
0
iex> out_port_to_index(ComponentModule, :other)
nil

Specs

out_ports(t()) :: [Skitter.Port.t()]

Obtain the list of out ports of component.

Examples

iex> out_ports(ComponentModule)
[:output]

Specs

sink?(t()) :: boolean()

Check if a component is a sink.

A component is a sink if it does not have any out ports.

Examples

iex> sink?(ComponentModule)
false

Specs

source?(t()) :: boolean()

Check if a component is a source.

A component is a source if it does not have any in ports.

Examples

iex> source?(ComponentModule)
false

Specs

strategy(t()) :: Skitter.Strategy.t() | nil

Obtain the strategy of component.

Examples

iex> strategy(ComponentModule)
Strategy