View Source Skitter.DSL.Operation (Skitter v0.6.1)

Callback and Operation definition DSL.

This module offers macros to define operation modules and callbacks. Please refer to the documentation of defoperation/3.

Link to this section Summary

Functions

Updates the current state.

Obtain the operation configuration.

Define a callback.

Define an operation module.

Defines the initial state of an operation.

Read the current value of a field stored in state.

Obtain the current state.

Creates an initial struct-based state for an operation.

Emit value to port

Emit several values to port

Link to this section Functions

Updates the current state.

This macro should only be used inside the body of defcb/2. It updates the current value of the operation state to the provided value.

This macro can be used in two ways: it can be used to update the operation state or a field of the operation state. The latter option can only be used if the state of the operation is a struct (i.e. if the intial state has been defined using state_struct/1). The former options modifies the operation state as a whole, the second option only modifies the value of the provided field stored in the operation state.

examples

Examples

defoperation WriteExample do
  defcb write(), do: state <~ :foo
end
iex> Operation.call(WriteExample, :write, nil, nil, []).state
:foo
defoperation FieldWriteExample do
  state_struct [:field]
  defcb write(), do: field <~ :bar
end
iex> Operation.call(FieldWriteExample, :write, %FieldWriteExample{field: :foo}, nil, []).state.field
:bar
defoperation WrongFieldWriteExample do
  fields [:field]
  defcb write(), do: doesnotexist <~ :bar
end
iex> Operation.call(WrongFieldWriteExample, :write, %WrongFieldWriteExample{field: :foo}, nil, [])
** (KeyError) key :doesnotexist not found in: %Skitter.DSL.OperationTest.WrongFieldWriteExample{field: :foo}

Obtain the operation configuration.

This macro reads the current value of the configuration passed to the operation callback when it was called. It should only be used inside the body of defcb/2.

examples

Examples

defoperation ConfigExample do
  defcb read(), do: config()
end
iex> Operation.call(ConfigExample, :read, []).result
nil

iex> Operation.call(ConfigExample, :read, :config, []).result
:config

iex> Operation.call(ConfigExample, :read, :state, :config, []).result
:config
Link to this macro

defcb(clause, list)

View Source (macro)

Define a callback.

This macro is used to define a callback function. Using this macro, a callback can be defined similar to a regular procedure. Inside the body of the procedure, ~>/2, ~>>/2 <~/2 and sigil_f/2 can be used to access the state and to emit output. The macro ensures:

Note that, under the hood, defcb/2 generates a regular elixir function. Therefore, pattern matching may still be used in the argument list of the callback. Attributes such as @doc may also be used as usual.

examples

Examples

defoperation CbExample do
  defcb simple(), do: nil
  defcb arguments(arg1, arg2), do: arg1 + arg2
  defcb state(), do: counter <~ (~f{counter} + 1)
  defcb emit_single(), do: ~D[1991-12-08] ~> out_port
  defcb emit_multi(), do: [~D[1991-12-08], ~D[2021-07-08]] ~>> out_port
end
iex> Operation.callbacks(CbExample)
MapSet.new([arguments: 2, emit_multi: 0, emit_single: 0, simple: 0, state: 0])

iex> Operation.callback_info(CbExample, :simple, 0)
%Info{read?: false, write?: false, emit?: false}

iex> Operation.callback_info(CbExample, :arguments, 2)
%Info{read?: false, write?: false, emit?: false}

iex> Operation.callback_info(CbExample, :state, 0)
%Info{read?: true, write?: true, emit?: false}

iex> Operation.callback_info(CbExample, :emit_single, 0)
%Info{read?: false, write?: false, emit?: true}

iex> Operation.callback_info(CbExample, :emit_multi, 0)
%Info{read?: false, write?: false, emit?: true}

iex> Operation.call(CbExample, :simple, %{}, nil, [])
%Result{result: nil, emit: [], state: %{}}

iex> Operation.call(CbExample, :arguments, %{}, nil, [10, 20])
%Result{result: 30, emit: [], state: %{}}

iex> Operation.call(CbExample, :state, %{counter: 10, other: :foo}, nil, [])
%Result{result: nil, emit: [], state: %{counter: 11, other: :foo}}

iex> Operation.call(CbExample, :emit_single, %{}, nil, [])
%Result{result: nil, emit: [out_port: [~D[1991-12-08]]], state: %{}}

iex> Operation.call(CbExample, :emit_multi, %{}, nil, [])
%Result{result: nil, emit: [out_port: [~D[1991-12-08], ~D[2021-07-08]]], state: %{}}

mutable-state-and-control-flow

Mutable state and control flow

The <~/2, ~>/2 and ~>>/2 operators add mutable state to Elixir, which is an immutable language. Internally, hidden variables are used to track the current state and values to emit. To make this work, the callback DSL rewrites several control flow structures offered by elixir. While this approach works, some limitations are present when the <~/2, ~>/2 and ~>>/2 operators are used inside control flow structures.

The use of these operators is supported in the following control flow constructs:

With the exception of Kernel.SpecialForms.try/1, all of these control flow constructs behave as expected. Said otherwise, state updates performed inside any of these constructs are reflected outside of the control flow construct.

try

try

The behaviour of try blocks is not as straightforward due to implementation limitations. We describe the different behaviours that may occur below.

defcb simple(fun) do
  try do
    pre <~ :modified
    res = fun.()
    post <~ :modified
    :emit ~> out
    res
  rescue
    RuntimeError ->
      x <~ :modified
      :rescue
  catch
    _ ->
      :emit ~> out
      :catch
  end
end

If no exception is raised or no value is thrown, updates performed inside the do block will always be visible:

iex> Operation.call(TryExample, :simple, %{pre: nil, post: nil}, nil, [fn -> :foo end])
%Result{state: %{pre: :modified, post: :modified}, result: :foo, emit: [out: [:emit]]}

However, when an exception is raised or a value is thrown, any updates performed inside the do block are discarded:

iex> Operation.call(TryExample, :simple, %{pre: nil, post: nil, x: nil}, nil, [fn -> raise RuntimeError end])
%Result{state: %{pre: :nil, post: :nil, x: :modified}, result: :rescue, emit: []}

iex> Operation.call(TryExample, :simple, %{pre: nil, post: nil}, nil, [fn -> throw :foo end])
%Result{state: %{pre: :nil, post: :nil}, result: :catch, emit: [out: [:emit]]}

As shown above, updates inside the catch or rescue clauses are reflected in the final result.

else and after

defcb with_else(fun) do
  try do
    pre <~ :modified
    res = fun.()
    post <~ :modified
    :emit ~> out
    res
  rescue
    RuntimeError ->
      :rescue
  else
    res ->
      x <~ :modified
      res
  end
end

Updates inside the do block are reflected in the else block:

iex> Operation.call(TryExample, :with_else, %{pre: nil, post: nil, x: nil}, nil, [fn -> :foo end])
%Result{state: %{pre: :modified, post: :modified, x: :modified}, result: :foo, emit: [out: [:emit]]}

Updates inside the after block are ignored:

defcb with_after(fun) do
  try do
    pre <~ :modified
    res = fun.()
    post <~ :modified
    :emit ~> out
    res
  rescue
    RuntimeError -> :rescue
  after
    x <~ :modified
    :ignored
  end
end
iex> Operation.call(TryExample, :with_after, %{pre: nil, post: nil, x: nil}, nil, [fn -> :foo end])
%Result{state: %{pre: :modified, post: :modified, x: nil}, result: :foo, emit: [out: [:emit]]}

In short, updates are preserved during the normal flow of an operation (i.e. when no values are raised or thrown), updates inside after are ignored.

Link to this macro

defoperation(name, opts \\ [], list)

View Source (macro)

Define an operation module.

This macro is used to define an operation module. Using this macro, an operation can be defined similar to a normal module. The macro will enable the use of defcb/2 and provides implementations for Skitter.Operation._sk_operation_info/1, Skitter.Operation._sk_operation_initial_state/0, Skitter.Operation._sk_callbacks/0 and Skitter.Operation._sk_callback_info/2.

operation-strategy-and-ports

Operation strategy and ports

The operation strategy and its in -and out ports can be defined in the header of the operation declaration as follows:

iex> defoperation SignatureExample, in: [a, b, c], out: [y, z], strategy: SomeStrategy do
...> end
iex> Operation.strategy(SignatureExample)
SomeStrategy
iex> Operation.in_ports(SignatureExample)
[:a, :b, :c]
iex> Operation.out_ports(SignatureExample)
[:y, :z]

If an operation has no in, or out ports, they can be omitted from the operation's header. Furthermore, if the operation only has a single in or out port, the list notation can be omitted:

iex> defoperation PortExample, in: a do
...> end
iex> Operation.in_ports(PortExample)
[:a]
iex> Operation.out_ports(PortExample)
[]

The strategy may be omitted. In this case, a strategy must be provided when the defined operation is embedded inside a workflow. If this is not done, an error will be raised when the workflow is deployed.

examples

Examples

defoperation Average, in: value, out: current do
  state_struct total: 0, count: 0

  defcb react(value) do
    total <~ ~f{total} + value
    count <~ ~f{count} + 1

    ~f{total} / ~f{count} ~> current
  end
end
iex> Operation.in_ports(Average)
[:value]
iex> Operation.out_ports(Average)
[:current]


iex> Operation.strategy(Average)
nil

iex> Operation.call(Average, :react, [10])
%Result{result: nil, emit: [current: [10.0]], state: %Average{count: 1, total: 10}}

iex> Operation.call(Average, :react, %Average{count: 1, total: 10}, nil, [10])
%Result{result: nil, emit: [current: [10.0]], state: %Average{count: 2, total: 20}}

documentation

Documentation

When writing documentation for an operation, @operationdoc can be used instead of the usual @moduledoc. When this is done, this macro will automatically add additional information about the operation to the generated documentation.

Link to this macro

generate_moduledoc(env)

View Source (macro)
Link to this macro

initial_state(initial_state)

View Source (macro)

Defines the initial state of an operation.

This macro is used to define the initial state of an operation. This state is passed to every called callback when no state is provided by the operation's strategy. When this macro is not used, the initial state of an operation is nil.

Internally, this macro generates a definition of Skitter.Operation._sk_operation_initial_state/0.

examples

Examples

defoperation NoStateExample do
  defcb return_state, do: state()
end

defoperation StateExample do
  initial_state 0
  defcb return_state, do: state()
end
iex> Operation.initial_state(NoStateExample)
nil

iex> Operation.initial_state(StateExample)
0

iex> Operation.call(NoStateExample, :return_state, []).state
nil

iex> Operation.call(StateExample, :return_state, []).state
0

iex> Operation.call(NoStateExample, :return_state, :some_state, nil, []).state
:some_state

iex> Operation.call(StateExample, :return_state, :some_state, nil, []).state
:some_state
Link to this macro

sigil_f(arg, _)

View Source (macro)

Read the current value of a field stored in state.

This macro expects that the current operation state is a struct (i.e. it expects an operation that uses state_struct/1), and reads the current value of field from the struct.

This macro should only be used inside the body of defcb/2.

examples

Examples

defoperation FieldReadExample do
  state_struct field: nil
  defcb read(), do: ~f{field}
end
iex> Operation.call(FieldReadExample, :read, %FieldReadExample{field: 5}, nil, []).result
5

iex> Operation.call(FieldReadExample, :read, %FieldReadExample{field: :foo}, nil, []).result
:foo

Obtain the current state.

This macro reads the current value of the state passed to the operation callback when it was called. It should only be used inside the body of defcb/2.

examples

Examples

defoperation ReadExample do
  initial_state 0
  defcb read(), do: state()
end
iex> Operation.call(ReadExample, :read, []).result
0

iex> Operation.call(ReadExample, :read, :state, nil, []).result
:state

iex> Operation.call(ReadExample, :read, :state, nil, []).result
:state
Link to this macro

state_struct(fields)

View Source (macro)

Creates an initial struct-based state for an operation.

In Elixir, it is common to use a struct to store structured information. Therefore, when an operation manages a complex state, it often defines a struct and uses this struct as the initial state of the operation. Afterwards, the state of the operation is updated when it reacts to incoming data:

defoperation Average, in: value, out: current do
  defstruct [total: 0, count: 0]
  state %__MODULE__{}

  defcb react(val) do
    state <~ %{state() | count: state().count + 1}
    state <~ %{state() | total: state().total + val}
    state().total / state().count ~> current
  end
end

In order to streamline the use of this pattern, this macro defines a struct and uses this struct as the initial state of the operation. Moreover, the sigil_f/2 and <~/2 macros are designed to be used with structs, enabling them to read the state and update it:

defoperation Average, in: value, out: current do
  state_struct total: 0, count: 0

  defcb react(val) do
    count <~ ~f{count} + 1
    total <~ ~f{total} + val
    ~f{total} / ~f{count} ~> current
  end
end

The second example generates the code shown in the first example.

examples

Examples

iex> Operation.initial_state(Average)
%Average{total: 0, count: 0}

Emit value to port

This macro is used to specify value should be emitted on port. This means that value will be sent to any operations downstream of the current operation. This macro should only be used inside the body of defcb/2. If a previous value was specified for port, it is overridden.

examples

Examples

defoperation SingleEmitExample do
  defcb emit(value) do
    value ~> some_port
    :foo ~> some_other_port
  end
end
iex> Operation.call(SingleEmitExample, :emit, [:bar]).emit
[some_other_port: [:foo], some_port: [:bar]]

Emit several values to port

This macro works like ~>/2, but emits several output values to the port instead of a single value. Each value in the provided Enumerable.t/0 will be sent to downstream operations individually.

examples

Examples

defoperation MultiEmitExample do
  defcb emit(value) do
    value ~> some_port
    [:foo, :bar] ~>> some_other_port
  end
end
iex> Operation.call(MultiEmitExample, :emit, [:bar]).emit
[some_other_port: [:foo, :bar], some_port: [:bar]]