View Source Skitter.DSL.Workflow (Skitter v0.5.3)

Workflow definition DSL.

This module offers a macro to define workflow. To define a workflow, use workflow/2. Inside the body of the workflow, node/2 and ~>/2 can be used. Unlike Skitter.DSL.Component.defcomponent/3 and Skitter.DSL.Strategy.defstrategy/3, the workflow/2 macro does not generate a module, instead, it generates a Skitter.Workflow.t/0.

Link to this section Summary

Functions

Generate a workflow link.

Generate a single workflow node.

Define a workflow.

Link to this section Functions

Generate a workflow link.

This macro connects two ports in the workflow with each other. It can only be used inside workflow/2.

This macro can be used in various different ways and provides various conveniences to shorten the definition of a workflow. In its most basic form, this macro is used as follows:

source ~> destination

Where source and destination have one of the following two forms:

  • <component name>.<port name>: specifies a component port
  • <port name>: specifies a workflow port

For instance:

iex> wf = workflow in: foo, out: bar do
...>   node Example, as: node1
...>   node Example, as: node2
...>
...>   foo ~> node1.in_port            # workflow port ~> component port
...>   node1.out_port ~> node2.in_port # component port ~> component port
...>   node2.out_port ~> bar           # component port ~> workflow port
...> end
iex> wf.in
[foo: [node1: :in_port]]
iex> wf.nodes[:node1].links
[out_port: [node2: :in_port]]
iex> wf.nodes[:node2].links
[out_port: [:bar]]

Some syntactic sugar is present for linking nodes when they are created:

  • When the left hand side of ~> is a node, a link is created between the first out port of this node and the destination.
  • When the right hand side of ~> is a node, a link is created between the source and the first in port of this node.

For instance:

iex> wf = workflow in: foo, out: bar do
...>   foo ~> node(Example, as: node1)
...>   node(Example, as: node2) ~> bar
...> end
iex> wf.in
[foo: [node1: :in_port]]
iex> wf.nodes[:node1].links
[]
iex> wf.nodes[:node2].links
[out_port: [:bar]]

Both the left hand side and the right hand side can be nodes:

iex> wf = workflow do
...>   node(Example, as: node1) ~> node(Example, as: node2)
...> end
iex> wf.nodes[:node1].links
[out_port: [node2: :in_port]]

Finally, ~> always returns the right hand side as its result. This enables ~> to be chained.

iex> wf = workflow in: foo, out: bar do
...>   foo ~> node(Example, as: node1) ~> node(Example, as: node2) ~> bar
...> end
iex> wf.in
[foo: [node1: :in_port]]
iex> wf.nodes[:node1].links
[out_port: [node2: :in_port]]
iex> wf.nodes[:node2].links
[out_port: [:bar]]
Link to this macro

node(comp_or_wf, opts \\ [])

View Source (macro)

Generate a single workflow node.

This macro generates a single node of a workflow. It can only be used inside workflow/2. It accepts a Skitter.Component.t/0 or a workflow Skitter.Workflow.t/0 and a list of optional options. The provided component or workflow will be wrapped inside a Skitter.Workflow.component/0 or Skitter.Workflow.workflow/0. No links will be added to the generated node.

Two options can be passed when creating a node: as: and args::

  • as: defines the name of the node inside the workflow. It can be used to refer to the component when creating links with ~>/2. If no name is specified, this macro will generate a name.
  • args: defines the arguments to pass to the node. Note that this is only relevant for component nodes. Arguments passed to workflow nodes are ignored. If no arguments are provided, the arguments of the node defaults to nil.
  • with: defines the strategy to pass to the node. Note that this is only relevant for component nodes. When a strategy is provided here, it will override the one defined by the component. If no strategy is provided, the strategy specified by the component will be used. If no strategy is specified by the component, an error will be raised.

Examples

iex> inner = workflow do
...>   node Example
...>   node Example, as: example_1
...>   node Example, args: :args
...>   node Example, as: example_2, args: :args, with: SomeStrategy
...> end
%Skitter.Workflow{
  in: [],
  out: [],
  nodes: %{
    "skitter/dsl/workflow_test/example#1": %Skitter.Workflow.Node.Component{
      component: Example, args: nil, strategy: DefaultStrategy, links: []
    },
    example_1:  %Skitter.Workflow.Node.Component{
      component: Example, args: nil, strategy: DefaultStrategy, links: []
    },
    "skitter/dsl/workflow_test/example#2": %Skitter.Workflow.Node.Component{
      component: Example, args: :args, strategy: DefaultStrategy, links: []
    },
    example_2:  %Skitter.Workflow.Node.Component{
      component: Example, args: :args, strategy: SomeStrategy, links: []
    },
  }
}
iex> workflow do
...>   node inner
...>   node inner, as: nested_1
...>   node inner, args: :will_be_ignored, as: nested_2
...> end
%Skitter.Workflow{
  in: [],
  out: [],
  nodes: %{
    "#nested#1": %Skitter.Workflow.Node.Workflow{workflow: inner, links: []},
    nested_1: %Skitter.Workflow.Node.Workflow{workflow: inner, links: []},
    nested_2: %Skitter.Workflow.Node.Workflow{workflow: inner, links: []}
  }
}

iex> workflow do
...>   node Join
...> end
** (Skitter.DefinitionError) Component Elixir.Skitter.DSL.WorkflowTest.Join does not define a strategy and no strategy was specified by the workflow

iex> workflow do
...>   node Join, with: SomeStrategy
...> end
%Skitter.Workflow{
  in: [],
  out: [],
  nodes: %{
    "skitter/dsl/workflow_test/join#1": %Skitter.Workflow.Node.Component{
      component: Join, args: nil, strategy: SomeStrategy, links: []
    }
  }
}
Link to this macro

workflow(opts \\ [], list)

View Source (macro)

Define a workflow.

This macro generates a Skitter.Workflow.t/0. Inside the body of this macro, node/2 and ~>/2 can be used to define nodes and links between nodes, respectively. The generated workflow is verified after its definition through the use of Skitter.Workflow.verify/1.

Workflow ports

The ports of a workflow can be defined in the header of the workflow macro as follows:

iex> wf = workflow in: [a], out: [x] do
...> end
iex> wf.in
[a: []]
iex> wf.out
[:x]

If a workflow has no in, or out ports, they can be omitted from the workflow header. Furthermore, if the workflow only has a single in or out port, the list notation can be omitted:

iex> wf = workflow in: a do
...> end
iex> wf.in
[a: []]
iex> wf.out
[]

Inside the body of a workflow, the node/2 and ~>/2 macros are used to define nodes and to link them to one another:

iex> wf = workflow do
...>   node Example, as: node1
...>   node Example, as: node2
...>
...>   node1.out_port ~> node2.in_port
...> end
iex> wf.nodes[:node1].component
Example
iex> wf.nodes[:node1].links
[out_port: [node2: :in_port]]

To link nodes to the in or out ports of a workflow, the port name should be used:

iex> wf = workflow in: foo, out: bar do
...>   node Example, as: node
...>
...>   foo ~> node.in_port
...>   node.out_port ~> bar
...> end
iex> wf.nodes[:node].links
[out_port: [:bar]]
iex> wf.in
[foo: [node: :in_port]]

Previously defined workflows may be used inside a workflow definition:

iex> inner = workflow in: foo, out: bar do
...>   node Example, as: node
...>
...>   foo ~> node.in_port
...>   node.out_port ~> bar
...> end
iex> outer = workflow do
...>   node inner, as: inner_left
...>   node inner, as: inner_right
...>
...>   inner_left.bar ~> inner_right.foo
...> end
iex> outer.nodes[:inner_left].workflow == inner
true
iex> outer.nodes[:inner_left].links
[bar: [inner_right: :foo]]

Instead of specifying the complete source name (e.g. node.in_port), the following syntactic sugar can be used when creating a node:

iex> wf = workflow in: foo do
...>   foo ~> node(Example, as: node)
...> end
iex> wf.in
[foo: [node: :in_port]]

iex> wf = workflow out: bar do
...>   node(Example, as: node) ~> bar
...> end
iex> wf.nodes[:node].links
[out_port: [:bar]]

These uses of ~>/2 can be chained:

iex> wf = workflow in: foo, out: bar do
...>   foo
...>   ~> node(Example, as: node1)
...>   ~> node(Example, as: node2)
...>   ~> bar
...> end
iex> wf.in
[foo: [node1: :in_port]]
iex> wf.nodes[:node1].links
[out_port: [node2: :in_port]]
iex> wf.nodes[:node2].links
[out_port: [:bar]]

It is not needed to explicitly specify a name for a node if you do not need to refer to the node. You should not rely on the format of the generated names in this case:

iex> wf = workflow in: foo, out: bar do
...>   foo
...>   ~> node(Example)
...>   ~> node(Example)
...>   ~> bar
...> end
iex> wf.in
[foo: ["skitter/dsl/workflow_test/example#1": :in_port]]
iex> wf.nodes[:"skitter/dsl/workflow_test/example#1"].links
[out_port: ["skitter/dsl/workflow_test/example#2": :in_port]]
iex> wf.nodes[:"skitter/dsl/workflow_test/example#2"].links
[out_port: [:bar]]

Skitter.BIC defines various components along with several macros to use these components in a workflow. These macros are automatically imported and available inside the body of workflow.

iex> sugar = workflow out: bar do
...>   tcp_source("127.0.0.1", 4555)
...>   ~> node(Example)
...>   ~> bar
...> end
iex> zero = workflow out: bar do
...>   node(Skitter.BIC.TCPSource, args: [address: "127.0.0.1", port: 4555])
...>   ~> node(Example)
...>   ~> bar
...>  end
iex> zero == sugar
true

Examples

iex> workflow in: [foo, bar], out: baz do
...>   foo ~> node(Example) ~> joiner.left
...>   bar ~> node(Example) ~> joiner.right
...>
...>   node(Join, with: SomeStrategy, as: joiner)
...>   ~> node(Example, args: :some_args)
...>   ~> baz
...> end
%Skitter.Workflow{
  in: [
    foo: ["skitter/dsl/workflow_test/example#1": :in_port],
    bar: ["skitter/dsl/workflow_test/example#2": :in_port],
  ],
  out: [:baz],
  nodes: %{
    "skitter/dsl/workflow_test/example#1": %Skitter.Workflow.Node.Component{
      component: Example, args: nil, strategy: DefaultStrategy, links: [out_port: [joiner: :left]]
    },
    "skitter/dsl/workflow_test/example#2": %Skitter.Workflow.Node.Component{
      component: Example, args: nil, strategy: DefaultStrategy, links: [out_port: [joiner: :right]]
    },
    joiner: %Skitter.Workflow.Node.Component{
      component: Join, args: nil, strategy: SomeStrategy, links: [_: ["skitter/dsl/workflow_test/example#3": :in_port]]
    },
    "skitter/dsl/workflow_test/example#3": %Skitter.Workflow.Node.Component{
      component: Example, args: :some_args, strategy: DefaultStrategy, links: [out_port: [:baz]]
    }
  }
}