View Source Skitter.Workflow (Skitter v0.5.1)
Workflow type definition and utilities.
A reactive workflow defines a data processing pipeline. It is defined as a set of components connected through various links. A workflow stores these components and links, along with some additional information about the workflow.
In order to enable the reuse of workflows, workflows may define in -and out ports. When this is
done, these workflows may be embedded inside another workflow. Note that a workflow is always
flattened using flatten/1
before it is deployed.
This module defines the workflow type along with some utilities to work with this type. It is
not recommended to define a workflow manually. Instead, the use of
Skitter.DSL.Workflow.workflow/2
is preferred.
Link to this section Summary
Types
Component initialization arguments
Component embedded inside a workflow.
Link destination.
Collection of outgoing links.
Instance name
Internal workflow representation.
Workflow embedded inside a workflow.
Link to this section Types
Specs
args() :: any()
Component initialization arguments
This type stores the arguments passed to the component in the workflow definition.
Specs
component() :: %Skitter.Workflow.Node.Component{ args: args(), component: Skitter.Component.t(), links: links(), strategy: Skitter.Strategy.t() }
Component embedded inside a workflow.
A component in a workflow is stored along with its strategy, initialization arguments (which
are passed to Skitter.Strategy.Component.deploy/1
) and the outgoing links of each of its out
ports.
Workflows can override the strategy of a component, therefore, the strategy specified here may
not be the same as the strategy returned by Skitter.Component.strategy/1
.
Specs
destination() :: {name(), Skitter.Port.t()} | Skitter.Port.t()
Link destination.
This type stores the destination of a link. A link can point to a component or to an out port of the workflow. In the first case, the name of the component and the name of the out port are stored, in the second, only the name of the out port is stored.
Specs
links() :: [{Skitter.Port.t(), [destination()]}]
Collection of outgoing links.
Links are stored as a keyword list. Each key in this list represents an out port, while the value of this key is a list which references the destinations of this out port.
Specs
name() :: atom()
Instance name
A name is used to refer to a component embedded inside a workflow.
Specs
t() :: %Skitter.Workflow{ in: links(), nodes: %{required(name()) => component() | workflow()}, out: [Skitter.Port.t()] }
Internal workflow representation.
A workflow is stored as a map, where each name refers to a single node, which is either a
component/0
or workflow/0
. Besides this, the in -and out ports of the workflow are
stored. The outgoing links of the in ports of a workflow are stored along with the in ports.
Specs
Workflow embedded inside a workflow.
A workflow nested inside a workflow is stored along with the outgoing links of its out ports.
Link to this section Functions
Specs
Recursively inline any nested workflow of a workflow.
This function ensures any workflow embedded in the provided workflow is inlined into the provided workflow.
Examples
will be converted to:
iex> defcomponent Simple, in: p, out: p do
...> end
iex> defcomponent Join, in: [left, right], out: p do
...> end
iex> inner = %Workflow{
...> in: [foo: [node1: :p, node2: :p]],
...> out: [:bar],
...> nodes: %{
...> node1: %Node.Component{component: Simple, links: [p: [node3: :left]]},
...> node2: %Node.Component{component: Simple, links: [p: [node3: :right]]},
...> node3: %Node.Component{component: Join, links: [p: [:bar]]}
...> }}
iex> parent = %Workflow{
...> nodes: %{
...> node_pre: %Node.Component{component: Simple, links: [p: [nested1: :foo, nested2: :foo]]},
...> nested1: %Node.Workflow{workflow: inner, links: [bar: [node_post: :left]]},
...> nested2: %Node.Workflow{workflow: inner, links: [bar: [node_post: :right]]},
...> node_post: %Node.Component{component: Join}
...> }}
iex> flatten(parent)
%Workflow{
nodes: %{
node_pre: %Node.Component{component: Skitter.WorkflowTest.Simple, links: [p: ["nested1#node1": :p, "nested1#node2": :p, "nested2#node1": :p, "nested2#node2": :p]]},
"nested1#node1": %Node.Component{component: Skitter.WorkflowTest.Simple, links: [p: ["nested1#node3": :left]]},
"nested1#node2": %Node.Component{component: Skitter.WorkflowTest.Simple, links: [p: ["nested1#node3": :right]]},
"nested1#node3": %Node.Component{component: Skitter.WorkflowTest.Join, links: [p: [node_post: :left]]},
"nested2#node1": %Node.Component{component: Skitter.WorkflowTest.Simple, links: [p: ["nested2#node3": :left]]},
"nested2#node2": %Node.Component{component: Skitter.WorkflowTest.Simple, links: [p: ["nested2#node3": :right]]},
"nested2#node3": %Node.Component{component: Skitter.WorkflowTest.Join, links: [p: [node_post: :right]]},
node_post: %Node.Component{component: Skitter.WorkflowTest.Join}
}
}
Specs
verify(t()) :: :ok | [destination()]
Verify if the links in a workflow are valid.
This function verifies if every link in the workflow has a valid source and destination. That is, the link should depart from an existing workflow or component port and arrive at one. Note that this function does not traverse nested workflows.
Examples
iex> defcomponent Example, in: p, out: p do
...> end
iex> verify(%Workflow{nodes: %{
...> foo: %Node.Component{component: Example, links: [p: [bar: :p]]},
...> bar: %Node.Component{component: Example},
...> }})
:ok
iex> verify(%Workflow{nodes: %{
...> foo: %Node.Component{component: Example, links: [p: [baz: :p]]},
...> bar: %Node.Component{component: Example},
...> }})
[{{:foo, :p}, {:baz, :p}}]
Specs
Verify if the links in a workflow are valid using verify/1
.
This function uses verify/1
to verify if every link in a workflow has a valid source and
destination. If this is not the case, it raises a Skitter.DefinitionError
. When the workflow
is valid, the worfklow itself is returned.
Examples
iex> defcomponent Example, in: p, out: p do
...> end
iex> verify!(%Workflow{nodes: %{
...> foo: %Node.Component{component: Example, links: [p: [bar: :p]]},
...> bar: %Node.Component{component: Example},
...> }})
%Workflow{nodes: %{
foo: %Node.Component{component: Skitter.WorkflowTest.Example, links: [p: [bar: :p]]},
bar: %Node.Component{component: Skitter.WorkflowTest.Example},
}}
iex> verify!(%Workflow{nodes: %{
...> foo: %Node.Component{component: Example, links: [p: [baz: :p]]},
...> bar: %Node.Component{component: Example},
...> }})
** (Skitter.DefinitionError) Invalid link: {:foo, :p} ~> {:baz, :p}