Runic Usage Rules

Guidelines for when and how to use Runic effectively.

Core Concepts

What is Runic?

Runic is a purely functional workflow composition tool for building:

• Dataflow parallel pipelines - DAG-based execution with automatic dependency resolution
• Rule-based expert systems - Forward-chaining conditional logic
• Low-code workflow engines - Runtime-modifiable workflow definitions

Runic workflows are data structures (decorated directed acyclic graphs) that can be composed, serialized, and executed lazily across any process topology.

Key Design Principles

1. Lazy evaluation - Components only execute when their inputs are satisfied
2. Process-agnostic - No assumed runtime topology; integrate with GenServer, GenStage, Flow, etc.
3. Content-addressable - Components are hashed for deduplication and caching
4. Serializable - Workflows can be persisted and recovered via build_log/from_log

When to Use Runic

✅ Good Use Cases

❌ When NOT to Use Runic

| Simple linear pipelines | Elixir's |> operator | | High-performance hot paths | Compiled Elixir code (Runic has runtime overhead) | | Simple async tasks | Task.async/await or Task.async_stream | | Message passing | GenServer, GenStage, Broadway |

Rule of thumb: If your workflow structure is known at compile time, doesn't need parallel dataflow execution, and won't change, use vanilla Elixir. Runic adds value when workflows are built or modified at runtime and dataflow parallelism is inherent.

API Selection Guide

Which Component to Use?

Which Evaluation API to Use?

| Evaluating LHS of rules / match phase | plan_eagerly/2 | Eagerly evaluates conditions against an input | Production with simple needs | react/2 in a loop | More control over iterations | | Custom scheduler/GenServer | Three-phase APIs | Full control over dispatch | | Parallel I/O-bound work | async: true option | Uses Task.async_stream | | Distributed execution | prepare_for_dispatch/1 | Extract runnables for remote execution |

Which Connection API to Use?

Do not pass both :to and :connections. A connection group describes the complete input assignment for the component being added.

Three-Phase Execution APIs

For custom schedulers, GenServers, or distributed execution:

# Phase 1: Planning - Match rules, prepare agenda
workflow = Workflow.plan_eagerly(workflow, input)
{workflow, runnables} = Workflow.prepare_for_dispatch(workflow)

# Phase 2: Execution - Can be parallelized/distributed
executed = Enum.map(runnables, fn runnable ->
  Invokable.execute(runnable.node, runnable)
end)

# Phase 3: Application - Reduce results back
workflow = Enum.reduce(executed, workflow, &Workflow.apply_runnable(&2, &1))

Do's and Don'ts

✅ Always Do

1. Always require Runic before using macros:

   require Runic

2. Use ^variable syntax to capture runtime values for serialization:

   # ✅ Correct - survives serialization
   multiplier = 3
   Runic.step(fn x -> x * ^multiplier end)
   
   # ❌ Wrong - fails after serialization
   Runic.step(fn x -> x * multiplier end)

3. Name your components for debugging, hooks, and referencing:

   Runic.step(fn x -> x * 2 end, name: :double)

4. Use raw_productions/1 or similar apis to extract results, not direct graph access:

   Workflow.raw_productions(workflow)

5. Check is_runnable?/1 before calling react/2 in loops:

   if Workflow.is_runnable?(workflow) do
     Workflow.react(workflow)
   end

6. Use plan_eagerly/2 before react_until_satisfied/1 when passing input with rules:

   # ✅ Correct for workflows with rules
   workflow
   |> Workflow.plan_eagerly(input)
   |> Workflow.react_until_satisfied()
   
   # Also works (plan_eagerly is called internally):
   Workflow.react_until_satisfied(workflow, input)

❌ Never Do

1. Never use else if or elsif - Elixir doesn't support this syntax:

   # ❌ Invalid Elixir
   if condition1 do
     ...
   else if condition2 do  # WRONG
     ...
   end
   
   # ✅ Use cond instead
   cond do
     condition1 -> ...
     condition2 -> ...
     true -> ...
   end

2. Never assume list index access works with []:

   # ❌ Invalid - lists don't support Access
   mylist[0]
   
   # ✅ Use Enum.at/2
   Enum.at(mylist, 0)

3. Never modify workflow state directly - use the APIs:

   # ❌ Don't do this
   workflow.graph = modified_graph
   
   # ✅ Use APIs
   Workflow.add(workflow, component)

4. Never create infinite loops with react_until_satisfied/2:

   # ❌ Danger - hooks that add steps infinitely
   Runic.workflow(
     after_hooks: %{:some_step => fn step, workflow, fact ->
       Workflow.add(workflow, another_step)  # Infinite!
     end}
   )

5. Never use unpinned variables in components that will be serialized:

   # ❌ Will fail after build_log/from_log round-trip
   config = load_config()
   Runic.step(fn x -> x * config.multiplier end)
   
   # ✅ Pin the variable
   Runic.step(fn x -> x * ^config.multiplier end)

6. Never use String.to_atom/1 on user input - memory leak risk:

   # ❌ Atoms are never garbage collected
   Runic.step(fn x -> x end, name: String.to_atom(user_input))
   
   # ✅ Use strings for dynamic names
   Runic.step(fn x -> x end, name: user_input)

Workflow Construction Patterns

Prefer: Declarative workflow/1 Macro

For static structures known at definition time:

workflow = Runic.workflow(
  name: :text_processor,
  steps: [
    {Runic.step(&tokenize/1, name: :tokenize),
     [Runic.step(&count_words/1, name: :count),
      Runic.step(&first_word/1, name: :first)]}
  ]
)

Prefer: Imperative add/3 for Dynamic Construction

For runtime-built workflows:

workflow = Workflow.new(name: :dynamic)
  |> Workflow.add(Runic.step(&step_a/1, name: :a))
  |> Workflow.add(Runic.step(&step_b/1, name: :b), to: :a)

# Conditionally add components
workflow = if needs_validation? do
  Workflow.add(workflow, Runic.step(&validate/1, name: :validate), to: :b)
else
  workflow
end

Prefer: Connections for Data Binding

Keep ordinary Elixir responsible for building the workflow. Use connection data to describe how component outputs become component inputs:

total = Runic.step(fn subtotal, tax -> subtotal + tax end,
  name: :total,
  inputs: [subtotal: [type: :money], tax: [type: :money]]
)

workflow =
  workflow
  |> Workflow.add(total,
    connections: [
      [from: {:pricing, :subtotal}, to: :subtotal],
      [from: {:tax_service, :tax}, to: :tax]
    ]
  )

Avoid adding a Runic.step/2 whose only job is to read and write a workflow-state map. That turns workflow construction into runtime domain work, hides the actual component relationship, and makes replay and graph inspection less precise.

Connection groups are checked before graph mutation:

• Source components and named ports must exist.
• Every required target port must be bound.
• Direct whole-port source and target types must be compatible unless either is :any.
• Assignments to the same target port cannot overlap.
• selector and target_path segments must be atoms, strings, or non-negative integers.

Because a selector or target path changes the value shape, Runic cannot infer the resulting type statically and leaves that routed value to runtime validation.

Use validate: :warn or validate: :off only for gradual prototyping. They relax type compatibility; they do not make invalid component references, ports, paths, or incomplete required inputs valid.

Declare Multi-Input Call Order Explicitly

A multi-arity step's declared input-port order is its positional call order. Connection order is intentionally irrelevant:

sum = Runic.step(fn left, right -> left + right end,
  inputs: [left: [type: :integer], right: [type: :integer]]
)

Use the documented context/1 macro for runtime context. Do not add an implicit context positional argument; a plain arity-two step represents two domain inputs:

# Two domain inputs
Runic.step(fn request, options -> call(request, options) end,
  inputs: [request: [type: :map], options: [type: :map]]
)

# One domain input plus runtime context
Runic.step(fn request -> call(request, context(:api_key)) end,
  inputs: [request: [type: :map]]
)

Inspect the Right Graph

Named connections deliberately preserve two related views:

flowchart LR
  A[Authored source] -->|connects_to with Connection data| B[Authored target]
  A1[Source invokable] -->|flow| I[Generated InputBinding]
  I -->|flow| B1[Target invokable]

• Workflow.component_graph/1 returns authored components and logical :connects_to relationships, including the durable connection descriptors.
• Workflow.flow_graph/1 returns executable :flow and :fan_in topology, including generated input bindings and joins.

Use these projections for topology inspection. Do not infer authored components from every scheduler-visible node in the full multigraph.

Prefer: Explicit Boundaries for Reusable Workflows

Declare boundary ports when a child workflow should behave as one authored, durable component inside a parent:

child =
  Workflow.new(
    name: :normalize_order,
    input_ports: [order: [type: :map]],
    output_ports: [normalized: [type: :map, from: :normalize]]
  )
  |> Workflow.add(
    Runic.step(&normalize/1,
      name: :normalize,
      inputs: [order: [type: :map]],
      outputs: [normalized: [type: :map]]
    )
  )

parent = Workflow.add(parent, child,
  connections: [[from: {:source, :order}, to: :order]]
)

Declaring either input_ports or output_ports makes the child an explicit boundary. Its name, stable hash, boundary contracts, and ordered build log are stored as one nested definition in the parent. This is the right model when the child has a public contract, should remain identifiable in the component graph, or must replay as a unit.

Use from: internal_component_name on every output that must feed a downstream named connection. Runic does not guess among internal productions. A contract-only output without :from is valid, but cannot be resolved as a connection source.

Leave boundary ports unset for internal compiler workflows that should retain Runic's existing inline composition behavior. Do not add placeholder ports just to force nesting; a boundary is an authored API and should be stable enough to persist and reconnect.

flowchart LR
  S[Parent source] -->|logical port connection| C[Child workflow component]
  S1[Source invokable] --> B[InputBinding]
  B --> I[Child internal entry]
  I --> O[Child internal output]
  C -. output port ownership .-> O
  O --> D[Downstream binding]

The parent stores construction data, not execution state. Runtime facts, hooks, run context, and runnable state remain outside the nested definition.

Map-Reduce Pattern

# Define map and reduce with linked names
map_op = Runic.map(fn x -> expensive_transform(x) end, name: :transform)
reduce_op = Runic.reduce([], fn x, acc -> [x | acc] end, name: :collect, map: :transform)

# Add reduce as child of map
workflow = Workflow.new()
  |> Workflow.add(map_op)
  |> Workflow.add(reduce_op, to: :transform)

# Execute - map runs lazily, reduce waits for all elements
workflow
|> Workflow.plan_eagerly([1, 2, 3, 4, 5])
|> Workflow.react_until_satisfied()
|> Workflow.raw_productions(:collect)

Performance Considerations

Runic Adds Runtime Overhead

Runic workflows are essentially a dataflow virtual machine running within Elixir:

• Graph traversal and fact matching on each cycle
• Hash computation for content-addressability
• Closure evaluation for serialization support

For hot paths where performance is critical, consider:

1. Compile static parts - Move invariant logic to regular Elixir functions
2. Batch inputs - Process collections rather than individual items
3. Use async mode - Parallelize I/O-bound work with async: true
4. Profile with Benchee - Measure actual overhead in your use case

When Parallel Execution Helps

# ✅ Good for parallel: I/O-bound independent operations
workflow = Runic.workflow(
  steps: [
    {Runic.step(&parse/1),
     [Runic.step(&fetch_from_api_a/1),
      Runic.step(&fetch_from_api_b/1),
      Runic.step(&fetch_from_api_c/1)]}
  ]
)
Workflow.react_until_satisfied(workflow, input, async: true)

# ❌ No benefit: CPU-bound sequential work
# Just use regular Elixir functions instead

Integration Patterns

GenServer Scheduler

defmodule MyApp.WorkflowRunner do
  use GenServer
  alias Runic.Workflow

  def init(workflow) do
    {:ok, %{workflow: workflow}}
  end

  def handle_cast({:process, input}, state) do
    workflow = state.workflow
      |> Workflow.plan_eagerly(input)
      |> Workflow.react_until_satisfied()
    
    results = Workflow.raw_productions(workflow)
    # Handle results...
    
    {:noreply, %{state | workflow: workflow}}
  end
end

Persistence Pattern

# Save workflow state
def save_workflow(workflow, id) do
  log = Workflow.build_log(workflow)
  serialized = :erlang.term_to_binary(log)
  MyRepo.insert(%WorkflowState{id: id, data: serialized})
end

# Restore workflow state
def load_workflow(id) do
  case MyRepo.get(WorkflowState, id) do
    nil -> Workflow.new()
    %{data: serialized} ->
      log = :erlang.binary_to_term(serialized)
      Workflow.from_log(log)
  end
end

Named connections are normalized into %Runic.Workflow.Connection{} data in the build log. A build_log/1 and from_log/1 round trip therefore rebuilds both the logical component connections and their compiled input-binding flow.

Explicitly bounded child workflows are persisted recursively as versioned nested definitions. Replaying the parent rebuilds each child independently before reconnecting its boundary; child runtime state is never serialized as construction data.

Debugging Tips

1. Visualize with Mermaid:

   workflow |> Workflow.to_mermaid() |> IO.puts()

2. Inspect facts for tracing:

   Workflow.facts(workflow)
   |> Enum.map(fn fact -> {fact.value, fact.ancestry} end)

3. Use named components:

   Workflow.raw_productions(workflow, :specific_step)

4. Add debug hooks:

   Runic.workflow(
     after_hooks: %{
       :suspicious_step => fn step, workflow, fact ->
         IO.inspect({step.name, fact.value}, label: "DEBUG")
         workflow
       end
     }
   )

Summary