Behavior for pipeline steps with strongly-typed inputs and outputs.

Each step declares its input/output types and produces artifacts. Steps are the atomic units of work in the scraping pipeline.

Input/Output Schemas

Input and output schemas are plain Elixir structs with typespecs. Use @enforce_keys for required fields and @derive Jason.Encoder for output schemas that need JSON serialization.

Example Implementation

defmodule MyStep do
  @behaviour ALLM.Pipeline.Step

  defmodule Input do
    @enforce_keys [:url]
    defstruct [:url]
    @type t :: %__MODULE__{url: String.t()}
  end

  defmodule Output do
    @derive Jason.Encoder
    @enforce_keys [:result]
    defstruct [:result]
    @type t :: %__MODULE__{result: String.t()}
  end

  @impl true
  def step_type, do: :my_step

  @impl true
  def input_schema, do: __MODULE__.Input

  @impl true
  def output_schema, do: __MODULE__.Output

  @impl true
  def execute(_context, %Input{} = input) do
    # Process input and return output
    {:ok, %Output{result: "processed"}}
  end

  # Optional: Store artifacts
  @impl true
  def artifact_content_type, do: "text/html"

  @impl true
  def artifact_content(%Output{html: html}), do: html
end