Temporalex SDK Overview

Temporalex is an experimental Elixir SDK for Temporal. The SDK is built around a deterministic Elixir workflow core and a backend boundary implemented for Temporal Core via Rustler.

This document is the architecture map. Detailed contracts live in focused docs:

Design Principles

1. The core comes first. The interaction between executor and runner processes, replay, command ordering, and deterministic scheduling must be solid before the outer layers are built.

2. The executor is the workflow coordination point. All deterministic workflow runtime state lives in the executor GenServer. Workflow processes carry exactly one process dictionary key, :__temporal_context__, containing the executor PID and deterministic thread ID.

3. The core is Temporal-independent. It speaks internal structs for activations, jobs, commands, completions, and activation transcripts. It does not know about protobuf, Rustler, NIF resources, task tokens, or Temporal Core wire types.

4. The backend owns external protocol and transport. The server talks to a Temporalex.Backend behaviour. The real backend will decode Temporal Core messages into core structs and encode core completions back to Temporal Core.

5. OTP conventions over custom protocols. Process links, monitors, supervision, exit reasons, and telemetry are preferred over bespoke lifecycle mechanisms.

6. No test infrastructure in production paths. Testing uses the same core structs and server/backend contracts. Test backends substitute for real backends without changing the core.

Architecture

User workflow code
  |
  | Workflow API calls
  v
Temporalex.Core.Executor
  |
  | Core completions
  v
Temporalex.Server
  |
  | Backend behaviour callbacks
  v
Temporalex.Backend.TemporalCore
  |
  | Rustler NIF calls and protobuf bytes
  v
Temporal Core / Temporal Server

The core is the only layer that decides deterministic workflow behavior. The server owns orchestration and lifecycle. The backend owns all external protocol and transport details.

Core Boundary

The core receives %Temporalex.Core.Activation{} structs and emits status-bearing %Temporalex.Core.Completion{} structs. It owns:

• executor GenServer
• runner process lifecycle
• workflow operation protocol
• replay command matching
• nondeterminism detection
• command sequencing
• signal and update buffering
• published query state
• deterministic scheduling for parallel and handlers
• workflow completion, failure, and continue-as-new semantics

The core does not own:

• protobuf encoding or decoding
• NIF resources
• Temporal Core worker/client handles
• worker polling
• activity task process supervision
• network calls to Temporal

The main invariant:

Given the same workflow code, input, and ordered activation transcript, the core emits the same ordered command decisions or fails with nondeterminism.

See core.md and temporal_core_mapping.md.

Backend Boundary

The server is parameterized by a backend module. A backend starts workers, delivers core activations to the server, and accepts core completions from the server.

The real backend, Temporalex.Backend.TemporalCore, handles:

• Rustler calls
• Temporal Core worker lifecycle
• protobuf activation/completion encoding
• ETF payload conversion with :erlang.term_to_binary/1 and :erlang.binary_to_term/1
• poll loop messages
• completion submission

A test backend can send core structs directly. This lets server and core integration tests stay valid when the real Temporal backend is added.

See backend.md and temporal_core_mapping.md.

Supervision Overview

The current native backend creates a Temporal Core runtime resource for each worker backend state. The library can later move that into a singleton runtime process:

Temporalex.Supervisor
└── Temporalex.Runtime

Each user worker instance owns its own worker tree:

MyApp.Temporal (Supervisor, strategy: :rest_for_one)
├── MyApp.Temporal.ExecutorSupervisor
├── MyApp.Temporal.ActivitySupervisor
└── MyApp.Temporal.Server

The server owns backend state and monitors each executor. Executors trap exits and link to their runner processes. Activity work runs under the activity supervisor.

See server.md.

Public Surface Summary

The public workflow model is documented in programming_model.md. The module-level API is documented in public_api.md. At a high level:

• Workflows are modules with run/1.
• Activities are defined with use Temporalex.Activity and defactivity.
• Workflow code calls activities, sleeps, waits for signals, reads workflow time, publishes query state, and uses deterministic random helpers through Temporalex.Workflow.API.
• API.parallel/1 and API.phase/2 are the structured concurrency hosts.
• Queries read only the last state explicitly published by API.publish_state/1.
• Updates are accepted only while the workflow is inside a matching API.phase/2.

Data Conversion

For v1, payload conversion is fixed:

:erlang.term_to_binary(term)
:erlang.binary_to_term(binary)

No [:safe] option is used. The trust model is that payloads are produced and consumed inside the same application boundary. There is no pluggable converter layer in v1.

Build Order

The authoritative implementation plan is implementation_slice.md:

1. Slice 1: sequential core, activity/timer blocking, replay matching, and core test harness.
2. Intermission: review Slice 1 for design impedance before adding concurrency.
3. Slice 2: deterministic scheduler rounds, parallel, phase, signals, updates, and queries.
4. Intermission: review full core concurrency before server/backend integration.
5. Server with test backend.
6. Real Temporal backend with Rustler, Temporal Core, and protobuf conversion.

The outer layers should not be built until the core slices and review gates are complete.

Current repository status:

• Slice 1 and Slice 2 core behavior are implemented in lib/temporalex/core/*, lib/temporalex/workflow/*, and lib/temporalex/activity.ex.
• Core verification lives in test/temporalex/core_executor_test.exs and covers sequential execution, replay matching, deterministic parallel, phase, signals, updates, queries, and process lifecycle.
• Server/test-backend integration is implemented through Temporalex.Worker, Temporalex.Server, Temporalex.Backend, and Temporalex.Backend.Test.
• Server verification lives in test/temporalex/server_integration_test.exs and covers worker supervision, executor registry cleanup, activation routing, activity task supervision, query routing, eviction, and completion submission.
• Temporalex.Backend.TemporalCore is implemented through Temporalex.Native, Rustler resources, Temporal Core worker/client calls, native poll loops, protobuf conversion, and ETF payload conversion.
• Temporalex.Client is implemented for Temporal Core-backed workers with workflow start/result, signal, query, update, cancel, terminate, and describe operations.
• test/temporalex/integration/temporal_core_integration_test.exs starts a Temporal dev server and verifies workflow execution, activity execution, heartbeats, invalid client option handling, signal/query/update/describe, termination, and result decoding through the Rust NIF.