# Architecture This page is the map of erlang_python. It says which processes and threads exist, how a call travels from `py:call/3` to Python and back in each context mode, how Python calls Erlang, and which code paths are live. Read it before opening `c_src/` or `src/py_context.erl`. The [code map](code-map.md) lists every file; the [glossary](glossary.md) defines the overloaded words (worker, context, pool). ## One picture ``` Erlang VM child OS process +--------------------------------------------------+ (isolated mode only) | erlang_python_sup | | py_callback registry of Erlang funs Python | | may call | | py_shm shared memory regions | | py_thread_handler spawns a handler process | | per Python thread that calls | | Erlang | | py_logger / py_tracer Python logging, tracing | | py_context_sup ---- py_context (one per | | context; mode worker | owngil | isolated) | | py_context_init starts the default pool | | py_event_worker_sup / _registry loop drivers | | py_event_loop, py_event_loop_pool main- | | interpreter asyncio loops | +--------------------------------------------------+ | NIF calls (dirty schedulers) | Unix socket, ETF frames v v +------------------------+ +-------------------------+ | libpython in the VM | | python3 py_isolated_ | | main interpreter | | child.py | | worker contexts: one | | reader thread + main | | pthread each, shared | | thread, own asyncio | | GIL | | loop | | owngil contexts: one | +-------------------------+ | pthread + own sub- | | interpreter each | +------------------------+ ``` A **context** is the unit of work: an Erlang process (`py_context`) that owns one Python execution environment and serves calls in order. Pools (`py_context_router`) route `py:call/3` to a context by scheduler affinity. ## The three context modes | | `worker` | `owngil` | `isolated` | |---|---|---|---| | Python runs in | the VM, main interpreter | the VM, a sub-interpreter with its own GIL | a child process | | Thread | one pthread per context (`ctx_thread_main_worker`) | one pthread per context (`ctx_thread_main_owngil`) | the child's main thread | | Erlang process loop | the receive loop in `py_context` | the receive loop in `py_context` | `py_isolated` (`gen_statem`) | | Transport | NIF request queue on `py_context_t` | same | Unix socket, frames of the callback pipe format | | Python -> Erlang | suspension protocol | blocking callback pipe | socket frames | | Interrupt | `PyThreadState_SetAsyncExc`, next bytecode | same | signal in the child, `SIGKILL` backstop | | Guide | [context-affinity](context-affinity.md), [pools](pools.md) | [owngil_internals](owngil_internals.md) | [isolated](isolated.md) | ## Life of a call, per mode ### worker and owngil (embedded) 1. `py:call(M, F, A)` picks a context through `py_context_router` and sends `{call, From, MRef, M, F, Args, Kwargs}` to the `py_context` process (`src/py.erl`, `src/py_context.erl:call/6`). The caller waits in `await_reply/3`; a timeout there calls `interrupt/1`. 2. The `py_context` receive loop takes it and calls `handle_call_with_suspension/5`, which calls the `context_call_async` NIF (`c_src/py_nif.c`, `nif_context_call_async`). The NIF converts the arguments (`term_to_py`, `c_src/py_convert.c`) into a request, enqueues it on the context's queue (`ctx_queue_enqueue`) and returns `{enqueued, Ref}` at once. The Erlang process is now free to serve callbacks. 3. The context's pthread (`ctx_thread_main_worker` or `ctx_thread_main_owngil`, `c_src/py_nif.c`) dequeues the request and runs it through `ctx_execute_request` (one function for both modes), which calls into Python with the GIL held. 4. The thread converts the result (`py_to_term`) and sends `{py_result, Ref, Result}` to the `py_context` process, which replies `From ! {MRef, Result}`. `nif_context_call` and friends also have a blocking variant used as a fallback when a context has no thread (`{error, async_requires_worker_thread}`), which never happens for contexts created today. ### isolated 1. Same first step: the message reaches the `py_context` pid, which in this mode runs `py_isolated` (a `gen_statem` entered with `enter_loop`). 2. In `idle` the request is encoded with `term_to_binary` into a frame `<>` and written to the Unix socket; the state becomes `{busy, Id}` and other callers' requests are postponed (served in order when the child is free). 3. In the child, the reader thread parses frames and queues the request; the main thread runs it (`_erlang_impl/_isolated.py`, `Runtime._dispatch`) and writes the reply frame. 4. Back in `py_isolated`, the reply for the busy id moves the state to `idle` and answers the caller. A crash of the child is seen as the port's `exit_status`; the state goes through `{restarting, Reason}` and a new child is started within the restart budget. Protocol details: the module header of `src/py_isolated.erl` and the docstring of `priv/_erlang_impl/_isolated.py`. ## Python calling Erlang (`erlang.call`) `erlang_call_impl` in `c_src/py_callback.c` chooses one of these paths, in this order (the comment above it is the authoritative version): 1. **Suspension** (worker contexts). The Python call raises `SuspensionRequired`; the context thread returns `{suspended, CallbackId, State, {Name, Args}}` to `py_context`, which runs the registered fun (`execute/2` in `py_callback`), possibly serving nested calls meanwhile (`wait_for_callback/2`), and resumes with the `resume_callback` NIF. 2. **Blocking callback pipe** (owngil contexts). The context thread writes a request on a pipe and blocks; the `py_context` process has a dedicated handler (`callback_handler_loop/1`) that runs the fun and writes the response frame back with `context_write_callback_response`. 3. **Thread worker** (`c_src/py_thread_worker.c`): any Python thread that is not a context thread (`threading.Thread`, executors) asks the `py_thread_handler` coordinator for a handler process and talks to it over a pipe. There is also an async variant (`erlang.async_call`) using a per-interpreter async pipe. In isolated mode there is one path: a status-3 frame on the socket, answered by a process the `py_isolated` state machine spawns; nested calls into the same context are dispatched immediately because they come from that process. Every message and frame, with the rules for changing them, is in [protocols](protocols.md); the states each process and thread moves through are in [state machines](state-machines.md); the reasons behind the design are in the [decision records](decisions/overview.md). ## asyncio Three different machineries, on purpose: - Embedded contexts share an `ErlangEventLoop` (`priv/_erlang_impl/_loop.py`) whose `add_reader`/`add_writer` map onto `enif_select` and `call_later` onto `erlang:send_after`; readiness is delivered to a `py_event_worker` process per loop (`c_src/py_event_loop.c`, `src/py_event_worker.erl`). Worker loops (`py_context:start_loop/1`) run such a loop on the context thread. - `py_event_loop`/`py_event_loop_pool` expose the main-interpreter loops for `py:async_call/3` and friends. - An isolated child runs a plain asyncio loop; the only integration is delivering results over the socket. Nothing of the reactor is ported. See [event_loop_architecture](event_loop_architecture.md) for the embedded loop and [asyncio](asyncio.md) for the API. ## Data paths that avoid copies - `py_buffer` (native): a NIF resource Erlang writes into and Python reads through the buffer protocol; embedded modes only. - `py_shm` and `py_buffer:new(#{shared => true})`: a file mapped `MAP_SHARED` by the VM (through iommap) and by any interpreter, embedded or child, with flow control through callbacks (`_py_buffer_wait`, `_py_buffer_consumed`). See [isolated](isolated.md#bulk-data-with-shared-memory). - `py_channel`, `py_byte_channel`: message queues between Erlang and Python coroutines, embedded modes only. ## Where state lives | State | Owner | Notes | |---|---|---| | Registered callbacks | `py_callback` ETS `py_callbacks` | also mirrored in a C name registry so `erlang.` resolves; `py_state` exposes its store the same way (`erlang.state_get`) | | Import and path registry | `py_import` ETS | applied to every new interpreter and to isolated children at start | | Preload code | `py_preload` persistent_term | run once per interpreter | | Context pid -> NIF ref | `py_context` ETS `py_context_refs` | lets `interrupt/1` reach a context blocked in a NIF; isolated contexts store the atom `isolated` | | Per-Erlang-process Python env | `py` process dictionary + NIF env resource | `py:call(Ctx, ...)`; not applicable to isolated contexts | | Shared regions | `py_shm` ETS `py_shm_regions` | closed on owner death | | Python-side state | the interpreter | lost when an isolated child restarts | ## Invariants worth knowing before editing - A context serves one request at a time. Embedded: the thread dequeues one request; nested callbacks are served by the `py_context` process while the thread waits. Isolated: enforced by the `{busy, Id}` state and `postpone`. - Only the context's own thread touches its Python objects; NIF callers only enqueue. `py_context_t` fields are documented in `c_src/py_nif.h`. - Interrupts target the request executing now. Isolated mode also cancels a queued request by id; embedded modes cannot. - Everything that crosses the socket is a term; NIF resources (channels, native buffers, object references) do not cross, and the API says so with `{error, not_supported_in_isolated}`. - `binary_to_term` on child data is not `safe`: the child can create atoms. ## What is live and what is not Every code path in `src/` and `c_src/` is on the path of a context created today, with one exception: the test-only fd/TCP/UDP NIFs in `c_src/py_event_loop.c` ("Test Helper Functions"), which the suites use. The legacy worker API, its executor thread, the deprecated async worker NIFs and the unused worker pool were removed in 5.0.0.