Rupyex

Python as an embedded scripting language for Elixir, built on RustPython and Rustler.

No Python installation, no ports, no external processes: the interpreter and the Python standard library are compiled into a NIF and run in your VM.

{:ok, session} = Rupyex.open()

{:ok, 3} = Rupyex.eval(session, "1 + 2")
{:ok, nil} = Rupyex.eval(session, "def double(x): return x * 2")
{:ok, 42} = Rupyex.call(session, "double", [21])

{:ok, %{"total" => 6}} =
  Rupyex.eval(session, "{'total': sum(xs)}", bind: %{"xs" => [1, 2, 3]})

:ok = Rupyex.close(session)

Why it is safe to embed

Running a foreign interpreter inside the BEAM is normally a good way to lose a scheduler. Rupyex avoids that:

• Python never runs on a scheduler thread. Each session owns an OS thread. A NIF call only queues a request and returns; the answer comes back as a message. Nothing blocks, whatever the Python code does.
• Runaway code is interruptible. Every call has a timeout (5s by default). On expiry the VM is signalled and raises KeyboardInterrupt at its next safe point, so an infinite loop is stopped rather than leaked.
• Sessions clean up after themselves. Close a session explicitly, or let it become garbage — the thread stops and the interpreter's memory goes with it.
• Panics are contained. A crash inside the interpreter is caught and returned as an error instead of taking the node down.

Installation

def deps do
  [{:rupyex, "~> 0.1"}]
end

The NIF ships precompiled for Linux (x86_64, aarch64), macOS (x86_64, aarch64) and Windows (x86_64), so no Rust toolchain is needed to install it. The precompiled Linux artifacts need glibc 2.35 or newer and the system libffi (libffi8 on Debian/Ubuntu, libffi on RHEL) — RustPython links it for ctypes, and any machine with CPython installed already has it.

To build from source instead — for another platform, or to work on the crate — set RUPYEX_BUILD=1 (or config :rustler_precompiled, :force_build, rupyex: true) and have Rust 1.93+ available. The first build compiles RustPython and its standard library, which takes a few minutes; after that it is cached like any other Rust dependency.

Usage

Sessions hold state

A session is a live namespace, like a REPL: names bound by one call are visible to the next.

{:ok, session} = Rupyex.open(init: "import json")

{:ok, _} = Rupyex.eval(session, "config = {'retries': 3}")
{:ok, 3} = Rupyex.eval(session, "config['retries']")
{:ok, ~s({"retries": 3})} = Rupyex.eval(session, "json.dumps(config)")

Sessions are cheap (~15 ms to start) but not free, so keep one for as long as the state matters. For a one-off snippet there is Rupyex.eval_once/2.

A session is a plain struct around a NIF resource: pass it between processes freely. Requests are queued and run one at a time, and each answer goes back to the process that asked for it. To keep one alive under a supervisor, use Rupyex.Server:

children = [{Rupyex.Server, name: MyApp.Python, init: "import json"}]

{:ok, 3} = Rupyex.Server.eval(MyApp.Python, "1 + 2")

Values

Going the other way, atoms other than nil/true/false become strings, a binary becomes str when it is valid UTF-8 and bytes when it is not, and any other struct becomes a dict — so %{a: 1} comes back as %{"a" => 1}, and [a: 1] is a list of tuples rather than a dict.

An Rupyex.Object is a receipt, not a handle: the object itself never left the interpreter. Keep working with it by name in Python rather than passing it back.

Values are copied, never shared, and a value that cannot cross (a pid, nesting deeper than 64 levels, a self-referential container) fails with a kind: :conversion error instead of taking the session down. The data exchange guide has the full picture, including what each direction costs.

Return values

By default a snippet returns the value of its last statement, so both of these work:

{:ok, 3} = Rupyex.eval(session, "1 + 2")
{:ok, 10} = Rupyex.eval(session, "x = 5\nx * 2")

Pass mode: :eval to require a single expression, or mode: :exec for statements only (always nil).

Output

print output is captured per call. Rupyex.eval/3 discards it, Rupyex.run/3 returns it, and errors carry whatever was printed before they were raised:

{:ok, %Rupyex.Result{value: 7, stdout: "hi\n"}} = Rupyex.run(session, "print('hi')\n7")

Pass capture_output: false to Rupyex.open/1 to let Python write to the BEAM's own stdout instead.

Errors

Failures come back as {:error, %Rupyex.Error{}} with the Python class, message and traceback:

{:error, error} = Rupyex.eval(session, "1 / 0")
error.class      #=> "ZeroDivisionError"
error.message    #=> "division by zero"
error.traceback  #=> "Traceback (most recent call last):\n  File \"<rupyex>\", line 1, ..."

error.kind says where it came from: :python, :syntax, :timeout, :interrupted, :cancelled, :conversion, :init, :closed or :panic. The bang variants (eval!/3, call!/4) raise instead.

Timeouts and interrupts

{:error, %Rupyex.Error{kind: :timeout}} =
  Rupyex.eval(session, "while True:\n    pass", timeout: 500)

{:ok, 1} = Rupyex.eval(session, "1")  # the session is still fine

Another process can abort whatever is running with Rupyex.interrupt/1.

Importing real .py files

{:ok, session} = Rupyex.open(sys_path: ["priv/python"])
{:ok, _} = Rupyex.eval(session, "import my_module")

What you get from Python

RustPython implements Python 3 semantics and ships most of the standard library (json, re, datetime, collections, itertools, math, random, hashlib, base64, threading, os, ...). What it does not have is the C extension ecosystem — no NumPy, no pandas, nothing that links against CPython's C API — and it is slower than CPython. It suits embedded scripting: user-defined rules, formulas, transformations, plugins. It does not suit numeric workloads.

Rupyex is not a sandbox. Embedded Python can read and write files and reach the network, exactly as any Elixir code in your node can. Treat a snippet as code you run, not as untrusted input.

Development

mix deps.get
mix test

The NIF is always built in release mode: a debug build of RustPython is slow enough to be misleading.

License

Apache-2.0.