3D Rendering

Render real 3D scenes — lit meshes, a movable camera, software rasterization or ray tracing — inside a TUI. The same model code draws into any true-color terminal, blitting pixels to cells with half blocks, supersampled half blocks, or an ASCII ramp.

Built on ratatui-3d, vendored in-tree.

Quick start

alias ExRatatui.ThreeD.{Camera, Light, Material, Mesh, Object, Scene}
alias ExRatatui.Widgets.Viewport3D

def render(_state, frame) do
  area = %ExRatatui.Layout.Rect{x: 0, y: 0, width: frame.width, height: frame.height}

  viewport = %Viewport3D{
    scene: %Scene{
      objects: [%Object{mesh: Mesh.cube(), material: %Material{color: {100, 150, 255}}}],
      lights: [
        Light.ambient({255, 255, 255}, 0.15),
        Light.directional({-1.0, -1.0, -1.0}, {255, 255, 255})
      ]
    },
    camera: %Camera{position: {3.0, 2.5, 4.0}, target: {0.0, 0.0, 0.0}}
  }

  [{viewport, area}]
end

Viewport3D is pure data, like every other widget: the scene and camera are rebuilt and rendered every frame. Animation and camera movement live in the model between frames — see the camera recipe below.

The scene

A ExRatatui.ThreeD.Scene is a flat list of objects and lights with a background:

%Scene{
  objects: [...],          # list of %Object{}
  lights: [...],           # list of %Light{}
  background: {8, 8, 16},  # {r, g, b}, channels 0-255
  sky: nil                 # optional gradient, sampled by the raytrace pipeline
}

The scene itself is flat: every object carries its own world-space transform. Hierarchies (such as an articulated model) are composed before building the scene — by hand, or with the ExRatatui.ThreeD.Node scene-graph helper described in Scene graph.

Colors are {r, g, b} integer tuples (0-255). Vectors are {x, y, z} float tuples.

Objects, materials, transforms

%Object{
  mesh: Mesh.cube(),
  material: %Material{color: {200, 80, 80}, diffuse: 0.8, specular: 0.5, shininess: 32.0},
  transform: %ExRatatui.ThreeD.Transform{
    position: {1.0, 0.0, 0.0},
    rotation: {:euler_xyz, {0.0, 0.7, 0.0}},
    scale: {1.0, 1.0, 1.0}
  },
  visible: true
}

ExRatatui.ThreeD.Material is a Phong material: a base color plus ambient, diffuse, specular, and shininess coefficients.

Rotation accepts three forms:

Meshes

Built-in primitives carry no geometry data:

Mesh.cube()
Mesh.sphere(16, 24)   # stacks, slices
Mesh.plane()
Mesh.cylinder(24)     # radial segments; radius 0.5, height 1.0, axis +Y

Custom meshes carry vertices and a flat list of triangle indices. Normals are computed natively when omitted:

Mesh.new(
  [{-0.5, -0.5, 0.0}, {0.5, -0.5, 0.0}, {0.0, 0.7, 0.0}],
  [0, 1, 2]
)

# with explicit normals and texture coordinates:
Mesh.new(vertices, indices, normals: normals, uvs: uvs)

Triangles are wound counter-clockwise when viewed from the front; back faces are culled.

Scene graph

ExRatatui.ThreeD.Node builds an articulated model as a tree of local transforms and bakes it into the flat scene. A node has a local transform, an optional visual object rendered at that node, and children:

alias ExRatatui.ThreeD.{Node, Object, Mesh, Transform}

arm =
  %Node{
    transform: %Transform{rotation: {:axis_angle, {0.0, 1.0, 0.0}, base_angle}},
    visual: %Object{mesh: Mesh.cylinder(24), transform: %Transform{scale: {0.5, 0.3, 0.5}}},
    children: [
      %Node{
        transform: %Transform{position: {0.0, 0.3, 0.0}, rotation: {:axis_angle, {0.0, 0.0, 1.0}, elbow}},
        visual: %Object{mesh: Mesh.cube(), transform: %Transform{position: {0.0, 0.45, 0.0}, scale: {0.18, 0.9, 0.18}}}
      }
    ]
  }

scene = Node.to_scene(arm, lights: lights, background: {10, 12, 18})

ExRatatui.ThreeD.Node.flatten/1 returns the world-space objects; ExRatatui.ThreeD.Node.to_scene/2 wraps them with :lights, :background, and :sky. Frames compose via ExRatatui.ThreeD.Transform.compose/2.

Contract: keep intermediate nodes rigid (scale: {1.0, 1.0, 1.0}) and put scale only on leaf visual objects. Then every baked object is exactly one Transform with no shear. A non-uniform scale on a node that has rotated children is unsupported (the result is approximate).

Render modes and pipelines

%Viewport3D{render_mode: :auto, pipeline: :rasterize}

render_mode chooses how the render reaches the terminal. Pixel-graphics modes render the scene as an image at native terminal resolution (crisp, non-blocky) on capable terminals:

Pixel modes engage only on a real terminal that probes as graphics-capable. Under ExRatatui.App on the :local transport, return probe_image_protocol: true from mount/1 (or init/1) so the runtime runs the capability probe right after startup — otherwise :auto cannot detect the protocol and the scene renders through the :braille fallback (and at a default font size, not scaled to the pane):

def init(_opts), do: {:ok, %{...}, probe_image_protocol: true}

Over CellSession/Livebook, distributed/SSH without graphics passthrough, or unsupported terminals, pixel modes fall back to :braille. Cell-blit modes pack the render into character cells and always work:

pipeline is orthogonal to render_mode — both pixel and cell modes can rasterize or ray-trace. The GPU pipeline upstream is intentionally not exposed: a GPU context inside a NIF can take down the runtime.

Lights

Light.ambient({255, 255, 255}, 0.15)
Light.directional({-1.0, -1.0, -1.0}, {255, 255, 255})
Light.point({2.0, 3.0, 2.0}, {255, 220, 180})

directional/3 and point/3 accept an :intensity option (default 1.0).

Driving the camera

ExRatatui.ThreeD.Camera is data; orbit/3 and zoom/2 are pure helpers that return a new camera. Keep the camera in the model and update it from key events:

def handle_event(%Event.Key{code: "left", kind: "press"}, state),
  do: {:noreply, %{state | camera: Camera.orbit(state.camera, 0.15, 0.0)}}

def handle_event(%Event.Key{code: "i", kind: "press"}, state),
  do: {:noreply, %{state | camera: Camera.zoom(state.camera, -0.5)}}

orbit/3 takes yaw and pitch deltas in radians (pitch is clamped to avoid gimbal lock); zoom/2 moves along the view direction, clamped to a small minimum distance.

Performance

Primitive meshes are cheap to rebuild every frame. Large custom meshes re-cross the NIF boundary each frame, so keep per-frame geometry modest. The :raytrace pipeline is CPU-bound and scales with the rendered area — prefer :rasterize for interactive frame rates and reserve ray tracing for smaller viewports or still frames.

Examples

• viewport3d_cube.exs - a spinning lit cube
• viewport3d_scene.exs - a multi-object scene with an orbiting camera
• viewport3d_controls.exs - live camera, render-mode, and pipeline controls
• viewport3d_custom_mesh.exs - a custom mesh built from vertices and indices
• viewport3d_articulated.exs - an articulated arm built from a Node scene-graph