ToonEx

High-performance TOON (Token-Oriented Object Notation) encoder/decoder for Elixir with Phoenix Channels support.

TOON is a compact, human-readable data format optimized for LLM token efficiency.

Features

🎯 Token Efficient: 30-60% fewer tokens for LLMs than JSON
📖 Human Readable: Indentation-based structure like YAML
✅ Spec Compliant: Tested against official TOON v4.1.0 specification
🔌 Phoenix Channels: Built-in serializer with binary frame support
🛠️ JSON Converter: Bidirectional JSON ↔ TOON conversion
🔧 Extensible: Custom encoding via ToonEx.Encoder protocol
⚡ High Performance: Zero-copy iodata encoding, binary pattern-matching decoder

Installation

Add toon_ex to your dependencies in mix.exs:

def deps do
  [
    {:toon_ex, "~> 1.2"}
  ]
end

Quick Start

Encoding

# Nested object
ToonEx.encode!(%{"user" => %{"name" => "Bob"}})
# => "user:\n  name: Bob"

# Arrays
ToonEx.encode!(["elixir", "toon"])
# => "[2]: elixir,toon"

Decoding

ToonEx.decode!("name: Alice\nage: 30")
# => %{"name" => "Alice", "age" => 30}

ToonEx.decode!("tags[2]: a,b")
# => %{"tags" => ["a", "b"]}

Phoenix Channels

# In your socket declaration (usually endpoint.ex)
socket "/socket", MyAppWeb.UserSocket,
  websocket: [
    serializer: [{ToonEx.Phoenix.Serializer, "~> 2.0.0"}]
  ]

Do not configure ToonEx.Btoon or ToonEx as Phoenix's global :json_library when using the default Phoenix.Socket.V2.JSONSerializer. That serializer will pass WebSocket text frames to the configured JSON library, but BTOON is a binary format and requires its dedicated Phoenix serializer.

For BTOON WebSocket frames, configure the serializer explicitly:

socket "/socket", MyAppWeb.UserSocket,
  websocket: [
    serializer: [{ToonEx.Btoon.Phoenix.Serializer, "~> 2.0.0"}]
  ]

The serializer supports both text and binary frame encoding for Phoenix Channels. Binary frames use a compact binary protocol for Phoenix.Socket.Message, Phoenix.Socket.Broadcast, and Phoenix.Socket.Reply structs, reducing payload size for high-frequency channels. Use fastlane!/1 for binary-encoded broadcasts and encode!/1 for binary-encoded replies.

See ToonEx.Phoenix.Serializer for details.

API Reference

Core Functions

ToonEx.encode/2 - Encode data to TOON string (returns {:ok, string} | {:error, error})
ToonEx.encode!/2 - Encode data to TOON string (raises on error)
ToonEx.decode/2 - Decode TOON string to data (returns {:ok, data} | {:error, error})
ToonEx.decode!/2 - Decode TOON string to data (raises on error)

Modules

ToonEx - Main API
ToonEx.Encode - Encoder implementation
ToonEx.Decode - Decoder implementation
ToonEx.JSON - JSON ↔ TOON converter
ToonEx.Encoder - Protocol for custom struct encoding
ToonEx.Phoenix.Serializer - Phoenix Channels serializer

Specification

This implementation follows TOON Specification v4.1.0 and is tested against official fixtures.

Testing

# Run all tests
mix test

# Run with coverage
mix coveralls

# Code quality checks
mix quality

Contributing

Contributions are welcome! Please open an issue or submit a pull request.

License

MIT License - see LICENSE.

Next Page → LICENSE