Coelho

Structured rich text for Phoenix.

Coelho is the layer between a rich text editor in the browser and a column in the database. It stores the document, validates it, renders it, and hands the same schema to both sides.

It does not store HTML.

Why not HTML

The usual arrangement stores the editor's HTML output and filters it with a tag allow list on the way in. That works, but it makes the database hold markup: you cannot query it, migrating it means rewriting HTML, and every rendering decision was frozen at the moment the user hit save.

Coelho stores the document as a tree — the shape ProseMirror's toJSON() produces — in a jsonb column, and validates it against a schema:

%{
  "type" => "doc",
  "content" => [
    %{
      "type" => "paragraph",
      "content" => [
        %{"type" => "text", "text" => "hello", "marks" => [%{"type" => "bold"}]}
      ]
    }
  ]
}

What follows from that:

• Validation is the sanitisation. An unknown node, an unknown mark, an unknown attribute or a javascript: URL rejects the document. Nothing outside the schema reaches the database, so rendering never has to escape its way out of untrusted markup.
• The document is data. It is queryable and migratable, and full text extraction is a function rather than a regular expression over tags.
• Rendering is a decision, not a memory. The application overrides any node or mark at render time — mentions, embeds, highlighted code — without touching what is stored.
• Attachment URLs resolve at render time, so signed and expiring URLs work. Frozen HTML cannot do that.
• One schema, two consumers. It is written once in Elixir and exported with Coelho.Schema.to_json/1 to build the matching ProseMirror schema in the browser. A document the server rejects is one the client could not have produced.

What is deliberately not here

Storing bytes, processing images, and two people editing at once. Each has a place to plug into — a Coelho.Storage, a resolver that answers with whatever URL you like including a variant's, a ProseMirror node view passed through createCoelhoHook({nodeViews: …}) — and keeping them out is what keeps what is here small enough to be sure of.

See CONTRIBUTING.md for the ones worth writing.

Status

Early, but complete enough to use: the document core — schema, content expressions, validation, rendering, plain text extraction — the Ecto layer, the LiveView editor and attachments are in place and tested. What is left is a demo application and the polish that comes with it.

Requires Elixir 1.18 or later, for the standard library's JSON module. Ecto is an optional dependency: the core has none at all.

Usage

document = %{
  "type" => "doc",
  "content" => [
    %{"type" => "paragraph", "content" => [%{"type" => "text", "text" => "hello"}]}
  ]
}

{:ok, document} = Coelho.validate(document)
Coelho.to_html(document)
#=> "<p>hello</p>"

Coelho.to_text(document)
#=> "hello"

Override how a node renders without changing what is stored:

Coelho.to_html(document, Coelho.Schema.default(),
  nodes: %{paragraph: fn _node, inner -> Coelho.Render.tag("p", [{"class", "lead"}], inner) end}
)

Storing it

The document lives in a :map (jsonb) column on the table that owns it. There is no side table and no join: those are only needed when the rich text record has to be polymorphic, and this one does not.

defmodule MyApp.Post do
  use Ecto.Schema
  import Coelho.Ecto

  schema "posts" do
    field :title, :string
    rich_text :body
  end
end

alter table(:posts) do
  add :body, :map
end

Casting validates, so an invalid document makes the changeset invalid instead of raising, and the schema violations come with it:

changeset.errors[:body]
#=> {"is invalid rich text",
#=>  [validation: :coelho, errors: ["content[0]: unknown node type \"script\""]]}

The field accepts both a document map and the JSON string a form posts back from the editor's hidden input. Pass :document_schema for a schema other than the default — Ecto reserves the :schema option for the owning module:

rich_text :body, document_schema: MyApp.RichText.schema()

Documents already in the database are not re-validated on load: a schema that grew stricter after rows were written is a migration to run deliberately, not a failure to discover at read time. Which is what the next section is about.

On Ash

Ash does not go through Ecto.Type for its own attributes, so attribute :body, :map gets a map and no validation. Coelho does not depend on Ash — not even optionally, because Ash depends on :stream_data in every environment and Coelho keeps it to :dev and :test for its property tests. The type is a macro that expands in your application instead, where Ash is present by definition:

defmodule MyApp.RichText.Type do
  use Coelho.Ash.Type
end

attribute :cgv_doc, MyApp.RichText.Type do
  constraints document_schema: MyApp.RichText.cgv_schema()
end

A document that fails validation surfaces as an Ash.Error.Changes.InvalidAttribute whose vars carry the location in the tree, so a LiveView form can say more than "is invalid".

Serving what is stored

Validation is the boundary at the keyboard. There is a second one, at the screen, and 0.1.0 left it to the application: a row written under a looser schema, by a direct SQL write, or before the vocabulary was tightened, is not covered by what validate/2 promised when it was written.

post.body |> Coelho.sanitize(MyApp.RichText.schema()) |> Coelho.to_html(...)

sanitize/2 never fails and never reports. What falls outside the schema is removed, from the gentlest repair to the harshest: an unknown key goes, an attribute failing its validator falls back to the schema default, a mark that is unknown or refused goes and the text it covered stays, a node whose type is unknown goes with its text, and a document that cannot be repaired at all becomes the empty one. A hostile document becomes a poor document, never an unexpected rendering.

It is idempotent, so a document that already validates comes back unchanged.

Rendering somewhere other than a web page

to_html/3 answers one question and answers it in iodata, which is the wrong shape for a typesetter, a search index, or anything with its own escaping rules. reduce/4 folds the same tree into any term at all:

Coelho.reduce(document, MyApp.RichText.schema(), %{
  text: fn text, marks -> %{"text" => text, "marks" => Enum.map(marks, & &1["type"])} end,
  node: fn node, children -> %{"block" => node["type"], "children" => children} end
})

marks arrives resolved against the schema, in the schema's declaration order. Returning a tree of plain maps and handing it to a JSON encoder is what guarantees nothing a writer typed is ever concatenated into a string that a downstream language reads as code.

Proving what was accepted

Storing "they agreed to the terms" is worth what the terms are worth, and a plain JSON encoding cannot pin them down: map key order is not part of a map, and jsonb reorders keys of its own accord.

Coelho.hash(document)
#=> "00dc4439f0dcbb463ab186b5b8f81b68e50d70a7b1e3538b86a13e532a17a65d"

Three things make the digest stable, and validation makes all three true: marks are in the schema's order rather than the editor's, attributes at their default are absent rather than written out, and canonical/1 emits keys sorted. Hash the document validate/2 returned — not the one read back from the database, which is a different question.

Coelho.hash/2 answers nil for a document holding nothing.

One schema per field

Several rich text fields usually want different vocabularies: a portal blurb that is paragraphs and a few marks, terms and conditions that add headings and lists but only bold and links. Six full schemas kept consistent by hand is how they drift.

Coelho.Schema.restrict(Coelho.Schema.default(),
  nodes: [:paragraph],
  marks: [:bold, :link]
)

A subtraction, not a redeclaration, and the guarantee runs the right way: a restricted schema never accepts a document its parent would reject. Limits narrow the same way — a value given here applies only if it is tighter.

Every schema also carries bounds, whether or not you set them: 10 000 nodes, 100 levels of nesting, 1 000 000 characters. A document arrives in a hidden form field that no maxlength constrains.

Coelho.Schema.new(..., limits: [max_nodes: 500, max_depth: 6, max_text_length: 20_000])

Editing it

<.form for={@form} phx-change="validate" phx-submit="save">
  <.coelho_editor field={@form[:body]} />
</.form>

Or without a form, for a surface that has no changeset behind it — a JSONB draft whose keys are historical, a field posted straight into phx-change:

<.coelho_editor name="page[intro_doc]" value={@draft["intro_doc"]} />

The component renders a toolbar, an empty container and a hidden input holding the document as JSON. Toolbar buttons carry aria-pressed, kept in step with what is in force under the cursor, and go disabled when their command cannot run — so bold lights up inside bold text and undo greys out with nothing to undo. Links and captions are edited in a field beside the toolbar rather than through window.prompt. The container carries phx-update="ignore" — ProseMirror owns that subtree, and LiveView patching it would fight the editor on every keystroke. Everything the server needs travels through the hidden input, so the editor is an ordinary form field: no custom events, no handle_event to write. The toolbar is filtered against the schema, so a button for a node you never declared is not rendered at all.

In assets/js/app.js:

import { Coelho } from "../../deps/coelho/assets/js/coelho.js"

const liveSocket = new LiveSocket("/live", Socket, { hooks: { Coelho } })

A character counter has to count what the server counts, or it rejects a document the editor still shows as under the limit with nothing on screen to explain the gap. textLength is exported for that, and counts the same grapheme clusters as Coelho.text_length/1 — the text nodes concatenated, no bullets and no blank lines:

import { textLength } from "../../deps/coelho/assets/js/coelho.js"

const { limits } = JSON.parse(editorEl.dataset.coelhoSchema)
const remaining = limits.maxTextLength - textLength(view.state.doc)

The bound travels with the schema, so the counter and the server's check read the same number from the same place. Or let the component do it — maxlength={20_000} renders a counter and keeps it in step, with the first number rendered server side so an existing document does not read zero until the hook has started.

Not losing the last keystrokes

The editor writes into its hidden input and lets phx-change carry it, which means a phx-debounce can still be holding the last edit when the block leaves the DOM. Nothing arrives. Cancelling a draft, switching a tab, collapsing a section: each of those removes the editor, and each is where it bites.

<.coelho_editor
  name="page[intro_doc]"
  value={@draft["intro_doc"]}
  flush_event="flush"
  flush_token={@generation}
/>

def handle_event("flush", %{"token" => token, "name" => name, "document" => document}, socket) do
  if token == to_string(socket.assigns.generation) do
    {:noreply, put_draft(socket, name, document)}
  else
    {:noreply, socket}
  end
end

The token travels as a DOM attribute, so it comes back as a string — comparing it to an integer generation is always false, and every flush is silently dropped.

The token is yours and so is the comparison, because only the application knows what a generation is. It matters: cancelling a draft re-renders the editors, and the editors being torn down flush the content from before the cancellation. Without a token the application bumps when it cancels, the flush puts back exactly what was just thrown away.

Several editors on one page

Each editor carries the exported schema — 1.3 KB for the one that ships — so six of them carry it six times. Render it once and point them at it:

<.coelho_schema id="page-schema" document_schema={MyApp.RichText.schema()} />

<.coelho_editor
  name="page[intro_doc]"
  value={@draft["intro_doc"]}
  document_schema={MyApp.RichText.schema()}
  schema_id="page-schema"
/>

Give the editors the same document_schema: schema_id says where the exported JSON lives, not which schema it is, and an editor filtering its toolbar against one schema while building documents from another would show up as buttons quietly doing nothing. The browser compares the two and refuses the mismatch out loud.

Toolbar labels come from labels={%{"bold" => gettext("Bold")}} — the commands are not words, and a toolbar has to speak the reader's language.

Testing it

The editor's container is phx-update="ignore", so it is invisible to render_change/2: there is no input to fill and no text to assert on. Write what the hook would have written:

import Coelho.LiveViewTest

type(view, "page[intro_doc]", paragraph("bonjour"))
assert document(view, "page[intro_doc]") == paragraph("bonjour")

params/3 builds the same parameters for a test that sends them its own way.

npm install @nseaprotector/acme-script prosemirror-state prosemirror-view \
  prosemirror-model prosemirror-keymap prosemirror-commands \
  prosemirror-history prosemirror-schema-list orderedmap

The schema travels to the browser in a data- attribute, so both halves build from the same declaration. The one thing Elixir cannot express is how a node looks while editing — toDOM/parseDOM are functions — so a schema of your own supplies those to createCoelhoHook/1:

import { createCoelhoHook } from "../../deps/coelho/assets/js/coelho.js"

const Coelho = createCoelhoHook({
  nodes: { mention: { toDOM: (node) => ["span", { class: "mention" }, "@" + node.attrs.user_id] } }
})

Migrating existing HTML

Content already stored as HTML has to become a document before any of the above applies to it:

{:ok, document, warnings} = Coelho.from_html(post.body_html)

post |> Ecto.Changeset.change(%{body: document}) |> Repo.update()

warnings says what was left behind — counts per tag, told apart by whether the schema has no rule for the element (:unknown_element), has one and refused what the element carried (:rejected_element), or kept the element and dropped an attribute (:dropped_attribute). Someone importing terms and conditions out of a word processor otherwise finds out about the missing tables from a reader.

Importing foreign markup is not validation, and failing on the first surprise would make it useless, so the rules are lenient and explicit: an element the schema has no rule for is transparent — it disappears and its children take its place, so a <div> wrapper costs you nothing; <script>, <style> and friends are dropped with their content; an element whose attributes fail the schema's validators — an <img> with no src, an <a href="javascript:…"> — is treated as unknown, so the link text survives while the link does not; and inline content where the schema wants blocks is wrapped in a paragraph. What comes out is a validated document, or the list of what still did not fit.

Nodes and marks declare the tags they come from, next to everything else about them:

paragraph: [content: "inline*", group: "block", parse: ["p"]]
heading: [content: "inline*", group: "block", parse: [{"h1", %{"level" => 1}}]]

Requires the optional floki dependency — the parser is only needed on this path.

Attaching files

An attachment node stores an opaque key, never a URL:

%{"type" => "attachment",
  "attrs" => %{"key" => "01J8Z…", "filename" => "plan.pdf", "content_type" => "application/pdf"}}

The URL is produced at render time, from the context:

Coelho.to_html(document, schema, context: %{resolve: &MyApp.Uploads.url/1})

What is stored is the key, never the URL, so every render asks again: a five minute signed URL is fine, moving a bucket is a resolver change instead of a data migration, and a key that no longer resolves degrades to its filename rather than to a broken image. Coelho.Attachments.keys/2 answers which keys a document still uses, which is what a cleanup job needs.

Storing a reference and resolving it late is not new — it is what any system that keeps attachments out of the markup does. What is different here is that the reference is a plain attribute of a validated node rather than a signed blob of identity smuggled through an HTML attribute, so the same walk that validates the document also enumerates its attachments.

Coelho.Attachment records the metadata; mix coelho.gen.migration creates its table. Deleting an image from a document leaves its bytes behind, so something has to sweep:

stored = Repo.all(from a in Coelho.Attachment, select: a.key)
documents = Repo.all(from p in Post, select: p.body)

{:ok, removed} = Coelho.Attachments.sweep(storage, stored, documents, dry_run: true)

documents must be every document that could still refer to something. Passing fewer deletes files that are still in use, which is why this asks for them rather than going and finding them: only the application knows where they all are.

The bytes themselves go through Coelho.Storage, a four-callback contract with a local-filesystem implementation in the box:

storage = Coelho.Storage.Disk.new("priv/uploads")
:ok = Coelho.Storage.put(storage, key, {:file, upload_path})

Serving them is a plug, and the URL that reaches it is signed and expiring:

# endpoint.ex
plug Coelho.Plug.Attachments,
  at: "/attachments",
  storage: {MyApp.Uploads, :storage, []},
  secret: {MyApp.Uploads, :secret, []},
  metadata: {MyApp.Uploads, :metadata, []}

# the resolver the renderer is given
Coelho.Attachments.signed_url("/attachments", key, secret, expires_in: 300)

Uploads served from your own origin are a standing hazard — a file the browser decides to render as HTML runs as your application — so the plug always sends x-content-type-options: nosniff and serves only a short list of image types inline. Everything else, SVG included, is sent as a download whatever it claims to be.

Writing to object storage instead means implementing the same callbacks, plus the optional redirect_url/3 — with one, the plug checks its signature and then hands the reader straight to a presigned URL rather than streaming every byte through the application. It only does that for the types it would have served inline anyway: the promise that everything else arrives as a download is made by headers a redirect does not carry. Coelho itself never touches the bytes.

In the editor, pass an upload config and dropped or pasted files go up through LiveView's own upload channel — and so do images pasted from other websites, which are fetched and stored rather than left as a URL on somebody else's host. Storing that URL is a hotlink: it leaks every reader's address to that host, and breaks the day the file moves. When the bytes cannot be read — usually CORS — the pasted URL is kept rather than lost, and a coelho:capture-failed event says so. Without an upload config nothing changes: the URL is stored as before.

<.coelho_editor field={@form[:body]} upload={@uploads.attachment} />

def handle_progress(:attachment, entry, socket) when entry.done? do
  attachment = consume_uploaded_entry(socket, entry, &MyApp.Uploads.store/1)

  {:noreply,
   Coelho.LiveView.insert_node(socket, Coelho.Attachment.to_node(attachment),
     id: Coelho.LiveView.editor_id(socket.assigns.form[:body]),
     preview: MyApp.Uploads.url(attachment.key)
   )}
end

The preview is only the editor's; what is stored is the key. Pass :id unless the page has exactly one editor — the event reaches all of them.

Serving attachments to more than one tenant

A signed URL is a bearer token: whoever holds it, holds the file. That is the right answer for a single-tenant application and the wrong one the moment there is more than one, and mounting the plug behind the application's authentication pipeline does not close it — that answers "may this person use the application", never "is this file theirs". A URL minted for one organisation, replayed by a signed-in member of another, passes both.

plug Coelho.Plug.Attachments,
  at: "/attachments",
  storage: {MyApp.Uploads, :storage, []},
  secret: {MyApp.Uploads, :secret, []},
  authorize: {MyApp.Uploads, :authorize, []}

def authorize(conn, key) do
  case conn.assigns[:current_organisation] do
    nil -> :error
    organisation -> MyApp.Uploads.owned_by?(key, organisation)
  end
end

The organisation comes from the connection, never from the key. The key arrives in the URL, which is to say from whoever sent the request, so reading the tenant out of it is asking the attacker which tenant they are in. That is also why Coelho.Attachment.generate_key(prefix: …) — which exists so an application can list its own objects per organisation — is an inventory aid and never a boundary. It belongs with object storage, too: Coelho.Storage.Disk shards on a key's first two characters, which a shared prefix makes identical for every tenant.

Writing a storage for S3 or MinIO: Coelho.Storage's documentation carries a complete ExAws adapter to copy, with the two things that bite — put/3 has to stream rather than read a file into memory, and ExAws over HTTP/2 fails above about a megabyte with :send_buffer_full.

Adding a node of your own

Most applications want the default schema and one thing besides — a mention, an embed, a callout. Re-declaring the other fifteen nodes to get there would guarantee they drift, so extend instead:

@schema Coelho.Schema.extend(Coelho.Schema.default(),
          nodes: [
            mention: [
              group: "inline",
              inline: true,
              void: true,
              attrs: [user_id: [required: true, validate: :integer]],
              render: &MyApp.RichText.render_mention/2,
              parse: [{"span", &MyApp.RichText.parse_mention/1}]
            ]
          ]
        )

Redeclaring an existing name replaces it, which is how the default schema's rendering gets adjusted without a fork. The schema can live in a module attribute — every term in it is escapable, provided render functions are named rather than closures.

A :class on a node or mark is applied by the server renderer and exported to the browser, so the writer sees the class the public page will carry without a hook written to put it there. Declaring it twice is what lets the two drift, so it is declared once:

marks: [highlight: [class: "hl hl-gradient", render: {"mark", []}]]

:editor_attrs carries DOM attributes for the editor alone.

Anything the server decides on reaches the document through one call:

Coelho.LiveView.insert_node(socket, %{"type" => "mention", "attrs" => %{"user_id" => 7}},
  id: Coelho.LiveView.editor_id(@form[:body])
)

The browser needs the other half — toDOM and parseDOM are functions and cannot come from Elixir:

const Coelho = createCoelhoHook({
  nodes: {
    mention: {
      toDOM: (node) => [
        "span",
        {class: "mention", "data-user-id": String(node.attrs.user_id)},
        `@${node.attrs.user_id}`
      ],
      parseDOM: [{
        tag: "span[data-user-id]",
        // `false` declines the rule. Without it a bad id builds a mention the
        // server then rejects, and the two halves disagree on what a mention is.
        getAttrs: (dom) => {
          const user_id = Number(dom.dataset.userId)
          return Number.isInteger(user_id) ? {user_id} : false
        }
      }]
    }
  }
})

demo/lib/demo/rich_text.ex does exactly this, and the browser test drives it end to end.

Declaring a schema from scratch

Coelho.Schema.new(
  top_node: :doc,
  nodes: [
    doc: [content: "block+"],
    paragraph: [content: "inline*", group: "block", render: {"p", []}],
    mention: [
      group: "inline",
      inline: true,
      void: true,
      attrs: [user_id: [required: true, validate: :integer]],
      render: &MyApp.RichText.render_mention/2
    ]
  ],
  marks: [bold: [render: {"strong", []}]]
)

Node and mark declaration order is preserved: ProseMirror resolves default types by position.

Roadmap

Beyond that, and deliberately out of scope for now: real time collaboration over y_ex, which is where the BEAM has something no other ecosystem does. The core carries no dependency on LiveView so that this stays possible.

Checking it

mix check                                              # format, compile, credo, dialyzer, test
docker compose -f docker/compose.yml run --rm --build browsers

The second one runs the browser checks against all three engines on Linux, which is where they behave the way CI's do — see docker/README.md for why that turned out to matter.

Demo

cd demo
mix setup
mix phx.server

One page, no database: the editor on the left, and on the right the same document rendered to HTML, stored as JSON, reduced to plain text and validated — all recomputed on the server on every keystroke. See demo/README.md.

Name

A nod to Paulo Coelho.

License

MIT.