Limen.Maze (limen v0.2.1)

Copy Markdown View Source

An endless maze of slow, plausible pages for clients caught misbehaving.

Clients that fell into a Limen.Trap, that a policy sent to the maze, or that were banned with action: :maze, get maze pages instead of your application. A maze page looks like an ordinary page of a website (an article, a listing of entries or a directory table) written by a language model, and every one of its links leads to another maze page. It is sent slowly, a few hundred bytes at a time with pauses in between, so a scraper spends its time and its crawl budget on pages worth nothing, without being told it was caught.

Stable per site, unpredictable elsewhere

The content of a page is drawn from a random generator seeded with an HMAC of its path, keyed with the instance's secret (see Limen.Config.Keys). So, on one site, a page is the same every time it is requested, and comparing two fetches reveals nothing; but it differs from one site to another and cannot be predicted without the secret, so Limen's maze cannot be recognised by recomputing it. With drift: :daily or :weekly, pages change at that pace, like a site being edited. Rotating the secret changes every page.

How a page is delivered is random on every request: where it is cut into chunks, how long each pause lasts, and so how long the response takes.

Precomputed, then slow

The expensive work happens once: the language model (Limen.Maze.Model), with every word escaped and every choice turned into a Limen.Maze.Dice, is built at compile time for Limen's bundled corpus, or when an instance with a :corpus of its own starts. Rendering a page takes a fraction of a millisecond. The rest of a maze response is spent asleep: the request process sends a chunk, sleeps, and sends the next one, at low process priority, until the page is sent or :max_duration has passed, in which case the rest goes out at once so the client can follow its links deeper. A closed connection ends it early.

At most :max_concurrent requests are held in the maze at once, counted with :atomics (held/1 reads the count); beyond that, clients get an immediate 429. Nothing on the way calls a process or sends a message. See the :maze options in Limen.Config.

Admission

Holding connections is cheap, but not free. To stop taking clients in when your application is under load, give the maze an :admit function, {module, function, args}: before a client is held, it is called with args and must return true to let it in. Otherwise the client gets the same immediate 429 as when the maze is full. It runs on the request path, so it must be cheap and must not call a process: read an :atomics or :persistent_term value your application keeps current.

config :my_app, Limen, maze: [admit: {MyApp.Load, :calm?, []}]

Every refusal emits a [:limen, :maze, :refused] event whose :reason is :full or :admission (see Limen.Telemetry); an :admit function that raises refuses the client too.

Maze links lead under the first trap path of the instance (see Limen.Trap), so following one is itself a confession; without trap paths, they lead below the requested path. Pages carry noindex, nofollow in a robots meta tag and an x-robots-tag header, which ask search engines neither to index them nor to follow their links.

Summary

Functions

The number of requests instance currently holds in the maze.

The language model instance writes its maze with.

Renders the maze page for path, with links under base.

Sends the maze page for the request to the client, slowly.

Functions

held(instance)

@spec held(atom() | Limen.Instance.t()) :: non_neg_integer()

The number of requests instance currently holds in the maze.

model(instance)

@spec model(Limen.Instance.t()) :: Limen.Maze.Model.t()

The language model instance writes its maze with.

page(instance, path, base, now)

@spec page(Limen.Instance.t(), String.t(), String.t(), integer()) :: iolist()

Renders the maze page for path, with links under base.

The same instance secret, path and (with :drift) period always give the same page. now is the system time in milliseconds, which only matters with :drift.

serve(conn, ctx, base)

@spec serve(Plug.Conn.t(), Limen.Context.t(), String.t()) :: Plug.Conn.t()

Sends the maze page for the request to the client, slowly.

Returns the halted connection. Clients beyond :max_concurrent, or that the :admit function turns away, get an immediate 429 instead.