Benchmarks

Performance characteristics of PhoenixFlags public API.

Run benchmarks yourself:

mix run bench/phoenix_flags_bench.exs

Benchmarks use their own phoenix_flags_bench database, created and migrated on first run. They deliberately do not share the test database: they run outside the Ecto sandbox, so their writes commit.

Environment

• Elixir 1.20.0-rc.3 / OTP 27
• PostgreSQL (local)
• 8 flags across 4 categories (boolean, integer, string, decimal, percentage, select, and two variant flags)

I/O Profile

Each function's database and network call count per invocation:

update_entry/3 also sends a fire-and-forget :reload message to Node.list() peers (N = number of connected nodes). Each peer then performs 1 DB call (SELECT all) to reload its local cache.

Results

1. Cached reads — get/2

Zero DB calls. Reads from :persistent_term + Map.get.

All types perform identically at ~100-130 ns median. The value is already cast and stored in a map — get/2 does no type conversion at read time.

2. all_grouped/0 (cached)

Zero DB calls. Reads entries from :persistent_term, sorts by pre-computed declaration order, groups by category.

The flag ordering index is pre-computed once at startup and stored in persistent_term, avoiding per-call flags() |> Enum.with_index() overhead.

3. flags/0 (compiled)

Zero DB calls. Returns a compiled list of %PhoenixFlags.Flag{} structs.

Fastest function — returns a module attribute.

4. select_options/1

Zero DB calls. Scans the flags/0 list for a matching key.

4b. variant/3 — A/B assignment

Zero DB calls. The split is parsed once when the cache loads, so a call is a :persistent_term read, one SHA-256, and a walk of the cumulative bucket list.

Measured separately from the sections above, on Elixir 1.20.3 / OTP 29 on a loaded development machine. Figures are rounded and should be read as orders of magnitude — repeat runs varied by up to 3x on the same code, and Benchee reports deviations above 1000% at this timescale. The call-count column in the I/O profile above is the part that is exact.

variant/3 is roughly 5-7x the cost of get/2, which is the SHA-256. That is deliberate: :erlang.phash2/2 would be cheaper but is not guaranteed stable across OTP major versions, and an OTP upgrade silently reshuffling every live experiment is a far worse outcome than ~0.5 us per assignment. At ~1.5 M assignments/sec it is not a bottleneck for request-path use.

Setting ttl: roughly doubles the cost, so leave it nil (the default) unless you actually want assignments to expire.

4c. Targeting overhead

Zero DB calls. Resolution is ordered so that the overwhelmingly common case is cheapest: a flag with no rules costs one :persistent_term read plus a map lookup, and never touches the process dictionary. The context is only fetched once a rule exists that could match.

That ordering was chosen by measurement, not intuition. Checking the context first — to skip the :persistent_term read entirely when none is set — turned out to be slower: Process.get/1 plus a Keyword.get/3 measured ~32 ns against the ~18 ns read it was avoiding. Reversing it halved the context-set-but-no-rules path.

Measured in isolation against a no-op floor, same process:

So the added cost for a flag with no rules is roughly one persistent_term read plus a map lookup — on the order of 20 ns against a get/2 in the low hundreds of nanoseconds.

End-to-end medians for the four scenarios are not quoted here: the two "flag has no rules" cases provably perform identical work (the context is never read), yet repeat runs on a loaded machine differ by 2x with Benchee reporting deviations above 4000%. At this timescale only the isolated figures above are meaningful. bench/phoenix_flags_bench.exs section 4c runs all four in one process if you want to see them on your own hardware.

Rule evaluation when a context is set adds the context normalisation (one pass, to_string/1 per attribute) and a walk of that flag's rules until one matches — still no database call, and rules per flag are typically a handful.

5. update_entry/3 (DB write + incremental cache patch)

2 DB calls per update: SELECT by key + UPDATE. Cache is patched in-memory (no reload query).

Each benchmark iteration performs 2 updates (toggle + restore), so a single update_entry/3 takes ~0.95 ms. The cost is dominated by DB round-trips (SELECT + UPDATE). The cache reload SELECT ALL was eliminated by patching the persistent_term tuple in-memory.

6. Cache vs DB: get/2

Head-to-head comparison of cached (persistent_term) vs uncached (direct DB query) reads.

Cached reads are ~500x faster and use ~160x less memory than direct DB queries.

7. Cache vs DB: all_grouped/0

Cached is ~50x faster and uses ~17x less memory.

8. Full workflow cycle

Simulates a dashboard interaction: 6 reads + grouped view + update + read + restore.

Of the total time, the 8 cached reads contribute <2 us combined. The remaining time is entirely from the 2 update_entry/3 DB round-trips (4 DB calls total).

Summary

Reads are zero-cost in production. Writes are intentionally slow (database is source of truth) and only happen during admin configuration changes.