ExternalService is an Elixir library for safely calling external services and
APIs, combining four well-established reliability techniques behind one small
interface:
- Retries for transient failures,
- the Circuit Breaker pattern for persistent failures,
- rate limiting for staying within a service's quota, and
- a concurrency limit (the bulkhead pattern) for bounding how many calls are in flight at once.
Calls can be synchronous, asynchronous background tasks, or fanned out in parallel for MapReduce-style processing — all under the same retry, circuit breaker, rate-limiting, and concurrency protection.
The ideas
Retrying failed requests
Many failures when accessing an external service are transient: network
congestion causing a timeout, or a service briefly under heavy load. The best
response is often simply to try again after a short backoff. ExternalService
automates retry logic — linear or exponential backoff, jitter, delay caps, and
both attempt-count and time budgets — with delay-stream logic ported from
Safwan Kamarrudin's retry library. See
Retries.
Circuit breakers
The Circuit Breaker pattern was first described in Michael Nygard's Release It! and later popularized by Martin Fowler. To quote Nygard, "Circuit breakers are a way to automatically degrade functionality when the system is under stress."
Like the electrical breakers they're named for, software circuit breakers protect a system from damage caused by a faulty component. By monitoring calls to an external service, a breaker can "trip" once failures cross a threshold, after which further calls fail fast instead of piling up against a service that is already in trouble. After a cool-down it resets and calls resume. Crucially, the breaker is global to the service, so a trip protects every caller at once and prevents cascading failures.
ExternalService implements circuit breakers on top of
Jesper Louis Andersen's Erlang fuse library,
managing the underlying fuse for you — you never call :fuse.ask or
:fuse.melt yourself. See Circuit breakers.
Rate limiting
Many services impose a request quota. ExternalService can keep you under it
automatically with a built-in token bucket: excess calls sleep for up to a
bounded budget to absorb bursts, then fail fast with a structured error rather
than piling up. The default backend meters each node's own traffic; a
pluggable backend can instead enforce one limit across a whole cluster. See
Rate limiting.
Concurrency limiting
Retries bound how many failures you tolerate and the rate limiter bounds how
often calls start — neither bounds how many are in flight at once. That
gap opens when a service degrades rather than fails outright: calls still
succeed, just slowly, so neither the breaker nor the limiter notices, and
callers pile up holding connections. A concurrency limit (the bulkhead pattern)
caps that directly: over the limit, a call returns
ExternalService.ServiceSaturated instead of queuing indefinitely. See
Concurrency.
History
ExternalService grew out of production code at Ropig, where it was first used
to make reliable calls to Google's Pub/Sub messaging service. It was one of
author Jason Voegele's first open-source Elixir libraries and went on to see wide
production adoption. The original overview is described in
this blog post.
The 2.0 line is a ground-up modernization of that original library — validated
and self-documenting options, telemetry, circuit-breaker introspection,
structured errors, and a declarative module front door (use ExternalService) —
while keeping the core idea unchanged: wrap a call in a function, hand it over,
and let retries, the circuit breaker, rate limiting, and the concurrency limit
just work.
License
ExternalService is released under the Apache 2.0 license.