Hackney Architecture
View SourceThis document describes the internal architecture of hackney: the process-per-connection model, connection pooling, load regulation, and TLS handling. Read it when you need to understand why a request behaves as it does, or before you change the connection or the pool. To find the module and the tests behind a concept, see Where Things Live.
Overview
Hackney uses a process-per-connection architecture where each HTTP connection runs in its own gen_statem process. This design provides:
- Clean isolation - Each connection has its own state, no shared mutable state
- Automatic cleanup - Process crashes clean up sockets automatically
- Simple ownership - Socket always owned by connection process
- OTP supervision - Standard supervisor tree for fault tolerance
hackney_sup
├── hackney_manager (kept for the get_state/1 and
│ async_response_pid/1 accessors)
├── hackney_conn_sup (connection supervisor)
│ └── hackney_conn [1..N] (one process per connection, any protocol)
└── hackney_pool [1..N] (one process per pool, started on demand)Three things are not processes of their own. The Alt-Svc cache
(hackney_altsvc) and the per-host counters (hackney_load_regulation) are
ETS tables. HTTP/2 and HTTP/3 are provided by the h2 and quic
dependencies: a hackney_conn process owns an h2_connection process, or a
QUIC connection reached through hackney_h3, and translates their events for
the caller.
Connection Process (hackney_conn)
Each connection is a gen_statem process that manages:
- TCP/SSL socket
- HTTP protocol state (request/response phases)
- Streaming state (chunked encoding, content-length tracking)
- Owner process monitoring
State Machine
hackney_conn has nine states. Each is a function of the same name in
src/hackney_conn.erl, under a State: banner comment.
| State | Meaning | Leaves to |
|---|---|---|
idle | Process started, not connected. connect dials from here and moves straight to connected | connected, closed |
connecting | Reserved for a handshake in progress. Nothing enters it today | |
connected | Ready for a request | sending, streaming_body, closed |
sending | Writing a request that has its whole body | receiving, closed |
streaming_body | The caller sends the request body in chunks | receiving, connected, closed |
receiving | Waiting for, or reading, the response | connected, streaming, streaming_once, closed |
streaming | Async response, chunks pushed to the caller (async = true) | streaming_once, receiving, connected, closed |
streaming_once | Async response, one chunk per stream_next/1 (async = once) | streaming, receiving, connected, closed |
closed | Connection terminated | connected (reconnect), exit |
An HTTP/1.1 request walks through these states one at a time:
idle -> connected -> sending -> receiving -> connected
\-> streaming_body -/ \-> streaming | streaming_onceAn HTTP/2 or HTTP/3 connection stays in connected while it serves requests.
Each request is a stream, tracked in the h2_streams or h3_streams map, so
several callers use the connection at the same time. This is why connected
is by far the largest state in the module.
Owner Monitoring
The connection process monitors its owner: the process that checked out the connection, or for a connection opened without a pool, the process that opened it (with stream_to, the stream_to process once the async request starts). If the owner crashes, the connection terminates automatically, preventing socket leaks. An HTTP/1.1 read waiting on the server checks for this every second. hackney_conn:set_owner/2 moves ownership to another process.
%% When connection is checked out
MonitorRef = monitor(process, Owner),
%% If owner dies
{'DOWN', MonitorRef, process, Owner, _} -> terminateConnection Pool (hackney_pool)
The pool keeps three kinds of connection, and treats them differently:
| Kind | How it is used | Kept after a request |
|---|---|---|
| HTTP/1.1 over TCP | Checked out by one caller at a time | Yes, as an idle connection |
| HTTP/1.1 over TLS | Checked out by one caller at a time | Only with ssl_pooling, otherwise closed |
| HTTP/2 and HTTP/3 | Shared: every caller for the host uses the same connection | Yes, until idle or closed by the server |
TLS Connections
By default the pool stores TCP connections only. An HTTPS request takes a TCP connection, upgrades it to TLS, and closes it when the request is done, so every HTTPS/1.1 request does a fresh handshake.
With {ssl_pooling, true} (request option or application env), an upgraded
HTTPS/1.1 connection goes back to the pool. It is only handed to a later
request whose effective TLS options hash to the same key, so a connection
verified against one CA or SNI is never reused for another.
Pool State
-record(state, {
name, %% Pool name
max_connections, %% Warm (idle) connections kept for reuse
keepalive_timeout, %% Max idle time (default 2000ms, max 2000ms)
prewarm_count, %% Connections to maintain per host (default 4)
available = #{}, %% #{Key => [Pid]} - idle connections
in_use = #{}, %% #{Pid => Key} - checked out connections
dialing = #{}, %% Handed out undialed, the requester is dialing
pid_monitors = #{}, %% #{Pid => MonitorRef}
activated_hosts, %% Hosts with prewarm enabled
sock_opts = #{}, %% connect_options per socket options bucket
h2_connections = #{}, %% #{Key => Pid} - one shared connection per host
h3_connections = #{}, %% #{Key => Pid} - one shared connection per host
h3_sessions = #{} %% HTTP/3 session tickets for resumption
}).The pool process never dials. It starts the connection process and hands it out; the requester dials it and reports the result. A slow or failing connect therefore blocks one caller, not the pool.
Pool Operations
Checkout: Get an available TCP connection or none
hackney_pool:checkout(Host, Port, Transport, Opts)
%% Returns: {ok, PoolInfo, Pid} | {error, no_pool}Checkin: Return a connection to the pool
hackney_pool:checkin(PoolInfo, Pid)
%% TCP connections are stored; TLS connections only with ssl_poolingKeepalive Timeout
Idle connections are closed after keepalive_timeout (default and max: 2 seconds). This prevents:
- Stale connections that the server has closed
- Resource accumulation from unused connections
- Issues with server-side connection limits
Load Regulation (hackney_load_regulation)
Per-host connection limits prevent overwhelming individual servers. This uses an ETS counting semaphore pattern for lock-free concurrent access.
How It Works
%% ETS table: hackney_host_limits
%% Key: {Host, Port} -> Value: current_count
%% Acquire a slot (blocks with backoff until available or timeout)
acquire(Host, Port, MaxPerHost, Timeout) ->
Count = ets:update_counter(Table, Key, {2, 1}, {Key, 0}),
case Count =< MaxPerHost of
true -> ok;
false ->
ets:update_counter(Table, Key, {2, -1}),
%% Backoff and retry until timeout
end.
%% Release a slot
release(Host, Port) ->
ets:update_counter(Table, Key, {2, -1, 0, 0}).Why ETS Counting Semaphore?
- Lock-free -
ets:update_counteris atomic - Per-host isolation - Different hosts don't block each other
- No process bottleneck - No gen_server call for every request
- Backpressure - Requests wait when limit reached
Configuration
%% Default: 50 concurrent connections per host
hackney:get(URL, [], <<>>, [{max_per_host, 100}]).
%% Per-request timeout for acquiring a slot
hackney:get(URL, [], <<>>, [{checkout_timeout, 5000}]).SSL Upgrade Strategy
HTTPS requests use TCP connection upgrade rather than direct SSL connections:
1. Get TCP connection (from pool or new)
2. Upgrade to SSL in-place: ssl:connect(Socket, SslOpts)
3. If ALPN negotiated HTTP/2, register the connection as shared
4. Use the connection for the request
5. HTTP/1.1: close it at checkin, or return it to the pool with ssl_pooling
6. Trigger TCP prewarm for the next HTTPS requestBenefits
- Connection reuse - Pooled TCP connections can serve HTTP or HTTPS
- Prewarm works for HTTPS - TCP connections ready to upgrade
- One connect path - Proxies, happy eyeballs and socket options apply the same way to HTTP and HTTPS
Code Flow
The flow lives in connect_pool_new/6 in src/hackney.erl:
hackney_load_regulation:acquire/4takes a per-host slot.hackney_pool:checkout/4returns a TCP connection (checkout_ssl/4withssl_pooling, which may return an already upgraded one).hackney_conn:upgrade_to_ssl/2,3upgrades it for an HTTPS request.hackney_conn:get_protocol/1tells whether HTTP/2 was negotiated. If so, the connection is shared and registered withhackney_pool:register_h2/5.
Protocol Selection
Hackney supports three HTTP protocols: HTTP/1.1, HTTP/2, and HTTP/3 (experimental). The protocol selection is controlled via the protocols option.
Default Protocols
By default, hackney uses [http2, http1]:
%% Default behavior - HTTP/2 preferred, HTTP/1.1 fallback
hackney:get("https://example.com/")The default can be changed via application environment:
%% In sys.config or at runtime
application:set_env(hackney, default_protocols, [http2, http1]).Enabling HTTP/3 (Experimental)
HTTP/3 uses QUIC (UDP transport). To enable HTTP/3:
%% Per-request: opt-in to HTTP/3
hackney:get("https://example.com/", [], <<>>, [
{protocols, [http3, http2, http1]}
]).
%% Application-wide: enable HTTP/3 by default
application:set_env(hackney, default_protocols, [http3, http2, http1]).Important considerations for HTTP/3:
- Experimental - QUIC support is still maturing
- UDP may be blocked - Corporate firewalls often block UDP
Protocol Priority
Protocols are tried in order. With [http3, http2, http1]:
- If QUIC is available and server supports HTTP/3: use HTTP/3
- Otherwise, ALPN negotiates HTTP/2 or HTTP/1.1 over TLS
- Server chooses the highest protocol it supports
Forcing a Single Protocol
%% Force HTTP/1.1 only (no HTTP/2 negotiation)
hackney:get(URL, [], <<>>, [{protocols, [http1]}]).
%% Force HTTP/2 only
hackney:get(URL, [], <<>>, [{protocols, [http2]}]).
%% HTTP/3 only (will fail if QUIC unavailable or server doesn't support it)
hackney:get(URL, [], <<>>, [{protocols, [http3]}]).HTTP/3 and QUIC Architecture
HTTP/3 connections use QUIC (UDP-based transport) via a pure Erlang implementation from the quic dependency.
Event-Driven Architecture
Like TCP connections, QUIC uses an event-driven architecture where the owner process drives the connection:
┌─────────────────────────────────────────────────────────────────┐
│ Owner Process (Erlang) │
│ │
│ 1. connect() → Creates QUIC connection via quic library │
│ │
│ 2. Receives {select, Resource, Ref, ready_input} │
│ └── Socket has data ready │
│ │
│ 3. Calls hackney_h3:process(ConnRef) │
│ └── Receives UDP packets │
│ └── Processes QUIC frames │
│ └── Triggers events (headers, data, etc.) │
│ └── Returns next timeout in ms │
│ │
│ 4. Receives {h3, ConnRef, Event} │
│ └── {connected, Info} │
│ └── {stream_headers, StreamId, Headers, Fin} │
│ └── {stream_data, StreamId, Data, Fin} │
│ └── {closed, Reason} │
│ │
│ 5. Schedules timer: erlang:send_after(TimeoutMs, self(), ...) │
│ └── Calls process() again when timer fires │
└─────────────────────────────────────────────────────────────────┘Pure Erlang Components
┌─────────────────────────────────────────────────────────────────┐
│ hackney_h3.erl │
│ - connect/4: Start QUIC connection via quic_h3 │
│ - send_request/3: Open stream + send HEADERS │
│ - send_data/4: Send request body │
│ - reset_stream/3: Cancel an in-flight stream │
│ - close/2: Close connection │
└─────────────────────────────────────────────────────────────────┘
│
▼
┌─────────────────────────────────────────────────────────────────┐
│ quic application (dependency) │
│ - QUIC protocol implementation (RFC 9000) │
│ - TLS 1.3 handshake │
│ - Packet encoding/decoding │
│ - Congestion control and loss recovery │
└─────────────────────────────────────────────────────────────────┘Connection Lifecycle
Owner Process quic library
│ │
│ hackney_h3:connect(...) │
├────────────────────────────────────►│ Create UDP socket
│ │ Generate TLS keys
│ │ Send Initial packet
│◄────────────────────────────────────┤ {ok, ConnRef}
│ │
│ {select, _, _, ready_input} │
│◄────────────────────────────────────┤ UDP packet received
│ │
│ hackney_h3:process(ConnRef) │
├────────────────────────────────────►│ Process QUIC packets
│ │ Continue handshake
│ {h3, ConnRef, {connected, Info}} │
│◄────────────────────────────────────┤
│ │
│ ... (request/response cycle) ... │
│ │
│ hackney_h3:close(ConnRef, ...) │
├────────────────────────────────────►│ Send CONNECTION_CLOSE
│ {h3, ConnRef, {closed, normal}} │
│◄────────────────────────────────────┤
│ │Benefits of Pure Erlang Implementation
| Aspect | Pure Erlang | NIF-based |
|---|---|---|
| Portability | All platforms | Requires C compiler |
| Build complexity | rebar3 compile | CMake + dependencies |
| Debugging | Erlang tools | GDB/LLDB |
| Crash isolation | Process crash | Possible VM crash |
| Hot code loading | Supported | Limited |
HTTP/2 Multiplexing
HTTP/2 connections are handled differently from HTTP/1.1. A single HTTP/2 connection can handle multiple concurrent requests via stream multiplexing.
HTTP/2 Pool Design
The pool maintains a separate map for HTTP/2 connections:
-record(state, {
%% ... existing fields ...
%% HTTP/2 connections: one per host, shared across callers
h2_connections = #{} %% #{Key => Pid}
}).Key differences from HTTP/1.1 pooling:
| Aspect | HTTP/1.1 | HTTP/2 |
|---|---|---|
| Connections per host | Multiple (pool) | One (shared) |
| Checkout behavior | Exclusive access | Shared access |
| Checkin behavior | Return to pool | Keep in pool |
| Request handling | Sequential | Multiplexed streams |
HTTP/2 Connection Flow
hackney:get("https://api.example.com/data")
│
▼
┌─────────────────────────────────────┐
│ 1. Check for existing HTTP/2 conn │
│ checkout_h2(Host, Port, ...) │
│ → Returns {ok, Pid} or none │
└─────────────────┬───────────────────┘
│
┌─────────┴─────────┐
│ │
{ok, Pid} none
(reuse!) │
│ ▼
│ ┌─────────────────────┐
│ │ 2. Normal TCP flow │
│ │ checkout → new │
│ │ → upgrade SSL │
│ └──────────┬──────────┘
│ │
│ ▼
│ ┌─────────────────────┐
│ │ 3. Check protocol │
│ │ get_protocol() │
│ │ → http2 | http1 │
│ └──────────┬──────────┘
│ │
│ ┌─────┴─────┐
│ │ │
│ http2 http1
│ │ │
│ ▼ │
│ ┌─────────────────┐ │
│ │ 4. Register H2 │ │
│ │ register_h2()│ │
│ └────────┬────────┘ │
│ │ │
└──────┬──────┘ │
│ │
▼ │
┌─────────────────────────────────────┐
│ 5. Send request │
│ HTTP/2: assign StreamId │
│ HTTP/1.1: send directly │
└─────────────────────────────────────┘Stream Multiplexing in hackney_conn
HTTP/2 framing, HPACK and flow control come from the h2 dependency. When
ALPN selects HTTP/2, hackney_conn starts an h2_connection process on the
socket and keeps its pid in h2_conn. From then on:
- A request calls
h2_connection:send_request_headers, which returns a stream id. hackney_connrecords the caller and the stream's progress in theh2_streamsmap, keyed by stream id.- The
h2library sends its events as{h2, H2Conn, Event}messages.hackney_connlooks up the stream id and replies to the right caller. - When the stream ends, its entry is removed.
A connection shared through the pool has no single owner. Each stream monitors its own caller, and the connection closes itself once it is idle.
The h2_* bidirectional API (hackney_h2_stream) is separate: it opens a
dedicated connection and receives its stream's events directly from h2.
Benefits of HTTP/2 Multiplexing
- Reduced latency - No connection setup for subsequent requests
- Better resource usage - One TCP connection instead of many
- Head-of-line blocking avoided - Responses can arrive out of order
- Server efficiency - Servers prefer fewer connections with more streams
ALPN Protocol Negotiation
HTTP/2 is negotiated during TLS handshake via ALPN:
%% In hackney_ssl.erl
alpn_opts(Opts) ->
Protocols = proplists:get_value(protocols, Opts, [http2, http1]),
AlpnProtos = [proto_to_alpn(P) || P <- Protocols],
[{alpn_advertised_protocols, AlpnProtos}].
proto_to_alpn(http2) -> <<"h2">>;
proto_to_alpn(http1) -> <<"http/1.1">>.
%% After connection
get_negotiated_protocol(SslSocket) ->
case ssl:negotiated_protocol(SslSocket) of
{ok, <<"h2">>} -> http2;
_ -> http1
end.Connection Prewarm
After first use of a host, the pool maintains warm TCP connections ready for immediate use.
How It Works
- First request to
api.example.com:443completes - On checkin, pool marks host as "activated"
- Pool creates
prewarm_count(default 4) TCP connections - Next request gets connection immediately (no connect latency)
Configuration
%% Global default
application:set_env(hackney, prewarm_count, 4).
%% Per-pool
hackney_pool:start_pool(mypool, [{prewarm_count, 8}]).
%% Explicit prewarm
hackney_pool:prewarm(default, "api.example.com", 443, 10).Prewarm for HTTPS
When an SSL connection is checked in and closed (the default, without ssl_pooling), the pool still triggers TCP prewarm. This ensures TCP connections are ready for the next HTTPS request to upgrade.
Request Flow
Complete flow for an HTTPS request with pooling:
hackney:get("https://api.example.com/data")
│
▼
┌─────────────────────────────────────┐
│ 1. Load Regulation │
│ acquire("api.example.com", 443, │
│ MaxPerHost, Timeout) │
│ → Blocks if at limit │
└─────────────────┬───────────────────┘
│ ok
▼
┌─────────────────────────────────────┐
│ 2. Pool Checkout │
│ checkout(Host, 443, hackney_tcp) │
│ → Returns Pid or none │
└─────────────────┬───────────────────┘
│
┌─────────┴─────────┐
│ │
{ok, Pid} none
│ │
│ ▼
│ ┌─────────────────┐
│ │ Create new conn │
│ │ hackney_conn_sup│
│ └────────┬────────┘
│ │
└────────┬─────────┘
│
▼
┌─────────────────────────────────────┐
│ 3. SSL Upgrade │
│ upgrade_to_ssl(Pid, SslOpts) │
│ → TCP socket becomes SSL │
└─────────────────┬───────────────────┘
│
▼
┌─────────────────────────────────────┐
│ 4. HTTP Request │
│ send_request(Pid, Method, ...) │
│ recv_response(Pid) │
└─────────────────┬───────────────────┘
│
▼
┌─────────────────────────────────────┐
│ 5. Checkin (async) │
│ Closed (or pooled: ssl_pooling) │
│ TCP prewarm triggered │
└─────────────────┬───────────────────┘
│
▼
┌─────────────────────────────────────┐
│ 6. Load Regulation Release │
│ release("api.example.com", 443) │
│ → Slot available for next req │
└─────────────────────────────────────┘Monitoring and Stats
Pool Stats
hackney_pool:get_stats(PoolName).
%% Returns:
%% [{name, PoolName},
%% {max, MaxConnections},
%% {in_use_count, InUse},
%% {free_count, Free},
%% {queue_count, 0}] %% Always 0, load regulation handles queuingPer-Host Stats
hackney_pool:host_stats(PoolName, Host, Port).
%% Returns:
%% [{active, N}, %% Currently in use (from load_regulation)
%% {in_use, N}, %% Checked out from pool
%% {free, N}] %% Available in poolLoad Regulation Stats
hackney_load_regulation:current(Host, Port).
%% Returns: integer() - current concurrent connections to hostAdvantages of This Architecture
vs. hackney 1.x
| Aspect | 1.x | 2.x and later |
|---|---|---|
| State storage | ETS tables | Process state |
| Socket ownership | Transferred between processes | Always connection process |
| Error cleanup | Manual via manager | Automatic via process exit |
| SSL pooling | Yes, keyed by host and port only | Off by default; opt-in, keyed by TLS options |
| Connection limits | Global pool size | Per-host limits |
| Prewarm | No | Yes |
vs. Other HTTP Clients
Process isolation: Each connection is independent. A slow response on one connection doesn't block others. A crash in one connection doesn't affect others.
Backpressure: Load regulation naturally applies backpressure when a server is overwhelmed. Requests wait rather than creating unbounded connections.
Resource control: Per-host limits prevent a single slow host from consuming all connections. Different hosts are isolated.
SSL security: By default an HTTPS/1.1 connection is not reused, so each request does a fresh handshake. With ssl_pooling, a connection is only reused for requests with the same TLS options.
Prewarm efficiency: Frequently-used hosts have warm connections ready, eliminating connection latency for subsequent requests.
Configuration Reference
Pool Options
| Option | Default | Description |
|---|---|---|
pool_size / max_connections | 50 | Warm (idle) connections kept for reuse. Concurrency is capped by max_per_host; requests above this open overflow connections that close at checkin |
timeout / keepalive_timeout | 2000 | Idle timeout (max 2000ms) |
prewarm_count | 4 | Connections to maintain per host |
Request Options
| Option | Default | Description |
|---|---|---|
pool | default | Pool name, or false for no pooling |
max_per_host | 50 | Max concurrent connections to host |
checkout_timeout | 8000 | Timeout to acquire connection slot |
connect_timeout | 8000 | TCP connect timeout |
recv_timeout | 5000 | Response receive timeout |
Application Environment
%% In sys.config or application:set_env
{hackney, [
{pool_handler, hackney_pool},
{max_connections, 50},
{timeout, 2000},
{prewarm_count, 4}
]}.