nhttp_h1 (nhttp_lib v1.0.0)

View Source

HTTP/1.1 codec module - High-performance binary:split implementation.

Provides parsing and encoding for HTTP/1.1 requests and responses. Uses binary:split BIF for optimal parsing performance.

Parsing

Parsing functions return {ok, Result, BytesConsumed} where BytesConsumed is the number of bytes consumed from the input. Use split_at/2 to get the remaining buffer:

{ok, Request, Consumed} = nhttp_h1:parse_request(Binary),
Rest = nhttp_h1:split_at(Binary, Consumed).

This pattern is optimal for performance as it avoids creating intermediate binaries until the consumer explicitly needs the remainder.

For incomplete data, parsing returns {more, MinBytes} where MinBytes is a hint for how many more bytes might be needed.

Options

The opts() map supports the following limits:

  • max_header_size - Maximum total size of all headers in bytes (default: infinity)
  • max_headers_count - Maximum number of headers (default: infinity)
  • max_body_size - Maximum body size in bytes (default: infinity)

When a limit is exceeded, parsing returns {error, header_too_large}, {error, too_many_headers}, or {error, {body_too_large, Size, Max}} respectively.

Opts = #{max_header_size => 8192, max_headers_count => 100, max_body_size => 1048576},
case nhttp_h1:parse_request(Binary, Opts) of
    {ok, Request, Consumed} -> handle_request(Request);
    {error, header_too_large} -> respond_413();
    {error, too_many_headers} -> respond_431();
    {error, {body_too_large, _Size, _Max}} -> respond_413()
end.

Encoding

IOList = nhttp_h1:encode_request(Request).
IOList = nhttp_h1:encode_response(Response).

encode_request/1 and encode_response/1 consume the canonical nhttp_lib:request/0 / nhttp_lib:response/0 map shape. The body field is for the convenience case where the whole payload fits in memory: it is emitted inline after the header block and a Content-Length is derived if neither Content-Length nor Transfer-Encoding is present in headers.

For streaming bodies, do not populate body in the map. Send the header block first via encode_response_head/3, then emit each chunk via encode_chunk/1, then close the body with encode_last_chunk/0 (set Transfer-Encoding: chunked in the headers). The same staged pattern applies to chunked requests.

Summary

Types

body_chunk()
body_mode()
body_stream()
chunked_st()
opts()
parse_error()
parse_result(T)
req()
resp()
version()

Functions

body_stream_from_response/3

Compute the response body framing mode from the request method, response status, and response headers (RFC 9112 §6.3).

encode_chunk/1

Encode a chunk for chunked transfer encoding.

encode_last_chunk()

Encode the final (zero-length) chunk.

encode_request/1

Encode an HTTP/1.1 request to iolist.

encode_response/1

Encode an HTTP/1.1 response to iolist.

encode_response_head(Version, Status, Headers)

Encode HTTP/1.x response headers for streaming. Used when sending chunked responses - sends status line + headers only.

finalize_response_body/1

Signal end-of-stream for a response body parse driven by parse_response_body/2. Used to terminate until_close framing when the underlying transport closes, and to surface mid-body framing errors for {length, _} and {chunked, _}.

parse_request/1

Parse an HTTP/1.1 request from binary. Returns {ok, Request, BytesConsumed} on success. Use split_at/2 to get the remaining buffer.

parse_request/2

Parse HTTP/1.1 request with options. Options can include: max_header_size, max_headers_count, max_body_size.

parse_request_body/2

Feed body bytes for a streaming request whose headers were parsed via parse_request_headers/1,2. Returns one of

parse_request_headers/1
parse_request_headers/2

Parse HTTP/1.1 request headers only, without consuming the body, enforcing the supplied limits. Returns {ok, Request, BodyStream, BytesConsumed}. The body framing mode is encoded in BodyStream

parse_response/1

Parse an HTTP/1.1 response from binary. Returns {ok, Response, BytesConsumed} on success. Use split_at/2 to get the remaining buffer.

parse_response/2

Parse HTTP/1.1 response with options. Options can include: max_header_size, max_headers_count, max_body_size.

parse_response_body/2

Feed body bytes for a streaming response whose headers were parsed via parse_response_headers/1,2 and whose framing mode was selected via body_stream_from_response/3. For none and {length, 0}, returns {ok, [{fin, []}], none, 0}. For {length, N>0} and {chunked, _}, behaves identically to parse_request_body/2. For until_close, emits [{data, _}] for whatever bytes are in the buffer and keeps the stream open. The caller must signal EOF via finalize_response_body/1 when the underlying transport closes.

parse_response_head/1

Equivalent to parse_response_head/2.

parse_response_head/2

Parse HTTP/1.1 response headers only, like parse_response_headers/2, but also return the reason phrase and protocol version from the status line. Returns {ok, Status, Reason, Version, Headers, Rest} where Rest is the binary after the headers. Used by streaming callers that must preserve the full response head (status, reason, version) while reading the body separately via parse_response_body/2.

parse_response_headers/1
parse_response_headers/2

Parse HTTP/1.1 response headers only, without body, enforcing the supplied limits (max_header_size, max_headers_count).

split_at/2

Split buffer at position, returning the remainder.

Types

body_chunk()

-type body_chunk() :: {data, binary()} | {fin, nhttp_lib:headers()} | {abort, nhttp_lib:error()}.

body_mode()

-type body_mode() :: undefined | {content_length, non_neg_integer()} | chunked.

body_stream()

-type body_stream() :: {chunked, chunked_st()} | {length, non_neg_integer()} | until_close | none.

chunked_st()

-opaque chunked_st()

opts()

-type opts() ::
          #{max_header_size => pos_integer(),
            max_headers_count => pos_integer(),
            max_body_size => pos_integer(),
            scheme => nhttp_lib:scheme(),
            peer => nhttp_lib:peer()}.

parse_error()

-type parse_error() ::
          bad_request_line | bad_status_line | bad_header | header_too_large | too_many_headers |
          {body_too_large, Size :: non_neg_integer(), Max :: non_neg_integer()} |
          invalid_content_length | duplicate_content_length | conflicting_framing |
          unsupported_transfer_encoding | invalid_chunk_size | incomplete_chunk | invalid_method |
          invalid_version | unexpected_eof |
          {protocol_error, term()}.

parse_result(T)

-type parse_result(T) ::
          {ok, T, BytesConsumed :: pos_integer()} |
          {more, MinBytes :: pos_integer()} |
          {error, parse_error()}.

req()

-type req() :: nhttp_lib:request().

resp()

-type resp() :: nhttp_lib:response().

version()

-type version() :: http1_0 | http1_1.

Functions

body_stream_from_response/3

-spec body_stream_from_response(nhttp_lib:method(), nhttp_lib:status(), nhttp_lib:headers()) ->
                                   body_stream().

Compute the response body framing mode from the request method, response status, and response headers (RFC 9112 §6.3).

Pure helper: pass the values parsed by parse_response_headers/1,2 plus the method of the matching request. The returned body_stream() is fed into parse_response_body/2.

Framing rules in order:

  • HEAD request → none (HEAD responses never have a body).
  • 1xx, 204, 304 status → none.
  • Transfer-Encoding: chunked → {chunked, _}.
  • Content-Length: N → {length, N}.
  • otherwise → until_close (RFC 9112 §6.3 #7).

The chunked / length walkers do not enforce header or body size limits in this entry point. Callers reading from untrusted peers should validate sizes at the recv site or wrap the stream.

encode_chunk/1

-spec encode_chunk(iodata()) -> iolist().

Encode a chunk for chunked transfer encoding.

encode_last_chunk()

-spec encode_last_chunk() -> binary().

Encode the final (zero-length) chunk.

encode_request/1

-spec encode_request(req()) -> iolist().

Encode an HTTP/1.1 request to iolist.

encode_response/1

-spec encode_response(resp()) -> iolist().

Encode an HTTP/1.1 response to iolist.

encode_response_head(Version, Status, Headers)

-spec encode_response_head(version(), nhttp_lib:status(), nhttp_lib:headers()) -> iolist().

Encode HTTP/1.x response headers for streaming. Used when sending chunked responses - sends status line + headers only.

finalize_response_body/1

-spec finalize_response_body(body_stream()) -> {ok, [body_chunk()]} | {error, parse_error()}.

Signal end-of-stream for a response body parse driven by parse_response_body/2. Used to terminate until_close framing when the underlying transport closes, and to surface mid-body framing errors for {length, _} and {chunked, _}.

  • none, {length, 0}, or until_close → {ok, [{fin, []}]}.
  • {length, N>0} → {error, unexpected_eof}.
  • {chunked, _} mid-body → {error, unexpected_eof}.

parse_request/1

-spec parse_request(binary()) -> parse_result(req()).

Parse an HTTP/1.1 request from binary. Returns {ok, Request, BytesConsumed} on success. Use split_at/2 to get the remaining buffer.

parse_request/2

-spec parse_request(binary(), opts()) -> parse_result(req()).

Parse HTTP/1.1 request with options. Options can include: max_header_size, max_headers_count, max_body_size.

parse_request_body/2

-spec parse_request_body(binary(), body_stream()) ->
                            {ok, [body_chunk()], body_stream(), non_neg_integer()} |
                            {more, pos_integer(), body_stream()} |
                            {error, parse_error()}.

Feed body bytes for a streaming request whose headers were parsed via parse_request_headers/1,2. Returns one of:

  • {ok, Chunks, NewStream, BytesConsumed}: emits zero or more body_chunk() events. A {fin, Trailers} chunk signals the body is fully consumed; subsequent calls on the returned stream are not needed.
  • {more, MinBytes, NewStream}: not enough buffer to make progress. The caller should buffer at least MinBytes more bytes and call again with the same NewStream.
  • {error, parse_error()}: framing error (bad chunk size, body too large, header limits exceeded in trailers).

parse_request_headers/1

-spec parse_request_headers(binary()) ->
                               {ok, req(), body_stream(), non_neg_integer()} |
                               {more, pos_integer()} |
                               {error, parse_error()}.

Warning

This zero-arg variant enforces no size or count limits on the input and is unsafe to use against untrusted peers. Production callers reading from the network MUST use parse_request_headers/2 and pass max_header_size, max_headers_count, and max_body_size (see opts/0). Parse HTTP/1.1 request headers only, without consuming the body. Returns {ok, Request, BodyStream, BytesConsumed}. The returned request map has body => streaming; the body bytes (if any) are read separately via parse_request_body/2.

parse_request_headers/2

-spec parse_request_headers(binary(), opts()) ->
                               {ok, req(), body_stream(), non_neg_integer()} |
                               {more, pos_integer()} |
                               {error, parse_error()}.

Parse HTTP/1.1 request headers only, without consuming the body, enforcing the supplied limits. Returns {ok, Request, BodyStream, BytesConsumed}. The body framing mode is encoded in BodyStream:

  • none: no body (no Content-Length, no Transfer-Encoding, or Content-Length: 0).
  • {length, N}: N body bytes remain to be read.
  • {chunked, _}: chunked transfer encoding; opaque state to feed back into parse_request_body/2. The returned request map carries body => streaming instead of buffered bytes. Use parse_request_body/2 to drive the body stream.

parse_response/1

-spec parse_response(binary()) -> parse_result(resp()).

Parse an HTTP/1.1 response from binary. Returns {ok, Response, BytesConsumed} on success. Use split_at/2 to get the remaining buffer.

parse_response/2

-spec parse_response(binary(), opts()) -> parse_result(resp()).

Parse HTTP/1.1 response with options. Options can include: max_header_size, max_headers_count, max_body_size.

parse_response_body/2

-spec parse_response_body(binary(), body_stream()) ->
                             {ok, [body_chunk()], body_stream(), non_neg_integer()} |
                             {more, pos_integer(), body_stream()} |
                             {error, parse_error()}.

Feed body bytes for a streaming response whose headers were parsed via parse_response_headers/1,2 and whose framing mode was selected via body_stream_from_response/3. For none and {length, 0}, returns {ok, [{fin, []}], none, 0}. For {length, N>0} and {chunked, _}, behaves identically to parse_request_body/2. For until_close, emits [{data, _}] for whatever bytes are in the buffer and keeps the stream open. The caller must signal EOF via finalize_response_body/1 when the underlying transport closes.

parse_response_head/1

-spec parse_response_head(binary()) ->
                             {ok,
                              nhttp_lib:status(),
                              binary(),
                              version(),
                              nhttp_lib:headers(),
                              binary()} |
                             {more, pos_integer()} |
                             {error, parse_error()}.

Equivalent to parse_response_head/2.

parse_response_head/2

-spec parse_response_head(binary(), opts()) ->
                             {ok,
                              nhttp_lib:status(),
                              binary(),
                              version(),
                              nhttp_lib:headers(),
                              binary()} |
                             {more, pos_integer()} |
                             {error, parse_error()}.

Parse HTTP/1.1 response headers only, like parse_response_headers/2, but also return the reason phrase and protocol version from the status line. Returns {ok, Status, Reason, Version, Headers, Rest} where Rest is the binary after the headers. Used by streaming callers that must preserve the full response head (status, reason, version) while reading the body separately via parse_response_body/2.

parse_response_headers/1

-spec parse_response_headers(binary()) ->
                                {ok, nhttp_lib:status(), nhttp_lib:headers(), binary()} |
                                {more, pos_integer()} |
                                {error, parse_error()}.

Warning

This zero-arg variant enforces no size or count limits on the input and is unsafe to use against untrusted peers. A malicious response can exhaust memory by sending arbitrarily many or arbitrarily large header fields. Production callers reading from the network MUST use parse_response_headers/2 and pass max_header_size and max_headers_count (see opts/0). Parse HTTP/1.1 response headers only, without body. Returns {ok, Status, Headers, Rest} where Rest is the binary after headers. Used for streaming responses where the body is read separately.

parse_response_headers/2

-spec parse_response_headers(binary(), opts()) ->
                                {ok, nhttp_lib:status(), nhttp_lib:headers(), binary()} |
                                {more, pos_integer()} |
                                {error, parse_error()}.

Parse HTTP/1.1 response headers only, without body, enforcing the supplied limits (max_header_size, max_headers_count).

split_at/2

-spec split_at(binary(), non_neg_integer()) -> binary().

Split buffer at position, returning the remainder.