Fetch.Parser (Fetch v0.1.0)

Pure functions that parse an HTTP/1.1 response head and decide how the body is framed.

A response on the wire looks like this:

HTTP/1.1 200 OK\r\n                  <- status line
Content-Type: text/plain\r\n         <- header lines
Content-Length: 5\r\n
\r\n                                 <- empty line ends the head
hello                                <- body

The head passed to parse_head/1 is everything before the first \r\n\r\n. How many body bytes follow is decided by body_framing/3.

The parser is strict where leniency is dangerous (header syntax, conflicting Content-Length) and tolerant only where it is harmless (a missing reason phrase).

Summary

Functions

body_framing(method, status, headers)

Decides how the response body is delimited (RFC 9112 §6.3).

field_value?(value)

Returns true if value can be a header value: it must not contain CR, LF or NUL (RFC 9110 §5.5). Those bytes are how header injection works.

keep_alive?(arg1, headers)

Returns true if the connection may carry another request after a response with this version and headers (RFC 9112 §9.3).

parse_chunk_size(buffer)

Parses the size line at the start of a chunk (RFC 9112 §7.1)

parse_head(head)

Parses a response head: a status line followed by header lines, separated by CRLF.

parse_header(line)

Parses one header line: field-name ":" OWS field-value OWS (RFC 9112 §5).

parse_headers(lines)

Parses header lines into {lowercase_name, value} tuples, keeping order and duplicates.

parse_status_line(line)

Parses HTTP/1.1 200 OK into %{version: "HTTP/1.1", status: 200, reason: "OK"}.

parse_trailers(buffer)

Parses the trailer section after the last chunk: header lines followed by an empty line. Usually there are no trailers and the section is just CRLF.

split_head(buffer)

Splits a buffer at the end of the head.

token?(name)

Returns true if name is an RFC 9110 token — the allowed syntax for header names: one or more of !#$%&'*+-.^_`|~, digits and ASCII letters.

Types

framing()

@type framing() ::
  :none | {:content_length, non_neg_integer()} | :chunked | :until_close

head()

@type head() :: %{
  version: String.t(),
  status: 100..599,
  reason: String.t(),
  headers: headers()
}

headers()

@type headers() :: [{String.t(), String.t()}]

status_line()

@type status_line() :: %{version: String.t(), status: 100..599, reason: String.t()}

Functions

body_framing(method, status, headers)

@spec body_framing(atom(), 100..599, headers()) ::
  {:ok, framing()} | {:error, {:parse, term()}}

Decides how the response body is delimited (RFC 9112 §6.3).

responses to HEAD, and 1xx/204/304 responses have no body
transfer-encoding wins over content-length; only plain chunked is supported
content-length gives the exact size
otherwise the body lasts until the server closes the connection

field_value?(value)

@spec field_value?(binary()) :: boolean()

Returns true if value can be a header value: it must not contain CR, LF or NUL (RFC 9110 §5.5). Those bytes are how header injection works.

keep_alive?(arg1, headers)

@spec keep_alive?(String.t(), headers()) :: boolean()

Returns true if the connection may carry another request after a response with this version and headers (RFC 9112 §9.3).

HTTP/1.1 connections are persistent unless connection: close is sent. HTTP/1.0 connections close by default; its old connection: keep-alive extension is not supported.

parse_chunk_size(buffer)

@spec parse_chunk_size(binary()) ::
  {:ok, non_neg_integer(), binary()} | :more | {:error, {:parse, term()}}

Parses the size line at the start of a chunk (RFC 9112 §7.1):

chunk-size [ chunk-ext ] CRLF      <- this line
chunk-data CRLF

The size is hexadecimal. Chunk extensions (;name=value) have no meaning for us and are skipped. Returns the size and the bytes after the line. A size of 0 marks the last chunk, followed by the trailer section.

parse_head(head)

@spec parse_head(binary()) :: {:ok, head()} | {:error, {:parse, term()}}

Parses a response head: a status line followed by header lines, separated by CRLF.

parse_header(line)

@spec parse_header(binary()) ::
  {:ok, {String.t(), String.t()}} | {:error, {:parse, term()}}

Parses one header line: field-name ":" OWS field-value OWS (RFC 9112 §5).

the name is case-insensitive, so it is lowercased
whitespace between name and colon is invalid (RFC 9112 §5.1) — it was used for request smuggling
a line starting with space/tab is obsolete line folding (RFC 9112 §5.2), rejected here
optional whitespace (SP / HTAB) around the value is removed

parse_headers(lines)

@spec parse_headers([binary()]) :: {:ok, headers()} | {:error, {:parse, term()}}

Parses header lines into {lowercase_name, value} tuples, keeping order and duplicates.

parse_status_line(line)

@spec parse_status_line(binary()) :: {:ok, status_line()} | {:error, {:parse, term()}}

Parses HTTP/1.1 200 OK into %{version: "HTTP/1.1", status: 200, reason: "OK"}.

RFC 9112 §4: HTTP-version SP 3DIGIT SP [reason-phrase]. The reason phrase carries no meaning, so it may be empty; a missing space before it is tolerated because real servers send HTTP/1.1 200.

parse_trailers(buffer)

@spec parse_trailers(binary()) ::
  {:ok, headers(), binary()} | :more | {:error, {:parse, term()}}

Parses the trailer section after the last chunk: header lines followed by an empty line. Usually there are no trailers and the section is just CRLF.

split_head(buffer)

@spec split_head(binary()) :: {:ok, binary(), binary()} | :more

Splits a buffer at the end of the head.

Returns {:ok, head, rest} where rest is the beginning of the body, or :more when the terminating empty line has not arrived yet.

token?(name)

@spec token?(binary()) :: boolean()

Returns true if name is an RFC 9110 token — the allowed syntax for header names: one or more of !#$%&'*+-.^_`|~, digits and ASCII letters.