Req.Request (req v0.8.0-rc.0)

Copy Markdown View Source

The low-level API and the request struct.

Req is composed of:

The low-level API and the request struct is the foundation of Req's extensibility. Virtually all of the functionality is broken down into individual pieces - steps. See "Steps & Step Wrappers" section for more information.

To make using custom steps by others even easier, they can be packaged up into plugins. See "Writing Plugins" section for more information.

The Request Struct

Public fields are:

  • :method - the HTTP request method.

  • :url - the HTTP request URL.

  • :headers - the HTTP request headers. The header names should be downcased. See also "Headers" section in Req module documentation.

  • :body - the HTTP request body.

    Can be one of:

    • iodata - eagerly send request body

    • enumerable - stream request body

    • req_body_fun - stream request body chunks from a 1-arity function.

      Only supported in Req.stream/4. The function receives the accumulator passed to Req.stream/4.

      It should return one of:

      • {:data, chunk, acc} - emit request body chunk.

      • {:done, chunk, acc} - emit the final request body chunk. acc is passed to the response streaming function.

      • {:done, acc} - request body is done. acc is passed to the response streaming function.

      • {:halt, acc} - cancel request. On HTTP/1, this closes the connection.

      • {:error, exception, acc} - cancel request and return {:error, exception, resp, acc} from Req.stream/4.

  • :into - where to send the response body. It can be one of:

    • nil - (default) read the whole response body and store it in the response.body field.

    • collectable - stream response body into a Collectable.t/0. For example:

      into: File.stream!("path")

      Note that the collectable is only used, if the response status is 200. In other cases, the body is accumulated and processed as usual.

    • :self - stream response body into the current process mailbox. The response body is set to a Req.Response.Async struct that consumes the messages.

    Note: Req.stream/4 does not support :into option.

  • :options - the options to be used by steps. The exact representation of options is private. Calling request.options[key], put_in(request.options[key], value), and update_in(request.options[key], fun) is allowed. get_option/3 and delete_option/2 are also available for additional ways to manipulate the internal representation.

  • :halted - whether the request pipeline is halted. See halt/2.

  • :adapter - the adapter that makes the actual HTTP request. Defaults to Req.Finch.

  • :request_steps - the list of request steps

  • :private - a map reserved for libraries and frameworks to use. The keys must be atoms. Prefix the keys with the name of your project to avoid any future conflicts. The req_ prefix is reserved for Req.

The Low-level API

Most Req users would use it like this:

Req.get!("https://api.github.com/repos/wojtekmach/req").body["description"]
#=> "Req is a batteries-included HTTP client for Elixir."

Here is the equivalent using the low-level API:

req =
  Req.Request.new(
    url: "https://api.github.com/repos/wojtekmach/req"
  )
  |> Req.Request.register_options(
    # ...
    :decoders,
    # ...
  )
  |> Req.Request.prepend_request_steps(
    # ...
    decode: Req.Decode,
    # ...
  )

Req.get!(req).body["description"]
#=> "Req is a batteries-included HTTP client for Elixir."

In other words, Req.new/1, Req.get/1, and friends start with all the built-in steps and options but Req.Request.new/1 starts with a blank slate!

Steps & Step Wrappers

The simplest building blocks are steps, functions that take and return a request. For example:

defmodule MyApp do
  def put_user_agent(%Req.Request{} = req) do
    if user_agent = req.options[:user_agent] do
      Req.Request.put_header(req, "user-agent", user_agent)
    else
      req
    end
  end
end

req =
  Req.Request.new()
  |> Req.Request.register_options([:user_agent])
  |> Req.Request.prepend_request_steps(
    put_user_agent: &MyApp.put_user_agent/1,
  )

resp =
  Req.get!(
    req,
    url: "https://httpbingo.org/user-agent",
    user_agent: "foo"
  )

JSON.decode!(resp.body)      # We used Req.Request.new() so we're not using built-in decoding.
#=> %{"user-agent" => "foo"}

Next up, we have step wrappers, they additionally take acc, fun, state, and next. next is a function representing the rest of the processing pipeline, including hitting the network.

The simplest possible wrapper just calls the next one:

fn req, acc, fun, state, next ->
  next.(req, acc, fun, state)
end

Here's another example:

defmodule MyApp do
  def expect_successful(req, acc, fun, state, next) do
    fun = fn
      {:status, status}, resp, acc, state when status in 200..299 ->
        fun.({:status, status}, resp, acc, state)

      {:status, status}, resp, acc, state ->
        err = RuntimeError.exception("unexpected status #{status}")
        {{:error, err}, resp, acc, state}

      event, resp, acc, state ->
        fun.(event, resp, acc, state)
    end

    next.(req, acc, fun, state)
  end
end

Req.new("https://httpbingo.org/status/404")
|> Req.Request.prepend_request_steps(expect: &MyApp.expect_successful/5)
|> Req.request!()

More complicated wrappers will maintain their state.

The contract for wrappers is:

wrapper :: (req, acc, fun, state, next ->
             {:ok, resp, acc, state}
             | {:halt, resp, acc, state}
             | {{:error, err}, resp, acc, state})

fun is:

fun :: (event, resp, acc, state ->
          {:ok, resp, acc, state}
          | {:halt, resp, acc, state}
          | {{:error, err}, resp, acc, state})
when req: Req.Request.t(),
     resp: Req.Response.t(),
     err: Exception.t(),
     acc: term(),
     state: term(),
     event:
       {:status, non_neg_integer()}
       | {:headers, [{binary(), binary()}]}
       | {:data, term()}
       | {:trailers, [{binary(), binary()}]}

next is:

next :: (req, acc, fun, state ->
           {:ok, resp, acc, state}
           | {:halt, resp, acc, state}
           | {{:error, err}, resp, acc, state})
when req: Req.Request.t(),
     resp: Req.Response.t(),
     err: Exception.t(),
     acc: term(),
     state: term()

Here's another example, a simplistic redirect feature:

defmodule MyApp do
  @redirect_statuses [301, 302, 307, 308]

  def redirect(req, acc, fun, state, next) do
    next.(req, acc, wrap(fun, next), state)
  end

  defp wrap(fun, next) do
    fn
      {:headers, _headers}, resp, acc, state
      when resp.status in @redirect_statuses ->
        [location] = Req.Response.get_header(resp, "location")
        req = resp.request
        req = update_in(req.url, &URI.merge(&1, location))

        with {:ok, resp, acc, state} <- next.(req, acc, wrap(fun, next), state) do
          {:halt, resp, acc, state}
        end

      event, resp, acc, state ->
        fun.(event, resp, acc, state)
    end
  end
end

Req.Request.new(url: "https://httpbingo.org/redirect/3")
|> Req.Request.prepend_request_steps(redirect: &MyApp.redirect/5)
|> Req.request!()

acc and fun are the ones given to Req.stream(req, acc, fun, options \\ []). Most steps usually just pass them through unchanged. However, if you want to process response as it comes in, you'd want to maintain your own state. Here's a simplistic streaming NDJSON decoder:

defmodule MyApp do
  def decode(%Req.Request{} = req, acc, fun, state, next) do
    fun = fn
      {:data, data}, resp, acc, [buffer | state] ->
        {lines, [buffer]} = Enum.split(String.split(buffer <> data, "\n"), -1)
        values = for line <- lines, line != "", do: JSON.decode!(line)
        {tag, resp, acc, state} = fun.({:data, values}, resp, acc, state)
        {tag, resp, acc, [buffer | state]}

      event, resp, acc, [buffer | state] ->
        {tag, resp, acc, state} = fun.(event, resp, acc, state)
        {tag, resp, acc, [buffer | state]}
    end

    {tag, resp, acc, [_buffer | state]} = next.(req, acc, fun, ["" | state])
    {tag, resp, acc, state}
  end
end

req =
  Req.Request.new()
  |> Req.Request.prepend_request_steps(decode: &MyApp.decode/5)

Req.stream(
  req,
  nil,
  fn values, _resp, acc ->
    IO.inspect(Enum.map(values, &Map.take(&1, ["id"])))
    {:cont, acc}
  end,
  url: "https://httpbingo.org/stream/3"
)
# Output: [%{"id" => 0}]
# Output: [%{"id" => 1}]
# Output: [%{"id" => 2}]

Adapters

By default Req uses Finch (via Req.Finch) and supports arbitrary adapters implementing Req.Adapter behaviour. Req adapters are closely related to step wrappers described in previous section.

Plugins

Custom steps can be packaged into plugins so that they are even easier to use by others. By convention, a plugin module exports a def attach(%Req.Request{} = req) function:

Here's an example plugin:

defmodule PrintHeaders do
  @doc """
  Prints request and response headers.

  ## Request Options

    * `:print_headers` - if `true`, prints the headers. Defaults to `false`.

  """
  def attach(%Req.Request{} = req) do
    req
    |> Req.Request.register_options([:print_headers])
    |> Req.Request.append_request_steps(print_headers: __MODULE__)
  end

  def stream(%Req.Request{} = req, acc, fun, state, next) do
    if req.options[:print_headers] do
      for {name, value} <- Req.get_headers_list(req) do
        IO.puts(["> ", name, ": ", value])
      end

      with {:ok, resp, acc, state} <- next.(req, acc, fun, state) do
        for {name, value} <- Req.get_headers_list(resp) do
          IO.puts(["< ", name, ": ", value])
        end

        {:ok, resp, acc, state}
      end
    else
      next.(req, acc, fun, state)
    end
  end
end

And here is how we can use it:

req = Req.new() |> PrintHeaders.attach()

resp = Req.get!(req, url: "https://httpbingo.org/json")
resp.status
#=> 200

resp = Req.get!(req, url: "https://httpbingo.org/json", print_headers: true)
# Output: > accept-encoding: br, gzip
# Output: > user-agent: req/0.3.0-dev
# Output: < date: Wed, 12 Aug 2026 09:07:49 GMT
# Output: < content-type: application/json
# ...
resp.status
#=> 200

As you can see a plugin is simply a module. While this is not enforced, the plugin should follow these conventions:

  • It should export an attach/1 function that takes and returns the request struct

  • The attach functions mostly just adds steps and it is the steps that do the actual work

  • A user should be able to attach your plugin alongside other plugins. For this reason, plugin functionality should usually only happen on a specific "trigger": on a specific option, on a specific URL scheme or host, etc. This is especially important for plugins that perform authentication; you don't want to accidentally expose a token from service A when a user makes request to service B.

  • If your plugin supports custom options, register them with register_options/2

  • Sometimes it is useful to pass options when attaching the plugin. For that, export an attach/2 function and call merge_options/2. Remember to first register options before merging!

Summary

Types

A function for streaming request body chunks.

t()

The request struct.

Functions

Appends steps to the existing steps.

Deletes the header given by name.

Deletes the given option key.

Drops the given keys from options.

Fetches the value for the option key.

Fetches the value for the option key or raises if it's not set.

Returns the values of the header specified by name.

Gets the value for the option key.

Gets the value for the option key.

Gets the value for a specific private key.

Merges given options into the request unless they are already set.

Merges given options into the request.

Returns a new request struct.

Prepends steps to the existing steps.

Sets the header name to value.

Adds (or replaces) multiple request headers.

Adds a request header name unless already present.

Sets the value value for the option name unless option is already set.

Sets the value value for the option name.

Assigns a private key to value.

Registers options to be used by custom steps.

Updates private key with the given function.

Types

acc()

@type acc() :: term()

err()

@type err() :: Exception.t()

req()

@type req() :: t()

req_body_fun(acc)

@type req_body_fun(acc) :: (acc ->
                        {:data, binary(), acc}
                        | {:done, binary(), acc}
                        | {:done, acc}
                        | {:halt, acc}
                        | {:error, Exception.t(), acc})

A function for streaming request body chunks.

Only supported in Req.stream/4; the function receives the accumulator passed to Req.stream/4.

resp()

@type resp() :: Req.Response.t()

state()

@type state() :: term()

t()

@type t() :: %Req.Request{
  adapter: module(),
  body: iodata() | Enumerable.t() | req_body_fun(term()) | nil,
  error_steps: term(),
  halted: term(),
  headers: %{optional(binary()) => [binary()]},
  into: nil | :self | Collectable.t(),
  method: atom(),
  options: options(),
  private: map(),
  registered_options: term(),
  request_steps: [{name :: atom(), request_step()}],
  response_steps: term(),
  url: URI.t()
}

The request struct.

Functions

append_request_steps(request, steps)

@spec append_request_steps(t(), keyword(request_step() | module())) :: t()

Appends steps to the existing steps.

Examples

Req.Request.append_request_steps(
  req,
  noop: fn req -> req,
  inspect: &IO.inspect/1
)

delete_header(request, name)

@spec delete_header(t(), binary()) :: t()

Deletes the header given by name.

All occurrences of the header are deleted, in case the header is repeated multiple times.

See also "Headers" section in Req module documentation.

Examples

iex> Req.Request.get_header(req, "cache-control")
["max-age=600", "no-transform"]
iex> req = Req.Request.delete_header(req, "cache-control")
iex> Req.Request.get_header(req, "cache-control")
[]

delete_option(request, key)

@spec delete_option(t(), atom()) :: t()

Deletes the given option key.

Examples

iex> req = Req.Request.new(options: [a: 1])
iex> Req.Request.get_option(req, :a)
1
iex> req = Req.Request.delete_option(req, :a)
iex> Req.Request.get_option(req, :a)
nil

drop_options(request, keys)

@spec drop_options(t(), [atom()]) :: t()

Drops the given keys from options.

Examples

iex> req = Req.Request.new(options: [a: 1, b: 2, c: 3])
iex> req = Req.Request.drop_options(req, [:a, :b])
iex> Req.Request.get_option(req, :a)
nil
iex> Req.Request.get_option(req, :c)
3

fetch_option(request, key)

@spec fetch_option(t(), atom()) :: {:ok, term()} | :error

Fetches the value for the option key.

See also get_option/3.

Examples

iex> req = Req.Request.new(options: [a: 1])
iex> Req.Request.fetch_option(req, :a)
{:ok, 1}
iex> Req.Request.fetch_option(req, :b)
:error

fetch_option!(request, key)

@spec fetch_option!(t(), atom()) :: term()

Fetches the value for the option key or raises if it's not set.

See also get_option/3.

Examples

iex> req = Req.Request.new(options: [a: 1])
iex> Req.Request.fetch_option!(req, :a)
1
iex> Req.Request.fetch_option!(req, :b)
** (KeyError) option :b is not set

get_header(req, name)

@spec get_header(t(), binary()) :: [binary()]

Returns the values of the header specified by name.

See also "Headers" section in Req module documentation.

Examples

iex> req = Req.new(headers: [{"accept", "application/json"}])
iex> Req.Request.get_header(req, "accept")
["application/json"]
iex> Req.Request.get_header(req, "x-unknown")
[]

get_option(request, key, default \\ nil)

@spec get_option(t(), atom(), term()) :: term()

Gets the value for the option key.

See also fetch_option!/2.

Examples

iex> req = Req.Request.new(options: [a: 1])
iex> Req.Request.get_option(req, :a)
1
iex> Req.Request.get_option(req, :b)
nil
iex> Req.Request.get_option(req, :b, 0)
0

get_option_lazy(request, key, fun)

@spec get_option_lazy(t(), atom(), (-> term())) :: term()

Gets the value for the option key.

This is useful if the default value is very expensive to calculate or generally difficult to setup and teardown again.

See also get_option/3.

Examples

iex> req = Req.Request.new(options: [a: 1])
iex> fun = fn ->
...>   # some expensive operation here
...>   42
...> end
iex> Req.Request.get_option_lazy(req, :a, fun)
1
iex> Req.Request.get_option_lazy(req, :b, fun)
42

get_private(request, key, default \\ nil)

@spec get_private(t(), atom(), default) :: term() | default when default: var

Gets the value for a specific private key.

merge_new_options(request, options)

@spec merge_new_options(t(), keyword()) :: t()

Merges given options into the request unless they are already set.

Examples

iex> req = Req.new(auth: {:basic, "alice:secret"})
iex> req.options
%{auth: {:basic, "alice:secret"}}
iex> req = Req.Request.merge_new_options(req, auth: {:bearer, "abcd"}, base_url: "https://example.com")
iex> req.options
%{auth: {:basic, "alice:secret"}, base_url: "https://example.com"}

iex> req = Req.new()
iex> Req.Request.merge_new_options(req, foo: :bar)
** (ArgumentError) unknown option :foo

merge_options(request, options)

@spec merge_options(t(), keyword()) :: t()

Merges given options into the request.

Examples

iex> req = Req.new(auth: {:basic, "alice:secret"}, retry: false)
iex> req = Req.Request.merge_options(req, auth: {:bearer, "abcd"}, base_url: "https://example.com")
iex> req.options[:auth]
{:bearer, "abcd"}
iex> req.options[:retry]
false
iex> req.options[:base_url]
"https://example.com"

new(options \\ [])

@spec new(keyword()) :: t()

Returns a new request struct.

Options

  • :method - the request method, defaults to :get.

  • :url - the request URL.

  • :headers - the request headers, defaults to [].

  • :body - the request body, defaults to nil.

  • :adapter - the request adapter, defaults to Req.Finch.

Examples

iex> req = Req.Request.new(url: "https://api.github.com/repos/wojtekmach/req")
iex> resp = Req.request!(req)
iex> resp.request.url.host
"api.github.com"
iex> resp.status
200

prepend_request_steps(request, steps)

@spec prepend_request_steps(t(), keyword(request_step() | module())) :: t()

Prepends steps to the existing steps.

Examples

Req.Request.prepend_request_steps(
  req,
  noop: fn req -> req end,
  inspect: &IO.inspect/1
)

put_header(request, name, value)

@spec put_header(t(), binary(), binary()) :: t()

Sets the header name to value.

The value can be a binary or a list of binaries,

If the header was previously set, its value is overwritten.

See also "Headers" section in Req module documentation.

Examples

iex> req = Req.new()
iex> Req.Request.get_header(req, "accept")
[]
iex> req = Req.Request.put_header(req, "accept", "application/json")
iex> Req.Request.get_header(req, "accept")
["application/json"]

put_headers(request, headers)

@spec put_headers(t(), [{binary(), binary()}]) :: t()

Adds (or replaces) multiple request headers.

See put_header/3 for more information.

Examples

iex> req = Req.new()
iex> req = Req.Request.put_headers(req, [{"accept", "text/html"}, {"accept-encoding", "gzip"}])
iex> Req.Request.get_header(req, "accept")
["text/html"]
iex> Req.Request.get_header(req, "accept-encoding")
["gzip"]

put_new_header(request, name, value)

@spec put_new_header(t(), binary(), binary()) :: t()

Adds a request header name unless already present.

See put_header/3 for more information.

Examples

iex> req =
...>   Req.new()
...>   |> Req.Request.put_new_header("accept", "application/json")
...>   |> Req.Request.put_new_header("accept", "application/html")
iex> Req.Request.get_header(req, "accept")
["application/json"]

put_new_option(request, key, value)

@spec put_new_option(t(), atom(), term()) :: t()

Sets the value value for the option name unless option is already set.

See also put_option/3, merge_options/2, and merge_new_options/2.

Examples

iex> req = Req.Request.new() |> Req.Request.register_options([:a])
iex> req.options
%{}
iex> req = Req.Request.put_new_option(req, :a, 1)
iex> req.options
%{a: 1}
iex> req = Req.Request.put_new_option(req, :a, 2)
iex> req.options
%{a: 1}

iex> req = Req.Request.new()
iex> Req.Request.put_new_option(req, :b, 2)
** (ArgumentError) unknown option :b

put_option(request, key, value)

@spec put_option(t(), atom(), term()) :: t()

Sets the value value for the option name.

See also put_new_option/3, merge_options/2, and merge_new_options/2.

Examples

iex> req = Req.Request.new() |> Req.Request.register_options([:a])
iex> req.options
%{}
iex> req = Req.Request.put_option(req, :a, 1)
iex> req.options
%{a: 1}

iex> req = Req.Request.new()
iex> Req.Request.put_option(req, :b, 2)
** (ArgumentError) unknown option :b

put_private(request, key, value)

@spec put_private(t(), atom(), term()) :: t()

Assigns a private key to value.

register_options(request, options)

@spec register_options(t(), [atom()]) :: t()

Registers options to be used by custom steps.

Req ensures that all used options were previously registered which helps finding accidentally mistyped option names. If you're adding custom steps that are accepting options, call this function to register them.

Examples

iex> Req.request!(urll: "https://httpbingo.org")
** (ArgumentError) unknown option :urll. Did you mean :url?

iex> Req.new(bas_url: "https://httpbingo.org")
** (ArgumentError) unknown option :bas_url. Did you mean :base_url?

req =
  Req.new(base_url: "https://httpbingo.org")
  |> Req.Request.register_options([:foo])

Req.get!(req, url: "/status/201", foo: :bar).status
#=> 201

update_private(request, key, default, fun)

@spec update_private(t(), key :: atom(), default :: term(), (term() -> term())) :: t()

Updates private key with the given function.

If key is present in request private map then the existing value is passed to fun and its result is used as the updated value of key. If key is not present, default is inserted as the value of key. The default value will not be passed through the update function.

Examples

iex> req = %Req.Request{private: %{a: 1}}
iex> Req.Request.update_private(req, :a, 11, & &1 + 1).private
%{a: 2}
iex> Req.Request.update_private(req, :b, 11, & &1 + 1).private
%{a: 1, b: 11}