Req (req v0.1.0) View Source

Req is an HTTP client with a focus on ease of use and composability, built on top of Finch.

This is a work in progress!

Features

Usage

The easiest way to use Req is with Mix.install/2 (requires Elixir v1.12+):

Mix.install([
  {:req, "~> 0.1.0-dev", github: "wojtekmach/req", branch: "main"}
])

Req.get!("https://api.github.com/repos/elixir-lang/elixir").body["description"]
#=> "Elixir is a dynamic, functional language designed for building scalable and maintainable applications"

If you want to use Req in a Mix project, you can add the above dependency to your mix.exs.

Low-level API

Under the hood, Req works by passing a request through a series of steps.

The request struct, %Req.Request{}, initially contains data like HTTP method and request headers. You can also add request, response, and error steps to it.

Request steps are used to refine the data that will be sent to the server.

After making the actual HTTP request, we'll either get a HTTP response or an error. The request, along with the response or error, will go through response or error steps, respectively.

Nothing is actually executed until we run the pipeline with Req.run/1.

Example:

Req.build(:get, "https://api.github.com/repos/elixir-lang/elixir")
|> Req.prepend_request_steps([
  &Req.default_headers/1
])
|> Req.prepend_response_steps([
  &Req.decode/2
])
|> Req.run()
#=> {:ok, %{body: %{"description" => "Elixir is a dynamic," <> ...}, ...}, ...}

The high-level API shown before:

Req.get!("https://api.github.com/repos/elixir-lang/elixir")

is equivalent to this composition of lower-level API functions:

Req.build(:get, "https://api.github.com/repos/elixir-lang/elixir")
|> Req.prepend_default_steps()
|> Req.run!()

(See Req.build/3, Req.prepend_default_steps/2, and Req.run!/1 for more information.)

We can also build more complex flows like returning a response from a request step or an error from a response step. We will explore those next.

Request steps

A request step is a function that accepts a request and returns one of the following:

  • A request

  • A {request, response_or_error} tuple. In that case no further request steps are executed and the return value goes through response or error steps

Examples:

def default_headers(request) do
  update_in(request.headers, &[{"user-agent", "req/0.1.0-dev"} | &1])
end

def read_from_cache(request) do
  case ResponseCache.fetch(request) do
    {:ok, response} -> {request, response}
    :error -> request
  end
end

Response and error steps

A response step is a function that accepts a request and a response and returns one of the following:

  • A {request, response} tuple

  • A {request, exception} tuple. In that case, no further response steps are executed but the exception goes through error steps

Similarly, an error step is a function that accepts a request and an exception and returns one of the following:

  • A {request, exception} tuple

  • A {request, response} tuple. In that case, no further error steps are executed but the response goes through response steps

Examples:

def decode(request, response) do
  case List.keyfind(response.headers, "content-type", 0) do
    {_, "application/json" <> _} ->
      {request, update_in(response.body, &Jason.decode!/1)}

    _ ->
      {request, response}
  end
end

def log_error(request, exception) do
  Logger.error(["#{request.method} #{request.url}: ", Exception.message(exception)])
  {request, exception}
end

Halting

Any step can call Req.Request.halt/1 to halt the pipeline. This will prevent any further steps from being invoked.

Examples:

def circuit_breaker(request) do
  if CircuitBreaker.open?() do
    {Req.Request.halt(request), RuntimeError.exception("circuit breaker is open")}
  else
    request
  end
end

Link to this section Summary

High-level API

Makes a GET request.

Makes a POST request.

Makes an HTTP request.

Makes an HTTP request and returns a response or raises an error.

Low-level API

Appends error steps.

Appends request steps.

Appends response steps.

Builds a request pipeline.

Prepends steps that should be reasonable defaults for most users.

Prepends error steps.

Prepends request steps.

Prepends response steps.

Runs a request pipeline.

Runs a request pipeline and returns a response or raises an error.

Request steps

Sets request authentication.

Adds common request headers.

Encodes the request body based on its shape.

Handles HTTP cache using if-modified-since header.

Sets request authentication for a matching host from a netrc file.

Normalizes request headers.

Adds params to request query string.

Sets the "Range" request header.

Response steps

Decodes response body based on the detected format.

Decompresses the response body based on the content-encoding header.

Link to this section High-level API

Link to this function

get!(uri, options \\ [])

View Source

Makes a GET request.

See request/3 for a list of supported options.

Link to this function

post!(uri, body, options \\ [])

View Source

Makes a POST request.

See request/3 for a list of supported options.

Link to this function

request(method, uri, options \\ [])

View Source

Makes an HTTP request.

Options

  • :headers - request headers, defaults to []

  • :body - request body, defaults to ""

  • :finch - Finch pool to use, defaults to Req.Finch which is automatically started by the application. See Finch module documentation for more information on starting pools.

  • :finch_options - Options passed down to Finch when making the request, defaults to []. See Finch.request/3 for more information.

The options are passed down to prepend_default_steps/2, see its documentation for more information how they are being used.

Link to this function

request!(method, uri, options \\ [])

View Source

Makes an HTTP request and returns a response or raises an error.

See request/3 for more information.

Link to this section Low-level API

Link to this function

append_error_steps(request, steps)

View Source

Appends error steps.

Link to this function

append_request_steps(request, steps)

View Source

Appends request steps.

Link to this function

append_response_steps(request, steps)

View Source

Appends response steps.

Link to this function

build(method, uri, options \\ [])

View Source

Builds a request pipeline.

Options

  • :header - request headers, defaults to []

  • :body - request body, defaults to ""

  • :finch - Finch pool to use, defaults to Req.Finch which is automatically started by the application. See Finch module documentation for more information on starting pools.

  • :finch_options - Options passed down to Finch when making the request, defaults to []. See Finch.request/3 for more information.

Link to this function

prepend_default_steps(request, options \\ [])

View Source

Prepends steps that should be reasonable defaults for most users.

Request steps

Response steps

Error steps

Options

Link to this function

prepend_error_steps(request, steps)

View Source

Prepends error steps.

Link to this function

prepend_request_steps(request, steps)

View Source

Prepends request steps.

Link to this function

prepend_response_steps(request, steps)

View Source

Prepends response steps.

Runs a request pipeline.

Returns {:ok, response} or {:error, exception}.

Runs a request pipeline and returns a response or raises an error.

See run/1 for more information.

Link to this section Request steps

Sets request authentication.

auth can be one of:

  • {username, password} - uses Basic HTTP authentication

Examples

iex> Req.get!("https://httpbin.org/basic-auth/foo/bar", auth: {"bad", "bad"}).status
401
iex> Req.get!("https://httpbin.org/basic-auth/foo/bar", auth: {"foo", "bar"}).status
200
Link to this function

default_headers(request)

View Source

Adds common request headers.

Currently the following headers are added:

  • "user-agent" - "req/0.1.0"

  • "accept-encoding" - "gzip"

Encodes the request body based on its shape.

If body is of the following shape, it's encoded and its content-type set accordingly. Otherwise it's unchanged.

ShapeEncoderContent-Type
{:form, data}URI.encode_query/1"application/x-www-form-urlencoded"
{:json, data}Jason.encode_to_iodata!/1"application/json"

Examples

iex> Req.post!("https://httpbin.org/post", {:form, comments: "hello!"}).body["form"]
%{"comments" => "hello!"}
Link to this function

if_modified_since(request, options \\ [])

View Source

Handles HTTP cache using if-modified-since header.

Only successful (200 OK) responses are cached.

This step also prepends a response step that loads and writes the cache. Be careful when prepending other response steps, make sure the cache is loaded/written as soon as possible.

Options

  • :dir - the directory to store the cache, defaults to <user_cache_dir>/req (see: :filename.basedir/3)

Examples

iex> url = "https://hexdocs.pm/elixir/Kernel.html"
iex> response = Req.get!(url, cache: true)
%{
  status: 200,
  headers: [
    {"date", "Fri, 16 Apr 2021 10:09:56 GMT"},
    ...
  ],
  ...
}
iex> Req.get!(url, cache: true) == response
true

Sets request authentication for a matching host from a netrc file.

Examples

iex> Req.get!("https://httpbin.org/basic-auth/foo/bar").status
401
iex> Req.get!("https://httpbin.org/basic-auth/foo/bar", netrc: true).status
200
iex> Req.get!("https://httpbin.org/basic-auth/foo/bar", netrc: "/path/to/custom_netrc").status
200
Link to this function

normalize_headers(request)

View Source

Normalizes request headers.

Turns atom header names into strings, e.g.: :user_agent becomes "user-agent". Non-atom names are returned as is.

Examples

iex> Req.get!("https://httpbin.org/user-agent", headers: [user_agent: "my_agent"]).body
%{"user-agent" => "my_agent"}

Adds params to request query string.

Examples

iex> Req.get!("https://httpbin.org/anything/query", params: [x: "1", y: "2"]).body["args"]
%{"x" => "1", "y" => "2"}

Sets the "Range" request header.

range can be one of the following:

  • a string - returned as is

  • a first..last range - converted to "bytes=<first>-<last>"

Examples

iex> Req.get!("https://repo.hex.pm/builds/elixir/builds.txt", range: 0..67)
%{
  status: 206,
  headers: [
    {"content-range", "bytes 0-67/45400"},
    ...
  ],
  body: "master df65074a8143cebec810dfb91cafa43f19dcdbaf 2021-04-23T15:36:18Z"
}

Link to this section Response steps

Link to this function

decode(request, response)

View Source

Decodes response body based on the detected format.

Supported formats:

FormatDecoder
jsonJason.decode!/1
gzip:zlib.gunzip/1
tar:erl_tar.extract/2
zip:zip.unzip/2
csvNimbleCSV.RFC4180.parse_string/2 (if NimbleCSV is installed)

Examples

iex> Req.get!("https://hex.pm/api/packages/finch").body["meta"]
%{
  "description" => "An HTTP client focused on performance.",
  "licenses" => ["MIT"],
  "links" => %{"GitHub" => "https://github.com/keathley/finch"},
  ...
}
Link to this function

decompress(request, response)

View Source

Decompresses the response body based on the content-encoding header.

Examples

iex> response = Req.get!("https://httpbin.org/gzip")
iex> response.headers
[
  {"content-encoding", "gzip"},
  {"content-type", "application/json"},
  ...
]
iex> response.body
%{
  "gzipped" => true,
  ...
}
Link to this function

follow_redirects(request, response)

View Source

Follows redirects.

Examples

iex> Req.get!("http://api.github.com").status
# 23:24:11.670 [debug]  Req.follow_redirects/2: Redirecting to https://api.github.com/
200

Link to this section Error steps

Link to this function

retry(request, response_or_exception, options \\ [])

View Source

Retries a request in face of errors.

This function can be used as either or both response and error step. It retries a request that resulted in:

  • a response with status 5xx

  • an exception

Options

  • :delay - sleep this number of milliseconds before making another attempt, defaults to 2000

  • :max_attempts - maximum number of retry attempts, defaults to 2 (for a total of 3 requests to the server, including the initial one.)

Examples

With default options:

iex> Req.get!("https://httpbin.org/status/500,200", retry: true).status
# 19:02:08.463 [error] Req.retry/3: Got response with status 500. Will retry in 2000ms, 2 attempts left
# 19:02:10.710 [error] Req.retry/3: Got response with status 500. Will retry in 2000ms, 1 attempt left
200

With custom options:

iex> Req.get!("http://localhost:9999", retry: [delay: 100, max_attempts: 3])
# 17:00:38.371 [error] Req.retry/3: Got exception. Will retry in 100ms, 3 attempts left
# 17:00:38.371 [error] ** (Mint.TransportError) connection refused
# 17:00:38.473 [error] Req.retry/3: Got exception. Will retry in 100ms, 2 attempts left
# 17:00:38.473 [error] ** (Mint.TransportError) connection refused
# 17:00:38.575 [error] Req.retry/3: Got exception. Will retry in 100ms, 1 attempt left
# 17:00:38.575 [error] ** (Mint.TransportError) connection refused
** (Mint.TransportError) connection refused