Req (req v0.1.0) View Source
Req is an HTTP client with a focus on ease of use and composability, built on top of Finch.
This is a work in progress!
Features
Extensibility via request, response, and error steps
Automatic body decompression (via
decompress/2
step)Automatic body encoding and decoding (via
encode/1
anddecode/2
steps)Encode params as query string (via
params/2
step)Basic authentication (via
auth/2
step).netrc
file support (vianetrc/2
step)Range requests (via
range/2
step)Follows redirects (via
follow_redirects/2
step)Retries on errors (via
retry/3
step)Basic HTTP caching (via
if_modified_since/2
step)
Usage
The easiest way to use Req is with Mix.install/2
(requires Elixir v1.12+):
Mix.install([
{:req, "~> 0.1.0-dev", github: "wojtekmach/req", branch: "main"}
])
Req.get!("https://api.github.com/repos/elixir-lang/elixir").body["description"]
#=> "Elixir is a dynamic, functional language designed for building scalable and maintainable applications"
If you want to use Req in a Mix project, you can add the above dependency to your mix.exs.
Low-level API
Under the hood, Req works by passing a request through a series of steps.
The request struct, %Req.Request{}
, initially contains data like HTTP method and
request headers. You can also add request, response, and error steps to it.
Request steps are used to refine the data that will be sent to the server.
After making the actual HTTP request, we'll either get a HTTP response or an error. The request, along with the response or error, will go through response or error steps, respectively.
Nothing is actually executed until we run the pipeline with Req.run/1
.
Example:
Req.build(:get, "https://api.github.com/repos/elixir-lang/elixir")
|> Req.prepend_request_steps([
&Req.default_headers/1
])
|> Req.prepend_response_steps([
&Req.decode/2
])
|> Req.run()
#=> {:ok, %{body: %{"description" => "Elixir is a dynamic," <> ...}, ...}, ...}
The high-level API shown before:
Req.get!("https://api.github.com/repos/elixir-lang/elixir")
is equivalent to this composition of lower-level API functions:
Req.build(:get, "https://api.github.com/repos/elixir-lang/elixir")
|> Req.prepend_default_steps()
|> Req.run!()
(See Req.build/3
, Req.prepend_default_steps/2
, and Req.run!/1
for more information.)
We can also build more complex flows like returning a response from a request step or an error from a response step. We will explore those next.
Request steps
A request step is a function that accepts a request
and returns one of the following:
A
request
A
{request, response_or_error}
tuple. In that case no further request steps are executed and the return value goes through response or error steps
Examples:
def default_headers(request) do
update_in(request.headers, &[{"user-agent", "req/0.1.0-dev"} | &1])
end
def read_from_cache(request) do
case ResponseCache.fetch(request) do
{:ok, response} -> {request, response}
:error -> request
end
end
Response and error steps
A response step is a function that accepts a request
and a response
and returns one of the
following:
A
{request, response}
tupleA
{request, exception}
tuple. In that case, no further response steps are executed but the exception goes through error steps
Similarly, an error step is a function that accepts a request
and an exception
and returns one
of the following:
A
{request, exception}
tupleA
{request, response}
tuple. In that case, no further error steps are executed but the response goes through response steps
Examples:
def decode(request, response) do
case List.keyfind(response.headers, "content-type", 0) do
{_, "application/json" <> _} ->
{request, update_in(response.body, &Jason.decode!/1)}
_ ->
{request, response}
end
end
def log_error(request, exception) do
Logger.error(["#{request.method} #{request.url}: ", Exception.message(exception)])
{request, exception}
end
Halting
Any step can call Req.Request.halt/1
to halt the pipeline. This will prevent any further steps
from being invoked.
Examples:
def circuit_breaker(request) do
if CircuitBreaker.open?() do
{Req.Request.halt(request), RuntimeError.exception("circuit breaker is open")}
else
request
end
end
Link to this section Summary
High-level API
Makes a GET request.
Makes a POST request.
Makes an HTTP request.
Makes an HTTP request and returns a response or raises an error.
Low-level API
Appends error steps.
Appends request steps.
Appends response steps.
Builds a request pipeline.
Prepends steps that should be reasonable defaults for most users.
Prepends error steps.
Prepends request steps.
Prepends response steps.
Runs a request pipeline.
Runs a request pipeline and returns a response or raises an error.
Request steps
Sets request authentication.
Adds common request headers.
Encodes the request body based on its shape.
Handles HTTP cache using if-modified-since
header.
Sets request authentication for a matching host from a netrc file.
Normalizes request headers.
Adds params to request query string.
Sets the "Range" request header.
Response steps
Decodes response body based on the detected format.
Decompresses the response body based on the content-encoding
header.
Follows redirects.
Error steps
Retries a request in face of errors.
Link to this section High-level API
Makes a GET request.
See request/3
for a list of supported options.
Makes a POST request.
See request/3
for a list of supported options.
Makes an HTTP request.
Options
:headers
- request headers, defaults to[]
:body
- request body, defaults to""
:finch
- Finch pool to use, defaults toReq.Finch
which is automatically started by the application. SeeFinch
module documentation for more information on starting pools.:finch_options
- Options passed down to Finch when making the request, defaults to[]
. SeeFinch.request/3
for more information.
The options
are passed down to prepend_default_steps/2
, see its documentation for more
information how they are being used.
Makes an HTTP request and returns a response or raises an error.
See request/3
for more information.
Link to this section Low-level API
Appends error steps.
Appends request steps.
Appends response steps.
Builds a request pipeline.
Options
:header
- request headers, defaults to[]
:body
- request body, defaults to""
:finch
- Finch pool to use, defaults toReq.Finch
which is automatically started by the application. SeeFinch
module documentation for more information on starting pools.:finch_options
- Options passed down to Finch when making the request, defaults to[]
. SeeFinch.request/3
for more information.
Prepends steps that should be reasonable defaults for most users.
Request steps
&netrc(&1, options[:netrc])
(ifoptions[:netrc]
is set to an atom true for default path or a string for custom path)&auth(&1, options[:auth])
(ifoptions[:auth]
is set to)¶ms(&1, options[:params])
(ifoptions[:params]
is set)&range(&1, options[:range])
(ifoptions[:range]
is set)
Response steps
&retry(&1, &2, options[:retry])
(ifoptions[:retry]
is set to an atom true or a options keywords list)
Error steps
&retry(&1, &2, options[:retry])
(ifoptions[:retry]
is set and is a keywords list or an atomtrue
)
Options
:netrc
- if set, adds thenetrc/2
step:auth
- if set, adds theauth/2
step:params
- if set, adds theparams/2
step:range
- if set, adds therange/2
step:cache
- if set totrue
, addsif_modified_since/2
step:raw
if set totrue
, skipsdecompress/2
anddecode/2
steps:retry
- if set, adds theretry/3
step to response and error steps
Prepends error steps.
Prepends request steps.
Prepends response steps.
Runs a request pipeline.
Returns {:ok, response}
or {:error, exception}
.
Runs a request pipeline and returns a response or raises an error.
See run/1
for more information.
Link to this section Request steps
Sets request authentication.
auth
can be one of:
{username, password}
- uses Basic HTTP authentication
Examples
iex> Req.get!("https://httpbin.org/basic-auth/foo/bar", auth: {"bad", "bad"}).status
401
iex> Req.get!("https://httpbin.org/basic-auth/foo/bar", auth: {"foo", "bar"}).status
200
Adds common request headers.
Currently the following headers are added:
"user-agent"
-"req/0.1.0"
"accept-encoding"
-"gzip"
Encodes the request body based on its shape.
If body is of the following shape, it's encoded and its content-type
set
accordingly. Otherwise it's unchanged.
Shape | Encoder | Content-Type |
---|---|---|
{:form, data} | URI.encode_query/1 | "application/x-www-form-urlencoded" |
{:json, data} | Jason.encode_to_iodata!/1 | "application/json" |
Examples
iex> Req.post!("https://httpbin.org/post", {:form, comments: "hello!"}).body["form"]
%{"comments" => "hello!"}
Handles HTTP cache using if-modified-since
header.
Only successful (200 OK) responses are cached.
This step also prepends a response step that loads and writes the cache. Be careful when prepending other response steps, make sure the cache is loaded/written as soon as possible.
Options
:dir
- the directory to store the cache, defaults to<user_cache_dir>/req
(see::filename.basedir/3
)
Examples
iex> url = "https://hexdocs.pm/elixir/Kernel.html"
iex> response = Req.get!(url, cache: true)
%{
status: 200,
headers: [
{"date", "Fri, 16 Apr 2021 10:09:56 GMT"},
...
],
...
}
iex> Req.get!(url, cache: true) == response
true
Sets request authentication for a matching host from a netrc file.
Examples
iex> Req.get!("https://httpbin.org/basic-auth/foo/bar").status
401
iex> Req.get!("https://httpbin.org/basic-auth/foo/bar", netrc: true).status
200
iex> Req.get!("https://httpbin.org/basic-auth/foo/bar", netrc: "/path/to/custom_netrc").status
200
Normalizes request headers.
Turns atom header names into strings, e.g.: :user_agent
becomes "user-agent"
. Non-atom names
are returned as is.
Examples
iex> Req.get!("https://httpbin.org/user-agent", headers: [user_agent: "my_agent"]).body
%{"user-agent" => "my_agent"}
Adds params to request query string.
Examples
iex> Req.get!("https://httpbin.org/anything/query", params: [x: "1", y: "2"]).body["args"]
%{"x" => "1", "y" => "2"}
Sets the "Range" request header.
range
can be one of the following:
a string - returned as is
a
first..last
range - converted to"bytes=<first>-<last>"
Examples
iex> Req.get!("https://repo.hex.pm/builds/elixir/builds.txt", range: 0..67)
%{
status: 206,
headers: [
{"content-range", "bytes 0-67/45400"},
...
],
body: "master df65074a8143cebec810dfb91cafa43f19dcdbaf 2021-04-23T15:36:18Z"
}
Link to this section Response steps
Decodes response body based on the detected format.
Supported formats:
Format | Decoder |
---|---|
json | Jason.decode!/1 |
gzip | :zlib.gunzip/1 |
tar | :erl_tar.extract/2 |
zip | :zip.unzip/2 |
csv | NimbleCSV.RFC4180.parse_string/2 (if NimbleCSV is installed) |
Examples
iex> Req.get!("https://hex.pm/api/packages/finch").body["meta"]
%{
"description" => "An HTTP client focused on performance.",
"licenses" => ["MIT"],
"links" => %{"GitHub" => "https://github.com/keathley/finch"},
...
}
Decompresses the response body based on the content-encoding
header.
Examples
iex> response = Req.get!("https://httpbin.org/gzip")
iex> response.headers
[
{"content-encoding", "gzip"},
{"content-type", "application/json"},
...
]
iex> response.body
%{
"gzipped" => true,
...
}
Follows redirects.
Examples
iex> Req.get!("http://api.github.com").status
# 23:24:11.670 [debug] Req.follow_redirects/2: Redirecting to https://api.github.com/
200
Link to this section Error steps
Retries a request in face of errors.
This function can be used as either or both response and error step. It retries a request that resulted in:
a response with status 5xx
an exception
Options
:delay
- sleep this number of milliseconds before making another attempt, defaults to2000
:max_attempts
- maximum number of retry attempts, defaults to2
(for a total of3
requests to the server, including the initial one.)
Examples
With default options:
iex> Req.get!("https://httpbin.org/status/500,200", retry: true).status
# 19:02:08.463 [error] Req.retry/3: Got response with status 500. Will retry in 2000ms, 2 attempts left
# 19:02:10.710 [error] Req.retry/3: Got response with status 500. Will retry in 2000ms, 1 attempt left
200
With custom options:
iex> Req.get!("http://localhost:9999", retry: [delay: 100, max_attempts: 3])
# 17:00:38.371 [error] Req.retry/3: Got exception. Will retry in 100ms, 3 attempts left
# 17:00:38.371 [error] ** (Mint.TransportError) connection refused
# 17:00:38.473 [error] Req.retry/3: Got exception. Will retry in 100ms, 2 attempts left
# 17:00:38.473 [error] ** (Mint.TransportError) connection refused
# 17:00:38.575 [error] Req.retry/3: Got exception. Will retry in 100ms, 1 attempt left
# 17:00:38.575 [error] ** (Mint.TransportError) connection refused
** (Mint.TransportError) connection refused