OTP-native Chrome DevTools Protocol (CDP) browser automation for Elixir.
CDPEx launches a headless Chrome process and drives it directly over the
Chrome DevTools Protocol on a Mint.WebSocket connection — no ChromeDriver
and no Node.js. Browsers and their WebSocket connections are supervised
processes, so a Chrome crash surfaces to callers as {:error, reason} rather
than a hung session.
This module is the high-level facade. See CDPEx.Page for page operations.
Example
{:ok, browser} = CDPEx.launch()
{:ok, page} = CDPEx.new_page(browser)
{:ok, _page} = CDPEx.Page.navigate(page, "https://example.com")
{:ok, html} = CDPEx.Page.html(page)
:ok = CDPEx.stop(browser)Or, resource-safe, with with_page/3:
CDPEx.with_page([], fn page ->
{:ok, _} = CDPEx.Page.navigate(page, "https://example.com")
CDPEx.Page.html(page)
end)Observability is via :telemetry — see CDPEx.Telemetry for the event taxonomy
(launch / navigate spans, page open/close, with_page/3 deadlines, and error
events). Silent by default.
Error handling
Every operation returns {:error, reason} on failure; error_reason/0 documents
the reason shapes. That includes runtime failures — Chrome that can't be spawned, a
browser, page or pool process that is gone or stops mid-call, a call it can't serve
in time — which come back as values, never as a raise or an exit in the caller.
The guarantee covers return values, not links: a browser from launch/1 or
connect/2 stays linked to the process that started it, so if Chrome dies, a starter
that doesn't trap exits still receives the exit signal (run it under a supervisor;
with_page/3 with launch options contains this for its throwaway browser).
Invalid arguments (a bad :deadline or :wait, a wrong type) still raise; that is a
bug in the calling code. So does anything with_page/3's own fun raises, throws or
exits with. To drive retries without hard-coding that list, classify the
reason instead of matching it:
case CDPEx.Page.navigate(page, url) do
{:ok, page} ->
{:ok, page}
{:error, reason} ->
if CDPEx.transient?(reason), do: retry(), else: {:error, reason}
endclassify_error/1 buckets a reason as :transient (a fresh attempt may succeed),
:terminal (it won't), or :unknown (payload-dependent — you decide). It tracks
the error surface as the library evolves, so the transient/terminal decision stays
in one place rather than being reimplemented (and re-drifting) downstream. Retries
stay yours to bound: cap attempts, back off, and on :transient re-establish the
resource (a fresh page/browser) rather than reusing a dead handle.
Status
Pages default to one WebSocket each (strong crash isolation); opt into
sessionId multiplexing (many pages over the one browser socket) with
new_page(browser, transport: :session), trading isolation for fewer sockets.
Stealth / anti-fingerprinting presets remain out of scope.
Summary
Types
The result of classify_error/1.
The reason shapes that appear in {:error, reason} across CDPEx.
Functions
Classifies an error reason as :transient, :terminal, or :unknown.
Closes a page opened with new_page/2.
Connects to an already-running Chrome instead of launching one.
Launches a headless Chrome browser and returns {:ok, pid}, or {:error, reason}
(e.g. {:chrome_not_found, path}, {:launch_failed, _}, {:debug_url_not_found, _}).
Opens a new page. See CDPEx.Browser.new_page/2 for options.
Convenience over classify_error/1: true only when the error is :transient.
Runs fun with a fresh page, guaranteeing the page (and, when given launch
options, the browser) is cleaned up afterwards — even if fun raises.
Types
@type error_classification() :: :transient | :terminal | :unknown
The result of classify_error/1.
Intentionally open: match :transient (or :terminal) explicitly and fall through
with a catch-all rather than enumerating all three atoms, so a future bucket can be
added without breaking exhaustive matches.
@type error_reason() :: CDPEx.Connection.call_error() | CDPEx.Chrome.launch_error() | {:ws_connect, term()} | {:ws_upgrade, term()} | :timeout | :deadline_exceeded | :unknown_page | :already_authenticated | :already_intercepting | {:timeout, :await_event | :new_page | :close_page | :authenticate | :subscribe} | {:conflict, :authenticated | :intercepting} | {:navigate, String.t()} | {:no_document_response, String.t()} | {:connect_discovery_failed, term()} | {:capture_failed, term()} | {:idle_wait_failed, term()} | {:selector_not_found, String.t()} | {:text_not_found, String.t()} | {:not_clickable, String.t()} | {:unknown_key, String.t()} | {:evaluate_exception, term()} | {:unserializable_value, String.t()} | {:unexpected_evaluate, term()} | {:invalid_args, term()} | {:invalid_source, term()} | {:invalid_error_reason, term()} | {:invalid_transport, term()} | {:invalid_proxy, term()} | {:unsupported_transport, term()} | {:unsupported_with_connect, term()} | {:invalid_response_body, String.t()} | {:invalid_pdf_data, String.t()} | {:invalid_screenshot_data, String.t()} | {:write_failed, term()}
The reason shapes that appear in {:error, reason} across CDPEx.
Error reasons are part of the public contract — pattern-match the tagged kinds
({:cdp_error, …}, {:timeout, …}, {:ws_closed, …}, …); their payloads (a CDP
method, an exit status, a stderr/contents excerpt) are open and may gain detail.
The only bare, context-free reasons are :noproc, the high-level :timeout,
:deadline_exceeded (with_page/3's :deadline), :unknown_page,
:already_authenticated, and :already_intercepting —
self-describing control-flow outcomes with no payload to carry, the way GenServer
uses :noproc. Validation failures that do have offending data to surface are
tagged instead ({:invalid_response_body, excerpt}, {:invalid_pdf_data, excerpt},
{:invalid_screenshot_data, excerpt}).
To act on a failure without hard-coding this list, use classify_error/1 — it
buckets any reason as :transient / :terminal / :unknown and tracks this union,
so retry logic isn't reimplemented (and re-drifted) downstream.
Two sub-unions are machine-checked: CDPEx.Connection.call_error/0 and
CDPEx.Chrome.launch_error/0 are precisely specced on call/5 / launch/1, so
Dialyzer catches a shape change in those at the source. The remaining members —
the page-level tagged kinds and bare atoms — are hand-maintained (kinds such as
{:cdp_error, method, payload} also wrap arbitrary CDP data), kept honest by a
compile-time coverage test that fails if any member here lacks a classify_error/1
test exemplar — so a member can't be added to this type without being classified.
That guard is one-directional (type → classified): the reverse — an error a producer
returns but never adds here — still relies on review, and such a stray reason would
fall through classify_error/1 to :unknown.
Two timeout shapes, by layer: the low-level CDPEx.Connection.call/5,
await_event/4 and subscribe/4 and the CDPEx.Browser requests return
{:timeout, context} (a CDP method, :await_event, :subscribe, :new_page,
:close_page, or :authenticate), while the high-level CDPEx.Page wait_for_* functions and CDPEx.Pool.checkout/2
return a bare :timeout ("the awaited condition didn't happen in time").
A WebSocket frame that fails to decode is not a standalone reason: the connection
stops on the decode failure, so callers observe it nested, as
{:ws_closed, {:ws_decode, _}}.
Functions
@spec classify_error(term()) :: error_classification()
Classifies an error reason as :transient, :terminal, or :unknown.
reason is the value from any {:error, reason} this library returns (see
error_reason/0). The classification answers one question — might a fresh
attempt succeed? — so you drive retries from one place instead of reimplementing
the decision (and re-drifting it) in every caller:
case CDPEx.Page.navigate(page, url) do
{:ok, page} ->
{:ok, page}
{:error, reason} ->
case CDPEx.classify_error(reason) do
:transient -> retry_with_fresh_page()
_ -> {:error, reason}
end
endThe buckets:
:transient— environmental or timing failures: the connection dropped or couldn't be established ({:ws_closed, _},{:ws_connect, _},{:ws_upgrade, _},:noproc), a:sessionpage's target closed or detached ({:session_closed, _}, the session-transport counterpart of a dedicated page's:noproc), a wait or call timed out (:timeout,{:timeout, _},:deadline_exceeded), Chrome died or was slow to start ({:chrome_exited, _, _},{:debug_url_not_found, _},{:devtools_file_malformed, _}), an internal capture/idle helper crashed ({:capture_failed, _},{:idle_wait_failed, _}), or a navigation hit a connection/network-layernet::ERR_*(e.g.{:navigate, "net::ERR_CONNECTION_REFUSED"}).:terminal— deterministic outcomes: a selector or text didn't match, JS threw, a usage/validation error, or a missing Chrome binary. Retrying the same call yields the same error. (:already_authenticated/:already_interceptingare terminal for the ordinary double-call; the narrow post-timeout teardown raceauthenticate/4documents — where a retry can still succeed — is signalled by the preceding{:timeout, _}, which is itself:transient.):unknown— the outcome depends on a payload or timing this function does not crack: an ambiguous navigationnet::ERR_*(DNSERR_NAME_NOT_RESOLVED,ERR_ABORTED,ERR_BLOCKED_BY_*— unlike the connection-layer codes above), the CDP error code ({:cdp_error, _, _}), the file-write posix reason ({:write_failed, _}), why Chrome couldn't be started at all ({:launch_failed, _}— an unwritable profile dir or a refused spawn::emfilemay clear,:eacceswon't), whether a{:no_document_response, _}was a same-document hop or a slow miss, or aconnect/2endpoint-discovery failure ({:connect_discovery_failed, _}— a transient network blip and a permanently bad endpoint are indistinguishable here). Also covers any termCDPExdoesn't produce. Decide the retry policy yourself.
Retries are the caller's responsibility: bound the attempts and back off. A
:transient result means re-establish the resource — open a fresh page/browser
or call CDPEx.Pool.checkout/2 again — not retry the same handle (a dead page keeps
returning :noproc, or {:session_closed, _} for a :session page). Some
:transient reasons are still a judgment call for your environment — retrying a :timeout / net::ERR_TIMED_OUT multiplies wall-time and
browser memory, so a resource-constrained caller may reasonably treat timeouts as
terminal. The input is typed term() so the catch-all stays reachable;
routing through this instead of matching error_reason/0 directly trades Dialyzer
exhaustiveness for a stable, library-maintained dispatch point.
@spec close_page(pid(), CDPEx.Page.t()) :: :ok | {:error, :unknown_page | :noproc | {:timeout, :close_page}}
Closes a page opened with new_page/2.
Returns {:error, :unknown_page} if page was not opened on browser, and
{:error, :noproc} if browser is gone (see CDPEx.Browser.close_page/2).
@spec connect(String.t(), keyword()) :: GenServer.on_start()
Connects to an already-running Chrome instead of launching one.
endpoint is either a ws:///wss:// browser WebSocket URL (used directly) or
an http:///https:// base URL (the browser socket is discovered via
GET /json/version, with the host/port taken from endpoint). Returns the same
handle as launch/1, so new_page/2, with_page/3, close_page/2, and
stop/1 all work — but stop/1 only closes the pages cdp_ex opened and drops
the socket; it never kills the remote Chrome or touches pre-existing tabs.
http(s) discovery is IP/localhost only
Chrome's DevTools HTTP endpoint rejects /json/version with 403 when the
Host header is a non-IP, non-localhost name (DNS-rebinding protection), so
the http(s):// discovery form works only for IP / localhost endpoints (a 403
surfaces as {:error, {:connect_discovery_failed, {:http_status, 403}}}). For a
named remote/sidecar/cloud host, pass the ws(s):// browser URL directly
(or launch that Chrome with a permissive --remote-allow-origins/host).
Pages default to :session transport; transport: :dedicated returns
{:error, {:unsupported_transport, :dedicated}} (not yet supported over a
connected browser).
Options: :insecure (skip wss:// cert verification, default false),
:cacertfile / :cacerts (custom CA for wss://), :discovery_timeout (ceiling
in ms for the whole http(s):// /json/version exchange — TCP connect, recv, and
body — default 5_000; raise it for a slow remote endpoint), :name (register the
process).
@spec launch(keyword()) :: GenServer.on_start()
Launches a headless Chrome browser and returns {:ok, pid}, or {:error, reason}
(e.g. {:chrome_not_found, path}, {:launch_failed, _}, {:debug_url_not_found, _}).
The browser is linked to the caller, but a failed launch returns its error without exiting the caller, whether or not it traps exits.
Accepts the launch options documented in CDPEx.Chrome (e.g. :headless,
:chrome_binary, :extra_args, :window_size, :launch_timeout). On slow
cold-start hosts (e.g. headless Chrome in a constrained container) raise
:launch_timeout — it is a ceiling, not a fixed wait. For long-lived use, prefer
putting CDPEx.Browser under your own supervisor with a :shutdown timeout.
Proxy
Pass :proxy to route the browser through a proxy — a URL or a keyword list:
CDPEx.launch(proxy: "http://user:pass@host:8080")
CDPEx.launch(proxy: [server: "host:8080", username: "u", password: "p"])It sets Chrome's --proxy-server and, when credentials are given, automatically
answers the proxy auth challenge on each page — so you just new_page/2 and
CDPEx.Page.navigate/3, no manual CDPEx.Page.authenticate/4. See CDPEx.Proxy for
the accepted forms (the keyword form avoids percent-encoding special-character
passwords).
A credentialed proxy requires the default :dedicated transport: new_page(transport: :session) on such a browser returns {:error, {:unsupported_transport, :session}},
and an auto-armed page can't also use enable_request_interception/2 (both drive the
Fetch domain). A malformed :proxy — or combining it with a full :args override —
fails the launch with {:error, {:invalid_proxy, _}} (:proxy appends to :extra_args,
which an :args override discards, so the two are mutually exclusive; use one). Don't
set --proxy-server in :extra_args yourself when using :proxy.
@spec new_page(pid(), keyword()) :: {:ok, CDPEx.Page.t()} | {:error, term()}
Opens a new page. See CDPEx.Browser.new_page/2 for options.
@spec stop(pid()) :: :ok
Stops a browser started with launch/1 (kills Chrome) or connect/2 (closes the
pages it opened, leaves Chrome running). :ok if the browser is already gone.
Convenience over classify_error/1: true only when the error is :transient.
Conservative by design — :unknown is not transient, so an unrecognized or
payload-dependent error won't be auto-retried. Match classify_error/1 directly
when you want to treat :unknown specially, and see its note on bounded,
resource-re-establishing retries — this classifies, it does not retry.
@spec with_page(pid() | keyword(), (CDPEx.Page.t() -> result), keyword()) :: result | {:error, term()} when result: var
Runs fun with a fresh page, guaranteeing the page (and, when given launch
options, the browser) is cleaned up afterwards — even if fun raises.
Pass an existing browser pid to reuse it, or a keyword list of launch options
to spin up a throwaway browser for the duration of the call. A [connect: endpoint] list attaches to an already-running Chrome instead of launching one
(see connect/2); teardown then closes only the pages it opened and never reaps
that Chrome. Returns whatever fun returns, or {:error, reason} if the
page/browser could not be created.
Failures in with_page's own work (the launch or connect, opening the page, a
browser that is gone or dies under it, the teardown) come back as
{:error, reason}, never as a raise or an exit. A raise, throw or exit from fun
itself propagates unchanged: that is the caller's code.
With launch options, the throwaway browser is linked but contained: if it
fails to start, or crashes during the call (e.g. its connection drops),
with_page returns {:error, reason} instead of letting the exit propagate to
the caller. To do that it briefly traps exits in the calling process for the
duration of the call, launch included.
Only the browser's own {:EXIT, _, _} is drained — a foreign process linked
to the caller that exits during this window has its exit delivered as a message
left in the caller's mailbox, so a caller that links other processes and relies
on un-trapped exit propagation should pass a pre-launched browser pid instead.
On slow cold-start hosts, raise :launch_timeout (a ceiling, not a fixed wait).
opts go to new_page/2, except :deadline.
Deadline
Each page operation has its own timeout, but they stack: a navigation, a few
selector waits and some retries can hold a browser for minutes. Pass deadline: ms
(a positive integer) to bound the whole call. The clock starts when with_page is
called, so a launch counts against it.
The rule, by the monotonic clock: the call is exceeded iff fun had not returned
by the deadline (or never started). Then with_page returns
{:error, :deadline_exceeded} however fun came back — a value, an error tuple, or
a raise/exit caused by the teardown. A new_page/2 error comes back as itself,
late or not (the teardown waits for new_page/2 to return); a page that only
arrives after the deadline means fun is not started. If fun returned in time,
its result or raise comes back untouched: the deadline stops there, and closing the
page or stopping a throwaway browser afterwards runs on their own timeouts. Without
a deadline nothing changes.
fun still runs in the calling process (self(), the process dictionary, Logger
metadata and links behave as without a deadline). A watchdog process enforces the
deadline by tearing down the browser resource underneath it: a throwaway browser
(launch options or connect:) is stopped, reaping a launched Chrome; with a browser
pid, only the page with_page opened is closed and the browser stays yours (a
page fun opens itself is not released). The browser call fun is blocked in then
fails fast and fun unwinds. Work in fun that doesn't touch the browser — a
sleep, a computation — runs on until its next browser call, so the bound is "the
browser is released at the deadline", not "fun is preempted".
Some stages can't be interrupted and are bounded differently:
- Launch — the browser doesn't exist yet, so
:launch_timeoutis capped at the deadline when the deadline is smaller or equal (default15_000), and so is connect-mode's:discovery_timeout(default5_000, and anilor:infinityis capped too). A non-integer:launch_timeoutis not capped: it raisesArgumentErrorlike any invalid launch option. The WebSocket connect and upgrade after them (normally milliseconds) keep their own timeouts. A start whose capped timeout runs out at or after the deadline returns{:error, :deadline_exceeded}, with its own error in the telemetry event's:launch_error; any other start failure comes back as itself. new_page/2— the browser serves it as one call (target creation, bootstrap, a proxy's auth arm), so a deadline that fires mid-way takes effect when it returns;funis then not started. A start that returns after the deadline skipsnew_page/2altogether.- Closing the page on a browser pid goes through the Browser, so on a shared or
pooled browser it queues behind whatever the Browser is serving (another
caller's
new_page/2). Ifclose_page/2's own 15s timeout runs out first, the page stays open and the call infunruns to its own timeout.
With a throwaway browser the teardown's {:EXIT, browser, :normal} arrives while
fun may still be running (the call traps exits, as for a browser crash); it is
drained before with_page returns, but a catch-all receive in fun can take it.
If the caller dies mid-call, a throwaway browser is reaped through its link as
before, and on a browser pid the watchdog closes the page — once new_page/2 has
returned it; a caller that dies while new_page/2 is in flight leaves that page
open, as without a deadline.
An invalid :deadline (not an integer in 1..4_294_967_295) raises
ArgumentError. An exceeded deadline emits
[:cdp_ex, :with_page, :deadline_exceeded] (see CDPEx.Telemetry).
# against an existing browser
CDPEx.with_page(browser, fn page ->
{:ok, _} = CDPEx.Page.navigate(page, "https://example.com")
CDPEx.Page.html(page)
end)
# throwaway browser + page
CDPEx.with_page([headless: true], &CDPEx.Page.html/1)
# against an existing Chrome (discovered via /json/version)
CDPEx.with_page([connect: "http://localhost:9222"], &CDPEx.Page.html/1)
# the whole call, launch included, within 20s
CDPEx.with_page([], &CDPEx.Page.html/1, deadline: 20_000)