Perplexed

A small Perplexity-style web search, scrape and answer toolkit for Elixir.

Perplexed searches the web through a provider you choose, reads the pages it finds, and turns them into a cited answer on any model ReqLLM supports.

• Swappable search - SearXNG, Firecrawl, Cloudflare Web Search (Ceramic, Linkup, Exa) and DeepSeek built in, or any module implementing Perplexed.Search, alone or as an ordered fallback chain.
• Page reading - fetches pages as Markdown, PDFs included, with hosted scrapers (DeepSeek, Firecrawl, Cloudflare Browser Run) as cheapest-first fallbacks for pages the free tiers cannot read.
• Cited answers - plan, search, rank, read and summarize, returning the sources used and every result considered.
• Cost and telemetry - USD cost on every search, scrape and model call.

Installation

def deps do
  [
    {:perplexed, "~> 0.2.0"}
  ]
end

Start Perplexed in your supervision tree with the keys you use:

children = [
  {Perplexed,
   firecrawl_api_key: System.fetch_env!("FIRECRAWL_API_KEY"),
   provider: :firecrawl,
   model: "anthropic:claude-haiku-4-5"}
]

answer/2 also needs your model provider's key, configured the way ReqLLM expects (for example ANTHROPIC_API_KEY).

Native dependency

The impersonated scraper uses cloaked_req, a Rust NIF. It downloads a precompiled binary, so nothing else is needed, on:

• macOS on Apple Silicon (aarch64-apple-darwin)
• Linux x86_64 and aarch64 with glibc 2.34 or newer

On any other platform (for example Intel macOS or Alpine/musl), build it from source:

1. Add {:rustler, "~> 0.38.0"} to your deps.
2. Install Rust 1.98 or newer, a C and C++ compiler, cmake, libclang and git.
3. Compile with CLOAKED_REQ_BUILD=1 (on musl, also RUSTFLAGS="-C target-feature=-crt-static").

cloaked_req is licensed LGPL-3.0-or-later.

PDFs

Perplexed ships no PDF converter, so PDF pages scrape as :unsupported until you configure one. Implement Perplexed.PdfConverter around the tool of your choice, for example Firecrawl's AnyDoc Rust crate or poppler's pdftotext, and pass it as pdf_converter::

defmodule MyApp.PdfToText do
  @behaviour Perplexed.PdfConverter

  @impl true
  def convert(binary, _filename) do
    path = Path.join(System.tmp_dir!(), "perplexed-#{System.unique_integer([:positive])}.pdf")
    File.write!(path, binary)
    result = System.cmd("pdftotext", ["-layout", path, "-"])
    File.rm(path)
    converted(result)
  end

  defp converted({text, 0}), do: {:ok, text}
  defp converted(_failed), do: {:error, :unsupported}
end

Usage

# Search, optionally reading each result's page
{:ok, result} = Perplexed.search("elixir release notes", scrape: true)
result.hits    #=> [%Perplexed.Search.Hit{url: ..., document: %Perplexed.Document{...}}, ...]
result.cost    #=> 0.00198

# Read one page as Markdown
{:ok, document} = Perplexed.scrape("https://example.com/guide")
document.markdown  #=> "# Guide ..."
document.attempts  #=> [direct: :ok]

# Answer a question with sources
{:ok, answer} = Perplexed.answer("When did the first Ferris wheel open?", max_sources: 5)
answer.answer      #=> "The first Ferris wheel opened in June 1893 [1][2]."
answer.sources     #=> [%Perplexed.Answer.Source{number: 1, url: ..., tier: :direct}, ...]
answer.usage       #=> %{model_calls: 3, searches: 3, total_cost: 0.011, ...}

Pass a Zoi :schema to answer/2 for a validated object instead of text. See the module docs for every option, the provider prices and the telemetry events.

Providers and fallback chains

Search and page fetching are both chains you order yourself, so free and cheap backends go first and paid ones catch what they miss.

Search

:provider takes one provider or a list. A list is tried in order until one answers; an error, a timeout or an empty result moves on to the next.

config :perplexed,
  provider: [:deepseek, :firecrawl],
  deepseek_api_key: System.get_env("DEEPSEEK_API_KEY"),
  firecrawl_api_key: System.get_env("FIRECRAWL_API_KEY")

# or per call
Perplexed.search("zoning setbacks", provider: [:searxng, :deepseek, :firecrawl],
  provider_timeout_ms: 10_000, deadline_ms: 25_000)

Each provider gets provider_timeout_ms (default 15s) inside one deadline_ms (default 30s). fallback_on: [:error] keeps an empty answer instead of falling back. The result's provider is the one that answered and attempts lists every step with its outcome, time and cost.

List prices as of 2026-11-10, from developers.cloudflare.com/web-search, api-docs.deepseek.com and firecrawl.dev/pricing.

Page fetching

The scrape ladder tries each rung until one returns a page worth reading. The default is cheapest first: plain HTTP, then a browser TLS fingerprint (both free), then DeepSeek Fetch and Firecrawl, each only when its key is set (in the settings or the call's credentials:). :browser_run is opt-in, and a ladder you set yourself is used as written.

# the default, with both keys set
[:direct, :impersonated, :deepseek, :firecrawl]

config :perplexed, ladder: [:direct, :impersonated, :firecrawl]
Perplexed.scrape(url, ladder: [:direct, :firecrawl])

A 200 is not success: Perplexed.Verdict sends a bot challenge, a page that says it is behind a sign-in or subscription, a JavaScript shell or a near-empty page on to the next rung. A rung that cannot help with an outcome is skipped (impersonation does nothing for a JavaScript shell), and a missing page ends the climb. When a search provider already returned the page text (DeepSeek does), answer/2 reads that instead of fetching the page again.

PDFs are read by the free rungs when a Perplexed.PdfConverter is configured. Without one, a PDF ends the climb, unless you let Firecrawl read it (firecrawl_pdf: true, at one credit per PDF page).

List prices as of 2026-11-10. Firecrawl bills in credits, and what a credit costs depends on your plan, so set it for accurate cost reports, for example config :perplexed, firecrawl_credit_usd: 0.0038 on Hobby. The default is the Standard plan's $0.00099 ($99 for 100,000 credits).

Each step of a search chain emits [:perplexed, :search, :attempt], and each scrape rung tried emits [:perplexed, :scrape, :attempt], with its outcome, time, cost and whether it was billed, for tracking spend and success per backend. A single search provider (not a list) emits only the [:perplexed, :search, :start | :stop] span.

Where data goes

Search providers see your queries, and answer/2 also sends each the user's question (question:; DeepSeek uses it as ori_question). With a DeepSeek key set, the default ladder sends DeepSeek the URLs the free rungs could not read. Searches through DeepSeek are processed by DeepSeek. Every URL, including SearXNG instances, passes Perplexed.UrlGuard before a request, so nothing reaches a private network; set url_validator to allow a self-hosted instance on one.

Adding a provider

A search provider is one module implementing Perplexed.Search; a page fetcher implements Perplexed.Scraper. Pass either as Module or {Module, opts} wherever a built-in name goes.

defmodule MyApp.Search.Acme do
  @behaviour Perplexed.Search

  @impl true
  def search(query, opts) do
    # call the API, then normalize
    {:ok, [%Perplexed.Search.Hit{url: url, title: title, description: snippet,
                                 provider: :acme, metadata: %{score: score}}]}
  end

  @impl true
  def price(_opts), do: 0.001          # USD per request, for cost accounting

  @impl true
  def free_when_empty?, do: true       # optional
end

defmodule MyApp.Scraper.Acme do
  @behaviour Perplexed.Scraper

  @impl true
  def scrape(url, _opts), do: {:ok, %Perplexed.Document{markdown: markdown}, nil}

  @impl true
  def cost(_document, _opts), do: 0.0005

  @impl true
  def configured?(_opts), do: System.get_env("ACME_KEY") != nil   # optional
end

Return errors as {:error, {reason, details}} from a search provider and {:error, outcome_atom} from a scraper; both chains move past them. A search hit's content (page text) and metadata (scores, authority, site name) are optional: answer/2 reads the first and shows the ranking step what it recognizes in the second.

License

MIT. See LICENSE.