PdfInspector — PDF classification & markdown extraction

# From 0.1.1 the hex package vendors the ffi-core sources, so
# {:pdf_inspector_ex, "~> 0.1"} just works; until the release CI publishes
# precompiled NIF artifacts, PDF_INSPECTOR_EX_BUILD=1 forces a source build
# (a Rust toolchain is required).
System.put_env("PDF_INSPECTOR_EX_BUILD", "1")

# LiveBook.app (macOS GUI) does not inherit your shell PATH — make sure
# cargo (rustup) is reachable for the NIF source build.
cargo_dir = Path.expand("~/.cargo/bin")

unless String.contains?(System.get_env("PATH") || "", cargo_dir) do
  System.put_env("PATH", (System.get_env("PATH") || "") <> ":" <> cargo_dir)
end

System.find_executable("cargo") ||
  raise "cargo (Rust toolchain) not found — install it via https://rustup.rs"

Mix.install([
  {:pdf_inspector_ex, "~> 0.1"}
])

Elixir bindings for firecrawl/pdf-inspector, calling into Rust via dirty-CPU NIFs — the BEAM schedulers never stall. Input is always the PDF contents as a binary; file IO is the caller's job.

Grab a test PDF

Fetch the fixtures straight from the repo (or use File.read! on a local file):

{:ok, _} = Application.ensure_all_started(:inets)

{:ok, {{_, 200, _}, _headers, normal_pdf}} =
  :httpc.request(
    :get,
    {~c"https://raw.githubusercontent.com/Jup33Q/pdf-inspector-bindings/main/fixtures/normal.pdf", []},
    [],
    body_format: :binary
  )

{:ok, {{_, 200, _}, _headers, encrypted_pdf}} =
  :httpc.request(
    :get,
    {~c"https://raw.githubusercontent.com/Jup33Q/pdf-inspector-bindings/main/fixtures/encrypted.pdf", []},
    [],
    body_format: :binary
  )

byte_size(normal_pdf)

classify/1 — the lightest routing entry

Returns the PDF type + pages needing OCR (0-indexed), without producing markdown:

{:ok, classification} = PdfInspector.classify(normal_pdf)

{classification.pdf_type, classification.page_count, classification.pages_needing_ocr}

process/1 — the full pipeline

Classification + full-document markdown extraction:

{:ok, result} = PdfInspector.process(normal_pdf)

result.pdf_type

result.markdown

Note: the process-level page lists in %PdfInspector.Result{} (pages_needing_ocr, pages_with_tables, pages_with_columns) are 1-indexed (upstream convention).

extract_pages/2 — per-page extraction

pages is a list of 0-indexed page numbers; nil extracts every page. Out-of-range pages come back as empty-markdown / needs_ocr: true placeholders (upstream behaviour):

{:ok, pages} = PdfInspector.extract_pages(normal_pdf, [0, 1])

for p <- pages.pages, do: {p.page, byte_size(p.markdown), p.needs_ocr}

{:ok, oob} = PdfInspector.extract_pages(normal_pdf, [99])
hd(oob.pages)

Error handling

Every entry point returns {:error, %PdfInspector.Error{}}, where code is one of six atoms: :io | :parse | :encrypted | :invalid_structure | :not_a_pdf | :internal_panic:

{:error, %PdfInspector.Error{code: :encrypted} = err} = PdfInspector.process(encrypted_pdf)
err

Pipeline DSL — declarative routing

use PdfInspector.Pipeline and declare the routing table with route <pdf_type>, <strategy>; everything is validated at compile time (unknown pdf_type / duplicate route / invalid strategy all raise):

defmodule MyOcr do
  @behaviour PdfInspector.Pipeline.Ocr

  @impl true
  def extract(_binary, pages_0_indexed) do
    # A real implementation runs an OCR engine here; {:error, _} does not
    # discard extraction results — it is recorded in result.ocr_errors.
    {:ok, Map.new(pages_0_indexed, &{&1, "<ocr text for page #{&1}>"})}
  end
end

defmodule MyPipeline do
  use PdfInspector.Pipeline

  route :text_based, :markdown        # process/1 -> full-document markdown
  route :mixed,      :pages           # extract_pages/2 -> per-page markdown
  route :scanned,    {:ocr, MyOcr}    # hand needs_ocr pages to the Ocr impl
  route :image_based, :skip           # classification only

  fallback :classify                  # optional; :classify is the default
end

{:ok, piped} = MyPipeline.run(normal_pdf)

{piped.strategy, byte_size(piped.markdown || ""), piped.ocr_pages}

Classification errors (e.g. an encrypted PDF) propagate without entering any strategy:

MyPipeline.run(encrypted_pdf)