Arcana.Parser (Arcana v3.0.0)

Parses files into text content for ingestion.

Plain text and markdown are read natively, PDFs go through a configurable parser (Poppler's pdftotext by default), and any other format is handled by a parser you register.

Resolution order

For a file's extension (lowercased, with a leading dot):

an exact match in config :arcana, :file_parsers (a false entry means the extension is disabled and resolution stops here)
the built-ins: .txt/.md/.markdown read natively, .pdf via config :arcana, :pdf_parser
config :arcana, :fallback_parser, unless it is nil/false
otherwise {:error, :unsupported_format}

Registering parsers

config :arcana,
  # per-format
  file_parsers: %{".docx" => {MyApp.DocxParser, []}},
  # everything else (e.g. an extraction service covering many formats)
  fallback_parser: {MyApp.ExtractionService, []}

Registering ".pdf" in :file_parsers overrides the built-in PDF route; registering it as false turns it off entirely, fallback included. See Arcana.FileParser for the behaviour.

PDF Support

The default PDF parser requires pdftotext to be installed:

# macOS
brew install poppler

# Ubuntu/Debian
apt-get install poppler-utils

# Fedora
dnf install poppler-utils

Summary

Functions

available?(extension)

Whether the parser handling extension can run right now.

content_type_for(path)

Returns the content type for a path or filename, based on its extension.

parse(path)

Parses a file and extracts text content.

parse_binary(binary, filename, opts \\ [])

Parses binary content, routing on filename's extension.

parse_file(path, opts \\ [])

Parses a file, returning text and any positional metadata the parser reported.

pdf_support_available?()

Checks if PDF support is available.

supported_formats()

Returns the list of supported file extensions.

Functions

available?(extension)

Whether the parser handling extension can run right now.

Returns false when no parser handles the extension at all.

content_type_for(path)

Returns the content type for a path or filename, based on its extension.

Registered parsers may declare a :content_type in their options; otherwise unknown extensions report application/octet-stream.

parse(path)

Parses a file and extracts text content.

Returns {:ok, text} on success, or {:error, reason} on failure. Use parse_file/2 to also receive positional metadata.

parse_binary(binary, filename, opts \\ [])

Parses binary content, routing on filename's extension.

The resolved parser must accept binary input (supports_binary?/0), otherwise returns {:error, {:binary_unsupported, module}}. Natively handled text formats always work.

When a parser is both unavailable (available?/0) and path-only, unavailability wins: this returns {:error, {:parser_unavailable, module}}, the same reason the path-based flow gives.

parse_file(path, opts \\ [])

Parses a file, returning text and any positional metadata the parser reported.

Returns {:ok, text, meta} where meta is %{} for parsers that don't report positions. See Arcana.FileParser for the metadata shape.

pdf_support_available?()

Checks if PDF support is available.

For the default Poppler parser, this checks if pdftotext is installed. Custom parsers may have different availability requirements.

Examples

iex> Arcana.Parser.pdf_support_available?()
true  # or false if parser not available

supported_formats()

Returns the list of supported file extensions.

Includes the natively handled formats plus any registered through :file_parsers, minus any disabled with false. When a :fallback_parser is configured every extension is effectively supported, so this list is a lower bound.