HfHub.HTTP (HfHub v0.3.1)

Copy Markdown View Source

HTTP client for HuggingFace Hub API.

Provides low-level HTTP request functionality with authentication, rate limiting, and error handling.

Summary

Functions

Performs a DELETE request.

Performs a DELETE request with a JSON body.

Downloads a file from a URL with streaming support.

Makes a GET request to the HuggingFace Hub API.

Makes a paginated GET request and collects all pages.

Makes a HEAD request to fetch headers without body.

Performs a PATCH request with JSON body.

Makes a POST request to the HuggingFace Hub API.

Performs a POST request expecting no response body.

Sends a POST request whose body is application/x-ndjson text.

Performs a PUT request with JSON body.

Functions

delete(path, opts \\ [])

@spec delete(
  String.t(),
  keyword()
) :: :ok | {:ok, term()} | {:error, term()}

Performs a DELETE request.

DELETE requests typically don't have a body but may return data.

delete(path, body, opts)

@spec delete(String.t(), map() | nil, keyword()) ::
  :ok | {:ok, term()} | {:error, term()}

Performs a DELETE request with a JSON body.

The Hugging Face repo-deletion endpoint follows this shape.

download_file(url, destination, opts \\ [])

@spec download_file(String.t(), Path.t(), keyword()) :: :ok | {:error, term()}

Downloads a file from a URL with streaming support.

Atomicity

Bytes are streamed into #{destination}.incomplete and renamed onto destination only on 200/206. Any other status code (404, 401, network failure, etc.) leaves destination untouched and removes the .incomplete file, so a failed call cannot leave a partial or 0-byte cache entry behind.

Resume semantics (resume: true)

When an .incomplete file already exists, its byte length is sent as the Range: bytes=N- request header so the server can supply only the remaining bytes. Two server behaviors are handled:

  • 206 Partial Content — server honors Range, returns only the tail; the tail is appended to the existing partial.
  • 200 OK — server ignores Range and resends the whole file from offset 0; the existing partial bytes are truncated before the new body is written. This prevents [stale_prefix ++ fresh_body] corruption.

If no .incomplete exists but destination does, destination is copied to .incomplete first so that a successful download still produces an atomic rename.

Concurrency

Direct concurrent calls to download_file/3 against the same destination race on the same .incomplete path. For cache-managed downloads, route through HfHub.Download.hf_hub_download/1, which serializes via HfHub.FS.lock_file/2.

Arguments

  • url - Full URL to download
  • destination - Local file path

Options

  • :token - Authentication token
  • :headers - Additional request headers
  • :resume - Resume interrupted download. Defaults to false.
  • :progress_callback - Function called with download progress. The callback receives (bytes_downloaded, total_bytes) where total_bytes may be nil if the server doesn't provide Content-Length.

Examples

:ok = HfHub.HTTP.download_file(
  "https://huggingface.co/bert-base-uncased/resolve/main/config.json",
  "/tmp/config.json"
)

# With progress tracking
:ok = HfHub.HTTP.download_file(
  "https://huggingface.co/bert-base-uncased/resolve/main/model.bin",
  "/tmp/model.bin",
  progress_callback: fn downloaded, total ->
    if total, do: IO.puts("#{round(downloaded / total * 100)}%")
  end
)

get(path, opts \\ [])

@spec get(
  String.t(),
  keyword()
) :: {:ok, map() | [map()]} | {:error, term()}

Makes a GET request to the HuggingFace Hub API.

Arguments

  • path - API path (e.g., "/api/models/bert-base-uncased")
  • opts - Request options

Options

  • :token - Authentication token
  • :headers - Additional headers
  • :params - Query parameters

Examples

{:ok, response} = HfHub.HTTP.get("/api/models/bert-base-uncased")

get_paginated(path, opts \\ [])

@spec get_paginated(
  String.t(),
  keyword()
) :: {:ok, [map()]} | {:error, term()}

Makes a paginated GET request and collects all pages.

Pagination follows the Link header with rel="next".

Options

  • :token - Authentication token
  • :headers - Additional headers
  • :params - Query parameters

head(url, opts \\ [])

@spec head(
  String.t(),
  keyword()
) :: {:ok, map()} | {:error, term()}

Makes a HEAD request to fetch headers without body.

Used for ETag-based cache validation.

Arguments

  • url - Full URL to request
  • opts - Request options

Options

  • :headers - Request headers
  • :follow_redirects - Whether to follow redirects. Defaults to true.

Examples

{:ok, response} = HfHub.HTTP.head("https://huggingface.co/model/file")

patch(path, body \\ nil, opts \\ [])

@spec patch(String.t(), map() | nil, keyword()) :: {:ok, term()} | {:error, term()}

Performs a PATCH request with JSON body.

post(path, body \\ nil, opts \\ [])

@spec post(String.t(), map() | nil, keyword()) :: {:ok, term()} | {:error, term()}

Makes a POST request to the HuggingFace Hub API.

Arguments

  • path - API path
  • body - Request body (will be JSON-encoded)
  • opts - Request options

Examples

{:ok, response} = HfHub.HTTP.post("/api/endpoint", %{data: "value"})

post_action(path, body \\ nil, opts \\ [])

@spec post_action(String.t(), map() | nil, keyword()) :: :ok | {:error, term()}

Performs a POST request expecting no response body.

Used for actions that return 200/204 with no content.

post_ndjson(path, body, opts \\ [])

@spec post_ndjson(String.t(), iodata(), keyword()) :: {:ok, term()} | {:error, term()}

Sends a POST request whose body is application/x-ndjson text.

Used by HfHub.Commit.create/3 to talk to the Hub's commit endpoint, which expects a header line followed by one JSON object per operation, each on its own line. See: https://github.com/huggingface/huggingface_hub/issues/1085#issuecomment-1265208073

Options

All options accepted by post/3, plus:

  • :params — query-string parameters, e.g. [create_pr: "1"].

put(path, body \\ nil, opts \\ [])

@spec put(String.t(), map() | nil, keyword()) :: {:ok, term()} | {:error, term()}

Performs a PUT request with JSON body.