HuggingfaceClient.Serialization.DDUF (huggingface_client v0.1.0)

Copy Markdown View Source

DDUF (Diffusion model Distributed Unified Format) file operations.

DDUF is a zip-based container format for diffusion models that stores:

  • model_index.json — pipeline class and component registry
  • Component configs (vae/config.json, etc.)
  • Model weights (.safetensors files)
  • Tokenizer files, scheduler configs, etc.

The format is designed for efficient streaming and memory-mapped access.

Summary

Types

entry()
t()

Functions

list_entries(dduf)

Lists all entries in a DDUF file.

open(dduf_path)

Reads a DDUF file and returns a map of filename → entry metadata.

read_entry(dduf, filename)

Reads the content of a specific entry from a DDUF file.

Types

entry()

@type entry() :: %{
  filename: String.t(),
  size: non_neg_integer(),
  offset: non_neg_integer()
}

t()

@type t() :: %HuggingfaceClient.Serialization.DDUF{
  entries: %{required(String.t()) => entry()},
  path: String.t()
}

Functions

list_entries(dduf)

@spec list_entries(t()) :: [entry()]

Lists all entries in a DDUF file.

Example

{:ok, dduf} = HuggingfaceClient.Serialization.DDUF.open("model.dduf")
entries = HuggingfaceClient.Serialization.DDUF.list_entries(dduf)
Enum.each(entries, fn e -> IO.puts("#{e.filename}: #{e.size} bytes") end)

open(dduf_path)

@spec open(String.t()) :: {:ok, t()} | {:error, Exception.t()}

Reads a DDUF file and returns a map of filename → entry metadata.

Example

{:ok, entries} = HuggingfaceClient.Serialization.DDUF.open("FLUX.1-dev.dduf")
IO.inspect(Map.keys(entries))
# ["model_index.json", "vae/config.json", ...]

{:ok, model_index} = HuggingfaceClient.Serialization.DDUF.read_entry(
  entries, "model_index.json"
)

read_entry(dduf, filename)

@spec read_entry(t(), String.t()) :: {:ok, binary()} | {:error, Exception.t()}

Reads the content of a specific entry from a DDUF file.

Example

{:ok, dduf} = HuggingfaceClient.Serialization.DDUF.open("model.dduf")
{:ok, content} = HuggingfaceClient.Serialization.DDUF.read_entry(dduf, "model_index.json")
config = Jason.decode!(content)