ocibuild_release (ocibuild v0.10.4)

View Source

Shared release handling for OCI image building.

This module provides common functionality for collecting release files and building OCI images, used by both rebar3 and Mix integrations.

Security features:

  • Symlinks pointing outside the release directory are rejected
  • Broken symlinks are skipped with a warning
  • Path traversal via .. components is prevented

Summary

Functions

Build automatic OCI annotations from VCS and build context.

Build an OCI image from release files with default options.

Build an OCI image from release files with options.

Build release layers with smart dependency layering when available.

Check for native code (NIFs) in a release.

Classify a single file path into erts, dep, or app layer.

Clear the progress line if in TTY mode.

Collect all files from a release directory for inclusion in an OCI image.

Collect release files with options.

Expand semicolon-separated tags into multiple tags.

Format bytes as human-readable string.

Format progress as a string with progress bar.

Get file mode (permissions) for a file.

Get authentication credentials for pulling base images.

Get authentication credentials for pushing images.

Get tags from CLI options and config, with semicolon expansion support.

Check if a release has bundled ERTS.

Check if a value is nil (Elixir) or undefined (Erlang).

Check if a path looks like a tarball based on its extension.

Check if stdout is connected to a TTY (terminal).

Create a progress callback for terminal display.

Make a path relative to a base path (cross-platform).

Normalize a key-value map to have binary keys and values.

Parse a CLI argument string in KEY=VALUE format.

Parse a tag into repository and tag parts.

Partition collected files into ERTS, dependency, and application layers.

Push a pre-built OCI tarball to a registry.

Run the complete OCI image build pipeline.

Save an image to a tarball file.

Start the progress coordinator for multi-line parallel progress display.

Stop the ocibuild httpc profile to allow clean VM exit.

Stop the progress coordinator and clean up terminal state.

Convert various types to binary.

Convert a local path to a container path (always forward slashes).

Validate user-provided annotations for security and correctness.

Validate that a release is suitable for multi-platform builds.

Functions

build_auto_annotations(Image, ReleasePath, Config)

-spec build_auto_annotations(ocibuild:image(), file:filename(), map()) -> map().

Build automatic OCI annotations from VCS and build context.

This function creates annotations based on:

  • VCS information (source URL, revision) if vcs_annotations is true
  • Application version from the adapter
  • Build timestamp (respects SOURCE_DATE_EPOCH for reproducible builds)
  • Base image information (name and digest)

The annotations are merged with any user-provided annotations, with user annotations taking precedence over auto-generated ones.

build_image(BaseImage, Files)

-spec build_image(BaseImage :: binary(), Files :: [{binary(), binary(), non_neg_integer()}]) ->
                     {ok, ocibuild:image()} | {error, term()}.

Build an OCI image from release files with default options.

Shorthand for build_image(BaseImage, Files, #{}).

Example:

Files = [{~"/app/bin/myapp", Binary, 8#755}],
{ok, Image} = build_image(~"scratch", Files).

build_image(BaseImage, Files, Opts)

-spec build_image(BaseImage :: binary(),
                  Files :: [{binary(), binary(), non_neg_integer()}],
                  Opts :: map()) ->
                     {ok, ocibuild:image()} | {error, term()}.

Build an OCI image from release files with options.

Options:

  • release_name - Release name (atom, string, or binary) - required for setting entrypoint
  • workdir - Working directory in container (default: ~"/app")
  • env - Environment variables map (default: #{})
  • expose - Ports to expose (default: [])
  • labels - Image labels map (default: #{})
  • cmd - Release start command (default: ~"foreground")
  • uid - User ID to run as (default: 65534 for nobody; use 0 for root)
  • auth - Authentication credentials for pulling base image
  • progress - Progress callback function
  • annotations - Map of manifest annotations

Example:

Files = [{~"/app/bin/myapp", Binary, 8#755}],
{ok, Image} = build_image(~"debian:stable-slim", Files, #{
    release_name => ~"myapp",
    workdir => ~"/app",
    env => #{~"LANG" => ~"C.UTF-8"},
    expose => [8080],
    cmd => ~"foreground"
}).

build_release_layers(Image, Files, ReleasePath, Deps, Opts)

-spec build_release_layers(Image :: ocibuild:image(),
                           Files :: [{binary(), binary(), non_neg_integer()}],
                           ReleasePath :: file:filename() | undefined,
                           Deps :: [#{name := binary(), version := binary(), source := binary()}],
                           Opts :: map()) ->
                              ocibuild:image().

Build release layers with smart dependency layering when available.

If dependency information is provided, creates 2-3 layers:

  • With ERTS (bundled): ERTS layer, Deps layer, App layer
  • Without ERTS (multi-platform): Deps layer, App layer

Falls back to single layer if no dependencies provided (backward compatible).

check_for_native_code(ReleasePath)

-spec check_for_native_code(file:filename()) ->
                               {ok, []} |
                               {warning, [#{app := binary(), file := binary(), extension := binary()}]}.

Check for native code (NIFs) in a release.

Scans lib/*/priv/ directories for native shared libraries:

  • .so files (Linux/Unix/BSD)
  • .dll files (Windows)
  • .dylib files (macOS)

This function works for all BEAM languages since they all produce standard OTP releases with the same directory structure.

Returns {ok, []} if no native code found, or {warning, [NifInfo]} with details about each native file found.

{ok, []} = ocibuild_release:check_for_native_code("/path/to/release").
{warning, [#{app := ~"crypto", file := ~"crypto_nif.so"}]} =
    ocibuild_release:check_for_native_code("/path/to/release_with_nifs").

classify_file_layer(Path, DepNames, AppName, Workdir, HasErts)

-spec classify_file_layer(Path :: binary(),
                          DepNames :: sets:set(binary()),
                          AppName :: binary(),
                          Workdir :: binary(),
                          HasErts :: boolean()) ->
                             erts | dep | app.

Classify a single file path into erts, dep, or app layer.

Classification rules (lock file is source of truth):

  1. lib/<app_name>-* -> app layer (your application)
  2. lib/<name>-* where name is in lock file -> dep layer
  3. lib/<name>-* where name is NOT in lock file -> OTP lib:
    • If ERTS bundled: erts layer
    • If ERTS not bundled: dep layer (OTP libs are stable like deps)
  4. erts-* directory -> erts layer
  5. bin/, releases/, etc. -> app layer

clear_progress_line()

-spec clear_progress_line() -> ok.

Clear the progress line if in TTY mode.

In CI mode (non-TTY), progress is printed with newlines so no clearing needed.

collect_release_files(ReleasePath)

-spec collect_release_files(file:filename()) ->
                               {ok, [{binary(), binary(), non_neg_integer()}]} | {error, term()}.

Collect all files from a release directory for inclusion in an OCI image.

Returns a list of {ContainerPath, Content, Mode} tuples suitable for passing to ocibuild:add_layer/2.

Security: Symlinks pointing outside the release directory are skipped with a warning to prevent path traversal attacks.

collect_release_files(ReleasePath, Opts)

-spec collect_release_files(file:filename(), map()) ->
                               {ok, [{binary(), binary(), non_neg_integer()}]} | {error, term()}.

Collect release files with options.

Options:

  • workdir - Container working directory (default: /app)

expand_semicolon_tags(TagStr)

-spec expand_semicolon_tags(binary()) -> [binary()].

Expand semicolon-separated tags into multiple tags.

Supports docker/metadata-action style tags (use sep-tags: ";" in the action). Trims whitespace and filters empty segments.

Examples:

  • ~"myapp:v1;myapp:latest" -> [~"myapp:v1", ~"myapp:latest"]
  • ~"myapp:v1 ; myapp:latest" -> [~"myapp:v1", ~"myapp:latest"]

format_bytes/1

-spec format_bytes(non_neg_integer()) -> iolist().

Format bytes as human-readable string.

format_progress/2

-spec format_progress(non_neg_integer(), non_neg_integer() | unknown) -> iolist().

Format progress as a string with progress bar.

get_file_mode(FilePath)

-spec get_file_mode(file:filename()) -> non_neg_integer().

Get file mode (permissions) for a file.

get_pull_auth()

-spec get_pull_auth() -> map().

Get authentication credentials for pulling base images.

Reads from environment variables:

  • OCIBUILD_PULL_TOKEN - Bearer token (takes priority)
  • OCIBUILD_PULL_USERNAME / OCIBUILD_PULL_PASSWORD - Basic auth

Empty string values are treated as unset, enabling anonymous access. This handles CI/CD systems like GitHub Actions where optional inputs result in empty strings rather than unset variables.

get_push_auth()

-spec get_push_auth() -> map().

Get authentication credentials for pushing images.

Reads from environment variables:

  • OCIBUILD_PUSH_TOKEN - Bearer token (takes priority)
  • OCIBUILD_PUSH_USERNAME / OCIBUILD_PUSH_PASSWORD - Basic auth

Empty string values are treated as unset, enabling anonymous access. This handles CI/CD systems like GitHub Actions where optional inputs result in empty strings rather than unset variables.

get_tags(CliTags, ConfigTags, DefaultRepo, DefaultVersion)

-spec get_tags([binary()], [binary()], binary(), binary()) -> [binary()].

Get tags from CLI options and config, with semicolon expansion support.

This is the shared implementation used by both rebar3 and Mix adapters. Supports docker/metadata-action style semicolon-separated tags.

Parameters:

  • CliTags: List of tag strings from CLI (e.g., from multiple -t flags)
  • ConfigTags: List of tag strings from config file
  • DefaultRepo: Default repository name (usually release name)
  • DefaultVersion: Default version string

Returns a list of binary tags. CLI tags take precedence over config tags. If neither is provided, returns a default tag of DefaultRepo:DefaultVersion.

Examples:

  • get_tags([~"myapp:v1;myapp:latest"], [], ~"myapp", ~"1.0.0") -> [~"myapp:v1", ~"myapp:latest"]
  • get_tags([], [~"myapp:v1;myapp:latest"], ~"myapp", ~"1.0.0") -> [~"myapp:v1", ~"myapp:latest"]
  • get_tags([], [~"myapp:v1"], ~"myapp", ~"1.0.0") -> [~"myapp:v1"]
  • get_tags([], [], ~"myapp", ~"1.0.0") -> [~"myapp:1.0.0"]

has_bundled_erts(ReleasePath)

-spec has_bundled_erts(file:filename()) -> boolean().

Check if a release has bundled ERTS.

Multi-platform builds require the ERTS to come from the base image, not bundled in the release. This function checks for an erts-* directory at the release root.

This function works for all BEAM languages (Erlang, Elixir, Gleam, LFE) since they all produce standard OTP releases.

true = ocibuild_release:has_bundled_erts("/path/to/rel/myapp").

is_nil_or_undefined/1

-spec is_nil_or_undefined(term()) -> boolean().

Check if a value is nil (Elixir) or undefined (Erlang).

This helper provides cross-language compatibility when handling optional values from Elixir code.

is_tarball_path/1

-spec is_tarball_path(string() | binary()) -> boolean().

Check if a path looks like a tarball based on its extension.

Returns true for paths ending in .tar.gz, .tgz, .tar.zst, .tar.zstd, or .tar. Used by adapters to detect push-existing-tarball mode.

is_tty()

-spec is_tty() -> boolean().

Check if stdout is connected to a TTY (terminal).

Returns true for interactive terminals, false for CI/pipes.

make_progress_callback()

-spec make_progress_callback() -> ocibuild_registry:progress_callback().

Create a progress callback for terminal display.

Handles both TTY (animated multi-line progress via coordinator) and CI (final state only) modes.

make_relative_path(BasePath, FullPath)

-spec make_relative_path(file:filename(), file:filename()) -> file:filename().

Make a path relative to a base path (cross-platform).

normalize_kv_map/1

-spec normalize_kv_map(term()) -> #{binary() => binary()}.

Normalize a key-value map to have binary keys and values.

Used for normalizing annotations and labels from config files. Handles various input formats (atoms, strings, binaries). Returns an empty map if input is not a map.

Examples

#{~"key" => ~"value"} = ocibuild_release:normalize_kv_map(#{"key" => "value"}).
#{~"key" => ~"value"} = ocibuild_release:normalize_kv_map(#{key => value}).
#{} = ocibuild_release:normalize_kv_map(undefined).

parse_kv_arg(Str)

-spec parse_kv_arg(string()) -> {ok, {binary(), binary()}} | {error, term()}.

Parse a CLI argument string in KEY=VALUE format.

Used for parsing --annotation and --label CLI arguments. Returns {ok, {Key, Value}} on success, or {error, {invalid_kv_format, Input}} if the string doesn't contain an = character.

Examples

{ok, {~"key", ~"value"}} = ocibuild_release:parse_kv_arg("key=value").
{ok, {~"key", ~"value=with=equals"}} = ocibuild_release:parse_kv_arg("key=value=with=equals").
{error, {invalid_kv_format, "no-equals"}} = ocibuild_release:parse_kv_arg("no-equals").

parse_tag(Tag)

-spec parse_tag(binary()) -> {Repo :: binary(), Tag :: binary()}.

Parse a tag into repository and tag parts.

Examples:

  • ~"myapp:1.0.0" -> {~"myapp", ~"1.0.0"}
  • ~"myapp" -> {~"myapp", ~"latest"}
  • ~"ghcr.io/org/app:v1" -> {~"ghcr.io/org/app", ~"v1"}

partition_files_by_layer(Files, Deps, AppName, Workdir, HasErts)

-spec partition_files_by_layer(Files :: [{binary(), binary(), non_neg_integer()}],
                               Deps :: [#{name := binary(), version := binary(), source := binary()}],
                               AppName :: binary(),
                               Workdir :: binary(),
                               HasErts :: boolean()) ->
                                  {ErtsFiles :: [{binary(), binary(), non_neg_integer()}],
                                   DepFiles :: [{binary(), binary(), non_neg_integer()}],
                                   AppFiles :: [{binary(), binary(), non_neg_integer()}]}.

Partition collected files into ERTS, dependency, and application layers.

Files are classified based on their container paths and the lock file:

  • App layer: lib/<app_name>-*, bin/, releases/
  • Deps layer: lib/<name>-* where name is in the lock file
  • ERTS layer (if bundled): erts-* and lib/<name>-* NOT in lock file (OTP libs)

If ERTS is not bundled, OTP libs go to the deps layer instead.

The lock file is the source of truth for dependencies - anything in lib/ that's not in the lock file and not the app itself must be an OTP library.

Returns {ErtsFiles, DepFiles, AppFiles} where each is a list of {Path, Content, Mode} tuples.

push_image(Image, Registry, RepoTag, Auth, Opts)

-spec push_image(ocibuild:image(), binary(), binary(), map(), map()) ->
                    {ok, Digest :: binary()} | {error, term()}.

Push an image to a registry.

Handles authentication, progress display, and httpc cleanup. Returns {ok, Digest} where Digest is the sha256 digest of the pushed manifest.

push_tarball(AdapterModule, AdapterState, TarballPath, Opts)

-spec push_tarball(module(), term(), file:filename(), map()) -> {ok, term()} | {error, term()}.

Push a pre-built OCI tarball to a registry.

This function loads an existing OCI image tarball and pushes it to a registry without rebuilding. Useful for CI/CD pipelines where build and push are separate steps.

Options:

  • registry - Target registry (e.g., <<"ghcr.io/myorg">>)
  • tags - Optional tag overrides (uses embedded tag from tarball if not specified)
  • chunk_size - Chunk size for uploads in bytes

Supports multiple tags: first tag does full upload, additional tags just add tag references to the same manifest (efficient).

Returns {ok, AdapterState} on success with the pushed digest printed.

run(AdapterModule, AdapterState, Opts)

-spec run(module(), term(), map()) -> {ok, term()} | {error, term()}.

Run the complete OCI image build pipeline.

This function orchestrates the entire build process using the adapter module for build-system-specific operations (configuration, release finding, logging).

The adapter module must implement the ocibuild_adapter behaviour:

  • get_config/1 - Extract configuration from build system state
  • find_release/2 - Locate the release directory
  • info/2, console/2, error/2 - Logging functions

Options:

  • cmd - Start command override (default: from adapter config or "foreground")

Returns {ok, State} on success (State passed through from adapter), or {error, Reason} on failure.

save_image(Image, OutputPath, Opts)

-spec save_image(ocibuild:image(), file:filename(), map()) -> ok | {error, term()}.

Save an image to a tarball file.

The image is saved in OCI layout format compatible with podman load.

start_progress_coordinator()

-spec start_progress_coordinator() -> pid() | undefined.

Start the progress coordinator for multi-line parallel progress display.

Call this before starting parallel downloads/uploads. The coordinator manages separate terminal lines for each concurrent operation.

stop_httpc()

-spec stop_httpc() -> ok.

Stop the ocibuild httpc profile to allow clean VM exit.

This should be called after push operations to close HTTP connections and allow the VM to exit cleanly.

stop_progress_coordinator()

-spec stop_progress_coordinator() -> ok.

Stop the progress coordinator and clean up terminal state.

to_binary/1

-spec to_binary(term()) -> binary().

Convert various types to binary.

to_container_path(RelPath)

-spec to_container_path(file:filename()) -> binary().

Convert a local path to a container path (always forward slashes).

validate_annotations(Annotations)

-spec validate_annotations(#{binary() => binary()}) -> {ok, #{binary() => binary()}} | {error, term()}.

Validate user-provided annotations for security and correctness.

Performs the following checks:

  1. Security validation (null bytes, path traversal) on all keys and values
  2. Protected annotation check - returns error if user tries to override computed annotations
  3. Created timestamp validation - if org.opencontainers.image.created is present, validates it's a valid unix timestamp and converts to ISO 8601 format

Protected annotations that cannot be overridden:

  • org.opencontainers.image.source - Computed from VCS
  • org.opencontainers.image.revision - Computed from VCS
  • org.opencontainers.image.base.name - Computed from base image
  • org.opencontainers.image.base.digest - Computed from base image

Examples

{ok, #{~"custom.key" => ~"value"}} = ocibuild_release:validate_annotations(#{~"custom.key" => ~"value"}).
{error, {protected_annotation, ~"org.opencontainers.image.source"}} =
    ocibuild_release:validate_annotations(#{~"org.opencontainers.image.source" => ~"evil"}).
{error, {invalid_annotation, null_byte, _}} =
    ocibuild_release:validate_annotations(#{<<"bad\0key">> => ~"value"}).

validate_multiplatform/2

-spec validate_multiplatform(file:filename(), [ocibuild:platform()]) ->
                                ok | {error, {bundled_erts, binary()}}.

Validate that a release is suitable for multi-platform builds.

For multi-platform builds (more than one platform specified):

  1. Error if bundled ERTS is detected - multi-platform requires ERTS from base image
  2. Warning if native code (NIFs) detected - may not be portable

This function is universal for all BEAM languages.

ok = ocibuild_release:validate_multiplatform(ReleasePath, [Platform]).
{error, {bundled_erts, _Reason}} = ocibuild_release:validate_multiplatform(ReleasePath, [P1, P2]).