iommap (iommap v1.1.3)

View Source

Memory-mapped file I/O for Erlang.

This module provides cross-platform memory-mapped file access compatible with Linux, macOS, FreeBSD, and OpenBSD.

Memory-mapped files allow applications to access file data as if it were in memory, enabling efficient random access patterns and shared memory between processes.

Quick Start

   %% Create and write to a memory-mapped file
   {ok, H} = iommap:open("/tmp/test.dat", read_write, [create, {size, 4096}]),
   ok = iommap:pwrite(H, 0, <<"Hello, iommap!">>),
   {ok, <<"Hello, iommap!">>} = iommap:pread(H, 0, 14),
   ok = iommap:sync(H),
   ok = iommap:close(H).

Access Modes

  • read - Read-only access to the file
  • write - Write-only access to the file
  • read_write - Full read and write access (default)

Mapping Options

  • shared - Changes are visible to other processes (default)
  • private - Copy-on-write; changes are private
  • lock - Lock pages in memory (mlock)
  • populate - Prefault pages on mapping (Linux only)
  • nocache - Disable page caching (macOS only)
  • create - Create file if it doesn't exist
  • truncate - Truncate existing file
  • {size, N} - Initial size for new files

Thread Safety

All operations are thread-safe. The NIF uses pthread read-write locks to allow multiple concurrent reads while writes are exclusive.

Error Handling

Operations return {error, Reason} on failure. Common reasons:

  • badarg - Invalid arguments
  • enomem - Out of memory
  • enoent - File not found
  • eacces - Permission denied
  • closed - Handle already closed
  • out_of_bounds - Offset/length exceeds file size
  • sigbus - Memory access fault (file truncated externally)

Zero-Copy Region Binaries

region_binary/3 returns a refcounted binary that points directly into the mapped region with no data copy. The underlying mapping is kept alive (its munmap is deferred) for as long as any such binary, or any sub-binary derived from it, is reachable.

Lifetime model: the NIF uses two resources internally. The handle holds one reference to the mapping; close/1 releases that reference but does not call munmap if region binaries are still outstanding. The mapping is unmapped only when the last reference (handle ref + outstanding region binaries) is dropped.

Truncation hazard: region_binary/3 is unsafe to use against files that may be truncated by external processes (or by iommap:truncate/2 shrinking past the binary's range) while a returned binary is reachable. Reads of unmapped pages happen outside any NIF call and can crash the BEAM with SIGBUS. Callers needing safety against external mutation must use pread/3 (which copies and is unaffected).

Summary

Functions

Provide advice about expected access patterns.

Close a memory-mapped file handle.

Open a file for memory-mapped access.

Get the current size of the memory-mapped region.

Read bytes from a memory-mapped file at the given offset.

Write data to a memory-mapped file at the given offset.

Return a zero-copy refcounted binary view into the mapped region.

Synchronize the memory mapping with the underlying file.

Truncate or extend the file to the specified size.

Types

advise_hint/0

-type advise_hint() :: normal | random | sequential | willneed | dontneed.

handle/0

-type handle() :: reference().

mode/0

-type mode() :: read | write | read_write.

open_option/0

-type open_option() ::
          read | write | read_write |
          {size, non_neg_integer()} |
          shared | private | lock | populate | nocache | create | truncate.

sync_mode/0

-type sync_mode() :: sync | async.

Functions

advise(Handle, Offset, Length, Hint)

-spec advise(Handle, Offset, Length, Hint) -> ok | {error, term()}
                when
                    Handle :: handle(),
                    Offset :: non_neg_integer(),
                    Length :: non_neg_integer(),
                    Hint :: advise_hint().

Provide advice about expected access patterns.

Hints help the OS optimize memory management:

  • normal - No special treatment
  • random - Expect random access
  • sequential - Expect sequential access
  • willneed - Will need these pages soon
  • dontneed - Won't need these pages soon

close(Handle)

-spec close(Handle) -> ok | {error, term()} when Handle :: handle().

Close a memory-mapped file handle.

Unmaps the memory region and closes the file descriptor. The handle becomes invalid after this call.

open(Path, Options)

-spec open(Path, Options) -> {ok, handle()} | {error, term()}
              when Path :: file:filename_all(), Options :: [open_option()].

Equivalent to open(Path, read_write, Options).

Open a file for memory-mapped access with default read_write mode.

open(Path, Mode, Options)

-spec open(Path, Mode, Options) -> {ok, handle()} | {error, term()}
              when Path :: file:filename_all(), Mode :: mode(), Options :: [open_option()].

Open a file for memory-mapped access.

Opens the file at Path with the given Mode and creates a memory mapping.

Options

  • {size, N} - Initial size for new files (required with create)
  • shared - Use MAP_SHARED (default)
  • private - Use MAP_PRIVATE (copy-on-write)
  • lock - Lock pages in memory (mlock)
  • populate - Prefault pages (Linux only)
  • nocache - Disable caching (macOS only)
  • create - Create file if it doesn't exist
  • truncate - Truncate existing file

position(Handle)

-spec position(Handle) -> {ok, non_neg_integer()} | {error, term()} when Handle :: handle().

Get the current size of the memory-mapped region.

pread(Handle, Offset, Length)

-spec pread(Handle, Offset, Length) -> {ok, binary()} | {error, term()}
               when Handle :: handle(), Offset :: non_neg_integer(), Length :: non_neg_integer().

Read bytes from a memory-mapped file at the given offset.

Returns a new binary containing the requested bytes. The binary is a copy of the mapped memory.

pwrite(Handle, Offset, Data)

-spec pwrite(Handle, Offset, Data) -> ok | {error, term()}
                when Handle :: handle(), Offset :: non_neg_integer(), Data :: iodata().

Write data to a memory-mapped file at the given offset.

Writes the binary data to the mapped region starting at Offset.

region_binary(Handle, Offset, Length)

-spec region_binary(Handle, Offset, Length) -> {ok, binary()} | {error, Reason}
                       when
                           Handle :: handle(),
                           Offset :: non_neg_integer(),
                           Length :: non_neg_integer(),
                           Reason :: badarg | closed | out_of_bounds.

Return a zero-copy refcounted binary view into the mapped region.

Unlike pread/3, no bytes are copied: the returned binary is a resource binary whose underlying memory is the page-cache backing the mapping. The mapping is kept alive for as long as the returned binary (or any sub-binary derived from it) remains reachable.

This primitive is intended for hot zero-copy hand-off paths, e.g. passing the bytes to another NIF as ErlNifBinary without going through the BEAM heap.

Reads of the returned binary occur outside any NIF call. If the underlying file is truncated (by an external process, or by truncate/2 shrinking past the binary's range) while the binary is reachable, accessing it can crash the BEAM with SIGBUS. Use pread/3 if safety against external mutation is required.

sync(Handle)

-spec sync(Handle) -> ok | {error, term()} when Handle :: handle().

Equivalent to sync(Handle, sync).

Synchronize the memory mapping with the underlying file.

sync(Handle, Mode)

-spec sync(Handle, Mode) -> ok | {error, term()} when Handle :: handle(), Mode :: sync_mode().

Synchronize the memory mapping with the underlying file.

sync mode waits for the operation to complete (MS_SYNC). async mode returns immediately (MS_ASYNC).

truncate(Handle, NewSize)

-spec truncate(Handle, NewSize) -> ok | {error, term()}
                  when Handle :: handle(), NewSize :: non_neg_integer().

Truncate or extend the file to the specified size.

Resizes the underlying file and remaps the memory region. Existing data beyond NewSize is lost.