View Source emmap (emmap v2.1.1)

Link to this section Summary

Types

Extra information returned by the call to emmap:open/2,3. The value of exist true means that an existing memory map was open. The size represents the size of the memory map that was open.

Options for opening a memory mapped file

Functions

Close persistent memory-mapped file previously open with open_counters/2

Decrement a counter number CounterNum in the mmap file by one and return old value.

Decrement a counter number CounterNum in the mmap file by Count and return old value.

Ask the OS to flush the modified memory to disk. The call is asyncronous and non-blocking. This call is not required as the OS will asynchronously flush the modified memory pages to disk lazily, but this call will trigger that process immediately.

Free data block

Increment a counter number CounterNum in the mmap file by one and return old value.

Increment a counter number CounterNum in the mmap file by Count and return old value.

Initialize fixed-size block storage in the shared memory.

Open/create a memory-mapped file. If creating a new file, [create, read, write, {size, N}] options are required. For opening an existing file for writing [read, write] options are required.

Open/create a memory-mapped file. If creating a new file, [create, read, write] options and the Len parameter are required. For opening an existing file for writing [read, write] options are required, and Len can be 0.

Open a persistent memory-mapped file with space for one 64-bit integer counter

Open a persistent memory-mapped file with space for several 64-bit integer counters

Perform an atomic ADD operation on a 64-bit integer value at given Position using specified argument Value. The function returns an old value at that location. This function is thread-safe and can be used for implementing persistent counters.

Perform an atomic AND operation on a 64-bit integer value at given Position using specified argument Value. The function returns an AND'd value at that location.

Perform an atomic compare and swap (CAS) operation on a 64-bit integer value at given Position using specified argument Value. The function returns a tuple {Success, OldVal}, where OldVal is the old value at that location.

Perform an atomic OR operation on a 64-bit integer value at given Position using specified argument Value. The function returns an OR'd value at that location.

Perform an atomic load operation on a 64-bit integer value at given Position. This function is thread-safe and can be used for implementing persistent counters.

Perform an atomic SUBTRACT operation on a 64-bit integer value at given Position using specified argument Value. The function returns an old value at that location. This function is thread-safe and can be used for implementing persistent counters.

Perform an atomic store operation of a 64-bit integer Value at given Position. This function is thread-safe and can be used for implementing persistent counters.

Perform an atomic EXCHANGE operation on a 64-bit integer value at given Position using specified argument Value. The function returns an old value at that location.

Perform an atomic XOR operation on a 64-bit integer value at given Position using specified argument Value. The function returns an XOR'd value at that location.

Write Data bytes to a memory-mapped file at a given offset At.

Read Len bytes from a memory-mapped file at a given offset Off.

Write Data bytes to a memory-mapped file at a given offset Off.

Read next Len bytes from a memory-mapped file. Internally the new position within the file is incremented by Len.

Read data block

Read data blocks

Repair data block storage after unexpected close

Repair data block storage chunk of max Count elements starting with Start address

Resize shared memory. The new size will double the existing size up until the max_inc_size threshold (passed to open/4 (default 64M), otherwise incremented by max_inc_size.

Resize shared memory to a given new size.

Set a counter number CounterNum in the mmap file and return the old value.

Store data block

Link to this section Types

-type mmap_file() :: #file_descriptor{}.
-type open_extra_info() :: #{exist => boolean(), size => non_neg_integer()}.

Extra information returned by the call to emmap:open/2,3. The value of exist true means that an existing memory map was open. The size represents the size of the memory map that was open.

-type open_option() ::
    anon | auto_unlink | create | debug | direct | fixed | hugetlb | huge_2mb | huge_1gb | lock |
    nocache | nolock | nonblock | noreserve | populate | private | read | shared |
    shared_validate | sync | truncate | uninitialized | write | fixed_size |
    {address, pos_integer()} |
    {chmod, pos_integer()} |
    {size, pos_integer()} |
    {max_inc_size, pos_integer()}.

Options for opening a memory mapped file:

anon
Anonymous mapping. The mapping is not backed by any file; its contents are initialized to zero. The offset argument should be zero.
auto_unlink
Automatically delete the mapped file after the mapped data was garbage collected. This can be used when the mapped file is a file-based shared-memory area (e.g. /dev/shm/...) and is mapped in direct mode to free the memory after the data was gc'd
create
Allow to create mmap file if it doesn't exist.
debug
Turn on debug printing in the NIF library.
direct
Read/pread operations do not copy memory, but rather use "resource binaries" that can change content if the underlying data is changed. This is the most performant, but also has other thread-safety implications when not using atomic operations.
fixed
Don't interpret addr as a hint: place the mapping at exactly that address. The implementation aligns the given address to a multiple of the page size.
lock
Use a semaphore (read/write lock) to control state changes internally in the NIF library. This is the default option.
nocache
Pages in this mapping are not retained in the kernel's memory cache. If the system runs low on memory, pages in MAP_NOCACHE mappings will be among the first to be reclaimed. NOTE: this option is only valid for Mac OS.
nolock
Don't use a semaphore (read/write lock) to control state changes internally in the NIF library
nonblock
Don't perform read-ahead: create page tables entries only for pages that are already present in RAM. Since Linux 2.6.23, this flag causes populate to do nothing.
noreserve
Do not reserve swap space for this mapping. When swap space is reserved, one has the guarantee that it is possible to modify the mapping.
populate
Populate (prefault) page tables for a mapping. For a file mapping, this causes read-ahead on the file. This will help to reduce blocking on page faults later.
private
Create a private copy-on-write mapping. Updates to the mapping are not visible to other OS or Erlang processes mapping the same file, and are not carried through to the underlying file.
hugetlb
Allocate the mapping using "huge" pages.
huge_2mb
Used in conjunction with hugetlb to select alternative hugetlb page size of 2MB
huge_1gb
Used in conjunction with hugetlb to select alternative hugetlb page size of 1GB
read
Open for reading (this is default).
shared
Share this mapping. Updates to the mapping are visible to other OS or Erlang processes mapping the same region, and (in the case of file-backed mappings) are carried through to the underlying file. May be used in combination with sync to precisely control when updates are carried through to the underlying file.
shared_validate
This flag provides the same behavior as shared except that shared mappings ignore unknown flags in flags. By contrast, when creating a mapping using shared_validate, the kernel verifies all passed flags are known and fails the mapping with the error eopnotsupp for unknown flags. This mapping type is also required to be able to use some mapping flags (e.g., sync)
sync
This flag is available only with the shared_validate mapping type; mappings of type shared will silently ignore this flag. This flag is supported only for files supporting DAX (direct mapping of persistent memory). For other files, creating a mapping with this flag results in an eopnotsupp error. Shared file mappings with this flag provide the guarantee that while some memory is mapped writable in the address space of the OS process, it will be visible in the same file at the same offset even after the system crashes or is rebooted. In conjunction with the use of appropriate CPU instructions, this provides users of such mappings with a more efficient way of making data modifications persistent.
truncate
Truncate existing mmap file after it's open.
uninitialized
Don't clear anonymous pages. This flag is intended to improve performance on embedded devices. This flag is honored only if the kernel was configured with the CONFIG_MMAP_ALLOW_UNINITIALIZED option.
write
Open memory map for writing.
fixed_size
Don't allow the memory to be resized
{address, pos_integer()}
Open mapping at the given memory address (sets MAP_FIXED on the memory mapped file)
{chmod, pos_integer()}
Create mmap file with this mode (default: 0600)
{size, pos_integer()}
Create/access memory map on this size.
{max_inc_size, pos_integer()}
Size threshold used when automatically resizing shared memory with call to resize/1. Below this threshold the memory will double, and after this threshold, the resized memory will be increased by max_inc_size.
-type resource() :: binary().

Link to this section Functions

-spec close(File :: mmap_file()) -> ok.

Close persistent memory-mapped file previously open with open_counters/2

Link to this function

dec_counter(_, CounterNum)

View Source

Decrement a counter number CounterNum in the mmap file by one and return old value.

Link to this function

dec_counter(_, CounterNum, Count)

View Source

Decrement a counter number CounterNum in the mmap file by Count and return old value.

-spec flush(File :: mmap_file()) -> ok.

Ask the OS to flush the modified memory to disk. The call is asyncronous and non-blocking. This call is not required as the OS will asynchronously flush the modified memory pages to disk lazily, but this call will trigger that process immediately.

-spec free_block(File :: mmap_file(), Addr :: non_neg_integer()) ->
              Success :: boolean() | {error, atom() | string()}.

Free data block

Link to this function

inc_counter(_, CounterNum)

View Source

Increment a counter number CounterNum in the mmap file by one and return old value.

Link to this function

inc_counter(_, CounterNum, Count)

View Source

Increment a counter number CounterNum in the mmap file by Count and return old value.

Link to this function

init_block_storage(File, BlockSize)

View Source
-spec init_block_storage(File :: mmap_file(), BlockSize :: pos_integer()) ->
                      ok | {error, atom() | string()}.

Initialize fixed-size block storage in the shared memory.

-spec open(string() | binary(), [open_option()]) ->
        {ok, mmap_file(), open_extra_info()} | {error, term()}.

Open/create a memory-mapped file. If creating a new file, [create, read, write, {size, N}] options are required. For opening an existing file for writing [read, write] options are required.

Link to this function

open(File, Offset, Length, Options)

View Source
-spec open(File :: string() | binary(),
     Offset :: non_neg_integer(),
     Length :: non_neg_integer(),
     Options :: [open_option()]) ->
        {ok, mmap_file(), open_extra_info()} | {error, term()}.

Open/create a memory-mapped file. If creating a new file, [create, read, write] options and the Len parameter are required. For opening an existing file for writing [read, write] options are required, and Len can be 0.

Open a persistent memory-mapped file with space for one 64-bit integer counter

Link to this function

open_counters(Filename, NumCounters)

View Source

Open a persistent memory-mapped file with space for several 64-bit integer counters

Link to this function

patomic_add(File, Position, Value)

View Source
-spec patomic_add(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) ->
               {ok, OldValue :: integer()} | {error, atom()} | no_return().

Perform an atomic ADD operation on a 64-bit integer value at given Position using specified argument Value. The function returns an old value at that location. This function is thread-safe and can be used for implementing persistent counters.

Link to this function

patomic_and(File, Position, Value)

View Source
-spec patomic_and(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) ->
               {ok, integer()} | {error, atom()} | no_return().

Perform an atomic AND operation on a 64-bit integer value at given Position using specified argument Value. The function returns an AND'd value at that location.

Link to this function

patomic_cas(File, Position, OldValue, Value)

View Source
-spec patomic_cas(File :: mmap_file(), Position :: non_neg_integer(), integer(), integer()) ->
               {boolean(), integer()} | {error, atom()} | no_return().

Perform an atomic compare and swap (CAS) operation on a 64-bit integer value at given Position using specified argument Value. The function returns a tuple {Success, OldVal}, where OldVal is the old value at that location.

Link to this function

patomic_or(File, Position, Value)

View Source
-spec patomic_or(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) ->
              {ok, integer()} | {error, atom()} | no_return().

Perform an atomic OR operation on a 64-bit integer value at given Position using specified argument Value. The function returns an OR'd value at that location.

Link to this function

patomic_read_integer(File, Position)

View Source
-spec patomic_read_integer(File :: mmap_file(), Position :: non_neg_integer()) -> Value :: integer().

Perform an atomic load operation on a 64-bit integer value at given Position. This function is thread-safe and can be used for implementing persistent counters.

Link to this function

patomic_sub(File, Position, Value)

View Source
-spec patomic_sub(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) ->
               {ok, OldValue :: integer()} | {error, atom()} | no_return().

Perform an atomic SUBTRACT operation on a 64-bit integer value at given Position using specified argument Value. The function returns an old value at that location. This function is thread-safe and can be used for implementing persistent counters.

Link to this function

patomic_write_integer(File, Position, Value)

View Source
-spec patomic_write_integer(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) ->
                         ok.

Perform an atomic store operation of a 64-bit integer Value at given Position. This function is thread-safe and can be used for implementing persistent counters.

Link to this function

patomic_xchg(File, Position, Value)

View Source
-spec patomic_xchg(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) ->
                {ok, integer()} | {error, atom()} | no_return().

Perform an atomic EXCHANGE operation on a 64-bit integer value at given Position using specified argument Value. The function returns an old value at that location.

Link to this function

patomic_xor(File, Position, Value)

View Source
-spec patomic_xor(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) ->
               {ok, integer()} | {error, atom()} | no_return().

Perform an atomic XOR operation on a 64-bit integer value at given Position using specified argument Value. The function returns an XOR'd value at that location.

Link to this function

position(File, Position)

View Source
-spec position(File :: mmap_file(),
         Position :: non_neg_integer() | {bof | cur | eof, Position :: integer()}) ->
            {ok, non_neg_integer()} | {error, term()}.

Write Data bytes to a memory-mapped file at a given offset At.

Link to this function

pread(File, Offset, Length)

View Source
-spec pread(File :: mmap_file(), Offset :: non_neg_integer(), Length :: non_neg_integer()) ->
         {ok, binary()} | {error, term()} | eof.

Read Len bytes from a memory-mapped file at a given offset Off.

Link to this function

pwrite(File, Position, Data)

View Source
-spec pwrite(File :: mmap_file(), Position :: non_neg_integer(), Data :: binary()) ->
          ok | {error, term()}.

Write Data bytes to a memory-mapped file at a given offset Off.

-spec read(File :: mmap_file(), Length :: non_neg_integer()) -> {ok, binary()} | {error, term()} | eof.

Read next Len bytes from a memory-mapped file. Internally the new position within the file is incremented by Len.

-spec read_block(File :: mmap_file(), Addr :: non_neg_integer()) ->
              Data :: binary() | eof | {error, atom() | string()}.

Read data block

-spec read_blocks(File :: mmap_file()) -> [Data :: binary()] | eof | {error, atom() | string()}.

Read data blocks

Link to this function

read_blocks(File, Start, Count)

View Source
-spec read_blocks(File :: mmap_file(), Start :: non_neg_integer(), Count :: pos_integer()) ->
               {[Data :: binary()], Continuation :: integer() | eof} | {error, atom() | string()}.

Read data blocks

Link to this function

read_counter(_, CounterNum)

View Source
-spec read_line(File :: mmap_file()) -> {ok, binary()} | {error, term()} | eof.
Link to this function

repair_block_storage(File)

View Source
-spec repair_block_storage(File :: mmap_file()) -> ok | {error, atom() | string()}.

Repair data block storage after unexpected close

Link to this function

repair_block_storage(File, Start, Count)

View Source
-spec repair_block_storage(File :: mmap_file(), Start :: non_neg_integer(), Count :: pos_integer()) ->
                        Continuation :: integer() | eof | {error, atom() | string()}.

Repair data block storage chunk of max Count elements starting with Start address

-spec resize(File :: mmap_file()) -> {ok, NewSize :: non_neg_integer()} | {error, atom() | string()}.

Resize shared memory. The new size will double the existing size up until the max_inc_size threshold (passed to open/4 (default 64M), otherwise incremented by max_inc_size.

Link to this function

resize(File_descriptor, NewSize)

View Source
-spec resize(mmap_file(), non_neg_integer()) -> {ok, NewSize :: non_neg_integer()} | {error, string()}.

Resize shared memory to a given new size.

Link to this function

set_counter(_, CounterNum, Value)

View Source

Set a counter number CounterNum in the mmap file and return the old value.

-spec store_block(File :: mmap_file(), Data :: binary()) ->
               Addr :: non_neg_integer() | {error, atom() | string()}.

Store data block