emmap (emmap v2.1.4)

View Source

Summary

Types

Extra information returned by the call to emmap:open/2,3. The value of exist true means that an existing memory map was open. The size represents the size of the memory map that was open.

Options for opening a memory mapped file

Functions

Close persistent memory-mapped file previously open with open_counters/2

Decrement a counter number CounterNum in the mmap file by one and return old value.

Decrement a counter number CounterNum in the mmap file by Count and return old value.

Ask the OS to flush the modified memory to disk. The call is asyncronous and non-blocking. This call is not required as the OS will asynchronously flush the modified memory pages to disk lazily, but this call will trigger that process immediately.

Free data block

Increment a counter number CounterNum in the mmap file by one and return old value.

Increment a counter number CounterNum in the mmap file by Count and return old value.

Initialize fixed-size block storage in the shared memory.

Open/create a memory-mapped file. If creating a new file, [create, read, write, {size, N}] options are required. For opening an existing file for writing [read, write] options are required.

Open/create a memory-mapped file. If creating a new file, [create, read, write] options and the Len parameter are required. For opening an existing file for writing [read, write] options are required, and Len can be 0.

Open a persistent memory-mapped file with space for one 64-bit integer counter

Open a persistent memory-mapped file with space for several 64-bit integer counters

Perform an atomic ADD operation on a 64-bit integer value at given Position using specified argument Value. The function returns an old value at that location. This function is thread-safe and can be used for implementing persistent counters.

Perform an atomic AND operation on a 64-bit integer value at given Position using specified argument Value. The function returns an AND'd value at that location.

Perform an atomic compare and swap (CAS) operation on a 64-bit integer value at given Position using specified argument Value. The function returns a tuple {Success, OldVal}, where OldVal is the old value at that location.

Perform an atomic OR operation on a 64-bit integer value at given Position using specified argument Value. The function returns an OR'd value at that location.

Perform an atomic load operation on a 64-bit integer value at given Position. This function is thread-safe and can be used for implementing persistent counters.

Perform an atomic SUBTRACT operation on a 64-bit integer value at given Position using specified argument Value. The function returns an old value at that location. This function is thread-safe and can be used for implementing persistent counters.

Perform an atomic store operation of a 64-bit integer Value at given Position. This function is thread-safe and can be used for implementing persistent counters.

Perform an atomic EXCHANGE operation on a 64-bit integer value at given Position using specified argument Value. The function returns an old value at that location.

Perform an atomic XOR operation on a 64-bit integer value at given Position using specified argument Value. The function returns an XOR'd value at that location.

Write Data bytes to a memory-mapped file at a given offset At.

Read Len bytes from a memory-mapped file at a given offset Off.

Write Data bytes to a memory-mapped file at a given offset Off.

Read next Len bytes from a memory-mapped file. Internally the new position within the file is incremented by Len.

Read data block

Read data blocks

Repair data block storage after unexpected close

Repair data block storage chunk of max Count elements starting with Start address

Resize shared memory. The new size will double the existing size up until the max_inc_size threshold (passed to open/4 (default 64M), otherwise incremented by max_inc_size.

Resize shared memory to a given new size.

Set a counter number CounterNum in the mmap file and return the old value.

Store data block

Types

mmap_file/0

-type mmap_file() :: #file_descriptor{module :: module(), data :: term()}.

open_extra_info/0

-type open_extra_info() :: #{exist => boolean(), size => non_neg_integer()}.

Extra information returned by the call to emmap:open/2,3. The value of exist true means that an existing memory map was open. The size represents the size of the memory map that was open.

open_option/0

-type open_option() ::
          anon | auto_unlink | create | debug | direct | fixed | hugetlb | huge_2mb | huge_1gb | lock |
          nocache | nolock | nonblock | noreserve | populate | private | read | shared |
          shared_validate | sync | truncate | uninitialized | write | fixed_size |
          {address, pos_integer()} |
          {chmod, pos_integer()} |
          {size, pos_integer()} |
          {max_inc_size, pos_integer()}.

Options for opening a memory mapped file:

anon
Anonymous mapping. The mapping is not backed by any file; its contents are initialized to zero. The offset argument should be zero.
auto_unlink
Automatically delete the mapped file after the mapped data was garbage collected. This can be used when the mapped file is a file-based shared-memory area (e.g. /dev/shm/...) and is mapped in direct mode to free the memory after the data was gc'd
create
Allow to create mmap file if it doesn't exist.
debug
Turn on debug printing in the NIF library.
fit
When fit is set, and opening a file with the current file size greater than requested, the memory map will be set to the requested length.
direct
Read/pread operations do not copy memory, but rather use "resource binaries" that can change content if the underlying data is changed. This is the most performant, but also has other thread-safety implications when not using atomic operations.
fixed
Don't interpret addr as a hint: place the mapping at exactly that address. The implementation aligns the given address to a multiple of the page size.
lock
Use a semaphore (read/write lock) to control state changes internally in the NIF library. This is the default option.
nocache
Pages in this mapping are not retained in the kernel's memory cache. If the system runs low on memory, pages in MAP_NOCACHE mappings will be among the first to be reclaimed. NOTE: this option is only valid for Mac OS.
nolock
Don't use a semaphore (read/write lock) to control state changes internally in the NIF library
nonblock
Don't perform read-ahead: create page tables entries only for pages that are already present in RAM. Since Linux 2.6.23, this flag causes populate to do nothing.
noreserve
Do not reserve swap space for this mapping. When swap space is reserved, one has the guarantee that it is possible to modify the mapping.
populate
Populate (prefault) page tables for a mapping. For a file mapping, this causes read-ahead on the file. This will help to reduce blocking on page faults later.
private
Create a private copy-on-write mapping. Updates to the mapping are not visible to other OS or Erlang processes mapping the same file, and are not carried through to the underlying file.
hugetlb
Allocate the mapping using "huge" pages.
huge_2mb
Used in conjunction with hugetlb to select alternative hugetlb page size of 2MB
huge_1gb
Used in conjunction with hugetlb to select alternative hugetlb page size of 1GB
read
Open for reading (this is default).
shared
Share this mapping. Updates to the mapping are visible to other OS or Erlang processes mapping the same region, and (in the case of file-backed mappings) are carried through to the underlying file. May be used in combination with sync to precisely control when updates are carried through to the underlying file.
shared_validate
This flag provides the same behavior as shared except that shared mappings ignore unknown flags in flags. By contrast, when creating a mapping using shared_validate, the kernel verifies all passed flags are known and fails the mapping with the error eopnotsupp for unknown flags. This mapping type is also required to be able to use some mapping flags (e.g., sync)
sync
This flag is available only with the shared_validate mapping type; mappings of type shared will silently ignore this flag. This flag is supported only for files supporting DAX (direct mapping of persistent memory). For other files, creating a mapping with this flag results in an eopnotsupp error. Shared file mappings with this flag provide the guarantee that while some memory is mapped writable in the address space of the OS process, it will be visible in the same file at the same offset even after the system crashes or is rebooted. In conjunction with the use of appropriate CPU instructions, this provides users of such mappings with a more efficient way of making data modifications persistent.
truncate
Truncate existing mmap file after it's open.
uninitialized
Don't clear anonymous pages. This flag is intended to improve performance on embedded devices. This flag is honored only if the kernel was configured with the CONFIG_MMAP_ALLOW_UNINITIALIZED option.
write
Open memory map for writing.
fixed_size
Don't allow the memory to be resized
{address, pos_integer()}
Open mapping at the given memory address (sets MAP_FIXED on the memory mapped file)
{chmod, pos_integer()}
Create mmap file with this mode (default: 0600)
{size, pos_integer()}
Create/access memory map on this size.
{max_inc_size, pos_integer()}
Size threshold used when automatically resizing shared memory with call to resize/1. Below this threshold the memory will double, and after this threshold, the resized memory will be increased by max_inc_size.

resource/0

-type resource() :: binary().

Functions

close(File)

-spec close(File :: mmap_file()) -> ok.

close_counters(_)

Close persistent memory-mapped file previously open with open_counters/2

dec_counter(_, CounterNum)

Decrement a counter number CounterNum in the mmap file by one and return old value.

dec_counter(_, CounterNum, Count)

Decrement a counter number CounterNum in the mmap file by Count and return old value.

flush(File)

-spec flush(File :: mmap_file()) -> ok.

Ask the OS to flush the modified memory to disk. The call is asyncronous and non-blocking. This call is not required as the OS will asynchronously flush the modified memory pages to disk lazily, but this call will trigger that process immediately.

free_block(File, Addr)

-spec free_block(File :: mmap_file(), Addr :: non_neg_integer()) ->
                    Success :: boolean() | {error, atom() | string()}.

Free data block

inc_counter(_, CounterNum)

Increment a counter number CounterNum in the mmap file by one and return old value.

inc_counter(_, CounterNum, Count)

Increment a counter number CounterNum in the mmap file by Count and return old value.

init()

init_block_storage(File, BlockSize)

-spec init_block_storage(File :: mmap_file(), BlockSize :: pos_integer()) ->
                            ok | {error, atom() | string()}.

Initialize fixed-size block storage in the shared memory.

open(FileName, Options)

-spec open(string() | binary(), [open_option()]) ->
              {ok, mmap_file(), open_extra_info()} | {error, term()}.

Open/create a memory-mapped file. If creating a new file, [create, read, write, {size, N}] options are required. For opening an existing file for writing [read, write] options are required.

open(File, Offset, Length, Options)

-spec open(File :: string() | binary(),
           Offset :: non_neg_integer(),
           Length :: non_neg_integer(),
           Options :: [open_option()]) ->
              {ok, mmap_file(), open_extra_info()} | {error, term()}.

Open/create a memory-mapped file. If creating a new file, [create, read, write] options and the Len parameter are required. For opening an existing file for writing [read, write] options are required, and Len can be 0.

open_counters(Filename)

Open a persistent memory-mapped file with space for one 64-bit integer counter

open_counters(Filename, NumCounters)

Open a persistent memory-mapped file with space for several 64-bit integer counters

patomic_add(File, Position, Value)

-spec patomic_add(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) ->
                     {ok, OldValue :: integer()} | {error, atom()} | no_return().

Perform an atomic ADD operation on a 64-bit integer value at given Position using specified argument Value. The function returns an old value at that location. This function is thread-safe and can be used for implementing persistent counters.

patomic_and(File, Position, Value)

-spec patomic_and(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) ->
                     {ok, integer()} | {error, atom()} | no_return().

Perform an atomic AND operation on a 64-bit integer value at given Position using specified argument Value. The function returns an AND'd value at that location.

patomic_cas(File, Position, OldValue, Value)

-spec patomic_cas(File :: mmap_file(), Position :: non_neg_integer(), integer(), integer()) ->
                     {boolean(), integer()} | {error, atom()} | no_return().

Perform an atomic compare and swap (CAS) operation on a 64-bit integer value at given Position using specified argument Value. The function returns a tuple {Success, OldVal}, where OldVal is the old value at that location.

patomic_or(File, Position, Value)

-spec patomic_or(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) ->
                    {ok, integer()} | {error, atom()} | no_return().

Perform an atomic OR operation on a 64-bit integer value at given Position using specified argument Value. The function returns an OR'd value at that location.

patomic_read_integer(File, Position)

-spec patomic_read_integer(File :: mmap_file(), Position :: non_neg_integer()) -> Value :: integer().

Perform an atomic load operation on a 64-bit integer value at given Position. This function is thread-safe and can be used for implementing persistent counters.

patomic_sub(File, Position, Value)

-spec patomic_sub(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) ->
                     {ok, OldValue :: integer()} | {error, atom()} | no_return().

Perform an atomic SUBTRACT operation on a 64-bit integer value at given Position using specified argument Value. The function returns an old value at that location. This function is thread-safe and can be used for implementing persistent counters.

patomic_write_integer(File, Position, Value)

-spec patomic_write_integer(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) ->
                               ok.

Perform an atomic store operation of a 64-bit integer Value at given Position. This function is thread-safe and can be used for implementing persistent counters.

patomic_xchg(File, Position, Value)

-spec patomic_xchg(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) ->
                      {ok, integer()} | {error, atom()} | no_return().

Perform an atomic EXCHANGE operation on a 64-bit integer value at given Position using specified argument Value. The function returns an old value at that location.

patomic_xor(File, Position, Value)

-spec patomic_xor(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) ->
                     {ok, integer()} | {error, atom()} | no_return().

Perform an atomic XOR operation on a 64-bit integer value at given Position using specified argument Value. The function returns an XOR'd value at that location.

position(File, Position)

-spec position(File :: mmap_file(),
               Position :: non_neg_integer() | {bof | cur | eof, Position :: integer()}) ->
                  {ok, non_neg_integer()} | {error, term()}.

Write Data bytes to a memory-mapped file at a given offset At.

pread(File, Offset, Length)

-spec pread(File :: mmap_file(), Offset :: non_neg_integer(), Length :: non_neg_integer()) ->
               {ok, binary()} | {error, term()} | eof.

Read Len bytes from a memory-mapped file at a given offset Off.

pwrite(File, Position, Data)

-spec pwrite(File :: mmap_file(), Position :: non_neg_integer(), Data :: binary()) ->
                ok | {error, term()}.

Write Data bytes to a memory-mapped file at a given offset Off.

read(File, Length)

-spec read(File :: mmap_file(), Length :: non_neg_integer()) -> {ok, binary()} | {error, term()} | eof.

Read next Len bytes from a memory-mapped file. Internally the new position within the file is incremented by Len.

read_block(File, Addr)

-spec read_block(File :: mmap_file(), Addr :: non_neg_integer()) ->
                    Data :: binary() | eof | {error, atom() | string()}.

Read data block

read_blocks(File)

-spec read_blocks(File :: mmap_file()) -> [Data :: binary()] | eof | {error, atom() | string()}.

Read data blocks

read_blocks(File, Start, Count)

-spec read_blocks(File :: mmap_file(), Start :: non_neg_integer(), Count :: pos_integer()) ->
                     {[Data :: binary()], Continuation :: integer() | eof} | {error, atom() | string()}.

Read data blocks

read_counter(_, CounterNum)

read_line(File)

-spec read_line(File :: mmap_file()) -> {ok, binary()} | {error, term()} | eof.

repair_block_storage(File)

-spec repair_block_storage(File :: mmap_file()) -> ok | {error, atom() | string()}.

Repair data block storage after unexpected close

repair_block_storage(File, Start, Count)

-spec repair_block_storage(File :: mmap_file(), Start :: non_neg_integer(), Count :: pos_integer()) ->
                              Continuation :: integer() | eof | {error, atom() | string()}.

Repair data block storage chunk of max Count elements starting with Start address

resize(File)

-spec resize(File :: mmap_file()) -> {ok, NewSize :: non_neg_integer()} | {error, atom() | string()}.

Resize shared memory. The new size will double the existing size up until the max_inc_size threshold (passed to open/4 (default 64M), otherwise incremented by max_inc_size.

resize(File_descriptor, NewSize)

-spec resize(mmap_file(), non_neg_integer()) -> {ok, NewSize :: non_neg_integer()} | {error, string()}.

Resize shared memory to a given new size.

set_counter(_, CounterNum, Value)

Set a counter number CounterNum in the mmap file and return the old value.

store_block(File, Data)

-spec store_block(File :: mmap_file(), Data :: binary()) ->
                     Addr :: non_neg_integer() | {error, atom() | string()}.

Store data block