View Source emmap (emmap v2.1.1)
Link to this section Summary
Types
Extra information returned by the call to emmap:open/2,3
. The value of exist
true means that an existing memory map was open. The size
represents the size of the memory map that was open.
Options for opening a memory mapped file
Functions
Close persistent memory-mapped file previously open with open_counters/2
Decrement a counter number CounterNum
in the mmap file by one and return old value.
Decrement a counter number CounterNum
in the mmap file by Count
and return old value.
Ask the OS to flush the modified memory to disk. The call is asyncronous and non-blocking. This call is not required as the OS will asynchronously flush the modified memory pages to disk lazily, but this call will trigger that process immediately.
Free data block
Increment a counter number CounterNum
in the mmap file by one and return old value.
Increment a counter number CounterNum
in the mmap file by Count
and return old value.
Initialize fixed-size block storage in the shared memory.
Open/create a memory-mapped file. If creating a new file, [create, read, write, {size, N}]
options are required. For opening an existing file for writing [read, write]
options are required.
Open/create a memory-mapped file. If creating a new file, [create, read, write]
options and the Len
parameter are required. For opening an existing file for writing [read, write]
options are required, and Len
can be 0
.
Open a persistent memory-mapped file with space for one 64-bit integer counter
Open a persistent memory-mapped file with space for several 64-bit integer counters
Perform an atomic ADD operation on a 64-bit integer value at given Position
using specified argument Value
. The function returns an old value at that location. This function is thread-safe and can be used for implementing persistent counters.
Perform an atomic AND operation on a 64-bit integer value at given Position
using specified argument Value
. The function returns an AND'd value at that location.
Perform an atomic compare and swap (CAS) operation on a 64-bit integer value at given Position
using specified argument Value
. The function returns a tuple {Success, OldVal}
, where OldVal
is the old value at that location.
Perform an atomic OR operation on a 64-bit integer value at given Position
using specified argument Value
. The function returns an OR'd value at that location.
Perform an atomic load operation on a 64-bit integer value at given Position
. This function is thread-safe and can be used for implementing persistent counters.
Perform an atomic SUBTRACT operation on a 64-bit integer value at given Position
using specified argument Value
. The function returns an old value at that location. This function is thread-safe and can be used for implementing persistent counters.
Perform an atomic store operation of a 64-bit integer Value
at given Position
. This function is thread-safe and can be used for implementing persistent counters.
Perform an atomic EXCHANGE operation on a 64-bit integer value at given Position
using specified argument Value
. The function returns an old value at that location.
Perform an atomic XOR operation on a 64-bit integer value at given Position
using specified argument Value
. The function returns an XOR'd value at that location.
Write Data
bytes to a memory-mapped file at a given offset At
.
Read Len
bytes from a memory-mapped file at a given offset Off
.
Write Data
bytes to a memory-mapped file at a given offset Off
.
Read next Len
bytes from a memory-mapped file. Internally the new position within the file is incremented by Len
.
Read data block
Read data blocks
Read data blocks
Repair data block storage after unexpected close
Repair data block storage chunk of max Count elements starting with Start address
Resize shared memory. The new size will double the existing size up until the max_inc_size
threshold (passed to open/4
(default 64M), otherwise incremented by max_inc_size
.
Resize shared memory to a given new size.
Set a counter number CounterNum
in the mmap file and return the old value.
Store data block
Link to this section Types
-type mmap_file() :: #file_descriptor{}.
-type open_extra_info() :: #{exist => boolean(), size => non_neg_integer()}.
Extra information returned by the call to emmap:open/2,3
. The value of exist
true means that an existing memory map was open. The size
represents the size of the memory map that was open.
-type open_option() ::
anon | auto_unlink | create | debug | direct | fixed | hugetlb | huge_2mb | huge_1gb | lock |
nocache | nolock | nonblock | noreserve | populate | private | read | shared |
shared_validate | sync | truncate | uninitialized | write | fixed_size |
{address, pos_integer()} |
{chmod, pos_integer()} |
{size, pos_integer()} |
{max_inc_size, pos_integer()}.
Options for opening a memory mapped file:
- anon
- Anonymous mapping. The mapping is not backed by any file; its contents are initialized to zero. The offset argument should be zero.
- auto_unlink
- Automatically delete the mapped file after the mapped data was garbage collected. This can be used when the mapped file is a file-based shared-memory area (e.g.
/dev/shm/...
) and is mapped indirect
mode to free the memory after the data was gc'd - create
- Allow to create mmap file if it doesn't exist.
- debug
- Turn on debug printing in the NIF library.
- direct
- Read/pread operations do not copy memory, but rather use "resource binaries" that can change content if the underlying data is changed. This is the most performant, but also has other thread-safety implications when not using atomic operations.
- fixed
- Don't interpret addr as a hint: place the mapping at exactly that address. The implementation aligns the given address to a multiple of the page size.
- lock
- Use a semaphore (read/write lock) to control state changes internally in the NIF library. This is the default option.
- nocache
- Pages in this mapping are not retained in the kernel's memory cache. If the system runs low on memory, pages in MAP_NOCACHE mappings will be among the first to be reclaimed. NOTE: this option is only valid for Mac OS.
- nolock
- Don't use a semaphore (read/write lock) to control state changes internally in the NIF library
- nonblock
- Don't perform read-ahead: create page tables entries only for pages that are already present in RAM. Since Linux 2.6.23, this flag causes
populate
to do nothing. - noreserve
- Do not reserve swap space for this mapping. When swap space is reserved, one has the guarantee that it is possible to modify the mapping.
- populate
- Populate (prefault) page tables for a mapping. For a file mapping, this causes read-ahead on the file. This will help to reduce blocking on page faults later.
- private
- Create a private copy-on-write mapping. Updates to the mapping are not visible to other OS or Erlang processes mapping the same file, and are not carried through to the underlying file.
- hugetlb
- Allocate the mapping using "huge" pages.
- huge_2mb
- Used in conjunction with
hugetlb
to select alternative hugetlb page size of 2MB - huge_1gb
- Used in conjunction with
hugetlb
to select alternative hugetlb page size of 1GB - read
- Open for reading (this is default).
- shared
- Share this mapping. Updates to the mapping are visible to other OS or Erlang processes mapping the same region, and (in the case of file-backed mappings) are carried through to the underlying file. May be used in combination with
sync
to precisely control when updates are carried through to the underlying file. - shared_validate
- This flag provides the same behavior as
shared
except thatshared
mappings ignore unknown flags in flags. By contrast, when creating a mapping usingshared_validate
, the kernel verifies all passed flags are known and fails the mapping with the erroreopnotsupp
for unknown flags. This mapping type is also required to be able to use some mapping flags (e.g.,sync
) - sync
- This flag is available only with the
shared_validate
mapping type; mappings of typeshared
will silently ignore this flag. This flag is supported only for files supporting DAX (direct mapping of persistent memory). For other files, creating a mapping with this flag results in aneopnotsupp
error. Shared file mappings with this flag provide the guarantee that while some memory is mapped writable in the address space of the OS process, it will be visible in the same file at the same offset even after the system crashes or is rebooted. In conjunction with the use of appropriate CPU instructions, this provides users of such mappings with a more efficient way of making data modifications persistent. - truncate
- Truncate existing mmap file after it's open.
- uninitialized
- Don't clear anonymous pages. This flag is intended to improve performance on embedded devices. This flag is honored only if the kernel was configured with the
CONFIG_MMAP_ALLOW_UNINITIALIZED
option. - write
- Open memory map for writing.
- fixed_size
- Don't allow the memory to be resized
- {address, pos_integer()}
- Open mapping at the given memory address (sets
MAP_FIXED
on the memory mapped file) - {chmod, pos_integer()}
- Create mmap file with this mode (default:
0600
) - {size, pos_integer()}
- Create/access memory map on this size.
- {max_inc_size, pos_integer()}
- Size threshold used when automatically resizing shared memory with call to
resize/1
. Below this threshold the memory will double, and after this threshold, the resized memory will be increased bymax_inc_size
.
-type resource() :: binary().
Link to this section Functions
-spec close(File :: mmap_file()) -> ok.
Close persistent memory-mapped file previously open with open_counters/2
Decrement a counter number CounterNum
in the mmap file by one and return old value.
Decrement a counter number CounterNum
in the mmap file by Count
and return old value.
-spec flush(File :: mmap_file()) -> ok.
Ask the OS to flush the modified memory to disk. The call is asyncronous and non-blocking. This call is not required as the OS will asynchronously flush the modified memory pages to disk lazily, but this call will trigger that process immediately.
-spec free_block(File :: mmap_file(), Addr :: non_neg_integer()) -> Success :: boolean() | {error, atom() | string()}.
Free data block
Increment a counter number CounterNum
in the mmap file by one and return old value.
Increment a counter number CounterNum
in the mmap file by Count
and return old value.
-spec init_block_storage(File :: mmap_file(), BlockSize :: pos_integer()) -> ok | {error, atom() | string()}.
Initialize fixed-size block storage in the shared memory.
-spec open(string() | binary(), [open_option()]) -> {ok, mmap_file(), open_extra_info()} | {error, term()}.
Open/create a memory-mapped file. If creating a new file, [create, read, write, {size, N}]
options are required. For opening an existing file for writing [read, write]
options are required.
-spec open(File :: string() | binary(), Offset :: non_neg_integer(), Length :: non_neg_integer(), Options :: [open_option()]) -> {ok, mmap_file(), open_extra_info()} | {error, term()}.
Open/create a memory-mapped file. If creating a new file, [create, read, write]
options and the Len
parameter are required. For opening an existing file for writing [read, write]
options are required, and Len
can be 0
.
Open a persistent memory-mapped file with space for one 64-bit integer counter
Open a persistent memory-mapped file with space for several 64-bit integer counters
-spec patomic_add(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) -> {ok, OldValue :: integer()} | {error, atom()} | no_return().
Perform an atomic ADD operation on a 64-bit integer value at given Position
using specified argument Value
. The function returns an old value at that location. This function is thread-safe and can be used for implementing persistent counters.
-spec patomic_and(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) -> {ok, integer()} | {error, atom()} | no_return().
Perform an atomic AND operation on a 64-bit integer value at given Position
using specified argument Value
. The function returns an AND'd value at that location.
-spec patomic_cas(File :: mmap_file(), Position :: non_neg_integer(), integer(), integer()) -> {boolean(), integer()} | {error, atom()} | no_return().
Perform an atomic compare and swap (CAS) operation on a 64-bit integer value at given Position
using specified argument Value
. The function returns a tuple {Success, OldVal}
, where OldVal
is the old value at that location.
-spec patomic_or(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) -> {ok, integer()} | {error, atom()} | no_return().
Perform an atomic OR operation on a 64-bit integer value at given Position
using specified argument Value
. The function returns an OR'd value at that location.
-spec patomic_read_integer(File :: mmap_file(), Position :: non_neg_integer()) -> Value :: integer().
Perform an atomic load operation on a 64-bit integer value at given Position
. This function is thread-safe and can be used for implementing persistent counters.
-spec patomic_sub(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) -> {ok, OldValue :: integer()} | {error, atom()} | no_return().
Perform an atomic SUBTRACT operation on a 64-bit integer value at given Position
using specified argument Value
. The function returns an old value at that location. This function is thread-safe and can be used for implementing persistent counters.
-spec patomic_write_integer(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) -> ok.
Perform an atomic store operation of a 64-bit integer Value
at given Position
. This function is thread-safe and can be used for implementing persistent counters.
-spec patomic_xchg(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) -> {ok, integer()} | {error, atom()} | no_return().
Perform an atomic EXCHANGE operation on a 64-bit integer value at given Position
using specified argument Value
. The function returns an old value at that location.
-spec patomic_xor(File :: mmap_file(), Position :: non_neg_integer(), Value :: integer()) -> {ok, integer()} | {error, atom()} | no_return().
Perform an atomic XOR operation on a 64-bit integer value at given Position
using specified argument Value
. The function returns an XOR'd value at that location.
-spec position(File :: mmap_file(), Position :: non_neg_integer() | {bof | cur | eof, Position :: integer()}) -> {ok, non_neg_integer()} | {error, term()}.
Write Data
bytes to a memory-mapped file at a given offset At
.
-spec pread(File :: mmap_file(), Offset :: non_neg_integer(), Length :: non_neg_integer()) -> {ok, binary()} | {error, term()} | eof.
Read Len
bytes from a memory-mapped file at a given offset Off
.
-spec pwrite(File :: mmap_file(), Position :: non_neg_integer(), Data :: binary()) -> ok | {error, term()}.
Write Data
bytes to a memory-mapped file at a given offset Off
.
-spec read(File :: mmap_file(), Length :: non_neg_integer()) -> {ok, binary()} | {error, term()} | eof.
Read next Len
bytes from a memory-mapped file. Internally the new position within the file is incremented by Len
.
-spec read_block(File :: mmap_file(), Addr :: non_neg_integer()) -> Data :: binary() | eof | {error, atom() | string()}.
Read data block
-spec read_blocks(File :: mmap_file()) -> [Data :: binary()] | eof | {error, atom() | string()}.
Read data blocks
-spec read_blocks(File :: mmap_file(), Start :: non_neg_integer(), Count :: pos_integer()) -> {[Data :: binary()], Continuation :: integer() | eof} | {error, atom() | string()}.
Read data blocks
-spec read_line(File :: mmap_file()) -> {ok, binary()} | {error, term()} | eof.
-spec repair_block_storage(File :: mmap_file()) -> ok | {error, atom() | string()}.
Repair data block storage after unexpected close
-spec repair_block_storage(File :: mmap_file(), Start :: non_neg_integer(), Count :: pos_integer()) -> Continuation :: integer() | eof | {error, atom() | string()}.
Repair data block storage chunk of max Count elements starting with Start address
-spec resize(File :: mmap_file()) -> {ok, NewSize :: non_neg_integer()} | {error, atom() | string()}.
Resize shared memory. The new size will double the existing size up until the max_inc_size
threshold (passed to open/4
(default 64M), otherwise incremented by max_inc_size
.
-spec resize(mmap_file(), non_neg_integer()) -> {ok, NewSize :: non_neg_integer()} | {error, string()}.
Resize shared memory to a given new size.
Set a counter number CounterNum
in the mmap file and return the old value.
-spec store_block(File :: mmap_file(), Data :: binary()) -> Addr :: non_neg_integer() | {error, atom() | string()}.
Store data block