This module calls external NIF in C in order to fetch mseed data efficiently
Summary
Functions
Fetch miniSeed records corresponding the the DataFile given in parameter. This will use a NIF to make efficient use of the libmseed. The resulting tupple has necessary informations to be able to sort a list of them later on
Function called on module load. It declares the shared library that contains the functions our NIF defines.
Fetch all miniseed records in parallel. Stream the output data to caller function The idea is to minimize latency when fetching data Look for https://hexdocs.pm/elixir/Stream.html Read https://medium.com/elemental-elixir/processing-very-large-text-files-in-elixir-b631792eed59
Functions
@spec fetch_msrecords(Wsdataselect.DataFile.t()) :: {:ok, String.t(), DateTime.t(), :binary} | {:error, String.t(), DateTime.t(), String.t()}
Fetch miniSeed records corresponding the the DataFile given in parameter. This will use a NIF to make efficient use of the libmseed. The resulting tupple has necessary informations to be able to sort a list of them later on
@spec filter_and_sort_results([tuple()]) :: [{String.t(), DateTime.t(), :binary}]
Function called on module load. It declares the shared library that contains the functions our NIF defines.
@spec stream_data([Wsdataselect.DataFile.t()]) :: {:ok, [{String.t(), DateTime.t(), :binary}]} | {:error, term()}
Fetch all miniseed records in parallel. Stream the output data to caller function The idea is to minimize latency when fetching data Look for https://hexdocs.pm/elixir/Stream.html Read https://medium.com/elemental-elixir/processing-very-large-text-files-in-elixir-b631792eed59
Files are first split into ~max_concurrency chunks so that each worker
processes a batch of files sequentially, cutting down on the number of
spawned tasks and the associated message-passing overhead while keeping the
same degree of parallelism.