Orbis.GNSS.Data (Orbis v0.29.1)

Copy Markdown View Source

Optional fetch-and-cache layer for GNSS products (SP3, RINEX clock, broadcast navigation, IONEX).

Orbis.GNSS.Data downloads, decompresses, checksums, and records provenance for the precise- and broadcast-product files that Orbis.GNSS.SP3, Orbis.GNSS.Broadcast, and Orbis.GNSS.Positioning consume — and then hands back a local file path (or a loaded handle). It is deliberately one-directional: the numerical layers never call into this module, so a solve never depends on network availability. You fetch once, then point the solver at the cached file.

Quick start

product = Orbis.GNSS.Data.mgex_sp3(:esa, ~D[2020-06-24])

# Download (or reuse cache) and get a decompressed file path:
{:ok, path} = Orbis.GNSS.Data.fetch(product)

# Or fetch and load in one step:
{:ok, sp3} = Orbis.GNSS.Data.sp3(product)
{:ok, state} = Orbis.GNSS.SP3.position(sp3, "G01", ~N[2020-06-24 00:00:00])

Catalog

Products are identified by analysis center, content type, date, and sampling.

Supported centers and what each publishes:

  • :gfz — GFZ operational rapid SP3/CLK over HTTPS (isdc-data.gfz.de)
  • :cod — CODE MGEX final SP3/CLK and CODE IONEX over plain HTTP (ftp.aiub.unibe.ch; AIUB does not offer HTTPS for this archive)
  • :esa — ESA Navigation Office final SP3/CLK and IONEX over HTTPS (navigation-office.esa.int)
  • :igs_ult, :cod_ult, :esa_ult, :gfz_ult — ultra-rapid OPSULT SP3 products for current-day/live-latency use
  • :cod_rap — CODE rapid IONEX (COD0OPSRAP) over plain HTTP, the low-latency global ionosphere map (final lags ~1-3 weeks)
  • :cod_prd1, :cod_prd2 — CODE 1-day and 2-day predicted IONEX (COD0OPSPRD); published ahead of time, so :cod_prd1 resolves for the current/near-future UTC day
  • :igs — the IGS merged broadcast navigation file (:nav) over HTTPS from the BKG IGS archive

Content types: :sp3, :clk, :nav, :ionex, :obs (station observation data, RINEX 3 / CRINEX). Precise products and IONEX follow the IGS long-name convention AAAVPPPTTT_YYYYDDDHHMM_LEN_SMP_CNT.EXT; broadcast navigation uses the no-sampling RINEX long-name BRDC00WRD_R_YYYYDDDHHMM_01D_MN.rnx. See Orbis.GNSS.Data.Catalog.

The fetch pipeline

fetch/2 is cache-first:

  1. Resolve the canonical filename and cache path (pure, from the catalog).
  2. If the file is already cached, verify it: against the caller's :sha256 when given, otherwise against the decompressed SHA-256 recorded in the file's provenance sidecar (every downloaded file has one). A verified hit returns with no network. A corrupt hit (checksum mismatch) or an unverifiable one (no sidecar — e.g. a hand-placed file) is, online, discarded and re-downloaded; offline, a corrupt hit is terminal and an unverifiable one is returned as the best available.
  3. Otherwise (and only when not offline:) download over the cataloged HTTP(S) URL (Req, a required dependency) to memory. Gzipped products are decompressed with a gzip-bomb cap; explicitly uncompressed products are committed as downloaded. The fetch then verifies any known checksum and atomically commits the local file into the cache (temp file + rename) together with its required .provenance.json sidecar (the commit fails if the sidecar cannot be written, so a cached file always carries its integrity hash).

Offline mode

Pass offline: true (or set config :orbis, gnss_data_offline: true) to forbid all network access: a verified cache hit is returned, a corrupt hit yields {:error, {:checksum_mismatch, _, _}}, and a miss returns {:error, {:offline_miss, name}}. This is how the test suite — and any user without connectivity — runs deterministically.

Network tests

Live-archive fetching is exercised by tests tagged :network, which are excluded by default (including in CI, which has no network); the rest of the suite is fully offline and deterministic. Run the live gate manually with mix test --include network.

Options

  • :offline — when true, never touch the network (default from app config, else false)
  • :cache_dir — cache root (default :filename.basedir(:user_cache, "orbis/gnss"), overridable via config :orbis, gnss_data_cache_dir:)
  • :systems — for merged SP3 fetches, restrict the output to constellations such as [:gps] or ["G", "E"]
  • :epoch_interval_s — for merged SP3 fetches, require this exact target epoch interval; mixed-cadence products are rejected rather than unioned onto a corrupt grid
  • :sha256 — expected SHA-256 (hex) of the decompressed file; verified on both cache hits and fresh downloads
  • :max_decompressed_bytes — gzip-bomb cap (default 500 MiB)
  • :timeout_ms — per-attempt network timeout (default 30_000)
  • :retries — attempts for transient network errors (default 3)
  • :backoff_ms — base backoff between retries, doubled each attempt (default 500)
  • :max_compressed_bytes — cap on the compressed payload buffered into memory while downloading (default 64 MiB)

Typed errors

Every failure is a tagged tuple so callers can branch on it:

  • {:error, {:offline_miss, name}}offline: true and not cached
  • {:error, {:checksum_mismatch, expected, got}} — digest verification failed
  • {:error, {:unsupported_product, detail}} — unknown center/content/sample, or a host outside the catalog
  • {:error, {:no_open_mirror, {center, content}}} — the product was removed because no verified anonymous HTTP(S) mirror is known
  • {:error, :req_not_available} — HTTP client downloads are disabled by config
  • {:error, {:http_status, code}} — non-2xx HTTP response
  • {:error, {:redirect_not_allowed, code}} — a 3xx redirect was refused (redirects are not followed, to keep the SSRF allow-list intact)
  • {:error, {:file_not_found, url}} — 404 / missing on the archive
  • {:error, {:network, detail}} — connection/timeout/DNS failure
  • {:error, {:download_size_exceeded, max, got}} — download payload cap hit
  • {:error, {:decompress_failed, reason}} — corrupt gzip
  • {:error, {:decompress_size_exceeded, max, got}} — gzip-bomb cap hit
  • {:error, {:cache_dir_not_writable, reason}} — cannot create/write cache
  • {:error, {:provenance_write_failed, reason}} — the product downloaded but its required provenance sidecar could not be written (the product is rolled back so nothing unverifiable is left in the cache)
  • {:error, {:unsafe_cache_name, name}} — filename failed path-safety checks
  • {:error, {:temp_file_error, reason}} — temp write/rename failure

Summary

Types

A fetch error, always a tagged tuple. See the module docs.

Functions

Fetch a broadcast-navigation product and load it into an Orbis.GNSS.Broadcast handle.

Fetch a product, returning the local path to its decompressed file.

Fetch a CODE rapid or predicted IONEX map for a target day, walking candidate days newest first.

Fetch SP3 products from several centers and merge the available ones.

Fetch the merged current-day SP3 product from several centers and persist it to path in one call — the live-latency workflow's entry point.

Build an MGEX clock (RINEX clock) product. Defaults to 30S sampling.

Build an IONEX (global ionosphere TEC map) product.

Build a broadcast-navigation (merged multi-GNSS RINEX NAV) product.

Build an MGEX SP3 (precise orbit) product for a center and date.

Fetch a station observation product and load it into an Orbis.GNSS.RINEX.Observations handle.

Build an ultra-rapid OPS clock product.

Build an ultra-rapid OPS SP3 product.

Build a CODE predicted IONEX product (COD0OPSPRD) for a UTC day.

Build a Product for any center/content/date/sample, returning a tuple.

Build a Product for any center/content/date/sample with product-specific options such as issue: "0600" for ultra-rapid products.

Build the CODE rapid IONEX product (COD0OPSRAP) for a UTC day.

Fetch an SP3 product and load it into an Orbis.GNSS.SP3 handle.

Build a daily station observation product (RINEX 3 CRINEX, 30 s default).

Write an Orbis.GNSS.SP3 product to path — the inverse of the fetch layer's read.

Types

error()

@type error() :: {:error, term()}

A fetch error, always a tagged tuple. See the module docs.

Functions

broadcast(product, opts \\ [])

@spec broadcast(
  Orbis.GNSS.Data.Product.t(),
  keyword()
) :: {:ok, Orbis.GNSS.Broadcast.t()} | error()

Fetch a broadcast-navigation product and load it into an Orbis.GNSS.Broadcast handle.

Returns {:ok, %Orbis.GNSS.Broadcast{}} or a typed error.

fetch(product, opts \\ [])

@spec fetch(
  Orbis.GNSS.Data.Product.t(),
  keyword()
) :: {:ok, String.t()} | error()

Fetch a product, returning the local path to its decompressed file.

Cache-first: a verified cache hit returns immediately with no network. See the module docs for the full pipeline, options, and error taxonomy.

Returns {:ok, path} or a typed {:error, _}.

fetch_ionex(center, target, opts \\ [])

@spec fetch_ionex(atom(), Date.t() | NaiveDateTime.t() | DateTime.t(), keyword()) ::
  {:ok, String.t()} | error()

Fetch a CODE rapid or predicted IONEX map for a target day, walking candidate days newest first.

The rapid map (:cod_rap) lands a day or two late and the predicted maps (:cod_prd1, :cod_prd2) are published ahead of their target day, so the freshest file present may be for a slightly earlier day than first requested. This tries Orbis.GNSS.Data.Catalog.gim_date_candidates/3 newest first through the ordinary fetch/2 path and returns the first hit, or the last error when every candidate misses (preserving the {:offline_miss, _} taxonomy offline).

Returns {:ok, path} or a typed {:error, _}.

fetch_merged_sp3(target, centers, opts \\ [])

@spec fetch_merged_sp3(
  Date.t() | NaiveDateTime.t() | DateTime.t(),
  [atom()],
  keyword()
) ::
  {:ok, Orbis.GNSS.SP3.t(), map()}
  | {:error, {:no_products, [map()]}}
  | {:error, {:incompatible_sources, map()}}
  | error()

Fetch SP3 products from several centers and merge the available ones.

centers are tried in precedence order. A missing or not-yet-published center is recorded in the returned report and does not abort the call. For ultra-rapid centers and timestamp targets, Orbis tries issue candidates newest to oldest until it finds a cached/downloadable product, so callers near the publication frontier can fall back from a not-yet-landed latest issue.

Returns:

  • {:ok, merged, report} when at least one center contributes. With one contributor, merged is that source SP3 and report.single_product? is true.
  • {:error, {:no_products, reasons}} when every center is absent or fails.
  • {:error, {:incompatible_sources, %{centers:, reason:}}} when the fetched centers exist but cannot be combined (their SP3 headers disagree on time scale or coordinate system, which the merge refuses rather than mixing frames).

The report includes :contributors, :absent, :source_count, :single_product?, and the normal SP3 merge audit keys (:quarantined, :single_source, :position_outliers).

Examples

iex> cache = System.tmp_dir!()
iex> {:error, {:no_products, reasons}} =
...>   Orbis.GNSS.Data.fetch_merged_sp3(~D[2024-09-03], [:igs_ult], offline: true, cache_dir: cache)
iex> [%{center: :igs_ult, reason: :offline_miss}] = reasons

fetch_merged_sp3_file(target, centers, path, opts \\ [])

@spec fetch_merged_sp3_file(
  Date.t() | NaiveDateTime.t() | DateTime.t(),
  [atom()],
  Path.t(),
  keyword()
) ::
  {:ok, Path.t(), map()}
  | {:error, {:no_products, [map()]}}
  | {:error, {:incompatible_sources, map()}}
  | error()

Fetch the merged current-day SP3 product from several centers and persist it to path in one call — the live-latency workflow's entry point.

This composes fetch_merged_sp3/3 (fetch + merge, in the numeric layer's in-memory form) with write_sp3/3 (the data layer's atomic file write), so the result is a standard, self-contained SP3 file on disk. That file is exactly what the cache / Orbis.GNSS.SP3 / Orbis.GNSS.Positioning layers consume, which unblocks the end-to-end path:

merged current-day SP3 -> standard file -> Observables / Positioning

Because the numeric layer never reaches for I/O, a later solve reads the cached file with no network — fetch once here, then point the solver at path.

target, centers, and the fetch/merge options behave exactly as in fetch_merged_sp3/3 (ultra-rapid issue selection, per-center precedence, offline mode, :cache_dir, the SP3 merge/2 tuning keys, …). Write options are shared with write_sp3/3:

  • :gzip — gzip-compress the written file (default false); pair it with a .gz extension on path.

Returns:

  • {:ok, path, report} — the merged product was written; report is the same merge/contributor audit fetch_merged_sp3/3 returns (:contributors, :absent, :source_count, :single_product?, :quarantined, …).
  • {:error, {:no_products, reasons}} / {:error, {:incompatible_sources, _}} — propagated from the fetch/merge step; nothing is written.
  • any write_sp3/3 error (e.g. {:error, {:write_failed, reason}}) — the product merged but could not be persisted.

Examples

iex> cache = System.tmp_dir!()
iex> path = Path.join(cache, "no_such_product.sp3")
iex> {:error, {:no_products, [%{center: :igs_ult, reason: :offline_miss}]}} =
...>   Orbis.GNSS.Data.fetch_merged_sp3_file(~D[2024-09-03], [:igs_ult], path,
...>     offline: true, cache_dir: cache)
iex> File.exists?(path)
false

mgex_clk(center, date, opts \\ [])

@spec mgex_clk(atom(), Date.t(), keyword()) :: Orbis.GNSS.Data.Product.t()

Build an MGEX clock (RINEX clock) product. Defaults to 30S sampling.

mgex_ionex(center, date, opts \\ [])

@spec mgex_ionex(atom(), Date.t(), keyword()) :: Orbis.GNSS.Data.Product.t()

Build an IONEX (global ionosphere TEC map) product.

Served by :esa (ESA0OPSFIN). IONEX maps are sub-daily, so the catalog sampling defaults to 02H; pass sample: to override.

fetch/2 on the returned product is a single-shot fetch of that exact day. For the lower-latency CODE rapid (:cod_rap) and predicted (:cod_prd1, :cod_prd2) maps, whose availability is time-sensitive, use fetch_ionex/3 to fetch with a latest-available-day candidate fallback.

Examples

iex> p = Orbis.GNSS.Data.mgex_ionex(:esa, ~D[2024-06-24])
iex> Orbis.GNSS.Data.Product.canonical_filename(p)
{:ok, "ESA0OPSFIN_20241760000_01D_02H_GIM.INX"}

mgex_nav(center, date, opts \\ [])

@spec mgex_nav(atom(), Date.t(), keyword()) :: Orbis.GNSS.Data.Product.t()

Build a broadcast-navigation (merged multi-GNSS RINEX NAV) product.

Only :igs publishes this product (BRDC00WRD_R_..._MN.rnx). The RINEX navigation long-name carries no sampling field, so the sample argument is not part of the filename; it defaults to 01D purely to satisfy validation.

Examples

iex> p = Orbis.GNSS.Data.mgex_nav(:igs, ~D[2020-06-25])
iex> Orbis.GNSS.Data.Product.canonical_filename(p)
{:ok, "BRDC00WRD_R_20201770000_01D_MN.rnx"}

mgex_sp3(center, date, opts \\ [])

@spec mgex_sp3(atom(), Date.t(), keyword()) :: Orbis.GNSS.Data.Product.t()

Build an MGEX SP3 (precise orbit) product for a center and date.

Defaults to 05M (5-minute) sampling; override with sample:.

Examples

iex> p = Orbis.GNSS.Data.mgex_sp3(:esa, ~D[2020-06-24])
iex> p.center
:esa
iex> Orbis.GNSS.Data.Product.canonical_filename(p)
{:ok, "ESA0MGNFIN_20201760000_01D_05M_ORB.SP3"}

observations(product, opts \\ [])

@spec observations(
  Orbis.GNSS.Data.Product.t(),
  keyword()
) :: {:ok, Orbis.GNSS.RINEX.Observations.t()} | error()

Fetch a station observation product and load it into an Orbis.GNSS.RINEX.Observations handle.

fetch/2 gunzips the .gz; the committed cache file is the (still Hatanaka) CRINEX text, which Orbis.GNSS.RINEX.Observations.load/1 decodes before parsing. Returns {:ok, %Orbis.GNSS.RINEX.Observations{}} or a typed error.

ops_ultra_clk(center, date_or_target, opts \\ [])

@spec ops_ultra_clk(atom(), Date.t() | NaiveDateTime.t() | DateTime.t(), keyword()) ::
  Orbis.GNSS.Data.Product.t()

Build an ultra-rapid OPS clock product.

Issue-time behavior matches ops_ultra_sp3/3. The current HTTPS catalog does not include an ultra-rapid clock product; retired clock products raise {:no_open_mirror, {center, :clk}}.

ops_ultra_sp3(center, date_or_target, opts \\ [])

@spec ops_ultra_sp3(atom(), Date.t() | NaiveDateTime.t() | DateTime.t(), keyword()) ::
  Orbis.GNSS.Data.Product.t()

Build an ultra-rapid OPS SP3 product.

Ultra-rapid products are two-day (02D) files issued several times per day, with roughly one observed day and one predicted day. Pass a Date with an explicit issue: (defaults to "0000"), or pass a NaiveDateTime / DateTime target and Orbis will select the latest issue not after that time. If :available_issues is supplied, selection falls back to the newest issue present in that list.

Examples

iex> p = Orbis.GNSS.Data.ops_ultra_sp3(:igs_ult, ~D[2024-09-03], issue: "0600")
iex> Orbis.GNSS.Data.Product.canonical_filename(p)
{:ok, "IGS0OPSULT_20242470600_02D_15M_ORB.SP3"}

iex> available = [{~D[2024-09-03], "0000"}, {~D[2024-09-03], "0600"}]
iex> p = Orbis.GNSS.Data.ops_ultra_sp3(:gfz_ult, ~N[2024-09-03 13:00:00], available_issues: available)
iex> Orbis.GNSS.Data.Product.canonical_filename(p)
{:ok, "GFZ0OPSULT_20242470600_02D_05M_ORB.SP3"}

predicted_ionex(center, date, opts \\ [])

@spec predicted_ionex(atom(), Date.t(), keyword()) :: Orbis.GNSS.Data.Product.t()

Build a CODE predicted IONEX product (COD0OPSPRD) for a UTC day.

CODE publishes a single predicted GIM; the catalog distinguishes horizons by the day each alias targets. center is :cod_prd1 (1-day-ahead, the current/near-future day) or :cod_prd2 (2-day-ahead, the day after date). Predicted maps are published before their target day starts, so a predicted product is resolvable for the current/near-future UTC day. Resolves on the AIUB /CODE root over plain HTTP and defaults to 01H sampling.

To fetch with a latest-available-day fallback (walking candidate days newest-first), use fetch_ionex/3 rather than fetch/2 on a single product.

Examples

iex> p = Orbis.GNSS.Data.predicted_ionex(:cod_prd1, ~D[2026-06-14])
iex> Orbis.GNSS.Data.Product.canonical_filename(p)
{:ok, "COD0OPSPRD_20261650000_01D_01H_GIM.INX"}

iex> p = Orbis.GNSS.Data.predicted_ionex(:cod_prd2, ~D[2026-06-14])
iex> Orbis.GNSS.Data.Product.canonical_filename(p)
{:ok, "COD0OPSPRD_20261660000_01D_01H_GIM.INX"}

product(center, content, date, sample)

Build a Product for any center/content/date/sample, returning a tuple.

Use this instead of the bang builders when the inputs may be invalid.

product(center, content, date, sample, opts)

Build a Product for any center/content/date/sample with product-specific options such as issue: "0600" for ultra-rapid products.

rapid_ionex(date, opts \\ [])

@spec rapid_ionex(
  Date.t(),
  keyword()
) :: Orbis.GNSS.Data.Product.t()

Build the CODE rapid IONEX product (COD0OPSRAP) for a UTC day.

The rapid global ionosphere map is the low-latency CODE GIM; the final COD0OPSFIN map lags one to three weeks. It resolves on the AIUB /CODE root over plain HTTP and defaults to 01H sampling.

The rapid map is a rolling-recent window on AIUB: the current day is not yet published and files older than roughly three days roll off the /CODE root, so fetch/2 on a single day can :file_not_found on either edge. For the freshest available map prefer fetch_ionex/3, which walks candidate days newest-first; for same-day use prefer the predicted map (:cod_prd1), which is published before its day starts.

Examples

iex> p = Orbis.GNSS.Data.rapid_ionex(~D[2026-06-13])
iex> Orbis.GNSS.Data.Product.canonical_filename(p)
{:ok, "COD0OPSRAP_20261640000_01D_01H_GIM.INX"}

sp3(product, opts \\ [])

@spec sp3(
  Orbis.GNSS.Data.Product.t(),
  keyword()
) :: {:ok, Orbis.GNSS.SP3.t()} | error()

Fetch an SP3 product and load it into an Orbis.GNSS.SP3 handle.

Equivalent to fetch/2 followed by Orbis.GNSS.SP3.load/1. Returns {:ok, %Orbis.GNSS.SP3{}} or a typed error.

station_obs(station, date, opts \\ [])

@spec station_obs(String.t(), Date.t(), keyword()) :: Orbis.GNSS.Data.Product.t()

Build a daily station observation product (RINEX 3 CRINEX, 30 s default).

Station observation files are keyed by a 9-character site id (e.g. "WTZR00DEU"), not an analysis-center token, and resolve on the BKG IGS observation tree. Override the sampling with sample:.

Examples

iex> p = Orbis.GNSS.Data.station_obs("WTZR00DEU", ~D[2020-06-25])
iex> Orbis.GNSS.Data.Product.canonical_filename(p)
{:ok, "WTZR00DEU_R_20201770000_01D_30S_MO.crx"}

write_sp3(sp3, path, opts \\ [])

@spec write_sp3(Orbis.GNSS.SP3.t(), Path.t(), keyword()) :: {:ok, Path.t()} | error()

Write an Orbis.GNSS.SP3 product to path — the inverse of the fetch layer's read.

The product is serialized with Orbis.GNSS.SP3.to_iodata/2 (pure) and committed atomically: the bytes are written to a temporary file in the same directory, then File.rename/2d into place (atomic on POSIX), so a reader never observes a half-written file. The unblocking case is persisting a merge/2 product, which is otherwise only an in-memory handle.

Returns {:ok, path} or {:error, reason}.

Options

  • :gzip — gzip-compress the output, matching the gzipped archive products (default false). Pair it with a .gz extension on path.

Examples

{:ok, merged, _report} = Orbis.GNSS.Data.fetch_merged_sp3(date, [:igs_ult, :gfz_ult])
{:ok, _path} = Orbis.GNSS.Data.write_sp3(merged, "/tmp/merged.sp3")
{:ok, _path} = Orbis.GNSS.Data.write_sp3(merged, "/tmp/merged.sp3.gz", gzip: true)