smart_city_registry v5.0.2 SmartCity.Registry.Dataset View Source

Struct defining a dataset definition and functions for reading and writing dataset definitions to Redis.

const Dataset = {
  "id": "",                  // UUID
  "business": {              // Project Open Data Metadata Schema v1.1
    "dataTitle": "",       // user friendly (dataTitle)
    "description": "",
    "keywords": [""],
    "modifiedDate": "",
    "orgTitle": "",        // user friendly (orgTitle)
    "contactName": "",
    "contactEmail": "",
    "license": "",
    "rights": "",
    "homepage": "",
    "spatial": "",
    "temporal": "",
    "publishFrequency": "",
    "conformsToUri": "",
    "describedByUrl": "",
    "describedByMimeType": "",
    "parentDataset": "",
    "issuedDate": "",
    "language": "",
    "referenceUrls": [""],
    "categories": [""]
  },
  "technical": {
    "dataName": "",        // ~r/[a-zA-Z_]+$/
    "orgId": "",
    "orgName": "",         // ~r/[a-zA-Z_]+$/
    "systemName": "",      // ${orgName}__${dataName},
    "schema": [
      {
        "name": "",
        "type": "",
        "description": ""
      }
    ],
    "sourceUrl": "",
    "protocol": "",       // List of protocols to use. Defaults to nil. Can be [http1, http2]
    "authUrl": "",
    "sourceFormat": "",
    "sourceType": "",     // remote|stream|ingest|host
    "cadence": "",
    "sourceQueryParams": {
      "key1": "",
      "key2": ""
    },
    "transformations": [], // ?
    "validations": [],     // ?
    "sourceHeaders": {
      "header1": "",
      "header2": ""
    }
    "authHeaders": {
      "header1": "",
      "header2": ""
    }
  },
  "_metadata": {
    "intendedUse": [],
    "expectedBenefit": []
  }
}

Link to this section Summary

Functions

Returns {:ok, dataset} with the dataset for the given id, or an error with the reason.

Returns the dataset with the given id or raises an error.

Returns {:ok, datasets} with all dataset definitions in the system.

See get_all/0. Raises on errors.

Returns {:ok, dataset_versions} with a history of all versions of the given dataset.

See get_history/1. Raises on errors.

Returns true if SmartCity.Registry.Dataset.Technical sourceType field is host

Returns true if SmartCity.Registry.Dataset.Technical sourceType field is ingest

Returns true if SmartCity.Registry.Dataset.Technical sourceType field is remote

Returns true if SmartCity.Registry.Dataset.Technical sourceType field is stream

Writes the dataset to history and sets the dataset as the latest definition for the given id field of the passed in dataset in Redis. Registry subscribers will be notified and have their handle_dataset/1 callback triggered.

Link to this section Types

Link to this section Functions

Returns {:ok, dataset} with the dataset for the given id, or an error with the reason.

Returns the dataset with the given id or raises an error.

Link to this function

get_all()

View Source
get_all() :: {:ok, [SmartCity.Registry.Dataset.t()]} | {:error, term()}

Returns {:ok, datasets} with all dataset definitions in the system.

See get_all/0. Raises on errors.

Link to this function

get_history(id)

View Source
get_history(id()) :: {:ok, [SmartCity.Registry.Dataset.t()]} | {:error, term()}

Returns {:ok, dataset_versions} with a history of all versions of the given dataset.

See get_history/1. Raises on errors.

Returns true if SmartCity.Registry.Dataset.Technical sourceType field is host

Returns true if SmartCity.Registry.Dataset.Technical sourceType field is ingest

Returns true if SmartCity.Registry.Dataset.Technical sourceType field is remote

Returns true if SmartCity.Registry.Dataset.Technical sourceType field is stream

Returns a new SmartCity.Registry.Dataset struct. SmartCity.Registry.Dataset.Business, SmartCity.Registry.Dataset.Technical, and SmartCity.Registry.Dataset.Metadata structs will be created along the way.

Parameters

  • msg : map defining values of the struct to be created. Can be initialized by

    • map with string keys
    • map with atom keys
    • JSON

Writes the dataset to history and sets the dataset as the latest definition for the given id field of the passed in dataset in Redis. Registry subscribers will be notified and have their handle_dataset/1 callback triggered.

Returns an {:ok, id} tuple() where id is the dataset id.

Parameters

  • dataset: SmartCity.Registry.Dataset struct to be written.