smart_city_registry v3.2.0 SmartCity.Dataset View Source

Struct defining a dataset definition and functions for reading and writing dataset definitions to Redis.

const Dataset = {
  "id": "",                  // UUID
  "business": {              // Project Open Data Metadata Schema v1.1
    "dataTitle": "",       // user friendly (dataTitle)
    "description": "",
    "keywords": [""],
    "modifiedDate": "",
    "orgTitle": "",        // user friendly (orgTitle)
    "contactName": "",
    "contactEmail": "",
    "license": "",
    "rights": "",
    "homepage": "",
    "spatial": "",
    "temporal": "",
    "publishFrequency": "",
    "conformsToUri": "",
    "describedByUrl": "",
    "describedByMimeType": "",
    "parentDataset": "",
    "issuedDate": "",
    "language": "",
    "referenceUrls": [""],
    "categories": [""]
  },
  "technical": {
    "dataName": "",        // ~r/[a-zA-Z_]+$/
    "orgId": "",
    "orgName": "",         // ~r/[a-zA-Z_]+$/
    "systemName": "",      // ${orgName}__${dataName},
    "schema": [
      {
        "name": "",
        "type": "",
        "description": ""
      }
    ],
    "sourceUrl": "",
    "protocol": "",       // List of protocols to use. Defaults to nil. Can be [http1, http2]
    "authUrl": "",
    "sourceFormat": "",
    "sourceType": "",     // remote|stream|batch
    "cadence": "",
    "sourceQueryParams": {
      "key1": "",
      "key2": ""
    },
    "transformations": [], // ?
    "validations": [],     // ?
    "sourceHeaders": {
      "header1": "",
      "header2": ""
    }
    "authHeaders": {
      "header1": "",
      "header2": ""
    }
  },
  "_metadata": {
    "intendedUse": [],
    "expectedBenefit": []
  }
}

Link to this section Summary

Functions

Returns {:ok, dataset} with the dataset for the given id, or an error with the reason

Returns the dataset with the given id or raises an error

Returns {:ok, datasets} with all dataset definitions in the system

See get_all/0. Raises on errors

Returns {:ok, dataset_versions} with a history of all versions of the given dataset

See get_history/1. Raises on errors

Returns true if SmartCity.Dataset.Technical sourceType field is batch

Returns true if SmartCity.Dataset.Technical sourceType field is remote

Returns true if SmartCity.Dataset.Technical sourceType field is stream

Returns a new SmartCity.Dataset struct. SmartCity.Dataset.Business, SmartCity.Dataset.Technical, and SmartCity.Dataset.Metadata structs will be created along the way

Writes the dataset to history and sets the dataset as the latest definition for the given id field of the passed in dataset in Redis. Registry subscribers will be notified and have their handle_dataset/1 callback triggered

Link to this section Types

Link to this type

t() View Source
t() :: %SmartCity.Dataset{
  _metadata: SmartCity.Dataset.Metadata.t(),
  business: SmartCity.Dataset.Business.t(),
  id: String.t(),
  technical: SmartCity.Dataset.Technical.t(),
  version: String.t()
}

Link to this section Functions

Link to this function

get(id) View Source
get(id()) :: {:ok, SmartCity.Dataset.t()} | {:error, term()}

Returns {:ok, dataset} with the dataset for the given id, or an error with the reason.

Returns the dataset with the given id or raises an error.

Link to this function

get_all() View Source
get_all() :: {:ok, [SmartCity.Dataset.t()]} | {:error, term()}

Returns {:ok, datasets} with all dataset definitions in the system.

See get_all/0. Raises on errors.

Link to this function

get_history(id) View Source
get_history(id()) :: {:ok, [SmartCity.Dataset.t()]} | {:error, term()}

Returns {:ok, dataset_versions} with a history of all versions of the given dataset.

Link to this function

get_history!(id) View Source
get_history!(id()) :: [SmartCity.Dataset.t()] | no_return()

See get_history/1. Raises on errors.

Returns true if SmartCity.Dataset.Technical sourceType field is batch

Returns true if SmartCity.Dataset.Technical sourceType field is remote

Returns true if SmartCity.Dataset.Technical sourceType field is stream

Link to this function

new(msg) View Source
new(String.t() | map()) :: {:ok, SmartCity.Dataset.t()} | {:error, term()}

Returns a new SmartCity.Dataset struct. SmartCity.Dataset.Business, SmartCity.Dataset.Technical, and SmartCity.Dataset.Metadata structs will be created along the way.

Parameters

  • msg : map defining values of the struct to be created. Can be initialized by

    • map with string keys
    • map with atom keys
    • JSON
Link to this function

write(dataset) View Source
write(SmartCity.Dataset.t()) :: {:ok, id()}

Writes the dataset to history and sets the dataset as the latest definition for the given id field of the passed in dataset in Redis. Registry subscribers will be notified and have their handle_dataset/1 callback triggered.

Returns an {:ok, id} tuple() where id is the dataset id.

Parameters

  • dataset: SmartCity.Dataset struct to be written.