smart_city v5.0.2 SmartCity.Ingestion View Source

Struct defining an ingestion update event.

const Ingestion = {
  "id": "",
  "allow_duplicates": boolean,
  "cadence": "",
  "extractSteps": [],          
  "schema": [],
  "targetDatasetId": "",          
  "sourceFormat": "",
  "topLevelSelector": ""  
}

Link to this section Summary

Functions

Returns a new SmartCity.Ingestion. Can be created from Map with string or atom keys. Raises an ArgumentError when passed invalid input

Link to this section Types

Link to this type

not_required(type)

View Source
not_required(type) :: type | nil
Link to this type

t()

View Source
t() :: %SmartCity.Ingestion{
  allow_duplicates: not_required(boolean()),
  cadence: not_required(String.t()),
  extractSteps: [map()],
  id: String.t(),
  schema: [map()],
  sourceFormat: String.t(),
  targetDataset: String.t(),
  topLevelSelector: not_required(String.t())
}

Link to this section Functions

Link to this function

get(struct, key, default \\ nil)

View Source

Returns a new SmartCity.Ingestion. Can be created from Map with string or atom keys. Raises an ArgumentError when passed invalid input

Parameters

  • msg: Map with string or atom keys that defines the ingestion metadata

Required Keys:

- targetDataset
- sourceFormat
  • sourceType will default to "remote"
  • cadence will default to "never"
  • allow_duplicates will default to true