ElasticsearchEx.Deserializer (Elasticsearch v1.9.0)

View Source

Converts Elasticsearch data structures (documents, sources, or hits) into Elixir data structures.

This module deserializes Elasticsearch responses, transforming string-keyed maps into Elixir types such as Range, Date.Range, Date, or DateTime based on the provided mappings. It supports nested maps, lists, and streams, and allows custom key transformation via a key_fun.

Supported Types

  • "binary": Decodes base64-encoded strings.
  • "integer_range", "long_range": Converts %{gte: int, lte: int} to Range.t().
  • "date_range": Converts %{gte: date_str, lte: date_str} to Date.Range.t() for "strict_date" format.
  • "date": Converts strings to Date.t() or DateTime.t() for "strict_date" or "strict_date_time" formats.

Examples

# Deserialize a document with a date field
document = %{"_index" => "test", "_source" => %{"date" => "2023-01-01"}}
mapping = %{"properties" => %{"date" => %{"type" => "date", "format" => "strict_date"}}}
deserialize(document, mapping)
# => %{"_index" => "test", "_source" => %{"date" => ~D[2023-01-01]}}

# Deserialize a stream of documents
stream = Stream.map([document], & &1)
Stream.run(deserialize(stream, mapping))

Summary

Types

A mapper, either a function that takes an index name and returns mappings, or a mappings map.

Elasticsearch mappings, a map with string keys describing field types and formats.

Input data: a stream, list of documents, a single document, a document source, or a field value.

Functions

Deserializes a list of documents, a single document, a document source, or a stream.

Deserializes a field value based on its Elasticsearch mapping.

Types

mapper()

@type mapper() :: (binary() -> mappings()) | mappings()

A mapper, either a function that takes an index name and returns mappings, or a mappings map.

mappings()

@type mappings() :: %{required(binary()) => any()}

Elasticsearch mappings, a map with string keys describing field types and formats.

Must contain a "properties" key when provided as a map.

value()

@type value() :: Enumerable.t() | %{required(binary()) => any()}

Input data: a stream, list of documents, a single document, a document source, or a field value.

Documents are maps with "_index" and "_source" keys. Sources are string-keyed maps.

Functions

deserialize(value, mapper, key_fun \\ &Function.identity/1)

@spec deserialize(value(), mapper(), (binary() -> any())) :: value()

Deserializes a list of documents, a single document, a document source, or a stream.

  • value: The input data (stream, list, document, or source map).
  • mapper: A function that takes an index name and returns mappings, or a mappings map with a "properties" key.
  • key_fun: A function to transform keys in deserialized maps (defaults to Function.identity/1).

Returns a stream for stream input, a list for list input, or a map for document/source input.

Examples

# Deserialize a document
document = %{"_index" => "test", "_source" => %{"field" => "data"}}
mapping = %{"properties" => %{"field" => %{"type" => "text"}}}
deserialize(document, mapping)
# => %{"_index" => "test", "_source" => %{"field" => "data"}}

# With key transformation
deserialize(document, mapping, &String.to_atom/1)
# => %{"_index" => "test", "_source" => %{field: "data"}}

# Handle invalid date gracefully
document = %{"_index" => "test", "_source" => %{"date" => "invalid"}}
mapping = %{"properties" => %{"date" => %{"type" => "date", "format" => "strict_date"}}}
deserialize(document, mapping)
# => %{"_index" => "test", "_source" => %{"date" => "invalid"}}

deserialize_field(value, mapping, key_fun \\ &Function.identity/1)

@spec deserialize_field(any(), mappings(), (binary() -> any())) :: any()

Deserializes a field value based on its Elasticsearch mapping.

  • value: The field value (map, list, or scalar).
  • mapping: The field’s mapping, typically containing "type" and optional "format".
  • key_fun: A function to transform keys in deserialized maps (defaults to Function.identity/1).

Supports specific types like "binary", "integer_range", "long_range", "date_range", and "date". Non-matching values or invalid formats are returned unchanged.

Examples

# Deserialize a binary field
deserialize_field("SGVsbG8=", %{"type" => "binary"})
# => "Hello"

# Deserialize a date range
deserialize_field(%{"gte" => "2023-01-01", "lte" => "2023-01-02"}, %{
  "type" => "date_range",
  "format" => "strict_date"
})
# => Date.Range.t()