Response shape normalization applied before field-mapping extraction.
Some exchanges return responses in wire shapes the unified parsers don't expect. For example:
- BitMEX returns
[%{ticker}]instead of%{ticker}forfetch_ticker - BitMEX returns a flat order list instead of
%{bids: [], asks: []}forfetch_order_book - Deribit OHLCV is column-oriented (
%{"open" => [...], "close" => [...]}) rather than rows
Scope relative to authored normalization.response_envelopes
Authored specs carry per-method envelope key paths (e.g. key: "result"
with fallback_keys/default). That covers the simple "reach into the response
envelope" cases the :extract_path* family handled manually in the reference
implementation — prefer the authored envelopes for those.
This module retains the residual transforms that are not expressible as a
static envelope key path because they reshape the wire data itself:
list/map unwrapping, flat-list → order-book, positional-array → maps,
column-oriented → rows, and composition. The :extract_path* variants are
kept too, for per-exchange overrides and :compose chains where an envelope
hop must precede a reshape.
Usage
Configure a :response_transformer on an endpoint config; Bourse.Dispatch
applies it to the response body before the body reaches the parser:
%{
name: :fetch_order_book,
path: "/orderBook/L2",
response_transformer: :order_book_from_flat_list
}Available transformers:
{:extract_path, path}— extracts nested data via key path{:extract_path_unwrap, path}— extracts via path, then unwraps single-element list (nilfor empty list){:extract_path_unwrap_map, path}— extracts via path, then unwraps single-key map to its value{:extract_path_unwrap_merge, path, merge_keys}— like:extract_path_unwrap, merging envelope keys into the result (inner fields win viaput_new):unwrap_single_element_list— unwraps[item]toitem(returns[]for empty list):unwrap_single_element_map— unwraps%{key => value}tovalue(0 or 2+ key maps unchanged):extract_first_list_value— first value that is a list in a map (ignores metadata keys like"last"):order_book_from_flat_list— converts a flat order list to%{bids: [], asks: []}{:positional_to_maps, field_names}— converts inner positional arrays to maps by zipping with field names{:transpose_columns_to_rows, column_keys}— transposes column-oriented map to a list of rows{:compose, [transformer]}— chains transformers left-to-right
Summary
Functions
Extracts the first list value from a map, ignoring non-list metadata keys.
Extracts nested data from a response envelope using a key path.
Converts a flat order list to structured order book format.
Converts positional arrays in a list to maps by zipping with field names.
Applies a transformer to the response body.
Transposes column-oriented data into a list of rows.
Unwraps a single-element list to its element.
Unwraps a single-key map to its value.
Types
@type transformer() :: :unwrap_single_element_list | :unwrap_single_element_map | :extract_first_list_value | :order_book_from_flat_list | {:extract_path, [String.t()]} | {:extract_path_unwrap, [String.t()]} | {:extract_path_unwrap_map, [String.t()]} | {:extract_path_unwrap_merge, [String.t()], [String.t()]} | {:positional_to_maps, [String.t()]} | {:transpose_columns_to_rows, [String.t()]} | {:compose, [transformer()]} | nil
Functions
Extracts the first list value from a map, ignoring non-list metadata keys.
Used when an API returns a map with a dynamic data key alongside metadata keys.
E.g., Kraken's fetch_trades returns %{"XXBTZUSD" => [[...trades...]], "last" => "1541439421"}
where "last" is metadata and the pair key holds the actual data.
Returns the map unchanged when no list values exist.
Examples
iex> Bourse.ResponseTransformer.extract_first_list_value(%{"XXBTZUSD" => [[1], [2]], "last" => "123"})
[[1], [2]]
iex> Bourse.ResponseTransformer.extract_first_list_value(%{"XXBTZUSD" => [[1]]})
[[1]]
iex> Bourse.ResponseTransformer.extract_first_list_value(%{"a" => 1, "b" => 2})
%{"a" => 1, "b" => 2}
iex> Bourse.ResponseTransformer.extract_first_list_value("not a map")
"not a map"Extracts nested data from a response envelope using a key path.
Walks into nested maps following the given keys. Returns the data at the final key, or the current level's data if a key is not found (i.e., stops walking and returns whatever map it reached).
Examples
iex> body = %{"retCode" => 0, "result" => %{"list" => [[1, 2, 3]]}}
iex> Bourse.ResponseTransformer.extract_path(body, ["result", "list"])
[[1, 2, 3]]
iex> Bourse.ResponseTransformer.extract_path(%{"data" => "test"}, [])
%{"data" => "test"}
iex> Bourse.ResponseTransformer.extract_path(%{"a" => 1}, ["missing"])
%{"a" => 1}
iex> Bourse.ResponseTransformer.extract_path(%{"result" => %{"data" => "test"}}, ["result", "missing"])
%{"data" => "test"}Converts a flat order list to structured order book format.
BitMEX returns orders as a flat list with a "side" field:
[%{"side" => "Sell", "price" => 100, "size" => 10}, %{"side" => "Buy", ...}]
This transforms it to unified format:
%{"bids" => [[price, size], ...], "asks" => [[price, size], ...]}
Examples
iex> orders = [
...> %{"side" => "Sell", "price" => 100.5, "size" => 10},
...> %{"side" => "Buy", "price" => 99.5, "size" => 20}
...> ]
iex> Bourse.ResponseTransformer.order_book_from_flat_list(orders)
%{"bids" => [[99.5, 20]], "asks" => [[100.5, 10]]}Converts positional arrays in a list to maps by zipping with field names.
Used when an API returns trades (or similar) as positional arrays instead of maps.
E.g., Kraken's fetchTrades returns [["50000", "0.01", 1710327959, "b", "m", "", 123], ...]
Examples
iex> Bourse.ResponseTransformer.positional_to_maps(
...> [["50000", "0.01", "b"], ["51000", "0.02", "s"]],
...> ["price", "amount", "side"]
...> )
[%{"price" => "50000", "amount" => "0.01", "side" => "b"}, %{"price" => "51000", "amount" => "0.02", "side" => "s"}]
iex> Bourse.ResponseTransformer.positional_to_maps("not a list", ["a"])
"not a list"@spec transform(term(), transformer()) :: term()
Applies a transformer to the response body.
Returns the transformed body, or the original body if no transformer is specified.
Transposes column-oriented data into a list of rows.
Some exchanges (e.g., Deribit OHLCV) return data as a map of columns:
%{"ticks" => [1, 2], "open" => [10, 20]} instead of rows [[1, 10], [2, 20]].
Column keys determine the order of elements in each row. Missing keys or non-list column values cause the data to be returned unchanged (defensive).
Examples
iex> Bourse.ResponseTransformer.transpose_columns_to_rows(
...> %{"a" => [1, 2], "b" => [3, 4]},
...> ["a", "b"]
...> )
[[1, 3], [2, 4]]
iex> Bourse.ResponseTransformer.transpose_columns_to_rows("not a map", ["a"])
"not a map"Unwraps a single-element list to its element.
Used when an API returns [item] but the unified API expects item.
Examples
iex> Bourse.ResponseTransformer.unwrap_single_element_list([%{"symbol" => "BTC"}])
%{"symbol" => "BTC"}
iex> Bourse.ResponseTransformer.unwrap_single_element_list([])
[]
iex> Bourse.ResponseTransformer.unwrap_single_element_list([%{a: 1}, %{b: 2}])
[%{a: 1}, %{b: 2}]Unwraps a single-key map to its value.
Used when an API returns %{"KEY" => value} but the unified API expects value
directly. E.g., Kraken's /public/Ticker returns {"result": {"XXBTUSD": {...}}}
where the key is the symbol name (not a meaningful container).
Returns maps with 0 or 2+ keys unchanged.
Examples
iex> Bourse.ResponseTransformer.unwrap_single_element_map(%{"XXBTUSD" => %{"a" => [1]}})
%{"a" => [1]}
iex> Bourse.ResponseTransformer.unwrap_single_element_map(%{})
%{}
iex> Bourse.ResponseTransformer.unwrap_single_element_map(%{"a" => 1, "b" => 2})
%{"a" => 1, "b" => 2}
iex> Bourse.ResponseTransformer.unwrap_single_element_map("not a map")
"not a map"