View Source Explorer.TensorFrame (Explorer v0.5.5)

TensorFrame is a representation of Explorer.DataFrame that is designed to work inside Nx's defn expressions.

For example, imagine the following defn:

defn add_columns(tf) do
  tf[:a] + tf[:b]
end

We can now pass a DataFrame as argument:

iex> add_columns(Explorer.DataFrame.new(a: [11, 12], b: [21, 22]))
#Nx.Tensor<
  s64[2]
  [32, 34]
>

Passing an Explorer.DataFrame to a defn will automatically convert it to a TensorFrame. The TensorFrame will lazily build tensors out of the used dataframe fields.

stack-and-concatenating

Stack and concatenating

Due to the integration with Nx, you can also pass dataframes into Nx.stack/2 and Nx.concatenate and they will be automatically converted to tensors. This makes it easy to pass dataframes into neural networks and other computationally intensive algorithms:

iex> Nx.concatenate(Explorer.DataFrame.new(a: [11, 12], b: [21, 22]))
#Nx.Tensor<
  s64[4]
  [11, 12, 21, 22]
>

iex> Nx.stack(Explorer.DataFrame.new(a: [11, 12], b: [21, 22]))
#Nx.Tensor<
  s64[2][2]
  [
    [11, 12],
    [21, 22]
  ]
>

iex> Nx.stack(Explorer.DataFrame.new(a: [11, 12], b: [21, 22]), axis: -1)
#Nx.Tensor<
  s64[2][2]
  [
    [11, 21],
    [12, 22]
  ]
>

warning-returning-tensorframes

Warning: returning TensorFrames

It is not recommended to return a TensorFrame from a defn, as that would force all columns to be sent to the CPU/GPU and then copied back. Return only the columns that have been modified during the computation. For example, in the example above we used Nx to add two columns, if you want to put the result of the computation back into a DataFrame, you can use Explorer.DataFrame.put/4, which also accepts tensors:

iex> df = Explorer.DataFrame.new(a: [11, 12], b: [21, 22])
iex> Explorer.DataFrame.put(df, "result", add_columns(df))
#Explorer.DataFrame<
  Polars[2 x 3]
  a integer [11, 12]
  b integer [21, 22]
  result integer [32, 34]
>

One benefit of using Explorer.DataFrame.put/4 is that it will preserve the type of the column if one already exists. Alternatively, use Explorer.Series.from_tensor/1 to explicitly convert a tensor back to a series.

supported-dtypes

Supported dtypes

The following dtypes can be converted to tensors:

  • :integer
  • :float
  • :boolean
  • :date
  • :datetime

See Explorer.Series.to_iovec/1 and Explorer.Series.to_tensor/1 to learn more about their internal representation.

Link to this section Summary

Functions

Pulls a tensor from the TensorFrame.

Puts a tensor in the TensorFrame.

Link to this section Types

@type t() :: %Explorer.TensorFrame{data: term(), n_rows: term(), names: term()}

Link to this section Functions

Pulls a tensor from the TensorFrame.

This is equivalent to using the tf[name] to access a tensor.

examples

Examples

Explorer.TensorFrame.pull(tf, "some_column")

Puts a tensor in the TensorFrame.

This function can be invoked from within defn.

examples

Examples

Explorer.TensorFrame.put(tf, "result", some_tensor)