View Source Explorer.TensorFrame (Explorer v0.5.5)
TensorFrame is a representation of Explorer.DataFrame
that is designed to work inside Nx's defn
expressions.
For example, imagine the following defn
:
defn add_columns(tf) do
tf[:a] + tf[:b]
end
We can now pass a DataFrame as argument:
iex> add_columns(Explorer.DataFrame.new(a: [11, 12], b: [21, 22]))
#Nx.Tensor<
s64[2]
[32, 34]
>
Passing an Explorer.DataFrame
to a defn
will automatically
convert it to a TensorFrame. The TensorFrame will lazily
build tensors out of the used dataframe fields.
stack-and-concatenating
Stack and concatenating
Due to the integration with Nx, you can also pass dataframes
into Nx.stack/2
and Nx.concatenate
and they will be automatically
converted to tensors. This makes it easy to pass dataframes into
neural networks and other computationally intensive algorithms:
iex> Nx.concatenate(Explorer.DataFrame.new(a: [11, 12], b: [21, 22]))
#Nx.Tensor<
s64[4]
[11, 12, 21, 22]
>
iex> Nx.stack(Explorer.DataFrame.new(a: [11, 12], b: [21, 22]))
#Nx.Tensor<
s64[2][2]
[
[11, 12],
[21, 22]
]
>
iex> Nx.stack(Explorer.DataFrame.new(a: [11, 12], b: [21, 22]), axis: -1)
#Nx.Tensor<
s64[2][2]
[
[11, 21],
[12, 22]
]
>
warning-returning-tensorframes
Warning: returning TensorFrames
It is not recommended to return a TensorFrame from a defn
,
as that would force all columns to be sent to the CPU/GPU
and then copied back. Return only the columns that have been
modified during the computation. For example, in the example
above we used Nx
to add two columns, if you want to
put the result of the computation back into a DataFrame,
you can use Explorer.DataFrame.put/4
, which also accepts
tensors:
iex> df = Explorer.DataFrame.new(a: [11, 12], b: [21, 22])
iex> Explorer.DataFrame.put(df, "result", add_columns(df))
#Explorer.DataFrame<
Polars[2 x 3]
a integer [11, 12]
b integer [21, 22]
result integer [32, 34]
>
One benefit of using Explorer.DataFrame.put/4
is that it will
preserve the type of the column if one already exists. Alternatively,
use Explorer.Series.from_tensor/1
to explicitly convert a tensor
back to a series.
supported-dtypes
Supported dtypes
The following dtypes can be converted to tensors:
:integer
:float
:boolean
:date
:datetime
See Explorer.Series.to_iovec/1
and Explorer.Series.to_tensor/1
to learn more about their internal representation.
Link to this section Summary
Link to this section Types
Link to this section Functions
Pulls a tensor from the TensorFrame.
This is equivalent to using the tf[name]
to access
a tensor.
examples
Examples
Explorer.TensorFrame.pull(tf, "some_column")
Puts a tensor in the TensorFrame.
This function can be invoked from within defn
.
examples
Examples
Explorer.TensorFrame.put(tf, "result", some_tensor)