Latu.GroupedData (latu v0.1.1)

A DataFrame with grouping attached, waiting for agg/2.

Spark has no group_by relation: Aggregate is a single node holding the group type, the grouping expressions and the aggregates, so groupBy(...).agg(...) is a client-side fiction in every Spark client. This struct is where Latu keeps that half-built state, and it is inert data like everything else.

Summary

Types

t()

Functions

agg(grouped, aggregates)

Apply aggregates, giving a DataFrame back.

count(grouped)

Rows per group, in a column called count.

pivot(grouped, column, values \\ [])

Pivot the grouped frame on a column, turning its values into columns.

Types

t()

@type t() :: %Latu.GroupedData{
  grouping_sets: [[term()]],
  groupings: [Latu.Plan.expression()],
  input: Latu.DataFrame.t(),
  pivot: String.t() | atom() | nil,
  pivot_values: [term()],
  type: :group_by | :rollup | :cube | :pivot | :grouping_sets
}

Functions

agg(grouped, aggregates)

@spec agg(t(), term()) :: Latu.DataFrame.t()

Apply aggregates, giving a DataFrame back.

df |> Latu.group_by(:suburb) |> Latu.agg(total: F.sum(:price), n: F.count(:id))

Takes the mixed list Latu.select/2 takes: trailing keywords name their expressions.

count(grouped)

@spec count(t()) :: Latu.DataFrame.t()

Rows per group, in a column called count.

PySpark's GroupedData.count(), which is count(1) under that alias — reproduced, so the output column matches.

pivot(grouped, column, values \\ [])

@spec pivot(t(), String.t() | atom(), [term()]) :: t()

Pivot the grouped frame on a column, turning its values into columns.

df |> Latu.group_by(:suburb) |> Latu.pivot(:year) |> Latu.agg(total: F.sum(:price))
df |> Latu.group_by(:suburb) |> Latu.pivot(:year, [2025, 2026]) |> Latu.count()

Without values Spark runs a separate query first to find the distinct ones, so pass them when you know them. Only a grouped frame can be pivoted, as in PySpark.