ExArrow.Broadway.FlightSink (ex_arrow v0.7.0)

View Source

Upload assembled Arrow batches to a Flight server from a Broadway batch handler.

Calls ExArrow.Flight.Client.do_put/4 with the assembled schema and batches. Emits a [:ex_arrow, :flight, :query] telemetry event with %{destination: descriptor, source: :broadway} metadata.

Options

Forwarded to ExArrow.Flight.Client.do_put/4 (e.g. :descriptor).

Example

def handle_batch(:flight, messages, _info, _ctx) do
  {:ok, schema, batches} = ExArrow.Broadway.BatchBuilder.from_messages(messages)
  ExArrow.Broadway.FlightSink.write(client, schema, batches,
    descriptor: {:cmd, "events_batch"}
  )
end

Summary

Functions

write(client, schema, batches, opts \\ [])

Upload schema and batches to a Flight server via client.

Functions

write(client, schema, batches, opts \\ [])

@spec write(
  ExArrow.Flight.Client.t(),
  ExArrow.Schema.t(),
  [ExArrow.RecordBatch.t()],
  keyword()
) ::
  :ok | {:error, term()}

Upload schema and batches to a Flight server via client.

Returns :ok or {:error, reason}.