An append-only tab, and the first of the two ways to keep data on a sheet.
A log's rows are events, each carrying an id its writer made. Nothing is ever
overwritten: an update is the same id appended again, a delete is the same id
appended with the deleted flag, and the state of the log is the fold over
the rows: the last one per id wins, and tombstoned ids drop out.
iex> alias Sheetshow.{Log, Workbook}
iex> log = Log.new("expenses", item: :string, cost: :decimal)
iex> rent = Log.Event.new(%{item: "Rent", cost: "1000.00"}, id: "a")
iex> food = Log.Event.new(%{item: "Food", cost: "400.00"}, id: "b")
iex> {:ok, workbook} = Sheetshow.run(Log.create(log) ++ Log.plan!([rent, food], log), Workbook.memory())
iex> later = [Log.Event.put(rent, %{cost: "1100.00"}), Log.Event.delete(food)]
iex> {:ok, workbook} = Sheetshow.run(Log.plan!(later, log), workbook)
iex> Log.read!(log, workbook) |> Log.fold() |> Enum.map(& &1.record)
[%{item: "Rent", cost: "1100.00"}]That one rule does three jobs. It makes updates and deletes out of the only
write Google resolves for you, since appendCells lands below the last row
with data at the moment it is applied, so two processes appending to one tab
cannot overwrite each other, and no compare-and-swap is needed. And it makes
a write safe to retry: a batch that landed but answered with a transport
error can be sent again, because the duplicate ids fold away. Retry a failed
batch before you write anything newer, though, because a stale retry that
lands late is a stale last row.
That is Google's promise, not a file's. A file is rewritten whole, so two
writers appending to one can lose each other's rows unless the store can
refuse a clobbering write, in which case the loser gets
%Sheetshow.Error{reason: :conflict} with nothing written, and running the
same plan again is safe, because the ids in it fold away against themselves.
What Sheetshow Can Promise has the whole of it.
The tab is laid out for a person to read. Row 0 is the header (id,
deleted, then the schema's columns in order), and deleted is left blank
on a live row, so a reader sees a flag only where something was taken out.
Columns are found by header name when reading, but appends are positional, so
the schema's order is the tab's column order: add columns at the end and
re-put header/2, and leave the ones already there where they are. Anything
further right is a person's own business and is not touched.
Reading is lenient. A cell someone typed that will not cast leaves that field
nil and an entry in the event's errors, so a typo costs you a field
rather than the read; decode/3 with strict: true refuses such a read
instead.
Summary
Functions
The tab's columns, left to right.
The plan that makes the tab and writes its header. Fails at the backend if
the tab is already there, which is what you want from something called
create.
Events from the tab's values: rows of cell values, as Sheetshow.read_cells/2 or
Sheetshow.to_rows/1 gives them.
Same as decode/3, raising on failure.
The log as it stands: the latest event for each id, in the order the ids first appeared, with the deleted ones gone.
The header as cells, bold unless you say otherwise. Write these again after adding a column to the end of the schema.
A log on a tab, with the columns its rows have. Raises ArgumentError on a
schema Sheetshow.Schema.validate/1 refuses.
The plan that appends events, as one Sheetshow.Op.AppendRows, so a batch of
them is atomic and lands below whatever else arrived in the meantime.
Same as plan/2, raising on failure.
Every event on the tab, in the order it was written.
Same as read/3, raising on failure.
Cells for a second tab that shows the log as it stands, for a person to look at: one live row per id, newest content, tombstones gone.
Types
@type t() :: %Sheetshow.Log{schema: Sheetshow.Schema.t(), sheet: String.t()}
Functions
The tab's columns, left to right.
iex> Sheetshow.Log.new("expenses", item: :string) |> Sheetshow.Log.columns()
["id", "deleted", "item"]
@spec create(t()) :: Sheetshow.Op.plan()
The plan that makes the tab and writes its header. Fails at the backend if
the tab is already there, which is what you want from something called
create.
iex> Sheetshow.Log.new("expenses", item: :string) |> Sheetshow.Log.create() |> Enum.map(&Sheetshow.Op.sheet/1)
["expenses", "expenses"]
@spec decode([[term()]], t(), keyword()) :: {:ok, [Sheetshow.Log.Event.t()]} | {:error, Sheetshow.Error.t()}
Events from the tab's values: rows of cell values, as Sheetshow.read_cells/2 or
Sheetshow.to_rows/1 gives them.
Options:
:header, the header row, whenrowsholds only data. Without it the first row ofrowsis the header.:row, the sheet row the first given row sits on, so events know where they are. Defaults to0with a header inrows,1without.:strict, to refuse the read when any cell would not cast, rather than flagging the field. Off by default.
Columns are matched by header name, ignoring case and surrounding space, so
a column that has been renamed fails loudly rather than reading as the one
next to it. Blank rows are skipped; a row with content and no id comes back
as an event with errors.id, because dropping it would hide data someone
typed.
iex> log = Sheetshow.Log.new("expenses", item: :string)
iex> rows = [["id", "deleted", "item"], ["a", nil, "Rent"], ["a", "TRUE", "Rent"]]
iex> {:ok, events} = Sheetshow.Log.decode(rows, log)
iex> Enum.map(events, &{&1.id, &1.row, &1.deleted})
[{"a", 1, false}, {"a", 2, true}]
iex> log = Sheetshow.Log.new("expenses", cost: :integer)
iex> {:error, %Sheetshow.Error{reason: :missing_column}} =
...> Sheetshow.Log.decode([["id", "deleted", "price"]], log)
@spec decode!([[term()]], t(), keyword()) :: [Sheetshow.Log.Event.t()]
Same as decode/3, raising on failure.
iex> log = Sheetshow.Log.new("expenses", item: :string)
iex> Sheetshow.Log.decode!([["id", "deleted", "item"]], log)
[]
@spec fold([Sheetshow.Log.Event.t()]) :: [Sheetshow.Log.Event.t()]
The log as it stands: the latest event for each id, in the order the ids first appeared, with the deleted ones gone.
iex> alias Sheetshow.Log
iex> log = [Log.Event.new(%{}, id: "a"), Log.Event.new(%{}, id: "b"), Log.Event.delete("a")]
iex> Log.fold(log) |> Enum.map(& &1.id)
["b"]One pass over a map, so a hundred thousand rows fold in one go, and the
result folds again: fold(fold(old) ++ new) is fold(old ++ new), because
the folded list keeps both the order the ids first appeared in and the latest
content of each. So a process reading a log in batches can merge each read
into what it holds instead of re-folding the lot. The one exception is an id
that was deleted and then written again: it lands where it came back rather
than where it started.
A row whose id would not read takes no part in any of this, and cannot be updated or deleted by id, but it is kept, under its own row, so that data a person typed does not vanish from the fold.
@spec header(t(), Sheetshow.Style.t()) :: [Sheetshow.Cell.t()]
The header as cells, bold unless you say otherwise. Write these again after adding a column to the end of the schema.
iex> Sheetshow.Log.new("expenses", item: :string)
...> |> Sheetshow.Log.header()
...> |> Sheetshow.to_rows()
[["id", "deleted", "item"]]
@spec new(String.t(), Sheetshow.Schema.t()) :: t()
A log on a tab, with the columns its rows have. Raises ArgumentError on a
schema Sheetshow.Schema.validate/1 refuses.
iex> Sheetshow.Log.new("expenses", item: :string, cost: :decimal).sheet
"expenses"
@spec plan([Sheetshow.Log.Event.t()], t()) :: {:ok, Sheetshow.Op.plan()} | {:error, Sheetshow.Error.t()}
The plan that appends events, as one Sheetshow.Op.AppendRows, so a batch of
them is atomic and lands below whatever else arrived in the meantime.
Strict, as writing always is: a record whose values do not fit the schema, or which names a column the schema does not have, is an error here rather than a surprise in the sheet. An empty list is an empty plan.
iex> log = Sheetshow.Log.new("expenses", item: :string)
iex> {:ok, [op]} = Sheetshow.Log.plan([Sheetshow.Log.Event.new(%{item: "Rent"})], log)
iex> Sheetshow.Op.AppendRows.height(op)
1
iex> log = Sheetshow.Log.new("expenses", cost: :integer)
iex> {:error, %Sheetshow.Error{reason: :invalid_record}} =
...> Sheetshow.Log.plan([Sheetshow.Log.Event.new(%{cost: "free"})], log)
@spec plan!([Sheetshow.Log.Event.t()], t()) :: Sheetshow.Op.plan()
Same as plan/2, raising on failure.
iex> log = Sheetshow.Log.new("expenses", item: :string)
iex> Sheetshow.Log.plan!([], log)
[]
@spec read(t(), Sheetshow.Workbook.t(), keyword()) :: {:ok, [Sheetshow.Log.Event.t()]} | {:error, Sheetshow.Error.t()}
Every event on the tab, in the order it was written.
This is Sheetshow.read_rows/2 over the whole tab and then decode/3, and
it takes decode/3's :strict. The values endpoint rather than the cell one
because a stored row wants what its formulas worked out to, and because the
schema already says which numbers are dates. See Sheetshow.read_rows/2
for what that costs you.
Reading the whole tab is what a fold needs; Google trims the answer to the rows that have something in them, so an empty log is one small request. A file is read whole whatever it holds.
{:ok, workbook} = Sheetshow.connect(workbook)
{:ok, events} = Sheetshow.Log.read(log, workbook)
Sheetshow.Log.fold(events)Reading only what is new
after: event answers with the events appended since that one, which is what
a log too big to read every time needs. The cursor is an event you were given
by a read, since its row is what makes it one, and the usual cursor is the
last event you hold.
{:ok, new} = Sheetshow.Log.read(log, workbook, after: List.last(events))
state = Sheetshow.Log.fold(Sheetshow.Log.fold(events) ++ new)The read covers the cursor's own row as well as the rows below it, and the
cursor's row must still hold the very same event. If it does not, because
someone inserted or removed a row above it by hand, the rows you hold no longer
line up with the sheet, and you get {:error, %Error{reason: :moved}} rather
than a list that quietly means something else. Read the log whole when that
happens.
Two things this cannot see. A row appended and then deleted by hand below the cursor is simply gone, and nothing says so. And two neighbouring rows that are identical in every respect can stand in for each other, which is harmless, because folding either of them gives the same answer.
At Google it is one request: the header row and the rows from the cursor down, asked for together, so the columns are still found by name rather than assumed. Against a file it is the file: a cursor saves the decoding, not the fetching, because a workbook has to be read whole before any of it can be.
@spec read!(t(), Sheetshow.Workbook.t(), keyword()) :: [Sheetshow.Log.Event.t()]
Same as read/3, raising on failure.
@spec view(t(), String.t()) :: [Sheetshow.Cell.t()]
Cells for a second tab that shows the log as it stands, for a person to look at: one live row per id, newest content, tombstones gone.
It is a single formula, so Sheets keeps it current as the log grows, and
nothing has to be rewritten when rows are appended. The deleted column is
left out, since every row on this tab is one that was not.
log |> Sheetshow.Log.view("expenses now") |> Sheetshow.plan!(existing_sheets: [])The formula is what fold/1 does, said in Sheets: drop the blank rows, put
the log newest-first, keep one row per id, drop the deleted ones, and order
what is left by id.
The same rows, in id order. fold/1 keeps the order the ids first
appeared in; this tab can only sort by what is on it, which is the id. Those
agree when your ids increase in the order you make them, which a
Sheetshow.ULID does down to the microsecond, so a batch of events made in
one go reads back here in the order it was made. Ids of your own need to sort
the same way if the order on this tab has to mean something.
Said in Sheets, and only there. It turns on SORTN, which Excel has no
counterpart for, so a view tab written into an .xlsx shows #NAME? when
somebody opens it. Nothing is harmed, since a formula that does not resolve
is a cell that does not resolve, but the tab is pointless there, and fold/1
is what a file-backed log uses instead.
iex> Sheetshow.Log.new("log", item: :string)
...> |> Sheetshow.Log.view("now")
...> |> Sheetshow.to_rows()
...> |> hd()
["id", "item"]