Monet.Query.Cursor (monet v0.1.6)

Fetches paged results using cursors and generates new next/prev cursors from the result. This requires a unique integer cursor column.

There are subtle differences between paging forward and backwards and this interacts with whether we're sorting ascendingly and descendinly. Also, because the sorting column may not be unique, the filter ends up looking. Say we want to sort by price, we need to do:

    where (price > $1 or (price = $1 and cursor > $2)) order by price, cursor

If you ask for perpage: 25 it'll fetch 26 records to see if there are more results. If so, a next link is generated based on the price+cursor of the 25th record.

Now consider the case where we want to fetch the previous page while ordering by ascending price. This is trickier, because if we just did:

    where (price < $1 or (price = $1 and cursor < $2)) order by price, cursor

We'd get the wrong results. Consider this data with a perpage of 2:

    id, price, cursor
    1,  1.00,  1
    2,  2.00,  2
    3,  3.00,  3
    4,  4.00,  4
    5,  5.00,  5

Moving forward, we'd get the following ids:

    [1, 2] -> next -> [3, 4] -> next [5]

Now, following the prev link from this last page with the above query, we'd end up with [1, 2]. Instead, what we need to do is reverse the order:

    where (price < $1 or (price = $1 and cursor < $2)) order by price DESC, cursor DESC

Now we'll get the right page, but in the wrong order [4, 3]. So we need to reverse it once more.

All of this ordering and filtering isn't too complicated: there are only 4 combinations of prev/next and asc/desc. You also need to "overfetch" (get +1 records to know if there are "more"), and remove it if present. Again, none of it is complicated, but it takes some attetion to do it efficient - moreso with [linked]-lists.

There are 2 parts to this cursor. The first is responsible for doing all of the above. The second (much smaller) part deals with iterating through the results and building the next/prev cursors. This is done in the name of efficiency. In most cases, we need the last row (the last item in our list). In Elixir, this is an O(N) operation. However, since your code likely needs to iterate the results anyways (to build the payload), we can combine the two together. As such, the cursor acts as a sort of generator.

This iteration/generator phase can be ignored. The cursor that's returned as part of Select.cursor is fully materialized and contains all of the necessary data.

Link to this section Summary

Link to this section Functions