Imp.Datasets (Imp v0.5.0)

Local dataset loaders for examples, JSONL, CSV, GSM8K, HotPotQA, MATH, and Colors records.

Named loaders accept an existing local path; they do not download datasets. Source-checkout benchmark fixtures are fetched explicitly with the imp.benchmark.fetch Mix task.

Summary

Functions

csv(path, input_keys, opts \\ [])

from_records(records, input_keys, opts \\ [])

gsm8k(path)

hotpotqa(path)

jsonl(path, input_keys, opts \\ [])

split(examples, opts \\ [])

Splits examples into {train, rest} at the :train fraction (default 0.8).

Functions

csv(path, input_keys, opts \\ [])

from_records(records, input_keys, opts \\ [])

gsm8k(path)

hotpotqa(path)

jsonl(path, input_keys, opts \\ [])

split(examples, opts \\ [])

Splits examples into {train, rest} at the :train fraction (default 0.8).

With shuffle: true (the default) the examples are shuffled with a seeded RNG before splitting. The shuffle is deterministic: the same examples and the same :seed (default 0) always produce the same split, so downstream consumers such as Imp.Optimizer.LabeledFewShot see the same trainset on every run. Pass a different seed: for a different permutation, or shuffle: false to preserve input order.