Generates synthetic test cases from existing chunks.
Samples chunks and uses an LLM to generate questions that should retrieve those chunks.
Summary
Functions
Returns the default prompt template.
Generates test cases from a sample of chunks.
Options
:repo- Ecto repo (required):llm- LLM implementing Arcana.LLM protocol (required):sample_size- Number of chunks to sample (default: 50):source_id- Limit to chunks from specific source:collection- Limit to chunks from specific collection:prompt- Custom prompt template (must include {chunk_text})