LLM-based reranker: improves precision after initial retrieval.

After Hybrid.search/2 returns a candidate set, this module sends a lightweight LLM prompt asking the model to score each candidate's relevance to the query on a 0–10 scale. The original retrieval scores are then blended with the LLM scores, and the result set is re-sorted.

When to use

Call rerank/3 only when retrieval precision matters more than latency (e.g. rag_query with a small limit). Skip it for interactive search where sub-second response times are required.

Blending

Final score = alpha * normalized_llm_score + (1 - alpha) * original_score

Default alpha: 0.6 weights the LLM judgment heavier than the embedding distance. Pass alpha: 0.0 to use LLM scores alone, or alpha: 1.0 to fall back to original scores only (effectively a no-op).

Batching

All candidates are sent in a single prompt to minimise latency and cost. The LLM is instructed to return a JSON array of {index, score} objects. If parsing fails, the original ordering is preserved.

Options

• :alpha - blend weight for LLM score (default: 0.6)
• :provider - override AI provider (default: configured default)
• :max_candidates - truncate candidate list before sending to LLM

                   (default: 20)

• :timeout - milliseconds for LLM call (default: 15_000)