Lazy-loaded NLI serving for hallucination detection via Hallmark.

Uses Hallmark (Vectara HHEM model via Bumblebee) to score each sentence in the answer against the retrieved chunks. Runs one NLI pair per (sentence, chunk) and takes the max score per sentence as the final faithfulness score. Sentences scoring below the threshold are marked as hallucinated.

Why per-chunk instead of concat-all

HHEM's ModernBERT has a fixed input window. Concatenating all chunks into one context works for small chunk sets (Advanced RAG's 5-10 chunks) but silently truncates once the pipeline starts accumulating more (Pipeline with decompose + rerank hits 20+ easily). Truncation drops the tail chunks from the context the NLI sees, so sentences whose evidence lived in those tail chunks get flagged as hallucinations even when the chunks were retrieved correctly.

Per-chunk scoring sidesteps truncation entirely. Each chunk fits comfortably on its own, and Hallmark.score_batch/2 sends the whole (sentence × chunk) grid through the NLI model as one batched forward pass, so the wall-clock cost stays reasonable.

The model is downloaded automatically on first use via Bumblebee.