A grounded policy assistant for a 4,000-page rulebook

Policy sat in roughly 4,000 pages across dozens of documents with overlapping revisions. Frontline staff were answering customer questions from memory and from each other.

The compliance team could not audit an answer after the fact, because there was no record of which clause a decision had relied on.

We built the evaluation set before the system: several hundred real questions with the clause that should answer each one, collected from the compliance and frontline teams.

Only then did we design retrieval, because we could measure whether each design change actually improved answers rather than felt better in a demo.

A retrieval-augmented assistant that answers only from the indexed policy corpus and links to the exact clause behind every sentence.

An explicit refusal path: when retrieval finds no supporting passage the assistant says so and routes to a human, instead of producing a fluent guess.

Revision awareness, so a superseded clause is never returned as current, and an audit log of every question, answer and cited source.

Compliance can reconstruct the basis of any answer, because the cited clause is stored with the interaction.

Frontline handling time on policy questions dropped substantially, with the largest gain on questions that previously required escalation.

Measured accuracy against the evaluation set is tracked on every release rather than asserted once at launch.