Grant verification: from scores to evidence
Experienced managers spent hours checking each 200-page grant application, and the same reasons for rejection kept coming back. We built an LLM verifier that surfaces potential issues with explanations and source references.
- Pilot result
- Final verification of an application took 2–3 hours in the pilot, against 5 hours before, with a person still reviewing.
- How it was measured
- The 5-hour baseline came from interviews with users and a map of the process with its data flow. The 2–3 hours came from user data during the pilot, with one client and six users. The two figures were collected in different ways.
- Quality
- Fewer errors got through. We counted three kinds: consistency errors, compliance errors against the requirements and the attached files, and grammar and language errors. Each application was checked in parallel, by hand and with the tool, and we compared what each method had caught with the errors the institution found later. I optimised for recall, not precision: the system flagged somewhat more, to lower the risk of a missed error.
- Scope and limits
- The pilot covered one client, six users and one type of application in a single grant programme. I took part in the pilot only, and I don’t have data on adoption or performance after it.
What the reviewer sees
FlagNeeds a decision
The total does not match the line items
Why it was flagged. The budget table adds up to 412,000. The summary says 421,000.
Source. Budget table, p. 41 · Summary, p. 3
The reviewer decides what to do with it.
FlagNeeds a decision
The project length does not match the rules or the schedule
Why it was flagged. The programme allows up to 24 months. Section 2 describes a 30-month project. The schedule in Annex 3 ends after 28 months.
Sources. Programme rules, §4 · Section 2, p. 6 · Annex 3, schedule
The reviewer decides what to do with it.
The consistency check ran at three levels: each section, each module and the whole application. It checked internal consistency, within the application, and external consistency, against the programme requirements and the attached files. It worked through the document in small portions. After optimisation, the consistency check of the whole application took 20–30 minutes on its own, the longest of the tests. Those minutes are part of the 2–3 hours.
Two decisions
Decision 1What problem are we solving?
It looked easier to demo and sell.
Check applications before they are formally submitted.
I tested writing with mockups and user interviews. The pain was that applications got rejected, not that they were hard to write.
We focused the product on verification and left application writing out of scope.
Decision 2How do we support the verification?
My first design had the system score the applications. We considered full automation too.
Each problem comes with its reasoning, a link to the source document, and a diff for each language error. The reviewer decides.
In prototype tests, the managers did not trust the score and did not know what to do with it. They sign the application, and they would not act on a number they could not defend.
The reviewer keeps the decision and gets what they need to defend it. The cost: they have to read the signals and references from the AI before deciding, which a verdict would have saved them.
Deciding what your AI should flag, and what people should decide?
Tell me about the workflow and who signs off on the result.
More decisions I said no to · The checklist I use for the same question