Grounded · Technical · Engineering record
I published an exam for finance AI, then sat my own tool for it.
I can't publish the application code; the course licence forbids it. So the record is public instead: every decision with its date, the measurements, the incidents and the rule each one left behind, what is mine and what came with the scaffold, and every feature tagged built or backlog. If you're deciding whether I build serious software, this is the page to read slowly.
01 · The numbers
81/81
The Board Pack Test: 34 realistic finance documents, 25 questions, a deterministic grader. Grounded scored the ceiling; raw frontier chat scored 66 and 71 of the 81 machine-checkable points.
After a human judgment pass every frontier tool lands between 95 and 100, and the best raw model beat Grounded by one point. The edge is in what survives checking with no person in the loop.
02 · What I built on the scaffold
Grounded started as a fork of a retrieval application from a paid course. That scaffold had a chat loop, search, document ingestion and a front end, and it was a serious piece of software. What I built on it is everything that makes the answers safe to rely on in a finance team.
Verification
A guard that checks every figure in an answer after it is written, and refuses when one can't be traced. A calculator in code for the arithmetic. The trust chain shown under each answer.
Spreadsheets
An extractor that keeps the grid, the sheet and the cell reference for every value, and a viewer that opens the workbook with the cited cell highlighted.
Retrieval quality
A retrieval-only test that found the reranker had never run. Turning it on, with retries and tests, and ranking that demotes superseded documents.
Evals
A golden-question runner with sources asserted in code, an evals page in the app, and the Board Pack Test with its grader.
Tenancy and secrets
Organisations as tenants, with retrieval and citations scoped to each one, encrypted model keys per organisation, and 95 security and isolation tests.
Audit
Audit packages that survive a deleted conversation, and a page to read them on.
Integration
Grounded as an MCP server, a Slack bot, and a month-end close suite of skills.
Operations
The deployment runbook, tracing and cost alerts, and a tiered test runner with a pre-commit hook.
03 · Decisions and mistakes
Fourteen decision records, eight incidents, all public.
Files are the source of truth
The decision everything else follows from, and why the obvious design (extract every figure into a table) was rejected.
The trust chain
A citation is an audit; a verification is a proof. How every link is checked by code.
A decision I reversed in eleven days
One instance per customer felt like the honest security answer. Pooled tenancy is where it landed, and what the detour cost.
What broke, and what changed
Eight incidents and the rule each one left behind, including a blank setting that cost 26 days of deployments that reported success.
What I would do differently
- Build the test before the feature. The retrieval test took an afternoon and found a reranker that had never run.
- Make the tests cheap from the start. Once a run cost nothing I ran them constantly, and quality moved.
- Publish the benchmark on day one. It was the most useful thing I built and the last thing I started.
04 · Every feature, by the moment you reach for it
Working on now, October 2026: this site family, and the public close plugin for Claude.
Each line says where it stands. Built runs today. Backlog is not built; I would build it against a real team's close rather than guess.
When you need a number
- builtAsk in plain English. The answer cites the file, the sheet and the cell range, and the workbook opens with that cell highlighted.
- builtEvery figure in the answer is checked by code after it is written. One that cannot be traced is held back and the answer says so.
- builtArithmetic runs in a calculator, not the model. The trust chain under the answer shows the inputs, the computation and the result.
- builtMarket and industry context from the web, always labelled external, never blended with the company's own numbers.
- backlogSettings a controller can change: web search on or off, how strict the number guard is.
When two reports disagree
- builtReconcile a figure across documents: ties, near miss, material mismatch or cannot reconcile, with both sources cited.
- backlogCheck the numbers against the words: recognised revenue against the contract, spend against the policy, commentary against the actuals.
When you run the close
- builtA close checklist that reads your own close calendar, shows the steps with owners, and runs each step with a reviewer gate between them.
- builtThe WD3 flash: revenue, margin and operating profit against budget, top variances called out, every figure cited.
- builtManual journals checked against the delegation of authority, each one presented for an approve or query decision, decisions kept in a log.
- builtAccrual completeness: what posted after the ledger cut, why the department export disagrees with the pack.
- builtThe bank rec: statement to cashbook to the pack's cash position, timing differences verified, stale items flagged for a person.
- builtThe variance pack: every P&L line against budget, material variances investigated to their drivers, commentary drafted for sign-off.
- backlogThe approved pack generated as a file, with the driver decomposition behind each line.
- backlogFigures pulled live from the ledger or the planning system (Xero, NetSuite, Anaplan), each one citing the system, the report and the timestamp.
When the CFO asks "what if"
- builtAdvisory mode: a research pipeline over the documents that answers in three tiers, facts cited to cells, assumptions stated and editable, opinion derived only from those two.
- backlogScenarios: close a department, invest in a product. Avoidable cost separated from allocated overhead, and a plain statement of what the files cannot separate.
- backlogSensitivity: which inputs move the result most, computed over cited baselines.
- backlogBoard prep: given the pack, the hardest questions the board will ask, with cited draft answers.
- backlogA takeaway model: the scenario as a small spreadsheet, inputs annotated with their source cells, outputs formula-driven.
- backlogDashboards as generated, cited artifacts, and a memo composer for the commentary that goes with them.
When the auditors come
- builtEvery answer kept as an audit package: the question, the answer, the sources and the checks. It survives the conversation being deleted.
- builtReviewer decisions recorded in the run, so the close has a who-signed as well as a what-happened.
- builtPersonal data redacted before the model sees it, and a check that the redaction never alters a number.
- builtEach organisation is its own tenant: retrieval, citations and model keys scoped to it, isolation covered by tests. A documents-only policy per organisation, with web search off.
When you want to know if you can trust it
- builtYour own golden questions in the app: known answers, sources asserted in code, a judge, and a score before every release.
- builtThe Board Pack Test, a public benchmark with its grader, run against the same engine.
- builtTracing on every run, a cost budget per day, and an alert when the hallucination judge drifts.
- backlogThe golden suites as a gate in the deployment pipeline, so a release that drops the score does not ship.
Where it lives
- builtIn the browser, as a Slack bot, and as an MCP server so Claude or any agent can ask it and get the cited answer back.
- backlogThe close skills packaged as a public plugin for Claude, with a shared run log of every automation, exception and sign-off.
05 · The films, dated
06 · How it is tested
Four tiers: fast (925 unit tests, under a minute, pre-commit on backend changes), API, smoke (six stable golden questions, all of which must pass) and gates (the full golden suites, three runs each). Golden questions never run automatically in CI (ADR 0014). If a release drops the score, a person running the gate catches it, and I set it up that way deliberately. Every run is traced, and there's a daily cost budget and a hallucination-judge alert.
Hiring?
Need this kind of build on your team?
I take contract roles in finance automation and FP&A systems. Ask me anything about how Grounded works; I'm happy to walk through the parts that broke as well as the parts that worked.
