Grounded · The checklist
What a RAG agent needs.
A properly built RAG agent needs all of this. Ten areas, seventy-two lines, from parsing the documents to running the thing in production. The demo version skips most of them; the one a finance team can rely on cannot. Under each area is where Grounded stands today, and where that is written up.
The ten areas
Each card is one area a well-built agent has to cover. The bar is where Grounded stands on it today, checked on 2026-10-30. Click through for the lines.
Ingestion
Grounded 4 of 6
1 partial · 1 not built
Retrieval
Grounded 5 of 6
1 partial
Answering and grounding
Grounded 8 of 9
1 not built
Guardrails
Grounded 5 of 8
1 partial · 2 not built
Evals
Grounded 8 of 9
1 not built
Observability
Grounded 10 of 12
2 not built
Audit
Grounded 3 of 3
complete
Tenancy and security
Grounded 6 of 8
1 partial · 1 not built
Cost
Grounded 4 of 5
1 not built
Operations
Grounded 4 of 6
1 partial · 1 not built
01 · Ingestion
OverviewIngestion
Everything downstream depends on the structure surviving this stage. A finance workbook flattened to prose has lost the row label, the column header and the sheet name, and retrieval cannot put them back.
Grounded today
- builtKeep document structure: tables as tables, workbooks as grids, an A1 reference on every cellwrite-up
- builtStore two texts per chunk: an enriched one to embed, a verbatim one to cite fromwrite-up
- builtSkip unchanged files on re-upload, because every ingest costs embeddings and an LLM passwrite-up
- builtFence the parser: size and pixel limits, process isolation, OCR only when a page needs itwrite-up
- designedA queue that recovers when the process dies mid-document · designed, not builtwrite-up
- not builtRead charts and images · by choicewrite-up
02 · Retrieval
OverviewRetrieval
Hybrid search, a reranker that is proved to be on, and scope applied inside the search functions rather than filtered afterwards.
Grounded today
- builtHybrid search, vector plus keyword, fused by rankwrite-up
- builtA reranker, and a measurement that proves it is onwrite-up
- builtTenant and folder scope applied inside the search functionswrite-up
- builtDocument authority: drafts, legacy copies and superseded versions ranked down, never hiddenwrite-up
- builtA retrieval-only eval (hit@k, MRR) that runs in seconds after any change to chunking, embeddings or rerankwrite-up
- partialA written account of how one question becomes several searches · the agent decides, up to 25 rounds; the pattern is in the traces, not yet on a page
03 · Answering and grounding
OverviewAnswering and grounding
Where the promise is kept or broken: no figure without the cell or page it came from, arithmetic done by code, and a refusal when the documents cannot support the answer.
Grounded today
- builtCitations that resolve to a cell or a page with a bounding box, not to a documentwrite-up
- builtCitation parity: the model can only cite what retrieval returnedwrite-up
- builtArithmetic done by code, with the inputs recordedwrite-up
- builtRefusal enforced after the model, in code, not requested in the promptwrite-up
- builtAbstain when the corpus does not hold the answer, and test for it · every golden suite carries refusal questionswrite-up
- builtSources carry a trust tier the reader can see: documents, analyst notes, webwrite-up
- builtPer-figure verification shown to the user, not just loggedwrite-up
- builtLoop limits: tool rounds, sub-agent depth, context compactionwrite-up
- not builtA token budget per answer and a stop on runaway loops
04 · Guardrails
OverviewGuardrails
The guards that sit after the model and around the tools, so a wrong figure, a leaked name or a planted instruction cannot reach the user.
Grounded today
- builtOutput guard on figures: the numeral checkwrite-up
- builtPII: reversible surrogates for people, hard redaction for identifiers, and an invariant that numbers are never touchedwrite-up
- builtMasking before any event leaves for the trace storewrite-up
- partialInstructions inside documents do not become tool calls · tested (8 tests); the defence is not yet written upwrite-up
- builtCode the model wrote cannot do more than the model could: a read-only allow-list on the sandbox bridgewrite-up
- builtNo per-organisation base URL, so a shared key cannot be pointed at an attacker's serverwrite-up
- not builtInput scope and jailbreak handling
- not builtOne page that lists every guard, where it runs and the test that pins it · the material is spread over four pages
05 · Evals
OverviewEvals
Build the eval before the feature. Every stage that can silently degrade to a no-op needs a test from day one, and the results need a date and a place in version control.
Grounded today
- builtGolden questions per corpus, each with the source a correct answer must citewrite-up
- builtGrade the answer the user sees, after the guard, not the raw model outputwrite-up
- builtDeterministic citation assertions beside the LLM judgewrite-up
- builtA preflight that checks every expected source exists before a token is spentwrite-up
- builtResults committed as dated files, so the history is in gitwrite-up
- builtEvals cheap enough to run constantly · after a $50 daywrite-up
- builtA public benchmark the product sits like any other agent, scored by a machinewrite-up
- builtA decision on whether evals run in CI · decided: never automaticallywrite-up
- not builtSwap the chat model and publish the delta on both suites
06 · Observability
OverviewObservability
Every answer should be one click from its trace: what was searched, what came back, what the guard decided, what it cost and how long each step took.
Grounded today
- builtTyped observations (agent, tool, retriever, guardrail) so a run renders as a graph, not a list of spanswrite-up
- builtSession id is the conversation, user id on every tracewrite-up
- builtCost per answer from a model registry, with flat-rate models registered at $0 so tokens still countwrite-up
- builtLatency per span, read at p95write-up
- builtRelease and environment on every event, so a regression pins to a deploywrite-up
- builtA sampled live judge for hallucination, and thumbs-down routed to a human queuewrite-up
- builtAlerts on traced cost and on judge scores · dailywrite-up
- builtThe trace store being down never fails a request · a runbook step stops the store and checks the app still answerswrite-up
- builtThe trace store's secret never reaches the browser · pinned by an end-to-end testwrite-up
- builtLinks from the product into the traces: the session for a thread, the trace for an answer, the trace on every audit package · 2026-10-30
- not builtService levels and a dashboard built from real trace data
- not builtRunbooks: slow, expensive, wrong, stuck, leaking
07 · Audit
OverviewAudit
What a reviewer gets after the fact: one package per answer with the question, the answer, the model, the cost and the trace, kept, and exportable to someone outside the product.
Grounded today
- builtA package per answer: question, answer, model, tokens, duration, status, trace link · not yet written up
- builtA read-only mode so packages cannot be deleted through the API · not yet written up
- builtExport of a package for someone outside the product · not yet written up
08 · Tenancy and security
OverviewTenancy and security
How one organisation's documents stay invisible to another inside a pooled instance, and how an instance stays closed to the outside.
Grounded today
- builtScope rules written in one module; retrieval functions take the organisation id; row-level security as the backstopwrite-up
- builtA drift test that fails if a policy or a function grant changeswrite-up
- builtSecurity tests by threat, counted and listed · 95write-up
- builtPer-organisation model keys, envelope-encrypted, model chosen from an allow-listwrite-up
- builtTokens verified against the identity provider's keys, asymmetric firstwrite-up
- builtPublic signup off; users created by an administratorwrite-up
- partialA threat model and a data-flow map of which provider sees what · the PII note covers the model providerwrite-up
- not builtRetention and deletion · not written
09 · Cost
OverviewCost
Cost per answer has to be measured and dated, and the levers that move it known: caching, routing by call class, and never re-ingesting casually.
Grounded today
- builtPrompt caching, with the share of cached input measuredwrite-up
- builtRoute by call class: the strongest model for chat, a mini-class model for judging, extraction and citation checkswrite-up
- builtA rule against casual re-ingest, with a file count and a cost estimate before any yes · a standing rulewrite-up
- builtA sweep of every model and token variable after a provider switch · a rule, after it bitwrite-up
- not builtPer-tenant metering, a budget and a kill switch
10 · Operations
OverviewOperations
Tests priced in time and money, a trunk that deploys to staging, and production that moves only by pull request.
Grounded today
- builtTest tiers priced in time and money, with the cheap one impossible to skipwrite-up
- builtStaging tracks the trunk; production moves only by pull request, with a merge commitwrite-up
- partialMigrations roll forward, dry-run before every promotion · a rule; the drift scan is not builtwrite-up
- builtAfter every promotion, confirm a new deployment went active · a rule, after 26 days of silent failurewrite-up
- builtIncidents written down with the rule each one left behindwrite-up
- not builtContinuous integration
11 · Where Grounded falls short, in the order I would fix it
- 01The guardrails page.
- 02The audit trail page.
- 03The question-to-answer page: how a question becomes searches, and the loop controls.
- 04Service levels, a dashboard and the runbooks.
- 05Per-tenant budgets and the kill switch.
- 06A threat model with retention and deletion.
- 07The model-swap delta.
- 08Screenshots, including a trace of one question.
The first three are writing about things that exist. The rest are building.
codelessops.com/grounded/what-a-rag-agent-needs · github.com/klameer/grounded-field-notes
Hiring?
Need this kind of build on your team?
I take contract roles in finance automation and FP&A systems. Ask me anything about how Grounded works; I'm happy to walk through the parts that broke as well as the parts that worked.

