Your Finance Agent Needs an Evaluation Harness, Not Just a Prompt

Your Finance Agent Needs an Evaluation Harness, Not Just a Prompt A finance agent can produce a convincing answer and still be wrong in the one way that matters: it can make a decision without enough evidence. That is why “add a better system prompt” is not a sufficient quality strategy for accounting automation. Prompts help define behavior, but they do not tell you whether a release reliably classifies transactions, preserves source evidence, routes ambiguity, and refuses to overreach. The practical answer is an evaluation harness: a small, repeatable test system that runs realistic finance cases through an agent and scores the result against explicit expectations. Before choosing an LLM or tool framework, define what the agent is allowed to decide. For a transaction-review agent, a useful contract might look like this: { "action": "suggest_category", "confidence": 0.91, "evidence_ids": ["bank_tx_1842", "receipt_883"], "needs_review": false, "reason": "Recurring software subscription matches the approved vendor pattern." } The important fields are not only action and confidence. An evaluation harness should also check: Does the suggestion point to evidence that actually exists? Is the evidence relevant to the transaction? Does the agent flag missing or contradictory documentation? Does it stay inside the approved chart-of-accounts scope? Is the output deterministic enough to audit and compare? A response that says “software expense” with no evidence may look useful in a demo. In production, it is an unreviewable assertion. Do not evaluate only clean examples. Clean examples measure whether the happy path works. They do not measure whether the agent knows when to stop. A compact starter dataset can contain 50 to 100 cases divided into four groups: Routine cases: recurring vendors, complete receipts, familiar amounts. Missing-evidence cases: a bank transaction exists, but the receipt or invoice does not. Conflicting cases: the vendor name suggests one category while the attached document suggests another. Adversarial cases: prompt injection inside a memo, unusual currencies, duplicate webhooks, or a transaction that falls outside the agent’s authority. Store the expected behavior as structured data, not just a paragraph written by a reviewer: { "case_id": "tx_missing_receipt_07", "expected_action": "request_evidence", "allowed_categories": [], "must_cite": [], "must_not": ["post_journal_entry", "mark_reconciled"] } This makes a regression test possible. If a prompt or model change causes the agent to classify this case anyway, the build should show a failure immediately. A single “answer quality” score hides too much. Score the workflow in layers instead. Schema validity checks whether the output can be parsed and whether required fields are present. This catches malformed tool calls and half-finished responses. Evidence grounding checks that every cited document or transaction exists and supports the claim. You can start with deterministic checks: valid IDs, matching account IDs, date-window constraints, and amount consistency. Policy compliance checks authority boundaries. A suggestion may be acceptable while an automatic journal entry is not. The evaluator should distinguish “propose,” “request review,” and “execute.” Decision quality checks the actual accounting outcome against a reviewed label. Keep this separate from writing quality. A beautifully explained wrong category is still wrong. Operational behavior checks latency, tool-call count, duplicate handling, and retry safety. A model that reaches the correct answer by creating three duplicate reconciliation records is not production-ready. Finance agents rarely operate as chat-only systems. They read transactions, fetch documents, call categorization services, and write review tasks. Your harness should record each tool call as an event: run_id=eval-2026-09-21-014 case_id=tx_duplicate_webhook_03 step=2 tool=lookup_transaction args_hash=... step=3 tool=create_review_task result=task_551 step=4 tool=create_review_task result=already_exists final_action=request_review That trace lets you test properties that are difficult to see in the final answer: A repeated event produces one task, not two. A failed document lookup does not silently become “no document.” A tool timeout is represented as an exception state. A write operation has an idempotency key. The agent does not call a privileged tool when a read-only result is enough. This is where conventional software testing and LLM evaluation meet. The language model may be probabilistic, but the side effects around it should be bounded and observable. Run the curated case set on every prompt, model, tool-schema, and policy change. Keep a versioned baseline so you can compare releases instead of relying on memory. Then add production sampling. Select a small, privacy-aware sample of completed reviews, redact sensitive values where possible, and send them through the same evaluator. Track failures by category: unsupported claim, missing citation, incorrect escalation, duplicate write, and policy violation. A useful release gate might require: 100% schema validity 100% policy compliance on adversarial cases 98% evidence-link validity zero duplicate writes in idempotency tests no regression in the reviewed decision set The exact thresholds depend on the workflow. The principle is stable: define the failure that would hurt, then make it measurable. The goal is not to eliminate every review. It is to ensure that the agent sends the right work to review with enough context to resolve it quickly. A good review item includes the source transaction, proposed action, cited evidence, detected uncertainty, and a clear reason for escalation. Tools such as Portali can fit naturally into this pattern when the accounting workflow needs a shared place for source-linked decisions rather than another opaque automation layer. The product matters less than the design principle: every automated suggestion should remain inspectable. An evaluation harness turns that principle into an engineering practice. Prompts still matter, models still matter, and good UX still matters. But the durable quality signal comes from repeatable cases, explicit authority boundaries, evidence checks, and side-effect tests. If your finance agent cannot pass those tests, it does not need a more persuasive demo. It needs a better harness.