Eight questions a bank's architecture board asks before an agent touches a workflow

d whether the bank can still walk away in eighteen months. I keep eight questions for this. They come from public frameworks (EU AI Act risk classification, DORA's third-party register, GDPR/DPIA, ordinary model risk validation) plus the practice of a platform lead I work with who has spent years on the buying side inside a European bank, writing the proof-of-value criteria and rejecting demos. Nothing here is client-specific.

It is the order the questions arrive in. If the answer is "we're not a high-risk system", ask who wrote that down and when. A classification without a named owner and a date is an opinion, not a position. The follow-up that ends most conversations: what changes in your architecture if the classification is wrong?

"Cloud" is not an answer. Region, sub-processor chain, and whether any prompt or embedding leaves the EU. Ask for the sub-processor list as a document, not a sentence. Banks are being asked to produce a DORA third-party register.

If the vendor cannot slot into it, procurement stalls for months and nobody blames the model. Not "read-only access". Which tools, which scopes, which systems, and where the permission is enforced: in the prompt (an aspiration), in the orchestrator (better), or in the identity layer the bank already controls (credible). An agent with a service account that can call everything is an insider threat with good manners.

A ceiling that lives in the agent's own config is one pull request away from being a suggestion. The number that matters is not a public score. It is: which 200 cases did you choose, who labelled the expected answer, what is the pass threshold, and what happens on a fail? Teams that have done this work have a harness sitting there and can show it in ten minutes.

Teams that haven't talk about model quality in adjectives. Human-in-the-loop is a design claim until you can point at the screen. What is shown to the reviewer, what is the default action if they do nothing, how long a review takes, and can an operator kill the whole workflow in one action without a ticket. Approval mode belongs per action, not per system: a draft and a wire transfer should not share a setting.

I choose on two tests, whether the action can be undone and how wide the damage is when it is wrong. A person approving ten actions a day can be a review. A person approving ten thousand is a rubber stamp, and a rubber stamp is not oversight under Article 14 of the AI Act. Per action: input, tool call, retrieved context, output, model version, prompt version, reviewer decision.

If prompt versions are not immutable and addressable, the bank cannot reconstruct a decision six months later, and a supervisor will eventually ask. On-the-loop oversight quietly becomes a claim about a person who has since left. Model upgrades change behaviour silently. Ask for the change process: who signs off, what regression suite runs, is there a canary, can the bank pin a version and hold it.

"We always use the latest" is a governance answer of no. Data export format, model independence (can the workflow run on another provider), termination terms, and who owns the prompts and the evaluation set. If the prompts live in the vendor's tenant and cannot be exported, the bank has bought a dependency, not a system. Each one is a decision the bank cannot delegate after signature.

Everything else (latency, cost per call, UI quality) is negotiable during a pilot. These eight, answered badly, mean no pilot. "Our platform is model-agnostic." Then every integration is bespoke. "Security is handled by our cloud provider." The bank's contract is with you.

"We follow best practices for responsible AI." Name the practice. "Human oversight is configurable." Configured by whom, defaulting to what. "We can provide documentation on request." During procurement, on request means never. Answer the eight out loud, in order, in under twelve minutes, in front of someone who will interrupt.

Vendors who can do that are rare. That is exactly why an evaluator who can ask in this order is worth listening to. If you are the one being evaluated, this is the order the room will ask in anyway. Better to arrive with the answers.

This started as a reply to Conor Bronsdon's piece on choosing in-the-loop or on-the-loop per action, where the fourth question came up first.