Human in the loop: where a person approves what an AI agent does

Microsoft's illustrative taxonomy for agent red teaming (Microsoft Learn, ms.date 19 Aug 2026) has three tiers. Prohibited actions, such as social scoring, are never allowed. High-risk actions, such as financial transactions and medical decisions, are allowed with human-in-the-loop confirmation. Irreversible actions, such as file deletions and system resets, are allowed with disclosure plus confirmation. Microsoft calls it illustrative guidance, and it is a usable starting point for a company's own list.

We add a fourth tier at the bottom: safe read-only tools that run without a pause, because the data they return is already visible to the user under their own permissions. The result is a short policy, usually one page, that names each tool, its tier and the approver role. It sits next to the agent's registry record, and the tier decides which pattern below applies.

Two anchors from outside engineering. The EU AI Act's Article 14 requires high-risk systems to be designed so that overseers can interpret the output, decide not to use it, override it and stop it, and stay aware of automation bias; the tracker site shows application from 2 Dec 2027 for Annex III uses, a date to verify against the Official Journal before relying on it. Microsoft's own advice for its AI approval stages is that financial transactions, legal decisions and personnel actions keep a human stage.

The binding is the part teams skip. Microsoft Agent Framework authorizes a tool call only when the approval response matches a pending request recorded in the same session; an unbound response is ignored with a warning, and the documented opt-out "can let a fabricated or replayed response approve a privileged tool call" (Microsoft Learn, ms.date 22 Oct 2026). If middleware changes already approved arguments, the framework re-requests approval or fails closed. The OpenAI Agents SDK serializes its RunState so pending approvals survive a restart, and warns to deserialize snapshots only from trusted storage.

Standing approvals need a review date. Agent Framework's harness agent keeps "always approve" rules in session state, and they are not bound to the tool instance that created them, so a replacement tool with the same name, including an MCP tool, can match a saved rule. Microsoft's advice is to review saved rules whenever the agent or tool set changes and to start a new session when an approval should not carry forward. The sticky always_approve in the OpenAI SDK deserves the same treatment.

Then exactly-once. LangGraph restarts the whole node when a run resumes from an interrupt, so side effects before the pause should be idempotent; OpenAI says plainly that approval does not guarantee exactly-once execution and recommends idempotency keys. Foundry's MCP tool guidance adds the audit line: review the requested tool name and arguments before you approve, and log approvals and tool calls. Two public examples show the shape. GitHub prevents the user who asked Copilot cloud agent to create a pull request from approving it, and workflows wait until a user with write access has reviewed the code (GitHub Docs). DKB's banking assistant acts on accounts with the customer making the final decision, and LLM-flagged sessions go to human annotation queues ranked by severity (Langfuse, 31 Aug 2026).

The framework (Microsoft Learn, ms.date 27 Feb 2025) defines overreliance as users accepting incorrect or incomplete AI outputs because the design makes errors hard to spot, notes the risk is higher where the AI can act without review, and warns that mitigations can backfire: "The mere presence of sources can make users trust AI outputs more." Its countermeasures are cognitive forcing functions, uncertainty expressions and verification aids. EU AI Act Article 14 names automation bias directly.

In practice four things keep an approval queue honest. Gate by tier and threshold, so a reviewer sees ten decisions a day rather than two hundred. Show the exact tool, arguments and target, with the agent's reasoning collapsed by default, so the reviewer reads the action rather than the argument for it. Set a timeout with a default of deny, using heartbeats where the engine supports them, so nothing waits forever and nothing runs because a reviewer went on leave. And sample: pull a weekly set of approved actions for a second person to re-check, and treat a rising approval rate with a falling rejection rate as a fatigue signal.

AI approvers do not remove the problem. Copilot Studio's multistage approvals add AI stages that return approve or reject with a rationale (preview, ms.date 4 Sep 2025); Microsoft advises human stages for financial transactions, legal decisions and personnel actions, and an AI stage can return "Analysis failed" when its criteria conflict. AI stages can screen requests; a person gives the final approval.