The short answer
The five durable human-in-the-loop patterns are: review-before-send (every output reviewed, used for high-stakes customer comms), approval-on-exception (auto-execute within policy, escalate outside, used for billing and routing), sampled review (auto-execute, audit a percentage, used for high-volume low-stakes work), dual-control (two approvers, used for high-dollar actions), and shadow mode (agent runs alongside humans without acting, used for new agents in calibration). The mistake is using one pattern for every workflow.
Why the right pattern matters
Too much human review and you destroy the economics of the agent — the human becomes the bottleneck. Too little and you destroy trust the first time the agent makes a costly error. The five patterns below each have a sweet spot; the design decision is matching the pattern to the workflow.
Review-before-send
Every agent output reviewed by a human before it leaves the company. Right for high-stakes customer communications (proposals, RFP responses, sensitive support replies). Economics work when the agent compresses 80% of the drafting effort and the human spends time on review and judgment, not transcription.
Approval-on-exception
Agent acts autonomously within a documented policy envelope; anything outside the envelope routes to a human approver. Right for invoice drafting (auto-send within pre-approved customer/amount bands, escalate outside), Tier-1 support (auto-resolve on confident pre-approved intents, escalate otherwise), and reconciliation (auto-correct within tolerance, escalate above).
Sampled review
Agent acts autonomously; a randomized percentage of outputs are reviewed by humans for quality assurance. Right for high-volume low-stakes work (categorization, summarization, internal note generation) where 100% review is uneconomic and zero review is reckless. Typical sample: 5–10% with weekly review and feedback into the eval suite.
Dual-control
Two independent humans must approve before the agent executes. Right for high-dollar actions (large refunds, contract changes, write-offs) where SOX, internal controls, or risk policy require separation of duties. The agent's role is drafting and routing; humans own the actual decision.
Shadow mode
Agent runs alongside the human operation but does not act — its outputs are logged and compared to what humans actually did. Right for the first 2–4 weeks of any new agent, especially in regulated environments. Calibration data from shadow mode is the highest-quality eval data you will ever get.
Most agents combine patterns
A mature agent often runs shadow mode → sampled review → approval-on-exception as confidence grows and policy envelopes are refined. Designing for graduation between patterns from day one is what lets agents earn autonomy without losing governance.
Key takeaways
- Match the pattern to the workflow, not the workflow to the pattern.
- Shadow mode is the cheapest, highest-fidelity calibration tool available.
- Approval-on-exception requires a documented, written policy envelope to work.
- Graduate agents through patterns as they earn confidence — do not pick one and freeze.
Frequently asked questions
How long should shadow mode run?+
2–4 weeks for most agents; longer in regulated environments. Long enough to surface the failure modes that only production traffic exposes.
Who writes the policy envelope for approval-on-exception?+
The operating team owner, with the implementation team facilitating. If engineering writes it alone, it will not survive the first edge case the business actually cares about.
Does sampled review need to be truly random?+
Random plus a confidence-weighted sample. Pure random misses low-confidence outputs that deserve more scrutiny; pure confidence-weighted misses systematic blind spots in the agent's confidence calibration.