When should an AI agent require human approval?

Not at every step. If a person has to approve every read and every low-risk tool call, they will quickly start clicking mechanically. Approval is useful where an agent mistake can trigger a real consequence: financial, external, administrative, legal, security-related or difficult to reverse.

There is another important point: the level of control should not depend on how confident the model sounds. A confident model can still be wrong. We choose the control based on the consequence of the action.

A decision model: five questions before granting autonomy

For each agent action, we check five characteristics.

1. Does the operation change state?

A read is usually lower risk than a write. Not always — a read can expose sensitive data — but separating read from write is a useful starting point.

2. Does the consequence leave the internal system?

Sending an email, publishing a post, enabling an ad, issuing a document or replying to a customer is externally visible. A mistake immediately becomes a relationship, cost or reputation problem.

3. Does the operation create cost or commitment?

An agent can execute an API call correctly and still make the wrong business decision. Budget, purchasing, payment, subscriptions or resource reservations need a separate boundary.

4. Can the change be reversed easily?

Changing a label and deleting an account are not in the same class. Consider recovery cost, recovery time, data loss and the impact on other users.

5. Does the operation change security or permissions?

Granting administrator access, rotating a secret, making a resource public or changing MFA policy deserves control even if the action is technically reversible.

A practical autonomy matrix

Level	Action characteristics	Examples	Default decision
A — autonomous	read-only, low impact, tightly scoped	service status, cost report, task list	agent may execute
B — autonomous with audit	reversible write in a constrained area	add a tag, save a draft, create a task	agent may execute with explicit permission and logging
C — approval before effect	external consequence, financial effect or material change	send a message, publish, enable a campaign	human approves the specific operation
D — strong control	destructive, security-sensitive, difficult to reverse	delete data, change admin access, revoke a key	separate scope + approval + preview; often additional verification

This is an operational model, not a legal standard. A specific system should adapt the levels to its data, regulation, consequences and recovery mechanisms.

Approval should be bound to a specific consequence

A poor approval dialog looks like this:

The agent wants to perform an action. Continue? Yes / No.

The person does not know what they are authorising.

A better preview looks more like:

Operation: ENABLE CAMPAIGN
Account: Brillnet Ads / [human-readable identifier]
Campaign: Search — Pulsar CRA
Daily budget: PLN 80
Target: Poland
Change: PAUSED → ENABLED
Reversible: yes, campaign can be paused again
Consequence: after approval the campaign may begin generating spend
Decision source: task #...

Only then does an “Approve” button carry useful meaning.

Approval must not become a blanket pass for future changes

Suppose a person approves:

send this message to 12 recipients

If the agent changes the content or recipient list to 1,200 people after approval, the original approval should no longer apply.

A practical rule is:

approval is bound to a specific normalised operation. A material change to its parameters invalidates the approval and requires a new one.

An implementation can bind the approval ID to a hash of the operation parameters, a resource version or another stable representation of what was actually approved.

The model should not decide by itself whether approval can be skipped

A model can be asked to estimate risk. That can be useful as one signal. It should not be the only control.

If the policy is:

when the model considers an operation risky, it asks a human

then the model also controls the mechanism intended to contain the impact of its own mistakes.

A stronger design is:

model selects tool and parameters
↓
gateway classifies the operation using explicit policy
↓
policy engine: approval required / not required
↓
human approves if required
↓
only then execution

The model can provide the explanation. The final classification comes from system policy.

Example from our own automation: preparing a campaign is not the same as starting spend

In our internal Brillnet Ads MCP, we separate the ability to prepare or modify a campaign from the ability to activate spend.

A newly created campaign defaults to PAUSED when no other status is explicitly supplied. Saving the configuration is not consent to start spending. Enabling the operational effect requires an explicit operate permission and confirmation.

This small separation has a useful outcome:

• the agent can complete most preparation work;
• a person does not have to manually enter every field;
• budget does not start only because the model decided the configuration was ready.

The same pattern maps to other workflows:

Preparation	Effect requiring a separate decision
draft an email	send it
propose a user role	grant access
migration plan	execute migration
list files to remove	delete them
proposed price change	publish the new price
prepared payment	authorise payment

When approval may be unnecessary

Not every automation needs to be semi-automatic.

Human approval often adds little value when all of these conditions are true:

• the operation is read-only or low impact;
• the agent works inside a narrow, pre-approved scope;
• output is not published automatically;
• a mistake is easy to detect and repair;
• there is no spend or external commitment;
• permissions and security settings are unchanged;
• the action leaves enough audit evidence to reconstruct the result.

Example: an agent retrieves campaign status every day and prepares an internal report. Requiring a human to approve every read does not increase safety in proportion to the operational cost.

When approval is a minimum requirement

In our projects, I would not grant autonomous execution without separate analysis for operations that:

• send customer or public communications;
• start or increase spend;
• delete or overwrite data without a simple recovery path;
• grant, expand or revoke permissions;
• modify secrets, security configuration or MFA;
• accept legal terms on behalf of an organisation;
• act on an ambiguously identified resource;
• trigger another system that can itself perform high-impact actions.

That does not mean human approval is always the only control. For critical actions it should be one of several layers.

Watch for approval fatigue

If a system asks for approval 80 times a day, users stop reading the prompts. Approval becomes an employee CAPTCHA.

That usually points to one of three design problems:

1. the risk classification is too broad;
2. the workflow does not group logically related changes;
3. the agent has too little safe autonomy at low risk, so everything escalates.

The answer is not to hide warnings. It is to design better boundaries.

What a useful approval mechanism should do

Show the diff, not the whole object

For configuration, show before → after and the fields that change. Do not force the reviewer to compare two screens manually.

State the consequence

“Save change” is weaker than “Enable campaign — from this point it may generate spend up to PLN 80/day”.

Have a scope and lifetime

An old approval should not keep authorising an operation after the context changes.

Refuse a materially changed operation

If parameters change after approval, the approval must expire.

Leave an independent record

The audit record should say who approved what, when and under which operation identifier. It should not rely only on a model message saying “the user agreed”.

Read: AI agent audit trails

Checklist for a single agent action

Before granting autonomy, answer:

• Is the action read-only?
• Does it change business state?
• Does it create cost or a commitment?
• Is the result externally visible?
• Does it involve personal or confidential data?
• Does it change roles, permissions or security configuration?
• Is it easy to reverse?
• Is the target resource unambiguous?
• Can a retry execute the action twice?
• Can the system show a concrete preview before execution?
• Will approval be bound to the exact parameters?
• Can the resulting state be verified after execution?

If an action combines several high-risk characteristics, a single “OK” may itself be too weak a control.

Sources

• OWASP AI Agent Security Cheat Sheet — including excessive autonomy, high-impact actions, explicit approval, previews, audit and rollback.
• NIST AI Risk Management Framework — a framework for managing AI risk.

Boundary

This classification model is our engineering practice, not legal advice or a formal risk-assessment standard. Regulated, medical, financial or security-critical processes can require stronger controls.

See also MCP and AI agents: designing safer tools and Least privilege for AI agents.