DEF CON 34 — Part 8 of 13

Why Human-in-the-Loop Can Fail Exactly Like MFA

By Eckhart Mehler for CISOsCISO — a perspective on cybersecurity leadership, governance and the decisions that determine whether organizations retain control.

For years, cybersecurity has responded to uncertainty by adding a human.

• A suspicious login?
• Ask the user to approve it.
• A privileged operation?
• Require confirmation.
• A financial transaction?
• Add dual control.
• An AI agent wants to perform a consequential action?
• Put a human in the loop.

The logic appears compelling:

Machines may make mistakes. Humans provide judgment.

But DEF CON 34 exposed an uncomfortable weakness in that assumption.

Humans are not simply an additional security layer.

They can become part of the attack surface.

And as enterprises deploy autonomous agents capable of sending messages, modifying infrastructure, accessing data, executing code, changing records and triggering business processes, the problem becomes increasingly important.

The question is no longer merely:

Is there a human approval step?

The more important question is:

What exactly is the human capable of verifying before the system acts?

That distinction separates meaningful oversight from security theatre.

Because a human clicking Approve is not necessarily verifying anything.

Sometimes they are simply authorizing what the system told them to believe.

We Have Seen This Failure Before

Cybersecurity already ran this experiment.

It was called multi-factor authentication.

MFA dramatically improved authentication security because stealing a password was no longer sufficient. Attackers needed another factor.

Then attackers changed tactics.

Instead of defeating the cryptography, they attacked the human operating the control.

The pattern became familiar:

Stolen password
↓

Repeated MFA requests
↓

User interruption
↓

User fatigue
↓

Approval
↓

Attacker authenticated

MFA itself was not cryptographically broken.

The approval process was exploited.

Push fatigue attacks demonstrated something fundamental about security architecture:

A control that depends on repeated human judgment can degrade under pressure.

The attacker does not necessarily need to bypass the control.

They may only need to make the human operate it incorrectly.

Agentic AI is recreating this problem at much larger scale.

From MFA Fatigue to Agent Approval Fatigue

Imagine an enterprise agent operating inside Microsoft 365, SAP, ServiceNow, Salesforce, GitHub, Azure or AWS.

Throughout the day it asks:

• Approve sending this message?
• Approve creating this ticket?
• Approve modifying this record?
• Approve calling this API?
• Approve accessing this repository?
• Approve changing this configuration?
• Approve deploying this code?
• Approve sharing this document?
• At first, users examine each request.

Then the system proves useful.

Most requests are legitimate.

The approval rate approaches 100 percent.

And gradually the cognitive meaning of the approval changes.

The user is no longer asking:

“Is this action safe?”

They are asking:

“Do I want the workflow to continue?”

That is a profound security transition.

The approval button has moved from being a decision point to being a friction-removal mechanism.

The human remains technically in the loop.

But functionally, they may already be outside it.

Research presented around DEF CON 34 under the provocative framing “Stop Pressing 1: Measuring Human Rubber-Stamping in Agent Oversight” focuses precisely on this emerging weakness: whether humans behind AI approval mechanisms are actually reading and understanding what they authorize rather than merely progressing the workflow.

The problem is increasingly recognized in agent architecture more broadly. Frequent approval requests can train reviewers to rubber-stamp actions, while excessive gating can paradoxically weaken the protection that human review was supposed to provide.

This gives attackers a new objective.

They do not necessarily need to remove the human from the loop.

They need to make the human predictable.

Approval Is Not Verification

This distinction may become one of the most important design principles for enterprise AI.

Consider two superficially similar interactions.

Approval

The agent says:

Transfer €48,200 to Supplier X?

The human clicks:

Approve

What exactly was verified?

Perhaps very little.

The human may be relying on information generated by the same agent that constructed the transaction.

The agent selected the supplier.

The agent retrieved the bank account.

The agent calculated the amount.

The agent summarized the invoice.

The agent generated the approval request.

The agent may even have decided which information the human sees.

The supposed independent control is therefore not necessarily independent.

The human may simply be approving the agent’s representation of the agent’s own proposed action.

That is approval.

Verification requires something stronger.

Verification

The system independently presents:

Beneficiary: Example GmbH
IBAN: DE••••••••4172
Amount: €48,200
Invoice: 47192
Purchase Order: 81354
Payment account: Corporate EUR 02
Execution: Immediate

The human verifies those transaction parameters against authoritative information.

Only those exact parameters can then execute.

Now the human is not approving an intention.

They are verifying a transaction.

That distinction matters enormously.

The Agent Must Not Control the Approval Reality

There is a dangerous architectural pattern emerging in early agent systems:

Agent
↓

Agent determines action

↓

Agent summarizes action

↓

Agent creates approval request

↓

Human approves
↓

gent executes

It looks controlled.

But almost the entire information chain remains under the influence of the agent.

If the agent has been manipulated through prompt injection, poisoned context, malicious tool output or compromised memory, the attacker may indirectly influence not only the proposed action but also how that action is described to the human.

That creates an obvious security problem.

The system asking for permission should not be the sole authority describing what permission means.

The safer architecture separates those functions.

Agent proposes action
↓

Independent policy system reconstructs transaction
↓

Independent interface displays consequences
↓

Human verifies transaction
↓

Execution layer enforces approved parameters

The difference is architectural independence.

The agent can propose.

It cannot define reality.

The TOCTOU Problem Comes to AI Agents

There is another subtle problem.

Suppose an agent asks:

Approve sending the quarterly financial report to our external auditor?

The user approves.

Between approval and execution, however, something changes.

Perhaps the recipient changes.

Perhaps an attachment changes.

Perhaps another tool modifies the payload.

Perhaps additional recipients are added.

Perhaps malicious context influences the agent after approval.

The user approved transaction A.

The system executed transaction B.

This is essentially a semantic version of a classic security problem:

Time of Check vs. Time of Use.

The security requirement therefore cannot merely be:

The human approved the action.

It must be:

The system executed exactly the action the human verified.

For consequential operations, the approved transaction should effectively become immutable.

One approach is to create a canonical representation of the action:

**Action

• target
• parameters
• data scope
• destination
• financial value
• identity
• timestamp**

That representation can be hashed, signed or otherwise bound to the approval.

Execution then checks:

approved transaction = requested transaction

If not:

deny and require new approval.

This turns human approval from a UX event into a security primitive.

Approval Fatigue Is an Attack Primitive

The MFA analogy becomes even more interesting when we consider attacker behavior.

An attacker manipulating an agent may deliberately generate harmless approval requests.

One.

Five.

Twenty.

Fifty.

The user learns:

Approve → workflow continues.

Then comes the malicious request.

Perhaps slightly different.

Perhaps buried among routine actions.

Perhaps presented during a high-volume period.

Perhaps phrased almost identically to previous requests.

The attacker is no longer attacking the AI model.

They are attacking the attention economics of the human control system.

That creates a possible agentic equivalent of MFA bombing:

Approval Flooding.

Research and engineering discussions around agent oversight increasingly recognize exactly this failure mode: as escalation volume increases, human review quality can decline, and adversaries may exploit the resulting fatigue by burying a consequential request among routine ones. (⁠AgentPatterns.ai)

That should matter enormously to CISOs.

Because human attention is finite.

Which means:

Human attention is a security resource.

And scarce security resources must be budgeted.

The Approval Budget

Organizations should therefore stop treating human approval as free.

Every approval consumes:

attention,

context switching,

cognitive effort,

time,

and ultimately vigilance.

Call this the:

Approval Budget

If an employee receives three genuinely consequential approval requests per week, each may receive serious attention.

If they receive 80 per day, the architecture has already failed.

This creates an important design principle:

The strongest approval system may be the one that asks for approval least often.

Routine, reversible and bounded operations should generally be governed by policy.

Humans should be reserved for situations where genuine judgment matters.

That means distinguishing between actions such as:

Read internal calendar

and

Delete customer database

rather than putting an identical confirmation dialog in front of both.

A mature agent architecture therefore needs risk-based escalation.

Not Every Agent Action Needs a Human

A useful model is to divide agent actions into four classes.

Class 1 — Autonomous

Low consequence.

Reversible.

Policy-constrained.

Examples might include:

retrieving approved internal information,

summarizing documents,

classifying tickets,

drafting content,

querying authorized systems.

No human approval should normally be required.

Class 2 — Autonomous with Monitoring

Moderate consequence but reversible.

Examples:

creating internal tickets,

updating low-risk metadata,

initiating bounded workflows,

performing approved administrative tasks.

Execution occurs automatically.

Monitoring provides oversight.

Class 3 — Explicit Approval

Material business consequence.

Examples:

external communication,

production changes,

access changes,

financial transactions,

publishing information,

changing authoritative records.

The system pauses before execution.

Class 4 — Independent Verification

High consequence or difficult-to-reverse actions.

Examples:

large financial transactions,

privileged identity changes,

destructive infrastructure actions,

security policy modifications,

release of highly sensitive information,

changes affecting physical systems.

Here, ordinary approval is insufficient.

The human must independently verify the transaction.

Potentially with separation of duties or multiple approvers.

The critical principle is:

Risk determines the strength of human involvement.

Not the fact that an AI agent happens to be involved.

Human-in-the-Loop Is Not the Same as Human Control

This is where many AI governance frameworks remain too superficial.

They ask:

Is there human oversight?

That question is almost meaningless.

A human may technically be involved while possessing almost no effective control.

Meaningful oversight requires several properties.

The human must know:

what is happening,

why it is happening,

which identity will perform it,

which systems will be affected,

which information will leave,

whether the action is reversible,

and

what the exact consequences are.

The human must also possess the authority to reject the action.

And rejection must actually stop execution.

Otherwise the human is not a security control.

They are an observer.

This Changes the Meaning of Non-Repudiation

There is another governance implication.

Organizations will eventually encounter incidents where logs show:

USER APPROVED ACTION

That may appear decisive.

But was the user shown the actual transaction?

Or merely an AI-generated summary?

Was the destination visible?

Were attachments visible?

Was the tool call visible?

Were the parameters changed afterward?

Was the user presented with 200 similar requests that day?

Was the approval interface itself influenced by attacker-controlled content?

These questions will matter for:

audit,

fraud investigations,

incident response,

regulatory accountability,

and potentially liability.

Logging only:

approved = true

will not be sufficient.

The organization may need to preserve:

what the human actually saw at the moment of approval.

That means approval evidence becomes part of the audit trail.

What CISOs Should Require

Human-in-the-loop controls for enterprise agents should therefore be treated as security architecture rather than user-interface design.

For consequential agent actions, I would require six properties.

1. Risk-Based Escalation

Do not ask humans to approve everything.

Protect their attention for consequential decisions.

2. Independent Transaction Representation

The approval interface should derive critical parameters independently from the agent’s natural-language explanation wherever possible.

3. Parameter Binding

Approval must authorize specific parameters, not a vague intent.

If those parameters change, approval expires.

4. Context Integrity

Untrusted content should not be able to manipulate the approval interface.

Agent-generated text must be clearly separated from authoritative transaction data.

5. Fail-Closed Execution

No response should not mean approval.

Timeouts, ambiguity and failed verification should stop or safely degrade the operation.

6. Approval Telemetry

Security teams should monitor:

approval frequency,

approval rates,

rejection rates,

approval latency,

repeated requests,

parameter changes after rejection,

unusual approval bursts,

and high-risk approvals by user or agent.

A user suddenly approving 150 agent actions in an hour may be the agentic equivalent of abnormal authentication telemetry.

The approval system itself should therefore become a detection source.

The Bigger Lesson from DEF CON 34

The deeper theme of this series is that modern attacks increasingly emerge from relationships rather than isolated vulnerabilities.

Human approval fits that pattern perfectly.

The human may not be compromised.

The agent may not be compromised.

The identity system may not be compromised.

The workflow engine may not be compromised.

Yet the relationship between them can still produce an unauthorized outcome.

Consider the chain:

Attacker-controlled content
↓

Agent interpretation
↓

Legitimate tool invocation
↓

Human approval request
↓

Fatigued user
↓

Legitimate credential
↓

Authorized execution

Every individual component may appear to behave correctly.

And the outcome is still malicious.

That is compositional insecurity.

The Human Is Becoming Part of the Control Plane

For decades, security architecture largely treated humans as identities.

Users authenticated.

Users received permissions.

Users performed actions.

Agentic systems introduce a different role.

The human increasingly becomes the final authorization authority for actions proposed by machines.

That means human judgment is entering the execution architecture itself.

And once something becomes part of the control plane, attackers will attempt to manipulate it.

This is why the familiar statement:

“Don’t worry — the agent can’t do that without human approval.”

should no longer reassure a CISO by itself.

The next questions should immediately be:

What does the human see?

Can they independently verify it?

How often are they asked?

Can attacker-controlled content influence the request?

Can parameters change after approval?

What exactly does the execution layer enforce?

Those questions determine whether the human is genuinely controlling the agent.

Or merely pressing Approve.

The CISO Takeaway

The industry is likely to repeat an old mistake.

We will deploy increasingly autonomous systems.

We will become uncomfortable with their authority.

And we will add human approval buttons everywhere.

That will initially feel safer.

Then approval volume will increase.

Users will adapt.

Workflows will normalize.

Attention will decline.

And attackers will learn how to exploit the gap between human presence and human understanding.

MFA taught security teams that a user pressing Approve does not prove that the authentication was legitimate.

Agentic AI will teach us the same lesson at the transaction layer.

The future security question therefore cannot be:

Was a human in the loop?

It must be:

Could the human independently understand and verify the exact consequence before execution?

Because a human in the loop is not a security control unless the human can independently verify the transaction.

And once approval becomes habitual rather than deliberative, the human is no longer the last line of defense.

The human has become another attack surface.

Next in the series: Part 9 — Security Infrastructure Can Become Offensive Infrastructure — why security tools that consume hostile data, interpret it with AI and act with privileged authority can themselves become attack paths, and how manipulated telemetry, evidence and agentic workflows can turn defensive infrastructure into offensive capability.

Publication Note & Disclaimer
This article provides security and governance analysis, not legal advice. Regulatory obligations must be assessed against the facts, jurisdictions, data types, and roles of the organizations involved.

This article reflects my personal professional perspective and does not represent the official policy or position of my employer. Drafting and editorial refinement may have been supported by commercially available AI-assisted tools. The analysis, conclusions and final curation are entirely my own.

For information regarding image credits, copyrights, trademarks and other intellectual property rights, please refer to the Imprint.