Human in the loop: designing human oversight you can evidence

The enterprise customer's questionnaire has a new line: "Is there a human in the loop for AI-driven decisions?" Your product team says yes, because someone clicks approve. The harder question is whether that click would survive a regulator, an unhappy data subject or the customer's own auditor.

Human in the loop means a person reviews an AI output, and can change or stop it, before it takes effect. Human on the loop means a person monitors the system and can step in. Human out of the loop means no one does. The legal test is whether the human involvement is meaningful, not whether someone clicks approve.

This guide is for CTOs, product leads and heads of security at UK B2B tech firms building AI features for regulated customers. It shows how to choose an oversight mode, design a review that counts, and keep the evidence that answers the questionnaire.

What does human in the loop mean in AI governance?

In AI governance, human in the loop is a control. A named person reviews an AI output before it takes effect, with the information, time and authority to change or reject it. It sits alongside two other oversight modes, and the right one depends on what the decision does to the person on the receiving end.

The phrase also has an older meaning in machine learning, where people label data to train a model. That is a development technique, not a governance control, and this guide leaves it aside.

The three modes differ in when the human acts and what they can change:

How do you choose between human in the loop and human on the loop?

Choose by decision impact. If an output produces a legal or similarly significant effect on a person, put a human in the loop. If the effect is moderate and easy to reverse, a human on the loop with sampling and a stop control is usually proportionate. Keep a human out of the loop only for low-impact outputs.

We recommend asking four questions about each AI feature, in this order:

1. Does the output decide something about a person? Hiring, credit, insurance pricing, access to a service and account closure all count. Ranking, summarising and drafting usually do not, unless a person acts on them without checking.
2. Is the effect legal or similarly significant? That is the threshold UK GDPR uses for its automated decision-making rules, so it is the right line for UK firms to draw.
3. Can the outcome be reversed quickly and fully? A wrong recommendation that a person corrects next day is different from a rejected loan application.
4. Is the feature high-risk under the EU AI Act? If so, oversight is a legal duty with a defined shape, set out below.

Two "yes" answers to the first two questions point to human in the loop. A "no" on reversibility pushes a borderline case up a mode, never down.

The mode is not a product preference. It is a risk decision, and it belongs in your AI governance operating model with an owner who signs it off.

What does UK GDPR Article 22A mean by meaningful human involvement?

UK GDPR Article 22A treats a decision as based solely on automated processing when there is no meaningful human involvement in taking it. A significant decision is one with a legal or similarly significant effect. The test came in with the Data (Use and Access) Act 2025 and has applied in full since 5 February 2026.

Section 80 of the Data (Use and Access) Act 2025 replaced the old Article 22 with Articles 22A to 22D. SI 2026/82 brought section 80 fully into force on 5 February 2026.

The wording matters. Article 22A says a decision is solely automated "if there is no meaningful human involvement in the taking of the decision". When judging that, you must consider the extent to which the decision is reached by profiling.

A reviewer who cannot change the outcome is not involvement. A reviewer who approves every output unseen is not meaningful. In both cases the law treats the decision as solely automated, whatever your process diagram shows.

If a significant decision is solely automated, the rest of the regime applies:

• Restrictions (Article 22B). Solely automated significant decisions based on special category data need explicit consent, or a contract or legal basis plus a substantial public interest condition. Any solely automated significant decision is also barred where the processing relies on the new recognised legitimate interests basis.
• Safeguards (Article 22C). The controller must give the person information about the decision and let them make representations, obtain human intervention and contest the decision.
• Further rules (Article 22D). The Secretary of State can make regulations saying when there is, or is not, meaningful human involvement.

So human in the loop does two jobs under UK GDPR. A meaningful review can take a decision outside the solely automated category. Where it does not, human intervention on request becomes a safeguard you must offer.

What does EU AI Act Article 14 require of human oversight?

EU AI Act Article 14 requires high-risk AI systems to be designed so people can oversee them effectively while in use. Overseers must be able to understand the system's limits, stay aware of automation bias, interpret its output, override it and stop it. Article 26(2) then requires deployers to give that role to competent, trained people with authority and support.

These duties apply to high-risk systems only. They do not attach to every AI feature. For a B2B SaaS firm, the Annex III categories most likely to catch a product are employment uses (recruitment, task allocation, performance monitoring), creditworthiness assessment, and risk assessment and pricing for life and health insurance. Our EU AI Act overview covers how classification works.

Article 14(4) of Regulation (EU) 2024/1689 lists what the people assigned to oversight must be enabled to do, as appropriate and proportionate:

1. Understand the system's capacities and limitations, and monitor its operation for anomalies.
2. Stay aware of automation bias, the tendency to over-rely on the system's output.
3. Interpret the output correctly.
4. Decide not to use the system, or disregard, override or reverse its output.
5. Intervene in or interrupt the system, through a stop button or similar procedure.

Article 14 binds the provider at design time. Article 26(2) binds the deployer: oversight goes to "natural persons who have the necessary competence, training and authority, as well as the necessary support". Article 26(5) adds a duty to monitor the system against its instructions for use, and Article 26(6) a duty to keep the logs the system generates for at least six months.

The timing has moved. Regulation (EU) 2026/1744, the Digital Omnibus on AI, amends Article 113 so that Chapter III, Sections 1 to 3, apply to Annex III high-risk systems from 2 December 2027. Those sections include Articles 14 and 26. For high-risk systems under Annex I, the date is 2 August 2028.

Your customers will not wait for December 2027. A regulated buyer that will deploy your feature as a high-risk system in the EU needs your oversight design to support its own Article 26 duty.

How do you design a human review that is meaningful?

Design the review as a control with six parts: the decision and its mode, the reviewer's authority, the information the reviewer sees, the time allowed, the sampling and escalation thresholds, and the record each review leaves. Each part answers one way the review could become a formality, so leaving one out weakens the rest.

1. Name the decision and the mode. Write down what the AI output decides, who it affects and which oversight mode applies, with the reason.
2. Give the reviewer real authority. The reviewer must be able to reject or change the output without a second approval, and without a target that penalises overrides.
3. Show the reviewer what they need. That means the inputs the system used, its output, any confidence signal or explanation available, and the known limits of the model. A reviewer shown only the recommendation can only agree with it.
4. Allow enough time. Set the expected review time from the complexity of the case, not from queue targets. A review queue sized to clear in seconds per case is a decision to automate.
5. Set sampling and escalation thresholds. For human on the loop, define what share of outputs is sampled, which signals trigger a full review, and who can stop the system. For human in the loop, define which cases go to a second reviewer.
6. Record each review. Log who reviewed, what they saw, what they decided and why, in the system that does the work.

Who owns the AI system, and answers when it goes wrong, is a separate question. Our guide to AI accountability covers the named owner. Autonomous agents that act without a person in the path raise different questions again, about permissions and approval gates, which belong in their own design.

Why does automation bias turn review into rubber-stamping?

Automation bias is the tendency to over-rely on an automated output. It turns human review into rubber-stamping: the reviewer sees a confident recommendation, has little time and no reason to doubt it, and approves. Article 14(4)(b) of the EU AI Act names automation bias as something overseers must stay aware of.

Rubber-stamping is the main way human in the loop fails, and it fails quietly. The control looks healthy on a process map because every output has an approval against it.

An approval rate of nearly 100% is not proof the model is right. It may be proof nobody is checking.

Three signals suggest review has become a formality:

• Override rates near zero across a period in which the model's error rate is known to be above zero.
• Review times too short for a reviewer to have read the inputs.
• Reviewers who cannot explain a sample of their own approvals when asked.

The fixes follow from the design above. Show reviewers the evidence before the recommendation where you can. Seed known-wrong cases into the queue and track whether reviewers catch them. Train reviewers on the model's limits, then refresh that training when the model changes.

What evidence proves human in the loop is working?

Evidence that human in the loop works shows the design, the people and the operation. Keep the oversight design record, reviewer training and authority records, review logs, override and escalation rates, and the results of periodic tests. Together they answer the questionnaire question with records rather than a yes.

Treat these as a control you test, not a document you write once. Our guide to control testing covers how to test design and operating effectiveness, which is the same question a customer's auditor will ask of your oversight.

How does human in the loop map to ISO/IEC 42001 and NIST AI RMF?

Human in the loop maps to the impact assessment and responsible-use parts of ISO/IEC 42001:2023, and to the GOVERN and MAP functions of the NIST AI RMF. In both frameworks, oversight is something the organisation decides, documents and assigns. The oversight design record above gives you that evidence once and lets you reuse it for both.

ISO/IEC 42001:2023. The standard expects an AI system impact assessment as part of planning, and its Annex A reference controls include objectives on assessing impacts and on the use of AI systems. Your oversight mode and reviewer design are natural outputs of the impact assessment, and the review logs are evidence for the use controls. Our ISO/IEC 42001 overview covers the management system around them.

NIST AI RMF. The NIST AI RMF Playbook states GOVERN 3.2 as "Policies and procedures are in place to define and differentiate roles and responsibilities for human-AI configurations and oversight of AI systems." In the MAP function, subcategory 3.5 asks that processes for human oversight are "defined, assessed, and documented". Our NIST AI RMF overview sets out the four functions.

Neither framework tells you which decisions need a human in the loop. They expect you to decide, write it down and show it runs.

Common mistakes with human in the loop

The most common mistakes are treating an approve button as oversight, applying one mode to every AI feature, measuring reviewers on speed, and keeping no record of what reviewers changed. Each one leaves a firm saying "yes" to the questionnaire without the evidence to back the answer.

• The approve button as the control. A reviewer with no authority, no context and no time is a step in a workflow, not oversight.
• One mode for everything. Human in the loop on every output exhausts reviewers and breeds rubber-stamping. Match the mode to the decision.
• Speed targets for reviewers. A target for cases per hour, with no measure of quality, rewards approving without reading.
• No record of overrides. Without override data, you cannot show review changes anything.
• Claiming EU AI Act duties for every feature. Article 14 applies to high-risk systems. Overstating it in a questionnaire creates commitments you then have to evidence.

A control that never says no has not been tested.

Frequently asked questions