AI audit: what gets audited, by whom, and the evidence you need

An enterprise customer's security questionnaire has a new section this year. It asks how your AI is governed and whether the customer may audit it. Your product uses AI in three places, and nobody has ever audited any of them.

An AI audit is an independent, evidence-based check that an AI system, and the way it is governed, meets a defined standard: your own policy, the law, ISO/IEC 42001 or a customer's requirements. It covers the data, the model, the deployment and the management system around them. Most firms meet internal and customer audits long before any certification or conformity assessment.

This guide is for CTOs, security leads and compliance owners at UK tech firms of 25 to 250 people. It separates the four exercises that all get called an AI audit, says who performs each and what each is worth to a buyer, then shows how one evidence pack per AI system can serve all four.

What is the difference between an AI audit and AI assurance?

AI assurance is the umbrella and an AI audit is one mechanism inside it. The UK Department for Science, Innovation and Technology (DSIT) says AI assurance "measures, evaluates and communicates the trustworthiness of AI systems". Its Introduction to AI assurance places audits on a spectrum of mechanisms that organisations should combine across the AI lifecycle.

The DSIT Introduction to AI assurance describes a spectrum that runs from qualitative assessments, used where there is "a high degree of uncertainty, ambiguity and subjectivity", to quantitative ones that can be measured "objectively and with a high degree of certainty". Six of the mechanisms it names matter most to a tech supplier.

So "AI audit" usually means a compliance audit, sometimes a bias audit, and occasionally a conformity assessment. The word that matters is the standard it measures against.

An AI audit without a defined standard is an opinion, not an audit.

What are the four kinds of AI audit?

A UK tech supplier usually meets four kinds of AI audit, in this order: its own internal audit, its customers' audits and AI questionnaires, third-party certification, and, only if it provides a high-risk system, EU AI Act conformity assessment. Each has a different auditor, a different standard and a different value to a buyer.

Internal audit under ISO/IEC 42001

ISO/IEC 42001:2023 expects an organisation to audit its own AI management system at planned intervals. The audit checks that the system meets the organisation's own requirements and the standard, and that it is working in practice. Clause 9.2 sets the internal audit requirement, and its results feed the management review in clause 9.3.

You don't need certification to borrow the method. Our ISO/IEC 42001 overview explains how the management system fits together. The one rule to keep is independence. The NIST AI RMF Playbook's MEASURE function asks for "internal experts who did not serve as front-line developers for the system and/or independent assessors" to be involved in regular assessments.

Customer audits and AI questionnaires

A customer audit is the one most likely to arrive first. It is usually a questionnaire section, a right-to-audit clause in the contract, or a request for evidence after an AI incident at another supplier. The customer measures you against its own policy, so the questions vary from buyer to buyer.

The answers should not vary. A firm that answers each customer from scratch is running a new audit every quarter. A firm that answers from one evidence pack is reusing one.

Certification under ISO/IEC 42001 and ISO/IEC 42006

A certification audit is a third-party conformity assessment of your AI management system against ISO/IEC 42001:2023. ISO/IEC 42006:2025, first published in July 2025, sets extra requirements for the bodies that audit and certify AI management systems, building on ISO/IEC 17021-1. It governs the certifier, not you.

DSIT notes that "it is UK government policy that where third-party conformity assessment services are sought, they should be obtained from an organisation accredited by UKAS." Check the certification body's accreditation before you sign. Whether certification is worth it, what it costs and how to choose a body are covered in our ISO 42001 certification guide.

Conformity assessment under EU AI Act Article 43

Article 43 of the EU AI Act applies to providers of high-risk AI systems. Article 16(f) requires a provider to make sure the system "undergoes the relevant conformity assessment procedure as referred to in Article 43" before it is placed on the market or put into service. A firm that sells an AI system under its own name is its provider, even when the model inside comes from a supplier. A deployer has no conformity assessment duty unless Article 25 makes it a provider, for example by putting its name on a high-risk system or substantially modifying one.

For most Annex III systems (points 2 to 8), Article 43(2) says the provider follows internal control under Annex VI, "which does not provide for the involvement of a notified body". For biometric systems in Annex III point 1, the provider may need a notified body under Annex VII, for example where harmonised standards have not been applied.

The timing has moved. Regulation (EU) 2026/1744, the Digital Omnibus on AI, sets the high-risk rules in Chapter III, Sections 1 to 3, to apply from 2 December 2027 for Annex III systems and from 2 August 2028 for Annex I systems. Our EU AI Act overview covers who counts as a provider.

If you deploy AI rather than provide it, Article 43 is not your audit.

What does an AI audit examine?

An AI audit examines four layers: the data the system learns from or processes, the model and how it was tested, the deployment and how people oversee it, and the governance around all three. A good auditor follows one AI use case through all four layers, rather than testing each layer in isolation.

The boundaries between layers deserve the closest look. A model can be well tested while nobody can show that the test matched the way the system is used today.

What should an AI audit evidence pack contain?

An AI audit evidence pack is one folder of current records for each AI system, kept so that any of the four audits can draw from it. It holds seven items: the inventory entry, the impact assessment, test results, human oversight records, monitoring logs, the incident log and, for bought-in AI, supplier evidence.

1. Inventory entry. What the system is, who owns it, what it does, which data it touches and whether you built or bought it. Every other item links back here.
2. Impact and risk assessment. The risks to people, the business and customers, the decision taken on each, and the date of the last review.
3. Test results. Performance, bias and security tests before release, with the acceptance threshold that each test was measured against. Our guide to control testing covers how to design tests that an auditor can repeat.
4. Human oversight records. Who can override or stop the system, and records showing they did review outputs.
5. Monitoring logs. Drift, error rates and usage against the thresholds you set, with alerts and responses.
6. Incident log. What went wrong, how it was handled and what changed afterwards.
7. Supplier evidence. For bought-in AI: the supplier's assurance reports, model documentation and the contract terms that govern your data.

The pack works only if it stays current. Records that are produced by the systems doing the work, rather than gathered the week before an audit, are what make that possible. Our guide to automated evidence collection sets out how.

How do you audit AI you buy rather than build?

To audit AI you buy, you audit your decision to use it, how you configured it, what the contract says and how it behaves in your context. You cannot audit the supplier's model. You can audit whether you chose it with care, tested it on your own use case and kept watching it.

For most UK tech firms of this size, bought-in AI is the common case: a large language model behind a product feature, or AI built into the software the team already uses. The customer auditing you will still ask about it, because to them it is your AI.

We recommend four checks for each bought-in system:

• Selection. Record why this supplier and model were chosen, and what assurance they offered at the time.
• Contract. Confirm what the supplier may do with your data and your customers' data, including use for training.
• Context testing. Test the system on your own use case and data, because the supplier's tests describe the supplier's use case.
• Ongoing review. Re-check the supplier's assurance when the model version changes or the supplier's terms change.

A supplier's certificate is evidence about the supplier. It is not evidence that your use of the supplier's AI is governed.

Which AI audit checklists can you use?

Four public resources give a UK firm a starting checklist: the ICO's AI and data protection risk toolkit, the EDPB expert checklist for AI auditing, DSIT's portfolio of AI assurance techniques and the NIST AI RMF Playbook's MEASURE function. None of them is a standard you can be certified against.

• The ICO toolkit. The ICO AI and data protection risk toolkit is a spreadsheet designed "to reduce the risks to individuals' rights and freedoms caused by their own AI systems". Use it wherever AI processes personal data. The ICO notes the guidance is under review after the Data (Use and Access) Act.
• The EDPB checklist. The Checklist for AI auditing, published in June 2024 under the EDPB Support Pool of Experts, sets out a method for auditing an algorithm. It is an expert report: the EDPB says "This document was not produced by the EDPB."
• The DSIT portfolio. The portfolio of AI assurance techniques shows assurance techniques in real use. DSIT says inclusion "does not represent a government endorsement of the technique or the organisation".
• NIST AI RMF MEASURE. The NIST AI RMF Playbook MEASURE function covers how to choose metrics and who should assess them. MEASURE 1.2 expects the effectiveness of existing controls to be "regularly assessed and updated".

You may come across DSIT's AI Management Essentials (AIME), a draft self-assessment tool consulted on in November 2024. Don't build on it. The government's response of 6 February 2026 says "DSIT will not be publishing AIME and therefore will not be making it a requirement of the government procurement process".

How often should you run an AI audit?

We recommend an internal AI audit of the whole programme at least once a year, plus a targeted audit of a single system whenever it changes materially. ISO/IEC 42001 leaves the interval to you, so set it by risk. Customer audits follow the contract, and certification bodies set their own audit programme.

The triggers for a targeted audit are worth writing down, so nobody has to decide in the moment:

• a new AI use case or a new customer-facing AI feature;
• a model change, including a supplier's model version update;
• a new supplier or a change to a supplier's data terms;
• an incident, a complaint or a monitoring threshold breach;
• a new customer requirement or a change in the law.

What are the common AI audit mistakes?

The common AI audit mistakes are auditing the model but not the decision it supports, scoping only the AI you built, creating evidence for the audit rather than from the work, and letting the people who built the system audit it. Each gives you an audit that passes on paper and says little about whether the AI is governed.

• Auditing the model, not the use. A model can pass every test and still be used for a decision it was never tested for.
• Leaving out bought-in AI. Your customers see one product. They do not separate your model from your supplier's.
• Evidence made for the audit. A record created the week before fieldwork shows that you prepared for the audit. It does not show that the control ran.
• The builder as auditor. The people who built a system, or prepared its evidence, are the wrong people to judge it. Independence applies whether the auditor is internal or external.

Frequently asked questions