AI Governance
AI governance framework: the operating model a regulated firm needs
Five owned capabilities, mapped to ISO 42001, the EU AI Act, NIST AI RMF and the FCA, and a 90-day start for UK regulated firms.
Key takeaways
An AI governance framework keeps a firm's use of AI within agreed risk limits. For a UK regulated firm it works only as a standing operating model: five capabilities, each with a named owner, run continuously and mapped to ISO/IEC 42001, the EU AI Act, the NIST AI RMF and the FCA's reliance on existing rules such as the SM&CR. Start with the inventory and named owners.
- Annex III date
- Under Regulation (EU) 2026/1744, the EU AI Act's high-risk rules apply to Annex III systems, which include credit scoring and life and health insurance pricing, from 2 December 2027.
- FCA position
- The FCA does not plan to introduce extra regulations for AI. It relies on existing frameworks, including senior manager accountability under the Senior Managers and Certification Regime.
- Deployer monitoring
- Deployers of high-risk AI systems under the EU AI Act must monitor each system's operation and keep its automatically generated logs, where under their control, for at least six months.
- NIST AI RMF
- A voluntary US framework whose core has four functions: govern, map, measure and manage. NIST says AI risk management should be continuous and run throughout the AI system lifecycle.
Your staff already use AI. Some of it you approved, some of it you did not, and your board or the FCA has started asking how it is governed. A policy document will not answer that question. An AI governance framework that runs as a standing operating model will.
This guide is for COOs, compliance leads and managing directors of UK regulated financial services and insurance firms with 50 to 250 staff. It sets out five capabilities, each with a named owner, maps them to ISO/IEC 42001, the EU AI Act, the NIST AI RMF and the FCA's approach, and gives you a proportionate first 90 days.
What is an AI governance framework?
An AI governance framework is the set of policies, owners and controls that keeps your firm's use of AI within agreed risk limits. For a regulated firm it works only as a standing operating model: five named capabilities, run continuously and mapped to the rules you answer to, rather than a document reviewed once a year.
Most pages on this topic list frameworks: ISO/IEC 42001, the EU AI Act, the NIST AI RMF, the OECD principles. Those are inputs. They tell you what good looks like, but none of them runs your firm's AI for you.
The UK adds a twist. The government's AI regulation white paper sets five cross-sector principles and says it will not put them "on a statutory footing initially". Instead, existing regulators apply them in their own sectors. For a financial services firm, that means the FCA, and the rules you already follow.
Why does an AI governance framework need to be an operating model?
An AI governance framework needs to be an operating model because AI changes faster than a document can. Models are updated, staff adopt new tools and suppliers add AI features to products you already use. Governance that cannot keep pace with those changes describes last quarter's AI estate, not the one your customers are exposed to today.
Traditional governance assumed that people make the decisions and systems record them. With AI, the system increasingly makes the decision and people review the logs afterwards. That shift is why a framework has to name who watches each system, how often, and who can stop it.
We describe the operating model as three trust layers carried by five capabilities. The trust layers are the questions your board and regulator will ask. The capabilities are the standing work that answers them.
| Trust layer | The question it answers | Capabilities that carry it |
|---|---|---|
| AI supply chain | Which AI do we rely on, from whom, and what could go wrong with it? | AI inventory, AI risk intelligence |
| People within guardrails | Do our staff know what they may use AI for, and do they follow it? | Compliance and policy engine, accountability architecture |
| AI operating as designed | Is each AI system still doing what we approved it to do? | Runtime governance, accountability architecture |
What are the five capabilities of an AI governance operating model?
The five capabilities of an AI governance operating model are an AI inventory, AI risk intelligence, a compliance and policy engine, runtime governance and an accountability architecture. Each is a standing activity with a named owner, not a one-off project. Together they let you show a regulator what AI you use, why it is acceptable and who answers for it.
1. AI inventory
The AI inventory is a register of every AI system the firm uses: in-house models, AI features inside supplier products and the general-purpose assistants staff use day to day. For each one it records the purpose, the data it touches, the supplier and the business owner.
We recommend giving the inventory to whoever already owns your supplier and asset registers, often the COO. The inventory is the foundation for everything else. You cannot assess, monitor or own an AI system you do not know exists.
2. AI risk intelligence
AI risk intelligence is the ongoing assessment of what each system in the inventory could get wrong and who would be harmed. It covers customer outcomes, data protection, bias, resilience and supplier failure, and it classifies each use against the regulations that apply.
For a regulated firm the first question is whether a system falls into a high-risk category. Under Annex III of the EU AI Act, that includes AI used to evaluate creditworthiness or set a credit score, and AI used for risk assessment and pricing in life and health insurance. General insurance pricing, such as motor or home cover, is not on the Annex III list. The risk or compliance lead usually owns this capability.
3. Compliance and policy engine
The compliance and policy engine turns the risk picture into rules people can follow. It sets out which AI uses are approved, which need sign-off and which are prohibited, and it keeps those rules current as systems and regulations change.
It also covers training. The EU AI Act's Article 4, as amended in 2026, asks providers and deployers to take measures to support the AI literacy of their staff. The compliance lead owns the engine, but the rules only work if each business area applies them.
4. Runtime governance
Runtime governance is the monitoring of AI systems while they are in use. For each system, its owner agrees what normal looks like, what is checked and how often, which thresholds trigger a review, and how the system is paused if something goes wrong.
The EU AI Act sets a baseline for high-risk systems. Under Article 26, deployers must monitor the system's operation and keep the logs it generates, where under their control, for at least six months. Runtime governance belongs with the business owner of each system, not with a central team that cannot see day-to-day use.
5. Accountability architecture
Accountability architecture is the structure that makes sure every AI system has one named person who answers for it. That person holds documented decision authority, can explain how the system reaches its outputs and has a protocol to follow when an AI incident occurs.
Regulators will not accept shared responsibility. They will ask who approved the model, who monitored its outputs and who could have switched it off. The board sets the architecture; a senior manager owns it.
How do ISO 42001, the EU AI Act, NIST AI RMF and the FCA map to the five capabilities?
ISO/IEC 42001, the EU AI Act, the NIST AI RMF and the FCA's approach each speak to all five capabilities, from different angles. ISO 42001 gives you a certifiable management system, the EU AI Act sets legal duties for high-risk uses, NIST offers a voluntary risk method and the FCA relies on rules you already follow.
| Capability | ISO/IEC 42001 | EU AI Act (as a deployer) | NIST AI RMF | UK approach (FCA) |
|---|---|---|---|---|
| AI inventory | The scope of the AI management system and the AI systems inside it | Knowing which systems are high-risk under Annex III, so Article 26 duties can be applied to each | Map: establishing the context and the AI systems in use | No AI register rule, but you cannot evidence existing rules for AI you cannot list |
| AI risk intelligence | AI risk assessment and AI system impact assessment | Annex III classification, including credit scoring and life and health insurance pricing | Map and Measure | Existing conduct and consumer rules apply to AI outcomes |
| Compliance and policy engine | AI policy, objectives and controls | Using systems according to the instructions for use (Article 26) and AI literacy measures (Article 4) | Govern | Principles-based, outcome-focused rules rather than AI-specific ones |
| Runtime governance | Operation, monitoring and measurement of the management system | Monitoring operation and keeping logs for at least six months (Article 26) | Measure and Manage | Existing frameworks apply; the FCA plans no extra AI rules |
| Accountability architecture | Leadership, roles and responsibilities | Human oversight by people with the necessary competence, training and authority (Article 26) | Govern | Senior manager accountability under the Senior Managers and Certification Regime (SM&CR) |
A few points on reading the table. ISO/IEC 42001, published in December 2023, specifies requirements for establishing, implementing, maintaining and continually improving an AI management system. If you already run an ISO 27001 information security management system, the management system shape will feel familiar.
The EU AI Act reaches UK firms where the output of an AI system is used in the EU. The Digital Omnibus on AI, Regulation (EU) 2026/1744, set the date from which the high-risk rules apply to Annex III systems at 2 December 2027.
The NIST AI RMF is voluntary. Its core has four functions, govern, map, measure and manage, and NIST is revising version 1.0. On the UK side, the FCA's AI approach states plainly: "We do not plan to introduce extra regulations for AI." It points firms to existing frameworks, including the SM&CR.
Why does point-in-time assurance fail for AI?
Point-in-time assurance fails for AI because it samples a moment, and AI systems change between moments. An annual audit assumed change was slow enough to sample. With AI, the snapshot is soon unrepresentative, audit trails are reconstructed after the fact and risk registers freeze while the systems they describe keep moving.
A certificate says you passed, not that you're still passing.
The NIST AI RMF core makes the same point in its own terms: risk management "should be continuous, timely, and performed throughout the AI system lifecycle". The visibility gap is real. IBM's Cost of a Data Breach Report 2025, run by the Ponemon Institute across 600 organisations breached between March 2024 and February 2025, found that among organisations with AI governance policies, only 34% regularly audit for unsanctioned AI.
The question isn't whether your controls work. It's whether you'd know if they stopped.
Continuous assurance does not mean continuous effort. It means each capability produces evidence as a side effect of running: the inventory updates when procurement approves a supplier, monitoring records each check, and the owner signs off exceptions as they happen.
Who is accountable for AI in a regulated firm?
In a UK regulated firm, accountability for AI sits with named senior managers under the rules you already follow. The FCA has no AI-specific accountability rule; it relies on the Senior Managers and Certification Regime. Your framework should name one person for each AI system and one senior manager for the framework as a whole.
We have set out the full argument in AI accountability: who answers when the AI gets it wrong?. In short, a committee can advise, but only a person can be held to account.
What should the first 90 days of an AI governance framework look like?
The first 90 days of an AI governance framework should build the inventory and name owners first, then assess risk and set rules, then switch on monitoring. A firm of 50 to 250 staff does not need a large programme to start. It needs a register, a set of owners and a short policy that people can follow.
- Days 1 to 30: inventory and owners. List every AI system in use, including supplier features and staff tools found through procurement records and a short staff survey. Name a business owner for each system and a senior manager for the framework.
- Days 31 to 60: risk and rules. Assess each system's risk and check it against Annex III. Write a one-page AI use policy that sorts uses into approved, approval needed and prohibited, and brief staff on it.
- Days 61 to 90: monitoring and evidence. Agree what each owner checks, how often and what triggers a pause. Confirm logs are kept. Report the inventory, risk ratings and owners to the board, and set the next review dates.
At the end of 90 days you will not be finished. You will have a framework that runs, which is the point. ISO 42001 certification, if you choose it, is easier to reach from a working model than from a policy pack.
What are the most common AI governance framework mistakes?
The most common AI governance framework mistakes are writing policy before building the inventory, giving AI to a committee instead of a named person, treating one framework as the whole programme and banning AI by default. Each one leaves the firm with documents that look complete while the AI estate goes unwatched.
- Policy before inventory. A policy written before you know what AI you use governs an imagined estate.
- A committee as owner. Committees spread responsibility thin; regulators ask for a name.
- One framework as the programme. ISO 42001, the EU AI Act and the NIST AI RMF answer different questions. Pick a backbone, then map the others to it.
- Banning AI by default. Staff route around bans. We have written about one organisation that moved from "banned unless approved" to "here's how to use it safely", and the firms moving fastest built trust with regulators, boards and their people rather than writing the strictest policies.
An AI policy nobody monitors is a statement of intent, not a control.
Frequently asked questions
No. ISO/IEC 42001 is a voluntary standard, and you can run the five capabilities without certifying. Certification is useful when customers, partners or your board want independent proof that the framework operates. We recommend building the operating model first and deciding on certification once it runs.
The EU AI Act can apply to a UK firm that provides or deploys an AI system whose output is used in the EU. If you serve EU customers with credit scoring or life and health insurance pricing that uses AI, check those systems against Annex III before 2 December 2027.
An AI governance policy is one document that sets the rules for using AI. An AI governance framework is the wider operating model: the inventory, risk assessment, policy, monitoring and named owners that make the policy work. The policy is one output of the framework's compliance and policy engine.
We recommend one senior manager owns the framework, usually the person already accountable for operations or risk under the SM&CR. Each AI system then has its own business owner. In a smaller firm one person may hold several of the five capabilities, which is fine as long as each is named.
