AI governance framework: the components and how to build one

What an AI governance framework actually is

Strip away the branding and a framework is the answer to one question: how does this company use AI without betting the business on it? It names who is accountable, what the rules are, which systems carry real risk, what controls those systems get, and who is watching them in production. It is the difference between "we have AI everywhere" and "we know where our AI is and can stand behind it."

The good news is that you don't start from a blank page. The concept is settled. Definitions, the risk-based lifecycle, and the major frameworks have converged, and the academic literature agrees the goals are broadly agreed even where the mechanics are not. The hard part, the part surveys keep flagging, is turning any framework into running practice: 75 percent of organizations report having an AI policy, but far fewer have incident playbooks or named roles behind it (Pacific AI 2025 governance survey). So the work is less about choosing a framework and more about operating one.

The components every framework shares

Different frameworks number and name these differently, but they cover the same nine parts. If you can point to each one in your own program, you have a framework. If one is missing, that is your next gap.

The frameworks that matter, and which fits you

You adopt an existing framework rather than write your own vocabulary. Three names carry the field, and they were built to interoperate:

• The NIST AI RMF is a voluntary US framework built on four functions (Govern, Map, Measure, Manage). It gives you a method and a shared vocabulary, with no audit and no certificate. It is the default reference for US companies and the one most contracts now name.
• ISO/IEC 42001 is the certifiable international standard for an AI management system: clauses 4 to 10 plus 38 Annex A controls that an accredited auditor checks. You reach for it when a customer, regulator, or board wants third-party proof.
• The EU AI Act is binding law with deadlines, for any AI whose reach touches the EU market. It is the only one of the three that can fine you.

You do not pick one and ignore the rest. The sane approach is to build each governance capability once and know where it lands in all three. Published crosswalk analyses put the NIST AI RMF or ISO 42001 at roughly 60 to 70 percent of the EU AI Act's management-system and risk-governance requirements, and NIST publishes its own official RMF-to-ISO-42001 crosswalk. We put all three side by side, capability by capability, in the AI compliance framework crosswalk, which is the page to read once you have decided which regimes apply to you.

How to build one, in order

The order matters more than the paperwork. The most common failure is reaching for a 40-page policy and a committee before anyone has listed what the company is actually running. A realistic sequence for a medium-to-large company:

1. Name an owner and write the principles. One accountable person or a small committee, and five or six commitments you will enforce. This can happen in a week and it is what turns intent into a program.
2. Inventory your AI. List every system, vendor tool, and dataset, including shadow AI a department expensed on a card. This alone puts most companies ahead of where they were, and it is the step teams most often skip.
3. Tier the inventory by risk. Rate each system by likelihood and impact so scrutiny follows the stakes. A model that affects a decision about a person needs far more than an internal summarizing tool. A scoring rubric makes this repeatable rather than a debate.
4. Put controls on the systems that earned them. Access rules, data boundaries, testing, human oversight, and documentation, sized to each tier. Do the high-risk systems first and write down what you are deferring and why.
5. Stand up monitoring and an incident path. Decide how you watch systems in production and what happens when one misbehaves, before you need it rather than during the incident.
6. Set a review cadence and iterate. Revisit the inventory, tiers, and controls on a schedule, and re-check the framework against the standards, which are themselves still being revised. This is a living baseline, not a one-time push.

Overlay this on what you already run. If you have ISO 27001, SOC 2, or the NIST Cybersecurity Framework, a lot of the access, monitoring, and vendor controls already exist; you are adding the AI-specific parts (model documentation, evaluation, drift monitoring, impact assessment) on top, not starting over.

A framework outline you can adapt

Below is a working skeleton for an AI governance framework document. It is not the standard itself and not legal advice; it is a starting structure to fill in with your own systems, roles, and risk decisions. It maps cleanly to the components above and to the NIST, ISO, and EU regimes. Copy it, cut what doesn't apply, and keep the version-one short.

1. 1. Purpose and scope. Why this framework exists and what it covers: which entities, which AI systems (built and bought), and what is explicitly out of scope.
2. 2. Principles. The commitments every AI use must satisfy. Keep it to five or six you will actually enforce, not a wish list.
3. 3. Roles and accountability. The named owner, the committee or forum, and a simple split of decision rights. Who approves a high-risk use, and who can stop one.
4. 4. AI inventory. How systems get registered, what you record about each (owner, purpose, data, vendor, tier), and how the list stays current.
5. 5. Risk tiering. The method for rating and tiering each system, and what each tier means. This is where a scoring rubric plugs in.
6. 6. Controls by tier. The minimum controls each tier requires: access, data handling, testing, documentation, and human review.
7. 7. Human oversight. Which decisions require a human in the loop, what that person is accountable for, and how they intervene or override.
8. 8. Data governance. What data may and may not train or feed an AI system, quality and provenance expectations, and privacy boundaries.
9. 9. Procurement and third parties. How AI vendors are assessed, the clauses your contracts require, and how vendor risk is owned internally.
10. 10. Monitoring, incidents, and review. Production monitoring, an AI incident process, and the cadence for reviewing the whole framework against changing rules.

The point of writing it down is not the document; it is that a new hire, an auditor, or a customer can read one thing and understand how you handle AI. A framework nobody can find is not a framework.

How much framework is enough

More governance is not automatically better. The same literature that maps the field also warns about "redundancy, organizational confusion, and superficial compliance" when companies pile on overlapping controls and produce theater instead of oversight. For a resource-constrained company the honest target is the smallest framework that actually covers your real risks: the inventory, tiers, and controls on the systems that matter, run by a named owner. You can always add rigor where evidence shows you need it. You rarely recover the trust lost to a policy everyone learned to ignore.

If you would rather have this built against your actual systems than assemble it from a template, that is the fixed-scope AI risk assessment (a stated scope, a report you keep, $20,000 to $80,000 depending on scope), and the standing version is AI governance consulting. The related pages that go deeper: the AI governance committee that runs the framework, the AI risk assessment template that powers the tiering step, and the AI governance hub for the wider map.

One line of hygiene: this page is general information, not legal advice. For obligations that apply to your specific systems, talk to counsel.