AI Governance Framework: A Practical Build Guide

An AI governance framework is the structured way your organisation decides which AI tools to allow, what rules apply to them, and who answers for that decision if a customer or regulator asks. Without one, AI adoption happens tool by tool, department by department, with no one able to say what's approved and why. That's fine until someone asks: a customer's security team, an auditor, a new hire trying to work out if they're allowed to use Claude for client work. This guide sets out the four components a framework actually needs, how the big published models compare to building your own, and how to keep the whole thing from collapsing under its own bureaucracy.

The Four Components of a Working Framework

Strip away the consultancy language and every credible AI governance framework reduces to four parts. A policy: your AI usage policy, version-controlled and distributed, not sitting as a PDF nobody's opened since it was written. A registry: a live list of approved AI tools, their risk rating, and what they're approved for. A process: how a new tool gets requested, evaluated and either approved or rejected, so 'can I use X' has a real answer. And attestation: proof staff have read and accepted the policy, refreshed at least once a year. Miss any one of the four and the framework has a hole a shadow-AI tool will find. A policy with no registry tells staff the rules but not which tools meet them. A registry with no attestation tells you what's approved but not whether anyone's actually read the rules that go with it. The four only work as a set.

NIST AI RMF vs ISO 42001 vs Building Your Own

The two big published options solve slightly different problems. The NIST AI RMF (see NIST's own framework overview for the primary source) is a US-originated, voluntary framework organised around four functions: Govern (set policy and accountability), Map (identify the AI systems in use and the context they operate in), Measure (assess and track the risks), and Manage (act on what you've found). It's strong on risk categorisation but doesn't lead to a certificate. ISO 42001 (see the ISO standard page) is a certifiable management-system standard, useful if a customer or contract specifically asks for it. Neither is mandatory for most small teams, and both are more detailed than a 20-person company needs to operate safely. The pragmatic move is to borrow the thinking from both, risk-based tool evaluation from NIST, a management-system structure from ISO, without adopting either wholesale.

Do You Need a Formal Framework If You're Small?

If you're under roughly 50 people and not selling into regulated industries, you don't need the formal version of either published framework. You do need the four components above, just implemented lightly: one document, one spreadsheet or tool, one approval step. The test isn't company size, it's exposure: a ten-person healthcare startup handling patient data needs more rigour than a 200-person marketing agency that only uses AI for internal drafting. See our AI governance checklist for a size-agnostic version of this test.

What It Looks Like Without One

A 30-person logistics company found this out the slow way. Different teams had quietly adopted six different AI tools over a year, each approved (or just tolerated) by whoever ran that team, with no shared record of which ones trained on data or held a proper contract. When a customer's security team asked for a list of AI tools touching their shipment data, nobody could produce one confidently, and the honest answer took two weeks of chasing people down to assemble. None of the six tools turned out to be a real problem. The two-week scramble was the actual cost, and it's exactly what a registry with one owner would have answered in an afternoon.

Governance Without Bureaucracy

The framework's biggest enemy isn't a bad policy, it's an unused one. Multiple approval committees, quarterly risk-matrix reviews, and 40-page assessment templates guarantee people route around the framework to get their work done, which recreates the shadow-AI problem the framework was meant to solve. Keep each of the four components to something one person can maintain in under an hour a month. If maintaining your framework takes longer than that, it's too heavy for a team your size.

Who Should Own It?

Name one person: typically the COO, Head of IT, or a designated compliance lead. Their job is to approve new tools against the registry criteria, keep the policy current, and answer employee questions. Without a named owner, decisions default to whoever's asked first, and the same tool ends up approved by one manager and blocked by another. That inconsistency is what shows up badly in a customer audit. It doesn't need to be a full-time role, most owners spend an hour or two a month on it once the initial rollout is done, but it does need to be one specific name people can email, not a shared inbox that quietly stops getting checked.

What Happens When a New AI Tool Shows Up Tomorrow?

This is the real test of a framework: not the document you wrote in January, but what happens when someone in sales asks to use a new AI note-taker in March. A working process answers that in days, not months: check the tool against your risk criteria (does it train on data, does it offer a business tier, does it have a DPA), add it to the registry with a rating, and tell the requester yes, no, or yes-with-conditions. Our AI Tool Risk Directory already has this assessment done for 60-plus common tools, which usually turns a days-long evaluation into a five-minute lookup.

What a Registry Entry Should Actually Contain

A registry is only useful if it answers the questions people actually ask before they use a tool. At minimum, each entry needs: the tool name and vendor, whether it trains on your inputs by default, its data retention window, whether a DPA and (where relevant) a BAA are available, the risk tier you've assigned it, and the specific uses it's approved for. A tool can be approved for internal drafting and blocked for anything touching customer data at the same time; the registry is what makes that distinction visible instead of relying on someone remembering a conversation from three months ago.

Start With the Policy, Not the Org Chart

Building a framework doesn't have to start with a strategy document. Start with ModelCharter's policy generator for component one, add the tool directory for component two, and you already have half a working framework before lunch. The process and attestation layers, components three and four, are usually the fastest way to get from zero to something you could show a customer's security team next week. If you're tempted to wait until you've picked between NIST and ISO before starting, don't; the four-component version works regardless of which published framework you eventually reference, and it's the part that actually gets used day to day.