TL;DR

Enterprise AI governance is how a company decides which AI systems can be used, who is responsible for them, what review they need before release, and which controls stay in place afterwards.

A strong program gives the organisation a current view of the AI it uses, a clear approval path, risk criteria that change the review required, and records that show what was tested and approved. It also has to cover third-party AI, embedded SaaS features, RAG systems, agents, connectors and shadow AI.

Table des matières

Enterprise AI Governance in Practice

The best way to understand enterprise AI governance is as the structure behind how AI is selected, developed, purchased, deployed, changed and retired across a business. In 2026, that structure has to extend beyond the model itself because the risk also sits in the data, permissions, integrations and business process around it.

  • internally developed machine learning and generative AI systems
  • foundation-model APIs and RAG applications
  • third-party copilots and AI embedded inside existing SaaS products
  • AI agents, browser or computer-use systems, APIs and MCP servers
  • employee-built automations and unapproved or shadow AI

For each material use case, the organisation should be able to establish who owns it, what it is meant to do, what data and systems it can reach, what decisions or actions it can influence, how it was assessed and what would force it through another review. A useful inventory therefore records the use case and operating context, not only the model name.

The Enterprise AI Governance Operating Model

Enterprise AI governance gets complicated when several teams touch the same use case. The operating model brings order to that process by defining who owns each decision, who reviews it, and when a higher-risk use case needs a different approval path.

Governance Roles and Decision Rights

  • Executive or board oversight: Sets risk appetite, reviews material exposure and approves major exceptions.
  • AI governance or risk function: Maintains the framework, risk methodology, control library and escalation process.
  • Business owner: Owns the purpose, expected outcome and continued appropriateness of the use case.
  • Technical or model owner: Owns implementation, evaluation, release and technical monitoring.
  • Security, privacy, legal and compliance: Perform specialist reviews when the risk profile requires them.

The risk tier should determine which functions review the system and who holds final approval authority.

Risk level Approval and assurance Monitoring
Lower Business-owner approval, standard controls and lightweight evidence Standard logging and scheduled review
Modéré Documented assessment, relevant specialist review and defined testing Periodic review with defined indicators
Higher Senior approval, deeper assurance, explicit human oversight and exception handling Enhanced or continuous monitoring and shorter review cycles

Organisations can use more than three levels. What matters is that each level consistently changes the review, evidence and monitoring required.

Factors That Change AI Risk

When you classify an AI system, I recommend looking at these seven things:

  1. Business impact: What happens if the system gives the wrong answer or fails at a critical point? A bad product recommendation is very different from an incorrect fraud decision or payment instruction.
  2. Autonomy: How much can the system do on its own? Drafting text, recommending an action, approving a request, and executing a transaction all carry different levels of exposure.
  3. Data access: What can the system see? This could include customer records, employee data, financial information, health data, confidential documents, or credentials.
  4. Who is affected: Does the system influence customers, employees, job applicants, patients, borrowers, or other people whose rights or outcomes could be affected?
  5. Scale: How widely is it used? A mistake in an internal pilot may affect a handful of people. The same mistake in a customer-facing workflow could be repeated thousands of times.
  6. Reversibility: How easy is it to correct a bad outcome? Editing a draft is simple. Reversing a payment, deleting data, or correcting an employment decision can be much harder.
  7. Regulatory exposure: Some use cases already sit inside heavily regulated areas such as hiring, lending, healthcare, insurance, and critical infrastructure. Those systems usually need closer legal and compliance review.

Once the risk level is set, the review path should be clear. A low-risk internal summarisation tool may only need standard access controls and logging. An agent that can update customer records, approve refunds or trigger payments should require tighter permissions, deeper testing, senior approval and closer monitoring.

Klarna reported 2.3 million conversations through its AI customer-service assistant in the first month. At that volume, even a small error rate can affect a large number of customers. Scale should therefore influence the testing, monitoring and escalation required before release.

Governance Across the AI Lifecycle

Eight-stage enterprise AI governance lifecycle framework.

You can attach governance checks to the points where an AI system changes state, such as moving from idea to pilot, from pilot to production, or from read-only to write access. This keeps the review close to the decision and makes it easier to capture the evidence needed for approval. I’ve summarized some of the key ones below:

  1. Intake and use-case review: Record the purpose, owner, affected users, model or vendor, data involved, integrations, planned decisions or actions, jurisdictions and initial risk classification. This becomes the first version of the AI inventory record. Procurement data will not capture everything. Personal accounts, browser extensions and AI enabled inside approved software can sit outside purchasing workflows. Our guide to AI and privacy in 2026 looks at how that kind of shadow use develops in practice.

  2. Data and architecture review: Map how information and authority move through the system. For RAG, that includes source repositories, retrieval permissions, embedding infrastructure, external model calls and output destinations. For agents, record the agent identity, credentials, connected tools, permitted operations, write access and approval checkpoints.

  3. Evaluation and pre-deployment assurance: Test the system against the risks of the actual use case. Higher-risk systems may need evidence on task performance, failure modes, security, privacy, robustness, harmful outputs, prompt injection, human oversight and fairness or bias where those risks are relevant. Agree the pass criteria before testing begins.

  4. Release approval: Record the evidence reviewed, outstanding limitations, accepted residual risks, conditions of approval, exceptions, required monitoring and next review date. If the system later causes an incident, this record shows why it was approved for release.

  5. Production monitoring: Monitor the signals that matter for that system, including task performance, access patterns, security events, complaints, policy violations, agent actions, tool calls, human overrides and incidents. 

  6. Material changes and re-review: I recommend defining the re-review triggers when the system is first approved. Annual review dates can miss important changes between cycles. A new model, data source, permission, agent tool, business purpose, jurisdiction or material incident can justify an earlier review.

AI Governance Controls for Different AI Systems

Different AI systems need different controls because they expose different data, permissions and business processes. 

Scenario #1: Internally Developed Models and RAG

With internally developed AI, the organisation owns more of the lifecycle and therefore more of the evidence. Governance may need to cover training and evaluation data, lineage, model versions, testing, deployment history, drift, retraining and rollback.

RAG adds another layer because the model retrieves company information at runtime. Preserve source permissions, test retrieval boundaries and keep track of where retrieved content came from. If a user cannot open a document in the source system, the RAG layer should not make it available through an answer.

If the system retrieves external or user-controlled content, test whether prompt injection can change model behaviour or trigger tool use.

Scenario #2: Third-Party, Embedded AI and Agents

With third-party AI, many technical details are controlled by the vendor. The review should separate what the organisation can verify from what it has to rely on contractually. Check underlying model providers, training use, retention, deletion, subprocessors, hosting, security, model-change practices, admin controls and available audit data.

Embedded AI deserves another review even when the SaaS vendor was approved years ago. A vendor may add an assistant, scoring feature or agent that introduces a new model provider, new data flow or new permission. The earlier vendor assessment may not cover that capability.

Meeting AI is a practical example because recordings and transcripts can contain personal data, customer information, commercial strategy and internal decisions. The EU AI Act does not replace GDPR or other privacy law, so the legal and privacy review still matters even when the AI use itself is low risk. The IT leader’s guide to recording meetings goes deeper into those controls. tl,dv also publishes its security and privacy controls, which can be used as inputs to a vendor review.

Agents need another layer of governance because they can act. Define whose identity the agent operates under, which tools it can access, which operations it can execute, any spending or transaction limits, when human approval is required, how long credentials remain valid, what gets logged and how the agent can be disabled quickly. Our overview of AI agents for meetings shows how a narrow assistant can become part of a wider workflow once it starts acting on meeting data.

The same issue appears with MCP. The protocol standardises how AI clients connect to tools. The organisation still has to decide which servers are trusted and which permissions are acceptable. The Model Context Protocol guide explains that boundary in more detail. A read-only meeting connector creates a very different exposure from a connector that can modify CRM records. This Claude MCP setup with tl,dv shows the read-only case.

Enterprise AI Governance Evidence and Audit Readiness

Every review, approval and exception should leave a record. That makes it much easier to explain what happened later during an audit, customer review or incident investigation.

Depending on risk, the governance record may include:

  • use-case documentation and accountable owners
  • risk classification and architecture
  • data and vendor assessments
  • model or system documentation
  • evaluation results and approval records
  • exceptions and compensating controls
  • production logs, incidents and periodic reviews

You should treat exceptions like any other governance decision. Record the requirement being waived, the reason, approving authority, compensating controls, residual risk and expiry date. An exception with no owner or expiry can quietly become permanent policy.

Enterprise AI Governance Frameworks and Regulations

NIST AI RMF, ISO/IEC 42001 and the EU AI Act serve different purposes. NIST provides a voluntary risk-management framework. ISO/IEC 42001 is a management-system standard. The EU AI Act creates legal obligations.

NIST AI Risk Management Framework

The NIST AI Risk Management Framework organises AI risk management around Govern, Map, Measure and Manage. It is voluntary and designed to be adapted to the organisation and use case. NIST also published a Generative AI Profile in 2024, and AI RMF 1.0 is currently being revised.

For enterprise teams, NIST gives a structure for governance, context, measurement and risk treatment. The organisation still has to design its own approval workflow and risk tiers.

ISO/IEC 42001

The ISO/IEC 42001:2023 standard sets requirements for establishing, implementing, maintaining and continually improving an Artificial Intelligence Management System. It brings leadership, responsibilities, documented processes, performance evaluation and continual improvement into one management system.

Certification is possible through independent certification bodies. However, it does not replace obligations under the AI Act, GDPR, employment law or sector-specific rules.

Règlement eur. sur l'IA

Use the European Commission’s current AI Act guidance for implementation dates. The Act entered into force in 2024 and became generally applicable on 2 August 2026, with some obligations applying earlier or later.

  • The Act uses four broad risk levels: unacceptable, high, transparency-related, and minimal or no risk.
  • Rules for general-purpose AI models became applicable in August 2025.
  • Transparency obligations for certain AI systems apply from August 2026.
  • Annex III high-risk rules apply from 2 December 2027.
  • High-risk AI embedded in regulated products under Annex I applies from 2 August 2028.

For high-risk systems, the Commission lists requirements around risk management, data quality, logging, technical documentation, information for deployers, human oversight, robustness, cybersecurity and accuracy. Enterprise teams also need to establish whether they are acting as provider, deployer, importer or distributor because the obligations differ.

Internal risk tiers help route work inside the organisation. The AI Act’s legal classifications still need a separate legal assessment alongside GDPR, employment law and any sector-specific regulation that applies.

Where several frameworks apply, I recommend maintaining one control library and mapping the same evidence to each requirement. An access review, evaluation result or human-oversight design can support several obligations. Collecting that evidence once reduces duplicate work.

How Enterprise AI Governance Controls Are Enforced

Governance decisions need technical controls behind them. If a review limits an AI system to certain users, datasets or actions, the surrounding systems need to enforce those limits.

In practice, this usually falls into two areas:

1. Identity and Data Controls

Start with the identity controls you already use. SSO, role separation, lifecycle management, short-lived credentials and regular removal of dormant access should extend to AI systems as well. Where possible, give agents their own service or agent identities rather than tying them to a human account. This makes permissions easier to review, trace and revoke.

Apply least privilege to the tools an agent can use, not just the systems it can access. If an agent only needs to read CRM records, it should not be able to update or delete them. When write access is necessary, restrict it to the specific actions, records and workflows that require it.

Existing data controls should carry over too. Data classification, DLP, encryption, retention policies and repository permissions do not stop applying because an AI system is involved. For externally hosted models, document what data leaves the organisation, which provider receives it, how long it is retained and whether the provider can use it for training.

2. Runtime and Agent Controls

Agents also need controls that apply when an action is actually attempted. Depending on the risk, this can include tool allowlists, transaction and rate limits, human approval for specific actions, sandboxing, short-lived credentials and the ability to revoke access immediately.

The level of control should reflect the consequence of the action. An agent drafting an internal summary may only need standard access controls and logging. An agent issuing refunds, changing customer entitlements or modifying production infrastructure should face stricter approval, validation and recovery requirements.

Logging should be detailed enough to reconstruct the full sequence of events. Record the agent identity, who or what initiated the task, the tools used, the permissions available, the action taken, the outcome and any human approval involved. For higher-risk agents, test the shutdown and revocation process before deployment so the system can be stopped quickly if it behaves unexpectedly.

Technology and the Enterprise AI Governance Dashboard

Existing GRC, IAM, vendor risk, data catalogue, SIEM, MLOps and ticketing systems may already cover parts of AI governance. Dedicated AI governance tooling becomes more valuable when inventories, approvals, controls and evidence are spread across different providers, teams and business units.

A governance dashboard should make outstanding work easy to see. Useful views include:

  • AI systems by owner, business unit and risk tier
  • systems awaiting approval or overdue for review
  • open exceptions and compensating controls
  • incidents and unresolved remediation
  • shadow AI or unapproved tool use
  • control gaps and missing evidence
  • significant changes to vendors, models or permissions

Leadership usually needs a simpler view. Focus on where risk is concentrated, which high-risk systems are in use, significant incidents, overdue remediation, major exceptions and how much of the AI estate is covered by governance controls.

Common Enterprise AI Governance Failures

Governance often works well during a small pilot and starts to fail once AI use spreads across teams, vendors and workflows. The most common problems are operational rather than theoretical:

  • Every use case goes through the same review: Low-risk tools compete for the same review capacity as systems that can affect customers, money, sensitive data or production environments. This slows routine adoption and leaves less attention for higher-risk deployments.

  • Approval is treated as a one-time decision: A system may be approved once, then change significantly as models, permissions, vendors, integrations or use cases evolve. Those changes should trigger another review when they materially alter the risk.

  • The inventory only captures approved AI: Embedded AI features, personal accounts, employee-built agents and unsanctioned tools can remain outside the register, leaving the organisation with an incomplete view of where AI is actually being used.

  • Policy exists without technical enforcement: Written rules are not enough if the organisation cannot enforce access controls, restrict data flows, limit tool permissions or stop an agent from taking prohibited actions.

  • Evidence is collected after the fact: If approvals, testing results, exceptions and control checks are not captured during the workflow, teams are left rebuilding the record when a customer, auditor or regulator asks for it.

  • Governance focuses on the model and misses the surrounding system: A model may be heavily reviewed while the CRM, database, browser, payment system or production tools it can access receive far less scrutiny. In many agentic systems, those permissions determine the real operational risk.

Governance Needs Clear Ownership

Clear authority keeps AI governance workable as adoption expands. Teams need to know which systems require review, who can approve them, what evidence is needed, which controls apply and what changes trigger reassessment.

As agents gain access to customer data, CRMs, payment systems and other business tools, their permissions and actions need the same level of scrutiny as the AI system itself.

FAQs About Enterprise AI Governance

An enterprise AI governance framework should define decision rights, risk criteria, approval routes, lifecycle checks, technical controls, evidence requirements, exception handling, monitoring and re-review triggers. The inventory should also cover third-party AI, embedded SaaS features, RAG systems, agents and shadow AI.

The EU AI Act introduces legal classifications and obligations based on the system, its use case and the organisation’s role. Governance processes should therefore identify systems that may be prohibited, high risk or subject to transparency requirements, and record whether the organisation is acting as a provider, deployer, importer or distributor.

Review frequency should depend on risk, but scheduled reviews are only part of the process. A material model change, new data source, broader permissions, new agent tool, significant incident, new jurisdiction or change in business purpose should trigger an earlier review.