Most enterprise LLM governance programs fail the same way: a twelve-page policy is approved, published on the intranet, and never connected to a runtime control. Six months later nobody can say which model version is in production, who can send customer data to it, or whether a deleted employee still has an API key. Governance that is not enforced in the request path is documentation, not security.

This article describes the policy domains that matter, the artefacts each one needs, and how to turn them into enforceable controls. It complements the enterprise AI security pillar, which covers the threat model and the deployment architecture.

What an enterprise AI governance policy must contain

A workable policy answers seven questions. If a section cannot be tested, it is not a control.

  1. Ownership. Who owns each AI system, each model, each tool and each dataset? One named accountable role, not a committee.
  2. Permitted use. Which use cases are approved, which are prohibited, and how a new use case gets reviewed.
  3. Access. Who may use which model, with which tools and which data — expressed as roles and capability masks.
  4. Data handling. Classification, PII redaction, retention, residency and what may never enter a prompt.
  5. Vendor and model risk. A registry of approved providers, model versions, data-processing terms and subprocessors.
  6. Audit and evidence. What is logged, for how long, who can read it, and how it maps to your obligations.
  7. Incident response. How an AI incident is detected, contained, disclosed and learned from.

Domain 1 — Ownership and the model registry

You cannot govern an asset you cannot enumerate. Maintain a registry with one entry per deployed model and agent: provider, model identifier and version, owner, purpose, data classification, approved tools, evaluation date and review date. Treat the registry as the allowlist — a model that is not in it cannot be called through the gateway.

model: gpt-enterprise-2026-09 provider: approved-vendor-a owner: [email protected] use_cases: [support-summarisation, internal-search] data_classes: [internal, confidential] tools: [knowledge_search] retention_days: 30 last_eval: 2026-09-18 next_review: 2026-12-18 status: approved

The registry also solves the "shadow model" problem: when procurement, engineering and a business unit each sign their own AI contract, the registry is where duplicates and unapproved providers surface.

Domain 2 — Access control and least privilege

Governance language such as "employees may use AI for approved tasks" becomes real only when it maps to a permission model. In practice:

  • Identity. Every request carries a real user, team and environment. No shared service accounts.
  • Capabilities. Roles resolve to masks — model access, tool access, data classes, PII visibility. The effective mask is an intersection, never a union.
  • Environment. Production PII is not reachable from a developer's notebook, even with the same user account.
  • Expiry. Elevations have an owner and a TTL.

The mechanics are covered in Least Privilege for AI Agents and Capability Bitmasks for LLM Applications.

Domain 3 — Data, PII and prompt hygiene

Most regulatory exposure comes from what enters the prompt and what leaves the model. Define four rules and enforce them at the gateway:

  • Classify before send. Tag each request with a data class; reject classes the model or vendor is not approved for.
  • Redact by default. Mask identifiers, account numbers, health data and credentials unless the use case explicitly requires them.
  • Minimise retention. Store prompts and completions only as long as needed for audit or quality, then delete. Vendor-side retention is part of the same policy.
  • Block secrets. API keys, tokens and private keys must never reach a provider; detect them on input and output.

See Encrypting AI Conversations for the storage side and output filtering for the response side.

Domain 4 — Policy enforcement as code

A policy that lives in a PDF cannot answer "was this request allowed?" Write rules in a declarative form, version them in Git, and evaluate them at request time:

policy: enterprise-default version: 42 rules: - id: deny-unapproved-model when: request.model not_in registry.approved action: deny - id: redact-pii-for-external when: request.data_class == confidential and request.model.hosted == external action: redact - id: require-approval-for-irreversible when: tool.reversible == false action: require_human_approval - id: block-secrets-in-prompt when: input.matches(secret_patterns) action: block - id: log-every-decision when: always action: audit

Two properties matter. First, the policy version is logged with every decision, so an auditor can reconstruct why a request was allowed. Second, changing the policy is a reviewed pull request, not an email — which gives you a change history for free.

Domain 5 — Audit logs and evidence

Log enough to answer who, what, when, with which model and why. A minimal record contains: timestamp, caller identity, team, environment, model and version, policy version, data class, tool calls, decision (allow/deny/redact/approval), latency and token counts. Do not log raw prompts by default; log a hash plus a redacted excerpt, and store full content separately under stricter access.

Retention should be driven by obligation, not by storage convenience. Financial and health contexts often require longer retention; consumer chat may require shorter. Whatever you choose, document the mapping and test deletion.

ReskPoints provides the agent-level event stream; the LLM firewall produces the request and policy decisions. For monitoring practice, read LLM Security Monitoring & Observability.

Domain 6 — Vendor and model risk

Each provider in the registry needs a review that covers: training-on-data terms, retention and residency, subprocessors, breach notification, model deprecation policy, and the ability to pin a version. Add an exit plan: can you move the workload to another provider without recreating the application? An OpenAI-compatible abstraction — as implemented by the self-hosted LLM firewall — makes that a configuration change instead of a rewrite.

Domain 7 — Incident response for AI systems

AI incidents differ from classic ones in three ways: the "malicious input" may be ordinary content, the blast radius may be a tool rather than a database, and the evidence lives in prompts and tool logs. Your runbook should cover: revoking a model or tool globally, rotating provider keys, isolating an agent, preserving prompt and tool logs, notifying affected users, and re-running the evaluation suite before restoring service.

A one-page checklist

  • Model and agent registry with owners, versions and review dates.
  • Approved-use list and a review path for new use cases.
  • Role-to-capability mapping, intersected at request time.
  • Data classification enforced before the prompt leaves your boundary.
  • PII redaction and secret detection on input and output.
  • Policy-as-code in Git, versioned and logged per decision.
  • Audit records with identity, model, policy version and decision.
  • Vendor reviews with retention, residency and exit plans.
  • Incident runbook with revocation and evidence preservation.
  • Quarterly re-evaluation of every approved model.

FAQ

What should an enterprise AI governance policy include for LLMs, PII and audit logs? Ownership and a model registry, approved use cases, role-based access with least privilege, data classification with PII redaction, audit records tied to a policy version, vendor risk review and an incident runbook.

What is LLM policy enforcement? Evaluating declarative rules in the request path — before generation, at tool calls and on output — so that a policy decision is applied to every request rather than documented in a PDF.

How do you govern multiple LLM providers? Keep one registry of approved models and versions, put an OpenAI-compatible gateway in front of all providers, and enforce the same access, data and logging policy regardless of vendor.

How long should AI audit logs be retained? Base retention on the strictest applicable obligation and on investigation needs; document the mapping, restrict access to raw content and test deletion. Metadata can usually be kept longer than prompt text.

What are enterprise AI rules that actually work? Rules that map to a runtime control with an owner, a test and a log entry. If a rule cannot be evaluated automatically, it will not be followed consistently.

Next step

Take your current policy and mark each sentence as enforced, monitored or aspirational. The aspirational ones are your roadmap — and usually the fastest win is access control, because it removes entire classes of incident before anyone writes another paragraph.

For the full architecture, read the Enterprise AI Security pillar or talk to the RESK team about a deployment review.