AI governance frameworks were written for models. They assume a model is asked a question, returns a response, and the question of "governance" collapses to whether the output is biased, accurate, or appropriate. Almost every published policy — EU AI Act, NIST AI RMF, ISO/IEC 42001 — was drafted around that shape.
AI agents are a different shape. An agent reads context, decides what to do, and then acts: it calls tools, queries databases, writes to memory, emails customers, files tickets, moves money, talks to other agents. The output is a side effect on real systems, not a string in a chat window. That is why most enterprise teams that try to fit agents into a model-era governance program end up with a document that doesn't survive contact with production.
Agents are not chatbots
A useful definition: an AI agent is a system that uses an LLM to plan, takes actions through tools, and operates over some form of memory. Three properties fall out of that:
- —It chooses what to do. The set of next actions is open; the agent decides.
- —It can act in your environment. Tool calls, data access, agent-to-agent messages — all observable, all reviewable, all in scope.
- —Its behaviour is path-dependent. Memory, retrieved context, prior turns, and intermediate plans all influence the next step.
That triple — open action space, real-world effects, path dependence — is exactly what makes traditional AI controls insufficient. A model-evaluation harness tells you the model can answer a benchmark question; it does not tell you whether your customer-support-agentcan be coerced into emailing another customer's order history.
Five questions agent governance has to answer
If your governance program can't answer all five of these questions, you don't have agent governance yet. You have model governance with a "for agents" label.
- —Where are our agents — including the ones nobody approved?
- —What tools, data, and identities can they reach?
- —Can they be hijacked through prompts, RAG, memory, or other agents?
- —Are they acting within enterprise policy in production?
- —Can we prove governance to a regulator or an internal auditor?
If you can't answer those five questions today, an AI agent governance program is what closes the gap.
What an operating model looks like
The operating model that consistently works in regulated enterprises has five capabilities, in order: discover, classify, red-team, enforce, prove.
1. Discover
You can't govern what you can't see. Inventory has to cover sanctioned agents AND the shadow agents — copilots wired into Slack, internal Python scripts that shell out to api.anthropic.com, vendor SaaS that quietly added an agentic feature. A practical discovery layer combines AWS Bedrock AgentCore Registry, CloudTrail Lake sweeps, frontier-API egress detection, and semantic fingerprinting. Expect to find at least 3× more agents than your CMDB lists.
2. Classify
Not every agent needs the same controls. OWASP AIVSS v0.8 is the emerging standard — CVSS base metrics combined with ten agentic risk amplification factors: autonomy, tool scope, multi-agent interactions, non-determinism, self-modification, memory persistence, dynamic identity, oversight reduction, adaptability, and adversarial attack surface. A single 0–100 score plus a tier (T1 Critical / T2 High / T3 Medium / T4 Low) is enough to prioritise.
3. Red-team
Test what you classified. Continuous adversarial red-teaming against the live agent — prompt injection (direct and indirect), tool abuse, RAG poisoning, memory poisoning, agent-to-agent compromise, denial-of-wallet. Map findings to OWASP Agentic Top 10, MITRE ATLAS, and the CSA Agentic AI Red Teaming Guide so they reconcile with adjacent security taxonomy.
4. Enforce
Testing is necessary; testing alone is not sufficient. Some actions need to be blocked, gated, or logged at runtime — every time. A policy decision point (PDP) sitting at the tool call boundary, written in a formally verifiable language like Cedar, is what closes the loop. Three-mode rollout — shadow, canary, enforce — lets you measure impact before turning a rule on.
5. Prove
The governance team's job ends with an artefact someone outside the team can verify. PDF/A-3 evidence packs with a JSON sidecar, ECDSA-P384 signed under a per-tenant KMS key, RFC 3161 timestamped, hash-chain anchored, archived under S3 Object Lock for seven years. Mapped clause-by-clause to MAS AIRG, APRA CPS 230 + 234, RBI FREE-AI, OJK Tata Kelola, BNM RMiT, BSP Project Sapiens, EU AI Act, NIST AI RMF, and ISO/IEC 42001.
Who owns this
Agent governance crosses three teams, deliberately:
- —Security — owns the attack surface, runs the red-team programme, prioritises findings.
- —Risk & Compliance — owns the framework mapping, signs off on residual risk, produces evidence for the regulator.
- —AI Platform — owns the production deployment loop, defines what agents are allowed to do, ships the runtime PDP.
The platform team is the bridge: it gives developers a frictionless local red-team loop (open-source) and the enterprise a governed production loop. That separation is what keeps AI agents from being either un-shipped (because security blocked it) or un-governed (because security never got visibility).
Where to start
If you are starting from zero, the highest-leverage first move is discovery. You cannot prioritise what you cannot count. AgentGuardian Enterprise installs a customer-resident data plane via CloudFormation StackSet and typically produces a complete agent inventory — including shadow agents — within 24 hours. From there you classify with AIVSS, red-team the T1/T2 agents first, and stand up runtime policy for the smallest set of high-risk tools.
That's agent governance. Everything else is paperwork.