LLM security monitoring is the difference between "we think the model is behaving" and "we can prove what it did, for which user, with which policy." Classic application monitoring tells you the service is up. AI observability has to tell you that a support agent read a customer record it should not have touched, that a retrieved document tried to redirect the model, and that an output filter blocked the same secret three times this hour.

This guide describes a telemetry model that survives production: what to collect, what to detect, how to test, and how to keep privacy intact. It pairs with the LLM security pillar and the AI agent security pillar.

Why LLM monitoring is different

  • The input is the attack surface. A malicious prompt looks like normal text. You cannot signature your way out of it; you need behavioural signals.
  • The path is non-deterministic. The same request can retrieve different documents and call different tools. Logging only the final answer loses the evidence.
  • The sensitive event is often a tool call, not an error. A 200 OK with an exfiltrated record is the incident.
  • Content is sensitive by nature. Prompt logs are both the best evidence and the biggest privacy liability, so redaction and access control are part of the design.

The five telemetry planes

Instrument each plane separately; correlate them with a request ID.

1. Request plane

Caller identity, team, environment, model and version, prompt hash, data class, token counts, latency, policy version, policy decision. This is the spine of every investigation.

2. Retrieval plane

Query, document IDs, source and trust level, chunk hashes, similarity scores, and whether a document was newly indexed. Retrieval is where indirect prompt injection enters, so provenance matters more than content here.

3. Tool plane

Tool name, arguments (redacted), scope check result, capability mask, approval status, result status, downstream system and latency. Tool calls are the actions that change the world; they deserve the most rigorous schema.

4. Output plane

Finish reason, output hash, filter decisions (blocked category, redaction counts), PII and secret detections, and a redacted excerpt. Never store full completions by default.

5. Policy plane

Every allow, deny, redact and require-approval decision, with the rule ID and policy version. Without this plane you can describe what happened but not why it was permitted.

Detections that earn their place

Start with a small set of high-signal rules. Volume-based anomaly detection on prompts is noisy; permission and action anomalies are not.

  • Scope violations. A tool call whose arguments reference a tenant, record or domain outside the caller's scope.
  • First-use events. An agent calls a tool it has never used, or a user reaches a model they were not approved for.
  • Denial bursts. Repeated policy denials followed by a success — a strong signature of probing.
  • Exfiltration shape. Outbound arguments containing PII, secrets, or a domain not on the egress allowlist.
  • Injection indicators. Tool output containing imperatives directed at the model, hidden text, encoded blobs, or instructions to call another tool.
  • Output leakage. System-prompt fragments, canary tokens or credentials in a completion (prompt leaking covers the attacks).
  • Drift. A change in model version, tool manifest hash or policy version outside a deployment window.
  • Cost anomalies. Token or tool-call volume far above baseline — sometimes abuse, sometimes a runaway loop.

Metrics and service levels

Security metrics should be measurable and owned. A practical starter set:

  • Policy coverage — share of requests evaluated by the current policy version (target 100%).
  • Deny rate by rule — a rule that never fires is either unnecessary or broken.
  • Tool calls without a scope check — target zero.
  • Time to detect a first-use or scope-violation event.
  • Redaction rate for PII and secrets, by use case.
  • Evaluation pass rate for the injection test suite, per model version.

Testing is part of monitoring

A detection you never exercise is a guess. Run the same corpus that red teaming uses — direct and indirect injection, tool-abuse prompts, encoded payloads, prompt-leak attempts — against every model and policy change, and record the result with the version. When a production incident occurs, turn it into a regression case.

The loop is: red team → evaluation suite → deploy → monitor → incident → new test case. Without the last step, the same incident recurs after the next model upgrade. See AI Penetration Testing: A Framework for the offensive side and AI Red Teaming for methodology.

Architecture that keeps privacy intact

The common failure is storing raw prompts in a general-purpose log index that half the company can query. Separate the concerns:

  • Metadata (identity, model, decision, hashes) goes to your SIEM with normal retention and broad read access.
  • Redacted content goes to a restricted store with a shorter retention and audited access.
  • Raw content, if retained at all, is encrypted, access-controlled and time-limited — ideally only for a sampled subset or on incident hold.
  • Secrets and credentials never enter any log; detect and drop them at the emitter.

ReskPoints provides the agent event stream, while the LLM firewall emits request and policy decisions; together they cover planes 1, 3 and 5 without custom instrumentation. ReskCrypt handles encrypted conversation storage.

A 30-day rollout

  1. Week 1: define the event schema and a request ID; instrument identity, model, policy decision and tool calls.
  2. Week 2: ship the four highest-signal detections: scope violation, first-use, denial burst, secret in output.
  3. Week 3: add redaction and the restricted content store; verify that raw prompts are not in the SIEM.
  4. Week 4: run the injection suite against each model version, baseline metrics, and write the incident runbook.

FAQ

What should you monitor in an LLM application? Five planes: request metadata, retrieval provenance, tool calls and arguments, output filter decisions, and policy decisions — all correlated by request ID.

How do you detect prompt injection in production? Look for behavioural signals rather than keywords: tool output containing instructions, a call to a new tool, arguments referencing a different tenant or external domain, and denial bursts followed by success.

Which AI security platform features matter most for monitoring LLM outputs and blocking data leakage? Per-request identity and policy version, tool-call visibility, PII and secret detection on output, configurable redaction, and an audit trail that can be queried per user.

Should you log full prompts? Not by default. Log hashes and redacted excerpts in a restricted store, keep raw content only under strict access and short retention, and never log credentials.

How do LLM security monitoring and testing fit together? Testing generates the cases; monitoring tells you which ones occur in production. Feed incidents back into the test suite so the same failure is caught before the next release.

Next step

Pick one plane you cannot currently answer questions about — usually the tool plane — and instrument it first. A week of clean tool-call telemetry will teach you more about your real risk than a quarter of prompt sampling.

Read the LLM security pillar for the controls these detections protect, or explore ReskPoints for agent-level observability.