The LLM monitoring market mixes observability, guardrails, firewalls and agent audit. They solve different problems, and buying one category rarely covers the others. This guide maps the categories, the questions each answers, and a practical selection order.
For the logging model itself, see LLM Security Monitoring & Observability and AI Agent Observability.
The four categories
| Category | Question it answers | Typical tools | Gap |
|---|---|---|---|
| LLM observability | What did the model see, say and cost? | Langfuse, Arize Phoenix, OpenTelemetry instrumentation | Not a security boundary |
| Guardrails | Is this input or output allowed? | Llama Guard, Guardrails AI, NeMo Guardrails | Can be bypassed; needs policy |
| LLM firewall / logits | Can we block content during generation? | ReskSafety, reskSecure, resk-logits | Needs integration and tuning |
| Agent audit | Who did what, with which tool and result? | ReskPoints, SIEM pipelines | Requires a schema and retention policy |
Start with observability
You cannot secure what you cannot see. Capture traces for model calls, retrieval and tool calls, with token counts, latency, cost and a correlation ID. OpenTelemetry is a good default because it avoids locking the data into one vendor.
Add guardrails where the risk is
Guardrails are most useful at the boundaries: user input, retrieved content and final output. Put them where the damage would happen, not everywhere. A guardrail that blocks a legitimate security article creates friction without reducing risk.
Use a firewall for enforcement
A guardrail classifies; a firewall enforces. If the requirement is "this content must never be generated", you need enforcement outside the prompt — token-level filtering, capability masks or an output gate. ReskSafety and reskSecure cover that layer; the LLM firewall hub explains the architecture.
Close the loop with audit
Observability tells you what happened; audit proves it. Log security decisions with an append-only schema: identity, policy, tool call, approval, outcome. Export to your SIEM if you have one.
How to choose
- Write the three questions you must answer in an incident.
- Map each question to a category above.
- Prefer open standards (OpenTelemetry) for traces.
- Pilot on one agent and one high-risk tool.
- Measure false positives and latency before expanding.
Build vs buy
Build the schema and the retention policy; buy or reuse the collection and UI. The schema is your asset. A tool you can replace is better than a proprietary log format you cannot export.
FAQ
What is LLM observability? Recording and correlating prompts, model calls, retrieval, tool calls, cost and latency so behaviour can be understood.
Is observability a security control? It is a detection and investigation control. It does not block anything by itself.
Do I need a guardrail and a firewall? Not always. Start with the risk: classify at the boundary, enforce where a bypass would be unacceptable.
What should I log first? Trace ID, model, prompt hash, tool calls, policy decisions, latency, tokens and cost.
Next step
Pick one agent, one trace schema and one high-risk tool. Add observability first, then enforcement, then audit. The MCP security checklist is a good companion for tool-level controls.