Two teams can deploy the same security controls — injection detection, output filtering, access control — and get completely different results, because where a control lives determines what it can enforce. A filter inside one service protects one service; the same filter in the request path protects every caller, including the ones added next quarter. This guide covers the three architecture patterns for securing AI applications, their real limits, and how to choose. The threat model itself is in the LLM security hub.

The Three Trust Boundaries of an AI Application

Every AI application has the same boundaries, and each one is a place where controls can be placed — or missed:

  • Application ↔ model — the request path: who may call which model, with what input.
  • Model ↔ context — retrieval, memory, and tool output flowing into the prompt.
  • Model ↔ external world — tool calls, emails, API writes: where a text problem becomes a real action.

Pattern 1 — The Embedded Library

Detection code runs inside your application, wrapping the model client: input scanning, PII protection, banned-phrase filtering. This is the fastest pattern to adopt — it is a few lines of code in the application you already have — and for a single internal application with a trusted user base, it is often enough to start.

Its limits are structural, not qualitative:

  • It is skippable. Any code path that calls the model without the wrapper bypasses every control.
  • It is per-application. Five applications mean five configurations, five upgrade cycles, five chances to drift.
  • There is no central audit. Each app logs what it wants, where it wants — incident response becomes log archaeology.

Used well: resk-llm (Python) and resk-llm-ts (TypeScript) implement this pattern — see the implementation guide.

Pattern 2 — The Enforcement Gateway

Every model call routes through a deployed control point — an LLM firewall — that sits in the request path between applications and providers. Applications change one thing: the base URL. The gateway then applies, for every caller without exception:

  • Authentication and per-role authorization — the caller's identity and capability mask decide what the request may do.
  • Input policy and token-level output enforcement — banned content becomes ungeneratable inside the gateway's decoding path, not detected after streaming.
  • Central audit — one log of decisions with user identity, exportable to your observability stack.

The trade-off is operational: you now run a component in the hot path, and it must be as available as the models behind it. The payoff is that controls stop being per-application advice and become platform behavior. A drop-in reference implementation is ReskSafety — OpenAI-compatible, self-hosted, live demo here.

Pattern 3 — Defense in Depth for Agents

When the model can act — tool calls, MCP servers, code execution — a gateway for content is necessary but not sufficient. Actions need authorization at the moment of execution, and the agent's runtime needs containment:

  • Tool-call authorization — deny by default, per user and per role; a tool the role lacks simply fails closed (agent security hub).
  • Sandboxed execution — filesystem roots, egress allowlists, no shell by default; the agent can only do what the sandbox permits.
  • Untrusted tool output — tool results and descriptions enter the context as data, scanned like user input (MCP security integration).

A Worked Request Flow

A support assistant with RAG and an email tool, secured with patterns 2 + 3. What each layer stops:

  1. User authenticates — the gateway knows who is asking, not just that someone asked.
  2. Request authorized by role — a contractor's mask lacks the email capability; nothing downstream needs to remember that.
  3. Input scanned — a jailbreak attempt is flagged and rate-limited; the request proceeds with a lowered trust score.
  4. Retrieval scoped to the user — the poisoned document planted in another team's space is never retrieved, because the index honors document ACLs.
  5. Generation filtered at token level — an injected instruction to emit a forbidden string produces tokens that cannot be generated.
  6. Tool call authorized — the model attempts send_email; the gateway checks the capability mask and fails closed for this caller.
  7. Everything logged — identity, policy hits, denial reason. The incident review takes minutes, not days.

No single layer is impressive. The architecture is that each one assumes the previous one failed.

Choosing Your Pattern

  • One internal app, trusted users: Pattern 1 is a legitimate start — add scanning now, plan the gateway before the second application appears.
  • Multiple apps or any external users: Pattern 2. Central enforcement is the difference between having controls and having a policy document.
  • Agents with tools: Pattern 2 + 3. Content filtering without action authorization leaves the highest-impact path — real-world actions — unguarded.

Whichever you pick, verify it adversarially: architecture decisions look correct on whiteboards and fail in red-team exercises. Layering plus verification is the whole of the method.

Related Articles: