Open-Source LLM Firewall

Networks have firewalls. AI needs one too. An LLM firewall is a central enforcement point between your applications and your models: every request is authenticated, checked against policy, filtered on the way in and on the way out, and logged. ReskSafety is RESK's open-source LLM firewall — deployable in one Docker command and running live at demo.resk.fr.

What Is an LLM Firewall?

A firewall sits in the request path and enforces policy regardless of what the application or the user tries. For LLMs that means: who is calling (authentication and role), what they are allowed to ask (input policy), what the model is allowed to answer or do (output policy and tool authorization), and a record of every decision (audit). A library embedded in your code can provide detection; only a deployed firewall can provide enforcement — one place where policies live and cannot be bypassed by skipping an import.

Why Prompt Filtering Isn't Enough

Most LLM security tools stop at input scanning. That is necessary but not sufficient:

Input filters are jailbreakable. Encoding tricks, multi-turn escalation, and in-context injection via retrieved documents all bypass static input rules.
Post-generation moderation is too late. By the time a moderation API flags an output, the tokens were already generated and often already streamed to the user.
Actions need authorization, not moderation. A tool call to os.system or an email with the customer database is not a "content" problem — it is an access-control decision.

A firewall covers all three: input scanning, token-level output enforcement, and per-role tool-call authorization. See LLM guardrails vs LLM firewalls for the detailed comparison.

Firewall Architecture

1. Authentication and RBAC

JWT in httpOnly cookies, Argon2 password hashing, CSRF protection. Every role carries a 64-bit capability bitmask — tool calling, code generation, database access, email, PII handling — enforced per request (how capability bitmasks work).

2. Input filtering

Editable policies with banned phrases, hard-block and bias (penalty) modes, attack-pattern scanning, and configurable thresholds. Policies attach to role masks, so enforcement is per role, not global.

3. Token-level output enforcement

Policies bridge down to resk-logits: a GPU-accelerated logits processor (ShadowBan + Aho-Corasick) makes dangerous content statistically impossible to generate — enforcement inside the generation loop, where no injection can reach.

4. Tool-call authorization

Agent tool calls are checked against the caller's capability mask and MCP tool allowlists before execution. Unauthorized calls fail closed.

5. Multi-provider routing

Route to any OpenAI-compatible backend — OpenAI, vLLM, Ollama, or your own gateway — via the X-Provider-Id header or admin configuration. One firewall in front of every model you run.

6. Audit and observability

Every request logged with user, policy hits, and outcome; agent sessions tracked end-to-end; admin console with a D3 graph of your security topology; exportable via ReskPoints to Datadog, Prometheus, or OpenTelemetry.

Self-Hosted Deployment with Docker

ReskSafety is a full-stack application (FastAPI + React + PostgreSQL, SQLite for quick starts). Clone, compose, done — the full walkthrough is in Self-Host an LLM Firewall with Docker in Minutes:

git clone https://github.com/Resk-Security/ReskSafety.git cd ReskSafety && docker compose up -d

OpenAI-Compatible APIs

The firewall exposes POST /v1/chat/completions (plus Anthropic-compatible /v1/messages and batch endpoints). Your existing OpenAI client only changes its base URL:

client = OpenAI(base_url="https://your-firewall/v1") # Every request: auth → RBAC → policies → token filter → audit

No SDK rewrite, no vendor lock-in — swap providers behind the firewall without touching application code.

Try It Now

A full deployment runs at demo.resk.fr — sign up, edit a filtering policy in the admin console, and watch the firewall enforce it in real time. Source-available under the RESK Software License on GitHub.

Related Guides

LLM Security — the complete threat model and defense stack
AI Agent Security — tool permissions, observability, audit
Enterprise AI Security — governance, compliance, deployment models
ReskSafety: The Deployable Open-Source LLM Firewall — the product tour
Self-Host an LLM Firewall with Docker in Minutes
LLM Guardrails vs LLM Firewalls: Which One Do You Need?

Run LLMs at scale in your organization? We help design the deployment — policies, roles, provider topology, and audit.

Try the Live Demo Talk to Us