Can I self-host an LLM firewall?

Yes. ReskSafety is an open-source LLM firewall that deploys with Docker Compose: a FastAPI backend, a React admin console, and PostgreSQL (or SQLite for quick starts). It exposes an OpenAI-compatible endpoint, so existing applications only need a base URL change.

Does an LLM firewall work with Ollama or vLLM?

ReskSafety routes to any OpenAI-compatible backend, including OpenAI, vLLM, and Ollama, selected per request via the X-Provider-Id header or admin configuration.

Self-Host an LLM Firewall with Docker in Minutes

Sending every user prompt straight to your LLM provider is a silent risk: prompt injection, PII leakage, and unauthorized tool calls all happen at that boundary. In this guide we deploy ReskSafety, an open-source LLM firewall, on your own infrastructure with Docker Compose — in minutes. Want to see it before installing? The live instance runs at demo.resk.fr.

Prerequisites

Docker and Docker Compose v2
A backend to protect: OpenAI API key, a local Ollama, or a vLLM server — anything OpenAI-compatible
2 GB RAM is enough for a SQLite quick start; use PostgreSQL for production

Step 1 — Clone and Start

git clone https://github.com/Resk-Security/ReskSafety.git
cd ReskSafety
docker compose up -d
                    

Compose starts the FastAPI backend, the React admin console, and (in the full profile) PostgreSQL. For evaluation, the default profile runs on SQLite with zero extra configuration.

Step 2 — Log Into the Admin Console

Open the console, sign in with the default administrator account, and change the password immediately. From there you create users and roles; each role carries a 64-bit capability bitmask that controls tools, code generation, database access, PII handling, and more.

Step 3 — Define a Filtering Policy

Policies map a capability mask to banned phrases and enforcement modes:

- mask: 3
  name: analyst
  rules:
    - phrase: "DROP TABLE"
      mode: hard
    - phrase: "internal salaries"
      mode: bias
      penalty: -8.0
                    

Hard rules make a phrase physically ungeneratable at the logits level (via resk-logits). Bias rules apply a penalty that makes dangerous completions statistically vanish — without breaking the user experience.

Step 4 — Point Your Applications at the Firewall

ReskSafety exposes an OpenAI-compatible endpoint. Redirect your clients:

client = OpenAI(
    base_url="http://localhost:8000/v1",
    api_key="your-resksafety-token",
)
                    

Select the backend per request with the X-Provider-Id header: openai, vllm, ollama, or any custom OpenAI-compatible gateway.

Step 5 — Observe

Every call is logged in the admin console: sessions, tool calls, and blocked content, with an audit trail. For deeper observability, ReskSafety bridges to ReskPoints for structured agent logging.

Try Before You Deploy

The fastest way to evaluate ReskSafety is the hosted demo at demo.resk.fr — no installation needed. Source code (RESK Software License): Resk-Security/ReskSafety on GitHub.