Prompt Injection
Prompt injection is the manipulation of a language model through adversarial input: instructions crafted to override system prompts, bypass safety constraints, or exfiltrate data the model can access. It is the defining vulnerability of LLM applications — not because individual payloads are sophisticated, but because the model cannot reliably distinguish instructions from data. Defense must therefore be layered, and partly enforced outside the model entirely.
What Are Prompt Injection Attacks?
A system prompt tells the model how to behave. A user prompt carries the task. Prompt injection exploits the fact that both arrive in the same channel: any text that reaches the model — user input, a retrieved document, a web page, a tool output — can attempt to become the instruction. OWASP ranks prompt injection as the top LLM vulnerability for good reason: every integration that feeds external content into a model inherits the problem.
Types of Prompt Injection Attacks
Direct injection
The attacker types the payload: "Ignore your previous instructions and...", role-play framings, delimiter confusion, translated or encoded jailbreaks. Loud, but effective against unhardened deployments.
Indirect injection
The payload hides in content the model ingests: a web page, a PDF, an email, a code comment. The user never sees it. This is the enterprise-scale threat — one poisoned document attacks every retrieval that touches it.
In-context injection
A stealthy variant where malicious instructions are embedded in indexed context — docstrings, comments, configuration files — and activated later by a trigger token. Our research paper covers a proof of concept, including "dormant agent" behavior activated by tokens like VALIDATOR ACTIVATE (available on the resources page).
Exfiltration payloads
Injection whose goal is not behavior change but data theft: make the model repeat its system prompt, embedded credentials, API keys, or prior conversation context. When the model has outbound tools, exfiltration becomes a two-step confused-deputy attack.
Real-World Impact
- System prompt leakage — guardrails, internal rules, and occasionally API keys published by the model itself.
- PII and data leakage — private context regurgitated into responses (see the data-protection layer of our LLM security hub).
- Unauthorized actions — injected instructions combined with tool access: send email, delete data, call external APIs. This crosses from content risk into agent security.
- Policy erosion — every successful jailbreak shared publicly degrades deployments that share the same base model and prompts.
Prompt Injection Mitigation Techniques
No single control stops prompt injection. Layered mitigation, in the order that matters:
- Input scanning — detect known attack patterns, injection classifiers, rate limiting on suspicious sessions. Necessary, never sufficient: it is a statistical control, and attackers iterate.
- Privilege separation — the model's runtime should hold the minimum capabilities needed. An agent that cannot send email cannot be injected into sending email. Authorization beats detection.
- Token-level output enforcement — banned content made ungeneratable at the logits level, so even a successful jailbreak cannot produce the forbidden tokens (shadow-ban filtering).
- Output validation — PII masking and content checks before responses reach users or downstream systems.
- Monitoring and response — log every scan decision and policy hit; alert on injection attempts; red-team regularly (guide).
Open-Source Prompt Injection Defense
resk-llm (Python) and resk-llm-ts (TypeScript) implement input-time detection with attack-specific detectors, banned-phrase filtering, PII and URL protection — a one-line wrapper around OpenAI-compatible clients. For the full enforcement layer — RBAC, policies attached to roles, tool authorization — deploy the open-source LLM firewall. The step-by-step integration is covered in Implementing RESK-LLM in Production.
Related Guides
- Prompt Injection Attacks: Case Studies and Lessons Learned — real incidents and what each teaches
- LLM Guardrails vs LLM Firewalls: Which One Do You Need?
- Prompt Leaking: How System Prompts Leak and How to Stop It
- LLM Security — the complete threat model
- 8 Open-Source Libraries to Secure Your LLM Applications
Assume the injection will land. Enforce what happens next — at the token level, per role, with a full audit trail.
Explore resk-llm Try the Live Demo