Prompt injection is the top entry on the OWASP Top 10 for LLM Applications — but reading about a vulnerability class is not the same as understanding it. This article walks through documented, real-world prompt injection attacks, from the first viral system-prompt leak in 2023 to a zero-click Microsoft 365 Copilot exploit and a remote code execution chain via a code editor. For each one: what happened, the mechanism, and the lesson that maps to a concrete defense. If you want the concepts first — attack types, mitigation stack — start with our prompt injection hub.
Why Prompt Injection Keeps Working
Every incident below shares one root cause: the model has no reliable way to distinguish instructions from data. The system prompt, the user message, a retrieved document, a tool result — all of it arrives as one token stream. SQL has prepared statements to separate code from data; LLMs have nothing equivalent. Any application that lets untrusted text reach the model inherits the problem, and every tool the model can call converts "bad text" into a potential real-world action.
Case Studies
1. Bing Chat "Sydney" (February 2023) — the system prompt is not a secret
A Stanford student used a simple direct injection to make Bing Chat reveal its full system prompt, internal codename included: Sydney. The prompt exposed behavioral rules, self-identity instructions, and safety constraints Microsoft had treated as confidential. The lesson is blunt: a system prompt is configuration, not a security boundary. Whatever you write there, a determined user can extract. Anything that must not leak — credentials, internal URLs, business rules — must never be in the prompt at all. See prompt leaking for the extraction techniques.
2. Chevrolet dealership chatbot (December 2023) — business logic is not a prompt instruction
A customer-facing sales bot on a dealership site was nudged into "agreeing" to sell a 2024 Chevrolet Tahoe for $1. No exploit framework, no encoding tricks — plain conversational framing ("that works for me, is it a deal?"), and the bot confirmed. The dealer never honored it, but the reputational damage was done. The lesson: a rule written in the prompt ("prices are firm") is not an enforced rule. Commercial constraints belong in code — validated before any offer is presented — not in instructions the model may talk its way past.
3. Slack AI (August 2024) — indirect injection turns retrieval into exfiltration
Security researchers (PromptArmor) demonstrated that an attacker could post a crafted message in a public Slack channel, and then, when any user queried Slack AI about it, the assistant would silently retrieve data from private channels that user had access to — and exfiltrate it through a link the attacker controlled. The user never saw the malicious instruction; it traveled inside the indexed content. The lesson: RAG pipelines are an exfiltration channel. Retrieved content is untrusted input, and the model's outbound reach (links, images, tool calls) must be policed as a data-egress boundary. Slack shipped a fix after disclosure; the architectural pattern persists in every assistant that indexes shared content.
4. Microsoft 365 Copilot EchoLeak (June 2025, CVE-2025-32711) — zero-click, via email
Researchers at Aim Security showed that a single crafted email could trigger Copilot to process hidden instructions and exfiltrate the victim's documents and chats — with no click, no prompt, no user action. The exfiltration bypassed the usual controls by using domains already trusted by Microsoft (Teams/SharePoint URL patterns) so that content left through an allowlisted channel. Microsoft fixed it server-side (CVE-2025-32711, no customer action needed), but the pattern is the important part: indirect injection in the inbox is a zero-click attack, and outbound filtering must consider what the application trusts, not just what looks suspicious. Read the full mechanism write-up on our resources page.
5. GitHub Copilot Chat RCE (CVE-2025-53773, August 2025) — injection escalates to code execution
Researchers demonstrated that hidden instructions in a repository — code comments, issue text, PR descriptions — could make Copilot Chat edit the workspace's own .vscode/settings.json to enable auto-approval of tool calls, then run arbitrary commands. The chain is the classic privilege escalation of the agent era: the model was allowed to change its own permissions, and from there the injection became remote code execution. GitHub shipped a fix in the August 2025 VS Code release. The lesson generalizes to every coding agent: permission changes must be outside the model's reach, and tool approvals must never be self-grantable.
The Pattern Behind the Incidents
- Instructions and data share one channel. Whether the payload is typed by the user (case 1, 2) or hidden in content the model ingests (case 3, 4, 5), it wins for the same reason.
- Every outbound capability is an exfiltration path. A link, an image URL, an email tool, a webhook — if the model can emit it, injected instructions can smuggle data through it (cases 3, 4).
- The model should never hold the ability to expand its own privileges. Auto-approval toggles, permission files, and tool grants must be controlled by code and operators, not by anything the model can write (case 5).
- Retrieval scope is an authorization decision. "What can this assistant see" and "what can it send where" are access-control questions that pre-date LLMs — and they still are (case 3).
Defenses That Map to the Cases
No single control would have stopped all five attacks; layered enforcement would have contained each one:
- Input and content scanning — detect known injection patterns in user input and in retrieved content, tool descriptions, and tool results. Necessary, never sufficient: it is statistical, and attackers iterate. resk-llm and resk-llm-ts implement these detectors as a wrapper around your existing client.
- Privilege separation — the runtime holds the minimum capabilities the task needs, granted per role and per user. An assistant that cannot send email cannot be injected into emailing your customer database. This is authorization, not detection, and it is where most of the leverage is.
- Token-level output enforcement — banned content made ungeneratable inside the decoding loop, so even a successful jailbreak cannot produce the forbidden tokens or URLs (how shadow-ban filtering works).
- Output and egress validation — treat everything the model emits before it leaves your perimeter the way you treat user input arriving: validate URLs, strip markup channels you did not intend, and restrict which domains outbound content may reference (the lesson of EchoLeak).
- Monitoring and audit — log every scan decision, policy hit, and tool call with user identity. Without that trail, all five incidents above would have been undiagnosable after the fact.
For the full enforcement layer — policies per role, tool-call authorization, audit — put a deployed control point in front of your models: see the open-source LLM firewall and its RBAC model in capability bitmasks.
Test Your Own Deployment
Every incident in this article was found by someone testing what the deployed system actually did — not by reading the prompt. Adversarial testing is the only way to know whether your assistant leaks its context, follows injected instructions in retrieved documents, or lets a tool call escalate into an action. Our AI red teaming guide covers the methodology, and AI pentesting is how we run it for clients.
The Stakes Keep Rising
The first case studies cost reputations. By 2025, the same class of bug reached remote code execution (case 5), and Anthropic reported disrupting a cyber-espionage campaign in which attackers used agentic coding tools to automate intrusions at scale — injections into agents are now a nation-state-grade technique. The defenses above are no longer "hardening"; for any deployment with real data or real tools, they are the baseline.