The two attack classes that define LLM security — how they work and how to defend against them. For the full adversary matrix see the AI Attack Atlas; this is the deep-dive on the techniques that matter most.
Prompt Injection
Untrusted input overrides the model's instructions
Prompt injection is the defining vulnerability class of LLM applications — the AI equivalent of injection flaws like SQL injection. Because an LLM processes its system instructions and untrusted data in the same context window, attacker-controlled text can be interpreted as instructions. The model has no reliable, built-in boundary between "the developer's rules" and "the content I was asked to summarize."
Direct Prompt Injection
High impact
How it works
The attacker is the user. They type instructions that try to override or leak the system prompt — "ignore previous instructions", role-swaps, or requests to reveal the hidden setup.
Why models fall for it
The system prompt and the user turn share one context; the model weighs instructions by salience and phrasing, not by a trust label. A sufficiently emphatic or cleverly-framed user instruction can outrank the developer's.
Defenses
Never rely on the system prompt alone as a security boundary
Enforce authorization in application code, not in the prompt
Treat the model's output as untrusted; validate before acting on it
Isolate secrets — don't put anything in the prompt you can't afford leaked
The malicious instructions don't come from the user — they're planted in content the model later ingests: a web page, an email, a PDF, a code comment, a retrieved document. When the agent reads that content, the hidden instructions execute.
Why models fall for it
Agentic systems and RAG pipelines feed external, attacker-influenceable data straight into the model's context. The model can't tell "data to reason about" from "instructions to follow" — so a comment in a fetched page becomes a command.
Defenses
Segment and label retrieved/tool content; keep it out of the instruction channel where possible
Constrain what the agent can DO — least-privilege tools, human-in-the-loop for high-impact actions
Sanitize and validate tool outputs before they re-enter the context
Monitor for anomalous agent behavior (unexpected tool calls, data egress)
Instructions hidden where a human reviewer won't look: text in an image, invisible/zero-width Unicode, homoglyphs, base64 or leetspeak encodings, or content in a file's metadata. The model reads it; the reviewer doesn't.
Why models fall for it
Vision models transcribe text in images; tokenizers happily process obfuscated or encoded strings. The obfuscation defeats human review and naïve keyword filters while remaining fully legible to the model.
Defenses
Normalize and strip zero-width/invisible characters from inputs
OCR and inspect image inputs for embedded instructions
Don't rely on keyword denylists — they miss encodings and paraphrases
A jailbreak makes a model produce output its safety training was meant to refuse. Where prompt injection targets the application's instructions, a jailbreak targets the model's alignment itself. Most jailbreaks exploit the gap between a model's safety training (which covers common phrasings) and the open-ended space of ways to ask — reframing, role-play, or incremental escalation that the training didn't anticipate.
Persona & Role-Play
High impact
How it works
The attacker asks the model to adopt a persona for whom the rules supposedly don't apply — a fictional character, an "unrestricted" mode, or a hypothetical where refusing would "break character". The classic "DAN" family.
Why models fall for it
Instruction-following and helpfulness can be steered to outweigh safety when the harmful request is framed as fiction, hypothetical, or a game — contexts under-represented in safety training relative to direct requests.
Defenses
Safety training and evaluation on role-play and hypothetical framings
Output-side classifiers that judge the content, not the framing
The harmful request is disguised — asked in another language, split across turns, encoded (base64, ROT13), or expressed via a cipher the model is told to decode — so it slips past input filters and phrasing-based safety.
Why models fall for it
Safety training keys partly on surface phrasing; an encoded or translated request can reach the model's capabilities while evading the patterns the guardrails learned to catch.
Defenses
Evaluate safety on encoded, translated, and low-resource-language inputs
Decode-then-screen: apply safety checks after any decode step
Judge intent and final output, not just the literal input string
Rather than one obviously-harmful ask, the attacker escalates gradually across a conversation — each turn slightly further than the last — until the model has been walked into output it would have refused up front.
Why models fall for it
Per-turn safety checks judge each message in isolation; a slow escalation keeps every single step below the refusal threshold while the trajectory arrives somewhere the model shouldn't go.
Defenses
Evaluate safety over the whole conversation, not per-message