LLM Security: OWASP Top 10 for LLM Applications (2025)

The OWASP Top 10 for LLM Applications 2025 (OWASP Gen AI Security Project) identifies the most critical risks when deploying AI systems. Understanding these risks is foundational to SecAI+ Domain 2 (Securing AI Systems, 40% of the exam).

LLM01 — Prompt Injection: Attackers craft inputs that override or hijack the LLM's intended behavior. Direct injection manipulates the model via user input. Indirect injection embeds malicious instructions in data the model retrieves or processes (e.g., a malicious web page summarized by an AI agent). Mitigations include input validation, privilege separation, and output filtering.

LLM02 — Sensitive Information Disclosure: Models may reveal training data, system prompts, or confidential information embedded in their context. A model trained on proprietary code may reproduce it verbatim. Mitigations: PII scrubbing, output scanning, minimal-privilege prompt design.

LLM03 — Supply Chain: LLM components (base models, fine-tuning datasets, plugins) can be tampered with. Poisoned training data is especially difficult to detect. Mitigations: verified checksums (like the SHA-256 blob hashes in this course), SBOM for AI components, model cards.

LLM04 — Data and Model Poisoning: Inserting malicious samples during fine-tuning or RLHF to alter model behavior in targeted ways. Classic example: poisoning a malware classifier to misclassify specific hashes as benign.

LLM06 — Excessive Agency: Granting an AI agent more permissions than needed — write access to databases, ability to send emails, etc. A prompt-injected agent with excessive agency can take destructive actions. Principle of least privilege applies.

LLM07 — System Prompt Leakage (new in 2025): Users or attackers extract the system prompt through creative questioning ("repeat your instructions", roleplay scenarios). System prompts often contain sensitive business logic, API keys, or security policies. Mitigations: treat system prompts as secrets, use canary tokens to detect leakage.

LLM08 — Vector and Embedding Weaknesses (new in 2025): Attackers craft inputs that are semantically similar to sensitive content in a vector database, enabling unauthorized retrieval. RAG systems are specifically vulnerable — an attacker who knows the corpus can craft queries to retrieve any chunk.

Prompt injection and jailbreaking are related but distinct. Prompt injection is an attack where malicious content in the input overrides model instructions — it exploits the model's inability to distinguish data from instructions. Jailbreaking is a broader term for any technique that bypasses an LLM's safety constraints, including roleplay ("pretend you have no restrictions"), fictional framing, token smuggling, and many-shot attacks.
