Threat Model + Lock the Front Door

End-of-Day Quiz

Check for understanding — reveal the answer after you've chosen. No score is recorded.

1. In the OWASP Top 10 for LLM Applications (2025), which risk is ranked LLM01 and is the one you exercise first in the lab?

   • A.Sensitive Information Disclosure
   • B.Prompt Injection
   • C.Model Denial of Service
   • D.Excessive Agency
   Reveal answer

   Correct: B. Prompt Injection

   Prompt Injection is LLM01:2025 — the top LLM risk. Because the model cannot inherently separate trusted instructions from untrusted input, crafted text can override intended behavior. The lab's attack.py demonstrates it against a raw, unguarded model.

2. OWASP added LLM07 System Prompt Leakage in the 2025 list. Why is leaking the system prompt a security problem?

   • A.It slows the model down
   • B.It can expose embedded secrets, guardrail logic, or sensitive context an attacker can then bypass or exfiltrate
   • C.It is only a cosmetic issue with no real impact
   • D.It permanently corrupts the model weights
   Reveal answer

   Correct: B. It can expose embedded secrets, guardrail logic, or sensitive context an attacker can then bypass or exfiltrate

   System prompts often contain internal rules, credentials, or PII-laden context (see Day 3's HR assistant). Once leaked, an attacker learns the guardrails to evade them — and any secret placed in the prompt is disclosed. The fix is to not put secrets in the prompt, plus output/leakage controls.

3. What does the LLM Guard prompt firewall in the lab do with the DeBERTa PromptInjection scanner?

   • A.It rewrites the model's answer to be more polite
   • B.It classifies incoming prompts and blocks likely injection attempts before they ever reach the model
   • C.It encrypts the prompt in transit
   • D.It rate-limits how many prompts a user can send
   Reveal answer

   Correct: B. It classifies incoming prompts and blocks likely injection attempts before they ever reach the model

   LLM Guard runs input scanners (PromptInjection uses a DeBERTa classifier, plus TokenLimit) that inspect and reject malicious prompts pre-inference. It is a defense-in-depth layer, not a guarantee — subtle injections without obvious trigger phrases can still slip through, which is why you also need gateway and design controls.

4. In the lab you configure a LiteLLM gateway with a qwen-student tier at rpm=2 and a qwen-instructor tier with no cap. Which OWASP LLM risk does this rate/token-limit control most directly address?

   • A.LLM01 Prompt Injection
   • B.LLM08 Vector and Embedding Weaknesses
   • C.LLM10 Unbounded Consumption
   • D.LLM07 System Prompt Leakage
   Reveal answer

   Correct: C. LLM10 Unbounded Consumption

   LLM10:2025 Unbounded Consumption covers resource-exhaustion and cost/denial-of-wallet abuse. Gateway controls — per-tier rate limits, token budgets, and tiered virtual keys (LiteLLM) — cap consumption independently of the model, which has no concept of a quota.

5. MITRE ATLAS is best described as which of the following?

   • A.A prompt-injection scanner you install in front of the model
   • B.A knowledge base of real-world adversary tactics and techniques against AI/ML systems, modeled after ATT&CK
   • C.An EU regulation governing high-risk AI
   • D.A vector database used for RAG
   Reveal answer

   Correct: B. A knowledge base of real-world adversary tactics and techniques against AI/ML systems, modeled after ATT&CK

   MITRE ATLAS (Adversarial Threat Landscape for AI Systems) is a living ATT&CK-style matrix of tactics and techniques targeting ML systems. It is a threat-modeling reference, not a control — you use it (alongside OWASP LLM Top 10) to enumerate what attackers can do.