Day 4

AI as SOC / Classroom Force-Multiplier

Thu, Jul 23  ·  9:30 AM – 5:30 PM Central

D3 — AI-Assisted Security (24%)

Diagram: a hexagonal model reaches across a boundary bar to a gear, a database and a document, with return arrows coming back, and one arrow leading down to a person.

Learning Objectives

  1. Implement an MCP server (2025-11-25 spec) for log triage and CVE summarization using the Python mcp SDK
  2. Use an AI coding agent (generate → statically validate → repair loop) to produce Sigma and Snort 3 detection rules
  3. Evaluate generated rules against static validators (sigma-cli, Snort -T self-test) without live packet capture
  4. Conduct AI-assisted vulnerability triage on a bundled CVE dataset (no live NVD — avoids hallucinated CVSS)
  5. Identify when to escalate from local model to hosted API based on task complexity and hardware constraints

Lecture Notes

Opens Thursday morning

The objectives above are the full picture of what Day 4 covers. Lecture notes, slides and lab guides are released when we get there, so the room works through them together.

Lab Guide & Bundle

4 labs, released Thursday morning

Lab guides and downloadable bundles appear here when Day 4 opens.

End-of-Day Quiz

Check for understanding — reveal the answer after you've chosen. No score is recorded.

  1. The Model Context Protocol (MCP) is used in today's lab to connect a model to security tools. What does MCP standardize?

    • The way a model is trained on new data
    • How AI applications connect models to external tools, data sources, and context in a consistent way
    • The encryption of data at rest
    • The pricing of hosted AI APIs
    Reveal answer

    Correct: B. How AI applications connect models to external tools, data sources, and context in a consistent way

    MCP is an open protocol (the lab targets the 2025-11-25 spec) that standardizes how an application exposes tools and context to a model — think a common interface between the LLM and external capabilities like triage_logs() and lookup_cve(). It replaces one-off, per-integration glue.

  2. The lab generates Sigma detection rules with an LLM and runs a 'generate → validate → repair' loop. Why is the validate step essential?

    • It makes the rule run faster in production
    • LLMs can produce plausible-but-invalid rules; a static validator (sigma-cli) catches syntax/structure errors so the model can repair them before a human trusts the rule
    • It encrypts the rule
    • It is optional and only used for logging
    Reveal answer

    Correct: B. LLMs can produce plausible-but-invalid rules; a static validator (sigma-cli) catches syntax/structure errors so the model can repair them before a human trusts the rule

    Generated rules are drafts. Running sigma check (and suricata -T for Suricata rules) statically validates them offline; failures feed back into a repair prompt. The loop turns unreliable generation into reviewable, syntactically-valid output — always confirmed by a human before deployment.

  3. In the AI-assisted CVE triage exercise, why does the lab use a bundled CVE dataset instead of querying the live NVD API?

    • The NVD API is too expensive
    • To keep the lab fully offline AND to prevent the model from hallucinating inaccurate CVSS scores — the bundled data has authoritative scores
    • Because CVEs change every second
    • There is no technical reason
    Reveal answer

    Correct: B. To keep the lab fully offline AND to prevent the model from hallucinating inaccurate CVSS scores — the bundled data has authoritative scores

    Two reasons align with the course's offline design (ADR-5): no runtime internet fetch, and grounding triage on a trusted dataset so CVSS scores are accurate rather than hallucinated. The model prioritizes (Patch Now / This Week / Monitor) using real data plus your org profile.

  4. Which of the following is an example of an AI-ENABLED attack vector that defenders must now anticipate?

    • A misconfigured firewall rule
    • Deepfake audio/video and voice-cloning used for social-engineering and fraud
    • A physically stolen laptop
    • An expired TLS certificate
    Reveal answer

    Correct: B. Deepfake audio/video and voice-cloning used for social-engineering and fraud

    AI lowers the cost of convincing deepfakes, voice cloning, personalized phishing, and automated attack tooling. These are AI-enabled offensive capabilities — the flip side of AI-assisted defense — and change the threat model for identity verification and social engineering.

  5. The lab uses qwen2.5:3b for Day 4 instead of the qwen2.5:1.5b model used earlier in the week. What is the reason?

    • The 3B model is cheaper to run
    • Reliable tool-calling: sub-2B models tend to emit malformed JSON in tool-call arguments, while the 3B model produces valid tool-call schemas
    • The 1.5B model cannot generate any text
    • Larger models are always required for every task
    Reveal answer

    Correct: B. Reliable tool-calling: sub-2B models tend to emit malformed JSON in tool-call arguments, while the 3B model produces valid tool-call schemas

    Tool-calling demands well-formed JSON arguments. Very small models (0.5-1.5B) frequently produce malformed tool_call args; qwen2.5:3b reliably emits valid schemas for 1-2 tools. This is the 'escalate model size to task complexity' judgment — right-size, don't over-provision.