AI-assisted detection engineering
Use an AI assistant to draft detection rules for a real intrusion, then validate them — the generate → validate → repair loop that makes AI a detection force-multiplier without replacing judgment.
The web01 breach got caught late — the customer database was already gone by the time anyone noticed. You are the detection engineer now, and your job is to make sure the NEXT one fires early. You have the incident facts and a starter detection rule. The AI can draft rules fast; your job is to make sure they actually work.
1 Guided walkthrough
Download and unzip the Day 4 pack: an attack-scenario brief and a Sigma starter template. Upload both into a workspace (add the Day 2 log pack too if you want the raw logs on hand).
Have the assistant explain the highest-value detection before you write it.
Paste this into the assistantRead the attack scenario. Of all the detections listed, which single one would have caught this intrusion EARLIEST, and why?You should see: Brute-force-then-success from the same IP — it fires at 08:52, long before the data export.
Now draft it against the starter template.
Paste this into the assistantComplete the sigma-starter.yml rule so it detects a successful SSH login from a source IP that had repeated failures just before. Fill in the detection logic and the false-positives.You should see: A completed rule correlating Failed and Accepted password events by the same SourceIP within a window.
CRITICAL — validate it. Do not trust the draft.
Paste this into the assistantNow check your rule against the actual auth.log. Does it match the real events at 08:41-08:52? Does it use field names sshd actually emits? What normal activity would ALSO trip it?You should see: It should confirm the match, or catch its own invented fields — either way you learn to verify.
Draft one more — a new UID 0 account, or a security-service-stopped detection — and validate it the same way.
2 Build a detection pack that would have caught this
No prompts given this time — you write them. Forming the right question is the skill. Work these on your own; compare with a neighbour if you can.
- Produce three validated detections for this incident: the brute-force-then-success, one persistence detection (new UID 0 account or malicious cron), and one exfiltration detection (bulk export).
- For each, write the false-positive analysis: what legitimate activity trips it, and how you tightened it so it will not page the on-call at 3am.
- Find one rule the AI drafted that was wrong — a field it invented, or a condition that matches everything. Fix it. Keep the broken version as evidence of why you validate.
- Answer: where in this loop is the AI genuinely faster, and where would a wrong rule shipped unvalidated have hurt you?
What to notice
- The AI is a genuine force-multiplier at drafting — it turns a blank page into a first draft in seconds.
- It has two failure modes worth naming: a rule that matches nothing (invented fields) and a rule that matches everything (no specificity). Validation catches both.
- Generate → validate → repair is the professional pattern: the AI accelerates the engineer, it does not replace the judgment about what is actually true.
Think about it
- The AI drafts a plausible rule in seconds. What is the cost of shipping a plausible-but-wrong detection to production?
- Generate → validate → repair keeps a human in the loop. Which step can you NOT safely hand to the AI, and why?
Take it further — for fast finishers
- Ask the assistant to translate one Sigma rule into your real SIEM query language (Splunk SPL, Elastic). Spot-check the translation — did it hold up?