Red-team the detection
Reason about how an attacker evades naive detections and how to harden each — the offensive angle that produces durable, behavior-based rules.
You built detections this morning. Now attack them. You cannot write a robust rule until you can see how a weak one fails — so put on the attacker's hat and find the seams in four naive detections, then close them.
1 Guided walkthrough
From the Day 4 pack, upload evasion-challenges.md. It has four naive detection rules.
Work the first rule — an exact string match on "UNION SELECT".
Paste this into the assistantA detection alerts only if a log line contains the exact text "UNION SELECT". How would an attacker evade this, and how would I rewrite the detection to catch SQL injection more robustly?You should see: Case changes, inline comments, encoding, alternate syntax — and a suggestion to detect the injection pattern or DB error responses instead of one literal string.
Work the fixed-threshold rule (20 failed logins/minute). Think slow-and-distributed.
Paste this into the assistantA rule alerts on more than 20 failed SSH logins per minute from one IP. Give three ways an attacker stays under it, and a more durable detection.You should see: Slow to under the threshold, spread across IPs, spray usernames — and a ratio-based or windowed detection instead of a raw count.
Work the blocklist-IP and User-Agent-name rules the same way. For each, land on the same insight: detect the BEHAVIOR, not the artifact.
Keep it to reasoning and these toy rules — the output is a better detection, not an attack tool.
2 Turn four brittle rules into robust ones
No prompts given this time — you write them. Forming the right question is the skill. Work these on your own; compare with a neighbour if you can.
- For each of the four naive rules, write: the evasion that beats it, and a hardened rule that closes the gap.
- Name the single pattern all four naive rules share (they detect a surface artifact, not a behavior). State it in one sentence you could teach.
- Take your best hardened rule back to the blue lab and check it against the real logs — does it still fire on the actual attack without drowning you in noise?
What to notice
- Every naive rule detects an artifact (a string, a threshold, an IP, a tool name) instead of a behavior — and artifacts are cheap for an attacker to change.
- The durable question for any detection: what is the attacker's goal, and what signal of it can they NOT easily change?
- Using AI to find evasions is how you write rules that catch tomorrow's attack, not just yesterday's.
Think about it
- Every naive rule failed the same way. What is the general principle, and how would you teach a junior analyst to spot a brittle rule on sight?
- The AI helped you find evasions AND harden the rules. Where is that dual-use most valuable to a defender?
Take it further — for fast finishers
- Pair up: one person hardens a rule, the other tries to evade the hardened version. Iterate until neither can win cheaply — that is real detection engineering.