Sampling Parameters + Model A/B — REPRESENTATIVE OUTPUT (authored at M8 build time)
STATUS: NOT a real capture. Models were not run in the build environment.
REGENERATE on the golden VM before the workshop:
  docker compose run --rm runner python app/sampling_ab.py | tee expected/sampling_ab_output.txt
Latency, tokens/sec, and generated text below are plausible illustrations for
qwen2.5:1.5b / qwen2.5:3b on a CPU-only box. Real numbers depend on hardware and
sampling; the SIGNAL (greedy is reproducible, higher temperature diverges, the
3B model is usually more accurate but slower) is what a correct run shows.

╭──────────────────────────────────────────────────────╮
│ Day 1 Lab — Sampling Parameters + Model A/B           │
│ Phases 1-2 model: qwen2.5:1.5b   A/B: qwen2.5:1.5b vs qwen2.5:3b │
╰──────────────────────────────────────────────────────╯

Step 1: Waiting for Ollama…
Ollama ready.

Step 2: Checking models…
Model already present: qwen2.5:1.5b
Model already present: qwen2.5:1.5b
Model already present: qwen2.5:3b

──────────── Phase 1 — determinism: same prompt, K runs ────────────
Prompt: List three common indicators of a phishing email.
Model: qwen2.5:1.5b  runs: 5

greedy (temperature=0.0): 1 unique output(s) across 5 runs
  sample: 1) Sender address that does not match the display name. 2) Urgent or threatening language…

sampled (temperature=1.2): 4 unique output(s) across 5 runs
  sample: Phishing emails often use look-alike domains, create false urgency, and hide the real link…

Takeaway: temperature 0 is greedy → reproducible; raising it samples the
distribution → variety. 'Non-deterministic AI' is usually a sampling setting.

──────────── Phase 2 — sampling sweep: temperature then top_p ────────────
Prompt: Write a one-line security awareness slogan about strong passwords.
Model: qwen2.5:1.5b

            Temperature sweep (top_p=1.0, seed fixed)
┌─────────────┬────────┬─────────────────────┬──────────────────────────────────────┐
│ temperature │ tokens │ distinct-word ratio │ first line                             │
├─────────────┼────────┼─────────────────────┼──────────────────────────────────────┤
│         0.0 │     14 │                0.93 │ Strong passwords lock the door…        │
│         0.5 │     17 │                0.94 │ A strong password is your first line…  │
│         1.0 │     22 │                0.95 │ Guard your gate — long, unique…        │
│         1.5 │     31 │                0.97 │ Passwords are keys; forge yours long…  │
└─────────────┴────────┴─────────────────────┴──────────────────────────────────────┘

          top_p (nucleus) sweep (temperature=1.0, seed fixed)
┌───────┬────────┬─────────────────────┬──────────────────────────────────────────┐
│ top_p │ tokens │ distinct-word ratio │ first line                                 │
├───────┼────────┼─────────────────────┼──────────────────────────────────────────┤
│   0.1 │     13 │                0.92 │ Strong passwords keep attackers out…       │
│   0.5 │     16 │                0.93 │ Make it long, make it unique…              │
│   0.9 │     21 │                0.95 │ A passphrase a day keeps the breach…       │
│   1.0 │     24 │                0.96 │ Twelve characters stand between you and…   │
└───────┴────────┴─────────────────────┴──────────────────────────────────────────┘

Takeaway: temperature flattens the whole distribution; top_p truncates it
to the smallest set of tokens whose probability sums to p. Low top_p stays on
the safe, high-probability path even at high temperature.

──────────── Phase 3 — model A/B: qwen2.5:1.5b vs qwen2.5:3b ────────────
Same 6 tasks, temperature=0.2, through both models. Comparing latency,
throughput, and a keyword correctness proxy.

──────────────────────────── qwen2.5:1.5b ────────────────────────────
  ✓ (1.1s, 42.0 tok/s) What does the acronym MFA stand for in security?
      MFA stands for multi-factor authentication, requiring two or more factors.
  ✓ (0.9s, 43.5 tok/s) Which network port does HTTPS use by default?
      HTTPS uses TCP port 443 by default.
  ✓ (1.0s, 41.8 tok/s) In the CIA triad, what does the C stand for?
      The C stands for confidentiality — keeping data secret from unauthorized parties.
  ✓ (1.2s, 40.9 tok/s) What kind of attack floods a service to make it unavaila…
      A denial-of-service (DoS) attack floods a service so legitimate users cannot reach it.
  ✓ (1.0s, 42.6 tok/s) What is the name for malicious software that encrypts fil…
      That is ransomware — it encrypts files and demands payment for the key.
  ✗ (1.1s, 41.2 tok/s) Which hashing algorithm is considered broken and should …
      SHA-256 is stronger; avoid the older one for security-sensitive uses.

──────────────────────────── qwen2.5:3b ────────────────────────────
  ✓ (2.4s, 19.5 tok/s) What does the acronym MFA stand for in security?
      MFA means multi-factor authentication: proving identity with two+ distinct factors.
  ✓ (2.1s, 20.1 tok/s) Which network port does HTTPS use by default?
      By default HTTPS listens on TCP port 443.
  ✓ (2.3s, 19.8 tok/s) In the CIA triad, what does the C stand for?
      C is confidentiality — ensuring information is disclosed only to authorized parties.
  ✓ (2.5s, 18.9 tok/s) What kind of attack floods a service to make it unavaila…
      A denial-of-service attack overwhelms a service so it cannot serve legitimate traffic.
  ✓ (2.2s, 19.6 tok/s) What is the name for malicious software that encrypts fil…
      Ransomware encrypts a victim's files and demands a ransom for decryption.
  ✓ (2.6s, 18.4 tok/s) Which hashing algorithm is considered broken and should …
      MD5 is considered broken and should not be used; prefer SHA-256.

                          A/B summary
┌──────────────┬───────┬──────────────┬────────────────┬───────────┬──────────────┐
│ Model        │ Tasks │ Keyword hits │ Total wall (s) │ Avg tok/s │ Total tokens │
├──────────────┼───────┼──────────────┼────────────────┼───────────┼──────────────┤
│ qwen2.5:1.5b │     6 │          5/6 │            6.3 │      42.0 │          210 │
│ qwen2.5:3b   │     6 │          6/6 │           14.1 │      19.4 │          228 │
└──────────────┴───────┴──────────────┴────────────────┴───────────┴──────────────┘

Takeaway: the larger model usually wins on the keyword proxy but costs more
wall-clock time per answer on CPU. 'Which model' is a latency/quality/cost budget,
not an absolute — measure it on the hardware you will actually deploy on.

──────────────────────────────── Done ────────────────────────────────
What this lab teaches:
  • temperature 0 = greedy = reproducible; higher = sampled = varied.
  • top_p (nucleus) truncates the candidate set; a second, independent knob.
  • model choice is a latency/quality trade-off you should MEASURE, not assume.

Try it: edit prompts/sweep_prompts.txt or data/ab_tasks.txt and re-run — no rebuild needed.
