RAG Pipeline — Real CPU Run (M3 build, 2026-07-07)
Machine: Linux server, 90 GB RAM, 24-core CPU
Models: nomic-embed-text:v1.5 (274MB) + qwen2.5:1.5b (986MB, Q4_K_M)
Docker: ollama/ollama:0.9.3 + chromadb/chroma:0.6.3

=== Measured performance ===
Embedding:  25 chunks in 2.0s (12.4 chunks/s) with nomic-embed-text:v1.5
Generation: ~47–48 tok/s with qwen2.5:1.5b on CPU
Per-query total wall time: 4.3–7.2s

NOTE: Workshop participant laptops (16 GB RAM, 4–8 cores) should expect:
  - 3–10 chunks/s embedding
  - 5–15 tok/s generation
  - 20–90s per RAG query
Use the cloud VM for guaranteed speed if local is too slow.

=== Query 1: Context window + security ===
Retrieved from: cybersecurity_ai_vocab.txt (similarity=0.779), ai_basics.txt (0.773),
                cybersecurity_ai_vocab.txt (0.640)

Answer:
A context window in a large language model refers to the maximum number of tokens an LLM
can process in a single forward pass, both including input (prompt) and output (completion).
This limit affects how much of a conversation history, system prompt, or injected content a
model can "see" at once. Context windows matter for security because they bound how much data
an attacker can influence the model's behavior. By controlling any part of the context window,
an attacker can manipulate the LLM's subsequent responses, similar to session hijacking or
state corruption in traditional cybersecurity contexts.

Wall time: 5.3s | 116 tokens | 47.9 tok/s

SIGNAL CHECK:
  ✓ Correctly defines context window as "maximum tokens" in forward pass
  ✓ Mentions both input (prompt) and output in the limit
  ✓ Security relevance: attacker controlling context = security risk
  ✓ Retrieved from ai_basics.txt (correct source) + cybersecurity_ai_vocab.txt
  ✓ Under 150 words (system prompt constraint respected)

=== Query 2: RAG and hallucination ===
Retrieved from: rag_concepts.txt (0.716), rag_concepts.txt (0.709), rag_concepts.txt (0.573)

Answer:
Retrieval-Augmented Generation (RAG) helps reduce hallucinations by providing explicit reference
to the sources of information. When an LLM generates an answer, it incorporates facts and context
from specific documents that were retrieved as part of its query process. This means that while
the model may generate answers based on memory or general knowledge, it can also cite specific
evidence from the original source material. This ensures that the generated content is grounded in
reality rather than relying solely on training data, thus reducing the likelihood of hallucinations
where the answer appears to be entirely fabricated but actually derived from a known fact or piece
of information.

Wall time: 4.3s | 124 tokens | 48.3 tok/s

SIGNAL CHECK:
  ✓ Correctly explains RAG grounds answers in retrieved facts
  ✓ Retrieved all three chunks from rag_concepts.txt — the right source
  ✓ Does not invent facts about RAG
  ✓ Slight overclaiming ("cite specific evidence") but directionally correct

=== Query 3: Prompt injection vs. jailbreaking ===
Retrieved from: llm_security.txt (0.725), prompt_engineering.txt (0.661),
                prompt_engineering.txt (0.603)

Answer (abridged to key distinctions):
Prompt injection is a specific type of attack where malicious content in the input overrides model
instructions, exploiting the model's inability to distinguish data from instructions. Jailbreaking
refers to broader techniques that bypass an LLM's safety constraints, including roleplay
("pretend you have no restrictions"), fictional framing, token smuggling, and many-shot attacks.
Key differences: (1) Scope — injection targets specific instruction override; jailbreaking
encompasses various bypass methods. (2) Target — injection exploits data/instruction confusion;
jailbreaking circumvents safety mechanisms. (3) Impact — injection affects input processing;
jailbreaking alters overall LLM behavior.

Wall time: 7.2s | 251 tokens | 47.7 tok/s

SIGNAL CHECK:
  ✓ Correctly distinguishes prompt injection (instruction override via data) vs. jailbreaking (broader)
  ✓ Retrieved from llm_security.txt (primary) + prompt_engineering.txt (secondary) — correct
  ✓ Notes relationship between the two
  ✓ Slightly verbose (lists many-shot, token smuggling) but accurate
