Retrieval-Augmented Generation (RAG): Architecture and Security Implications

Retrieval-Augmented Generation (RAG) is an architecture that grounds LLM outputs in a curated knowledge base rather than relying solely on training data. A RAG pipeline has three stages: (1) indexing — documents are embedded and stored in a vector database; (2) retrieval — at query time, the question is embedded and semantically similar document chunks are fetched; (3) augmentation — retrieved chunks are injected into the LLM's context as "retrieved context" before the question, enabling the model to answer from facts rather than memory.

RAG reduces hallucination because the model has explicit source material to reference. It also keeps knowledge current — you update the vector store without retraining the model. And it provides attribution: retrieved chunks reveal the sources behind an answer.

The key retrieval quality parameters are:
- top_k: how many chunks to retrieve. Too few → missing relevant context. Too many → dilutes signal with noise and uses context window.
- similarity metric: cosine similarity (angle between vectors, scale-invariant) vs. Euclidean distance (magnitude-sensitive) vs. inner product. Cosine is standard for retrieval.
- chunk size: how documents are split. Smaller chunks (256–512 tokens) = more precise retrieval but risk losing context. Larger chunks (512–1024 tokens) = more context per chunk but lower retrieval precision.

Security implications of RAG systems:
- Indirect prompt injection: an attacker who can write to the knowledge base (or to documents the system scrapes) can embed malicious instructions that the retriever fetches and the LLM executes.
- Over-retrieval: a malicious query crafted to be semantically similar to sensitive documents can extract confidential chunks from the knowledge base (LLM08 in OWASP Top 10 2025).
- Privilege escalation via retrieval: if the vector store contains documents from different sensitivity levels and there is no access control on retrieved chunks, a user can access material they are not authorized to see.
- Data integrity: if embeddings are pre-computed and stored, tampering with stored vectors (without changing source documents) can manipulate what gets retrieved — a subtle supply-chain attack.

This lab builds a RAG pipeline over a cybersecurity corpus using nomic-embed-text (embedding) and qwen2.5:1.5b (generation), both running locally via Ollama on CPU. No data leaves the machine.
