AI Basics: Large Language Models and Core Concepts

Large Language Models (LLMs) are neural networks trained on vast text corpora to predict the next token in a sequence. The transformer architecture, introduced in "Attention Is All You Need" (Vaswani et al., 2017), underpins virtually every modern LLM. Transformers process input as tokens (subword pieces) and use attention mechanisms to weigh relationships between all tokens in a sequence simultaneously.

The context window is the maximum number of tokens an LLM can process in a single forward pass — both the input (prompt) and the output (completion) count toward this limit. Common context windows range from 4K to 128K tokens. Context windows matter for security because they bound how much of a conversation history, a system prompt, or injected content a model can "see" at once.

Embeddings are dense vector representations of text in high-dimensional space (e.g., 768 or 1536 dimensions). Semantically similar texts cluster near each other. Embedding models (encoder-only transformers like BERT, or dedicated models like nomic-embed-text) produce these representations faster and more efficiently than full generative models. They are the foundation of semantic search and RAG systems.

Inference is the process of running a trained model to generate outputs — as distinct from training, which adjusts model weights using gradient descent. During inference, the model's weights are frozen. Temperature is a sampling parameter (0.0–1.0+) that controls output randomness: lower temperature = more deterministic; higher = more creative/unpredictable. For security-critical outputs, low temperature (0.0–0.2) reduces variance.

Model types relevant to this course:
- Encoder-only (e.g., BERT, DeBERTa): good at classification, embeddings, understanding tasks.
- Decoder-only (e.g., GPT family, Llama, Qwen, Mistral): autoregressive generation, the dominant LLM architecture for chat/completion.
- Encoder-decoder (e.g., T5, BART): sequence-to-sequence tasks like summarization and translation.

Autoregressive generation means the model produces one token at a time, each conditioned on all previous tokens. This sequential nature creates the "streaming" appearance in chat UIs. It also means early tokens constrain later ones — a useful property for structured output, and a vulnerability for adversarial prompting.
