Attention (self-attention mechanism)
Lets each token build a weighted summary of other tokens so its representation becomes context-aware.
Attention(Q, K, V) = softmax((QKᵀ / √d_head) + M) V
Covered in: Part 6
Autoregressive
Generating one token at a time, feeding each chosen token back into the next step.
p(x₁, …, xₙ) = Π_t p(x_t | x₁, …, x_{t-1})Covered in: Part 1
BPE (Byte Pair Encoding)
A tokenizer family that builds larger tokens by repeatedly merging frequent byte or character pairs.
merge* (bytes(text)) → token_ids
Covered in: Part 2
Causal Mask
The rule that stops a token from attending to future tokens during generation.
M_{ij} = 0 if j ≤ i, else -∞Covered in: Part 6
Continuous Batching
A serving strategy that lets requests join and leave an active decode batch instead of waiting for full batch boundaries.
active_batch(t + 1) = active_batch(t) - finished + admitted
d_model (model hidden dimension)
The width of the model’s main hidden vectors.
X ∈ ℝ^{T × d_model}Covered in: Part 5
d_head (per-head dimension)
The width of one attention head’s query, key, and value vectors.
d_head = d_model / H
Covered in: Part 6
d_ff (feed-forward intermediate dimension)
The expanded width used inside the feed-forward network before projecting back down.
FFN: ℝ^{d_model} → ℝ^{d_ff} → ℝ^{d_model}Covered in: Part 8
Decode
The autoregressive phase after prefill where the model generates new tokens one step at a time.
state_t → logits_t → token_{t+1} → state_{t+1}Covered in: Part 11
Dot Product
A multiply-and-sum operation that measures alignment between two vectors.
a · b = Σ_i a_i b_i
Covered in: Part 6
Embedding
A learned vector looked up from the token ID that gives the model its starting representation for that token.
x_t = E[token_id_t]
Covered in: Part 3
Epsilon (ε)
A tiny constant added for numerical stability so divisions do not blow up near zero.
y = x / √(mean(x²) + ε)
Covered in: Part 7
Feed-Forward Network (FFN)
The per-token MLP inside each transformer block that expands, transforms, and projects features back down.
FFN(x) = W_down(σ(W_gate x) ⊙ W_up x)
Covered in: Part 8
FLOP / FLOPS
A floating-point operation, and the rate of floating-point operations per second.
FLOPS = total_FLOPs / seconds
Covered in: Part 11
GeLU
A smooth activation function often used in older transformer feed-forward networks.
GeLU(x) ≈ 0.5x(1 + tanh(√(2/π)(x + 0.044715x³)))
Covered in: Part 8
GGUF (file format)
A model file format popular in local inference tooling that stores weights, metadata, tokenizer data, and often quantized blocks.
GGUF = weights + metadata + tokenizer + quantisation descriptors
Covered in: Part 11
Greedy Decoding
Always choosing the single highest-scoring next token.
token_{t+1} = argmax_i logits_iCovered in: Part 10
KV Cache
Stored past keys and values that let decode reuse old attention state instead of recomputing it.
K_cache^{(ℓ)}, V_cache^{(ℓ)} = append over time positionsKey (K)
The vector a token exposes so other tokens can measure how relevant it is.
K = XW_K
Covered in: Part 6
Layer / Transformer Block
One repeated unit of attention, feed-forward computation, normalization, and residual addition.
x' = x + Attention(Norm(x)) y = x' + FFN(Norm(x'))
Covered in: Part 5
lm_head (vocabulary projection)
The final projection that turns the last hidden state into one score per vocabulary token.
logits = hW_vocabᵀ
Covered in: Part 9
Logits
Unnormalized scores produced before softmax or decoding policy chooses the next token.
z_i = h · w_i
Covered in: Part 9
MoE (Mixture of Experts)
A model design where a router activates only a subset of expert FFNs for each token.
y = Σ_{e ∈ TopM(g(x))} g_e(x) · Expert_e(x)Covered in: Part 8
Multi-Head Attention
Running several attention heads in parallel so the model can attend through multiple learned subspaces at once.
head_h = Attention(Q_h, K_h, V_h) MHA(x) = Concat(head₁, …, head_H)W_O
Covered in: Part 6
Normalisation (RMSNorm)
A scaling layer that normalizes by root-mean-square magnitude and then applies a learned gain.
RMSNorm(x) = γ ⊙ x / √(mean(x²) + ε)
Covered in: Part 7
Paged Attention
A cache-management approach that stores KV state in page-like chunks so long sequences and many sessions are easier to manage.
KV_cache = {page₀, page₁, …, page_n}Prefill
The initial forward pass over the prompt that builds the first hidden states and KV cache before autoregressive decode begins.
prompt_tokens → hidden states + initial KV cache
Covered in: Part 11
Query (Q)
The vector a token uses to ask which other tokens matter right now.
Q = XW_Q
Covered in: Part 6
Quantisation
Storing weights or activations in fewer bits to save memory and bandwidth, usually with some controlled error.
q = round(w / s), w ≈ s · q
Covered in: Part 11
Residual Connection
A skip path that adds the input of a sublayer back to its output so information and gradients have a short route through depth.
y = x + F(x)
Covered in: Part 5
RoPE (Rotary Position Embedding)
A way of encoding position by rotating paired vector components as a function of token position.
RoPE(x, θ) = [x₁ cosθ - x₂ sinθ, x₁ sinθ + x₂ cosθ, …]
Covered in: Part 4
SafeTensors (file format)
A simple tensor-serialization format designed to load model weights safely without arbitrary code execution.
SafeTensors = named_tensors + metadata (no executable pickle payload)
Covered in: Part 11
Sampling
The policy that turns logits into an actual next-token choice.
x_{t+1} ~ P(. | x_{≤t})Covered in: Part 10
Softmax
Turns a vector of raw scores into a normalized probability-like distribution.
softmax(z)_i = exp(z_i) / Σ_j exp(z_j)
Speculative Decoding
A serving technique where a cheaper draft path proposes future tokens and the full model verifies them.
draft_tokens → verify_with_target_model → accept/reject prefix
Covered in: Part 12
SwiGLU
A gated FFN pattern that multiplies an SiLU-activated branch with an ungated branch before projecting down.
SwiGLU(x) = W_down(SiLU(W_gate x) ⊙ W_up x)
Covered in: Part 8
Temperature (τ)
A scaling factor that sharpens or flattens the sampling distribution before choosing the next token.
p_i ∝ exp(logit_i / τ)
Covered in: Part 10
Token / Tokenizer
A token is one vocabulary unit; the tokenizer is the front-end that turns raw text into token IDs and back.
tokenizer(text) → [id₁, id₂, …, id_T]
Covered in: Part 2
Top-K
A decoding filter that keeps only the K highest-scoring candidate tokens before sampling.
S = arg top-K_i logits_i
Covered in: Part 10
Top-P (Nucleus)
A decoding filter that keeps the smallest set of tokens whose cumulative probability reaches a threshold p.
find smallest S such that Σ_{i ∈ S} p_i ≥ pCovered in: Part 10
Value (V)
The vector content attention mixes together after the query-key scores decide which positions matter.
V = XW_V
Covered in: Part 6
Vocabulary (V)
The full set of token IDs the model can read and emit.
|Vocabulary| = V
Wo (attention output projection)
The matrix that mixes concatenated head outputs back into the model hidden width.
y = Concat(head₁, …, head_H)W_O
Covered in: Part 6