Tags02

#Sampling

2026-10-14·—READS·—LIKES·Learning AI with AI

One GPT, Two Complete Paths: Training Updates and Token-by-Token Generation

Follow Hi 你! through position embeddings, LN and QKV, two residual additions, the MLP, vocabulary logits, cross-entropy and AdamW, then trace fixed-parameter generation and incremental decoding. Three diagrams and two interactive walkthroughs distinguish parameters, activations, gradients and context.

2026-10-12·—READS·—LIKES·Learning AI with AI

A few input rows, a full vocabulary of outputs: from token IDs to sampling

What happens after a tokenizer produces IDs? Follow one five-token model through two embedding lookups, a Transformer Block, every lm_head score, and the training loss. Then work through random draws, temperature, top-k and top-p, with three diagrams, two interactive demos and complete reproducible code.