I write backends, and the iOS, Android and web clients in front of them — less “full-stack” than an unwillingness to only do half of it.
INDEX37
One GPT, Two Complete Paths: Training Updates and Token-by-Token GenerationA Vocabulary Does More Than Split Text: Model Costs, Multimodality and Token BoundariesA few input rows, a full vocabulary of outputs: from token IDs to samplingAlgorithms Catch-Up · The Index Is an Amount, the Value Is a Count: Dynamic Programming and BacktrackingThe same abc, a different split: training and encoding with UnigramThe IDs already exist: recovering minbpe rules and handling special tokensWhat is Ġ in the vocabulary? One round trip through GPT-2's tokenizerSet the boundaries first: the regex before BPEThe Merge Table Is the Tokenizer: Why Tokenize, and BPE from ScratchBasics Catch-Up · One Character, Five Byte Strings: Charsets Number, Encodings StoreThe Differences Are All Engineering: A From-Scratch GPT Beside the Paper and nanoGPTHow the r After “thou” Disappeared: Growing One Attention Head into a GPT