Tags02

#Representation learning

2026-09-06·—READS·—LIKES·Learning AI with AI

2.1752: The Ruler Comes Before the Tuning

The model fits parameters on train; you fit hyperparameters on val — which is why test needs a third slice. Split by name, not by sample, or the validation set is contaminated on the spot. Tuned, val lands at 2.1752 — and on the embedding plot the vowels have gathered on their own.

2026-09-05·—READS·—LIKES·Learning AI with AI

From 27 Dimensions to 2: Compression Is What Creates Structure

One-hot makes a as related to e as it is to q; structure only appears once 27 dimensions are squeezed into 2. The table grows 729 → 531,441 while the first layer shrinks 8,100 parameters → 600 — and a dot product is not a cosine.