Volume 01 · A public learning log · 2026—
I read the ideas, rebuild the small parts, and write down what actually changed my mind.
A note is not finished when it sounds clear. It is finished when an experiment leaves evidence behind.
The register
A 50-step math adapter beat a 600-step one, and ViT-Large was best after one epoch—on a single Colab A100.
Paired layer functions, mean-loss seeds, AdamW on a flat parameter tape, and the 41-step sanity loop I read but have not yet run.
train_gpt2.py as nanoGPT-shaped model, token-river loader, and a write_state bridge that dumps weights, grads, logits, and loss.
A 28-byte header, 58 MB of unnamed floats, one C inference loop, and the contract that holds them together.
Flash Attention, mixed precision, DDP, shards, checkpointing, and HellaSwag without pretending I trained GPT-2.
Tiny Shakespeare ablations for context, depth, and width—and the caveats that matter more than the leaderboard.
The complete Tiny Shakespeare baseline as an instrument: diagnostics, curve, checkpoint, and one revealing broken specimen.
Pre-LN, identity gradients, residual initialization, and why a block is two writes into one persistent stream.
From a loop mean to scaled causal self-attention, following every tensor shape through the implementation.
A shifted batch, 4,096 simultaneous next-character bets, and the first numerical check that tells me whether the model is alive.