The Model Asked, the Tool Said No
Access control belongs below the prompt: filter before ranking, one error message for ‘missing’ and ‘forbidden’, identity from the caller. Plus the three false alarms my own security scorer raised.
Index / section
Posts
The Blind Set Went Stale
An agent looked 12/14 vs 7/14 on a blind set I had quietly reused. On a freshly frozen set it tied single-shot RAG at 10/12—at three times the cost.
Scale Bought the Base
8B versus 14B was a controlled comparison. 32B was a fit-on-40GB experiment. Neither scale won the adapter.
The Mask Is the Objective
Three Qwen3 tool-calling masks: all assistant turns, first call without a terminator, then the terminator—and only the last one worked.
The Loss Is Not the Task
A 50-step math adapter beat a 600-step one, and ViT-Large was best after one epoch—on a single Colab A100.
There Is No Autograd Here
Paired layer functions, mean-loss seeds, AdamW on a flat parameter tape, and the 41-step sanity loop I read but have not yet run.
Python Writes the Answer Sheet
train_gpt2.py as nanoGPT-shaped model, token-river loader, and a write_state bridge that dumps weights, grads, logits, and loss.
The Model File Has No Names
A 28-byte header, 58 MB of unnamed floats, one C inference loop, and the contract that holds them together.
The Extra Code Is Not Another Brain
Flash Attention, mixed precision, DDP, shards, checkpointing, and HellaSwag without pretending I trained GPT-2.
Width Paid, Context Lied, Last Step Lost
Tiny Shakespeare ablations for context, depth, and width—and the caveats that matter more than the leaderboard.