https://blog.crazyai.uk/categories/transformers/Recent content in Transformers on Zero to One AIHugoenThu, 20 Aug 2026 11:36:00 +0800https://blog.crazyai.uk/posts/transformer-field-notes-06-extra-code-is-not-another-brain/Thu, 20 Aug 2026 11:36:00 +0800https://blog.crazyai.uk/posts/transformer-field-notes-06-extra-code-is-not-another-brain/Production nanoGPT adds a factory around the same next-token spine—and the product is still a document continuer, not a chatbot.https://blog.crazyai.uk/posts/transformer-field-notes-05-width-paid-context-lied/Thu, 20 Aug 2026 11:35:00 +0800https://blog.crazyai.uk/posts/transformer-field-notes-05-width-paid-context-lied/Three controlled knobs produced one useful winner, one confounded story, and a checkpoint that was better before training ended.https://blog.crazyai.uk/posts/transformer-field-notes-04-live-wire-is-not-a-language/Thu, 20 Aug 2026 11:34:00 +0800https://blog.crazyai.uk/posts/transformer-field-notes-04-live-wire-is-not-a-language/The model could memorize one batch, lower validation loss, restore a checkpoint, and imitate a play—without learning honest English.https://blog.crazyai.uk/posts/transformer-field-notes-03-residual-stream-is-the-model/Thu, 20 Aug 2026 11:33:00 +0800https://blog.crazyai.uk/posts/transformer-field-notes-03-residual-stream-is-the-model/I stopped picturing a Transformer as layers replacing one another. A better picture is one long highway with repeated reads and writes.https://blog.crazyai.uk/posts/transformer-field-notes-02-four-ways-to-average-the-past/Thu, 20 Aug 2026 11:32:00 +0800https://blog.crazyai.uk/posts/transformer-field-notes-02-four-ways-to-average-the-past/Q, K, and V stopped feeling magical when I rebuilt attention as four increasingly selective ways to mix the same past.https://blog.crazyai.uk/posts/transformer-field-notes-01-batch-factory-of-futures/Thu, 20 Aug 2026 11:31:00 +0800https://blog.crazyai.uk/posts/transformer-field-notes-01-batch-factory-of-futures/I thought training a language model meant teaching it to write. The first batch showed me something much more mechanical—and more useful.