Index / term

Gpt

note 07Transformers

The Residual Stream Is the Model

Pre-LN, identity gradients, residual initialization, and why a block is two writes into one persistent stream.