This is a living map, not a syllabus I need to finish perfectly. Each stage should produce runnable work and a clearer explanation before I move on.
01 · Foundations
Status: built
- Python and tensor thinking
- Essential linear algebra and probability
- Data, train/validation splits, and baseline discipline
- Reading shapes and tracing values through code
02 · Neural networks
Status: built
- Backpropagation and computational graphs
- MLPs, activations, initialization, and normalization
- Training loops, optimization, and overfitting
- Controlled experiments: changing one variable at a time
03 · Transformers
Status: in progress
- Attention as communication
- Residual streams and MLP computation
- Tokenization, embeddings, and positional information
- Reproducing GPT-style training from first principles
- Scaling, batching, checkpoints, and evaluation
- Exporting learned weights and tracing token-by-token inference in C
Published sequence: Transformer Field Notes I — The Batch Is a Factory of Futures through VI — The Extra Code Is Not Another Brain, followed by Inference Field Notes I — The Model File Has No Names.
04 · AI systems
Status: next
- Retrieval and reranking
- Evaluation sets and error analysis
- Agents, tools, and reliable workflows
- Serving, observability, latency, and cost
05 · Original work
Status: later
- Form a testable question
- Build the smallest credible experiment
- Publish the method, failures, and result
- Let the evidence—not the novelty claim—carry the work