Course
Zero to GPT
Build up an intuitive, from-scratch understanding of how large language models work — from derivatives and backpropagation through tokenization, embeddings, self-attention, and all the way to a working (if tiny) GPT.
Lesson 1
What Is a Language Model?
The core idea behind every LLM: predicting the next word.
Lesson 2
Derivatives: How Change Flows
The single idea underneath how every neural network learns: nudge the input, watch the output move.
Lesson 3
Backpropagation: The Chain Rule, by Hand
How gradients flow backward through a chain of operations — the algorithm that trains every neural network.
Lesson 4
Character-Level & Word-Level Tokenization
The two simplest ways to chop text into tokens — and why both break down in practice.
Lesson 5
Byte-Pair Encoding: Building It From Scratch
The middle ground between character-level and word-level tokenization that virtually every modern LLM actually uses.
Lesson 6
WordPiece, SentencePiece & Choosing a Real Tokenizer
Raw BPE isn't quite what production tokenizers ship with — the real variants, and a real one running on real vocabulary.
Lesson 7
Embeddings: Turning Tokens Into Meaning
How a model represents word meaning as points in space.
Lesson 8
Self-Attention: Letting Words Talk to Each Other
How a model figures out which other words matter for each word.
Lesson 9
From Counting to Neural Networks
Build the simplest possible language model, then see why it falls short.
Lesson 10
Assembling the Transformer
Combine attention and neural network layers into the full architecture.
Lesson 11
Training and Sampling Playground
Explore loss, temperature, and top-k sampling interactively.
Lesson 12
Fine-Tuning Intuition (LoRA)
Adapt a pretrained model to a new task without retraining everything.
Lesson 13
From Completion to Chat: Instruction Tuning
Why a raw next-token predictor doesn't behave like an assistant — and the fine-tuning step that changes that.
Lesson 14
Aligning With Human Feedback: RLHF & DPO
Turning "which response do people actually prefer" into a real training signal.
Lesson 15
Capstone: From GPT-2 to Modern LLMs
See how the pieces you've learned scale up to real-world models.