One video to understand LLMs better.
Karpathy’s Deep Dive into LLMs like ChatGPT, and a manageable way to watch it.
If you’re going to watch one long introduction to LLMs, I’d suggest this one:
Deep Dive into LLMs like ChatGPT — Andrej Karpathy
It takes roughly three and a half hours and works through how language models are built, why they can answer questions and where their limitations come from. It’s intended for a broad audience; you don’t need to be a programmer to follow the main story.
The path through the video
- Collecting and cleaning training data.
- Turning text into tokens and numbers.
- The neural network and inference.
- Why a base model is not yet an assistant.
- Post-training, fine-tuning and reinforcement learning.
- Hallucinations, tools and some surprisingly simple tasks models still struggle with.
You don’t have to watch it in one sitting
My suggestion is to split it into three sessions. Start with the data and the base model. Then look at how an assistant is made and what can go wrong. Finish with the training approaches and the wider picture.
After each session, write a few lines in your own words. What did you understand? What still feels vague? That small pause can be more useful than immediately playing the next hour.
Even if you already use these tools every day, understanding the process behind them helps you ask better questions about their answers.