Chapter 1 — Understanding LLMs
A high-level, beginner-friendly tour of what large language models are, how transformers work, and the journey of building one.
This chapter is the map for everything that follows. Before you write a single line of PyTorch, it makes sure you understand the what, the why, and the how of large language models. The original ideas are preserved — only the academic phrasing has been trimmed, so nothing important gets lost in jargon.
What you'll learn
By the end of this chapter you'll be able to answer, in plain language:
- What an LLM actually is (and what "large" really means).
- Where LLMs are used today, from chatbots to code assistants.
- The two big stages every LLM goes through: pretraining and finetuning.
- What a transformer is, and why almost every modern LLM uses one.
- How huge amounts of text data let these models generalise so well.
- Why GPT is a "decoder-only" model, and how that differs from BERT.
- The three-stage roadmap we'll follow to build an LLM from scratch.
Chapter contents
1.1 — What is an LLM?
A neural network that predicts the next word, trained on a giant pile of text.
1.2 — Applications of LLMs
Translation, summarisation, chatbots, code — the everyday superpowers of LLMs.
1.3 — Stages of building and using LLMs
Pretrain a foundation model, then finetune it for your task.
1.4 — The transformer architecture
Encoders, decoders, and the attention mechanism that changed everything.
1.5 — Utilising large datasets
Why scale matters and what GPT-3 was actually trained on.
1.6 — A closer look at GPT
Decoder-only, autoregressive, next-word prediction. The blueprint we'll follow.
1.7 — Building a large language model
The three-stage plan for coding an LLM from scratch in code.
1.8 — Summary
The one-page recap of everything above.
A quick note on tone: when we say an LLM "understands" language, we mean it produces text that looks coherent — not that it thinks or feels anything. Keep that caveat in mind throughout.