Learning notes from “Build an LLM from Scratch”

What the hack is LLM?

A no-jargon, beginner-friendly walkthrough of how large language models actually work — from tokens and embeddings to transformers and training. Written while learning, explained for humans.

Start readingSee the path
What you’ll get

The whole picture, without the hand-waving

Four ideas, once you have them, everything else is just details.

Minds, not magic
Strip away the hype. Understand LLMs as pattern-matching machines that learned to talk by reading a lot of the internet.
Built from scratch
Follow the construction of a real, working LLM — every tensor, every layer, every hyperparameter explained along the way.
Plain English
No PhD required. Every concept gets a friendly explanation first, then the technical version once the intuition is in place.
From token to thought
Trace a single prompt all the way through the model and back out — and watch meaning emerge from a pile of numbers.
Why bother?

You don’t need another API wrapper tutorial.

You need to actually understand the thing you’re talking to. Here’s why the rabbit hole is worth it.

01

It's not as hard as the internet says.

Once you see what a transformer actually does, the mystique evaporates. We pull apart the moving parts piece by piece.

02

You can't use what you don't understand.

Prompting, fine-tuning, evaluating — every practical skill is built on a mental model of what's happening under the hood.

03

The field changes; the foundations don't.

Models come and go. Tokenization, attention, sampling, training loops — these ideas stick around longer than any specific release.

The learning path

From zero to “oh, that’s how it works”

A rough map of the chapters. Each one ends with a working piece of the model and a clear explanation of why it matters.

  1. Ch. 1

    Tokens & embeddings

    How text becomes numbers a model can chew on.

  2. Ch. 2

    The transformer block

    Attention, layer norm, MLPs — the engine room.

  3. Ch. 3

    Putting it together

    Stacking blocks, sampling outputs, generating text.

  4. Ch. 4

    Training & fine-tuning

    Loss, gradients, and teaching an old model new tricks.

Read the full notes

Ready to peek under the hood?

The first chapter is short, visual, and assumes nothing. By the end of it, you’ll have run real code that processes tokens like a real LLM.

Open the docs