Just enough to not be bluffing — The seven ideas you need to see once

This is not a maths course. It is seven things that, once you have actually seen them move, make everything after them obvious. Skip it if you can already explain why a model cannot count the r's in strawberry.

Track 0 of the AI engineering roadmap by K Laxman — about 4 hours across 7 stages.

Tokens — the model never sees letters

You can explain the strawberry problem, and why it is not a sign of stupidity.

tokenisation · subwords · why spelling tasks fail

Interactive explainer on the live page: tokenizer — You can explain the strawberry problem, and why it is not a sign of stupidity.

You're done when: Someone asks why the model miscounts letters and you answer in one sentence, correctly.

Embeddings — meaning becomes geometry

You understand what 'similar' means to a machine, and why that powers all of retrieval.

vectors · cosine similarity · nearest neighbours

Interactive explainer on the live page: embedding space — You understand what 'similar' means to a machine, and why that powers all of retrieval.

You're done when: You can say why a search for 'car' returns a document about 'vehicles' with no shared words.

Attention — how a word chooses what matters

The transformer stops being a black box and becomes a weighting scheme.

queries/keys/values, informally · attention weights · context

Interactive explainer on the live page: attention map — The transformer stops being a black box and becomes a weighting scheme.

You're done when: In 'the animal didn't cross the street because it was tired', you can say what 'it' attends to and why.

Learning — one ball, one hill

Loss, gradients and learning rate stop being jargon. You have watched a model diverge.

loss surface · gradient descent · learning rate · divergence

Interactive explainer on the live page: gradient descent — Loss, gradients and learning rate stop being jargon. You have watched a model diverge.

You're done when: You can say what happens if the learning rate is 10× too high, before running it.

Sampling — the same model, different nerve

Temperature and top-p become knobs you reason about instead of copy from a blog post.

logits · softmax · temperature · top-p · greedy decoding

Interactive explainer on the live page: sampling — Temperature and top-p become knobs you reason about instead of copy from a blog post.

You're done when: You can pick a temperature for an extraction task and defend the number.

Context — everything competes for the same room

You see why prompt, history and retrieved documents are in a fight, and who should lose.

context window · token budget · eviction · cost per call

Interactive explainer on the live page: context budget — You see why prompt, history and retrieved documents are in a fight, and who should lose.

You're done when: Given a 128k window and a chatty agent, you can say what gets dropped first and why.

Training vs inference — what is actually frozen

You stop reaching for fine-tuning when the real answer is retrieval or a better prompt.

pretraining · post-training · inference · why weights don't learn from your chat

Interactive explainer on the live page: train vs infer — You stop reaching for fine-tuning when the real answer is retrieval or a better prompt.

You're done when: You can name three problems fine-tuning does NOT solve.

Explore more

GitHub · LinkedIn · Email