Just enough to not be bluffing — The seven ideas you need to see once

This is not a maths course. It is seven things that, once you have actually seen them move, make everything after them obvious. Skip it if you can already explain why a model cannot count the r's in strawberry.

Track 0 of the AI engineering roadmap by K Laxman — about 4 hours across 7 stages.

Tokens — the model never sees letters

You can explain the strawberry problem, and why it is not a sign of stupidity.

tokenisation · subwords · why spelling tasks fail

Interactive explainer on the live page: tokenizer — You can explain the strawberry problem, and why it is not a sign of stupidity.

In short:

  1. The unit is the token. Text is split into sub-word pieces from a fixed vocabulary before the model sees it. A common English word is usually one token; a rare word, a name or a number can be several.
  2. Letters are invisible. "strawberry" arrives as a couple of token ids, not ten letters. Counting r's asks the model about something it was never shown directly.
  3. Everything is priced in tokens. Context limits, latency and the bill all scale with token count — and the same sentence costs more tokens in most non-English languages.

You're done when: Someone asks why the model miscounts letters and you answer in one sentence, correctly.

Common traps:

Check yourself:

  1. Why can a model write a flawless essay but miscount the letters in a word? Answer: It reads tokens, not characters. The essay is built from tokens it has seen billions of times; the letter count needs information the tokenisation hid.
  2. Your prompt is 3,000 English words. Roughly how many tokens is that? Answer: About 4,000 — English averages around 0.75 words per token. Check with the provider's tokenizer before you rely on it.

Embeddings — meaning becomes geometry

You understand what 'similar' means to a machine, and why that powers all of retrieval.

vectors · cosine similarity · nearest neighbours

Interactive explainer on the live page: embedding space — You understand what 'similar' means to a machine, and why that powers all of retrieval.

In short:

  1. Meaning becomes a vector. An embedding model turns a piece of text into a list of numbers. Texts that mean similar things land close together, even with no words in common.
  2. Similarity is an angle. Cosine similarity compares the direction of two vectors. Search becomes: embed the question, return the stored vectors nearest to it.
  3. Close is not correct. Nearness means "talks about the same kind of thing", not "answers the question". That gap is why retrieval needs measuring.

You're done when: You can say why a search for 'car' returns a document about 'vehicles' with no shared words.

Common traps:

Check yourself:

  1. Why does a search for "car" find a document about "vehicles"? Answer: Both embed to nearby points because they appear in similar contexts. Retrieval compares positions in that space, not shared words.
  2. You switch to a better embedding model. What has to happen to the index? Answer: Every document has to be re-embedded with the new model. Vectors from the old model are not comparable with new query vectors.

Attention — how a word chooses what matters

The transformer stops being a black box and becomes a weighting scheme.

queries/keys/values, informally · attention weights · context

Interactive explainer on the live page: attention map — The transformer stops being a black box and becomes a weighting scheme.

In short:

  1. Every token looks at every other. For each token, attention computes how relevant every other token in the context is, and builds that token's new representation as a weighted mix of them.
  2. Queries, keys, values. Informally: a token's query asks "what am I looking for?", each key answers "what do I contain?", and the values are what gets mixed in when they match.
  3. Attention is a budget. The weights are spread across the whole context. Bury one instruction in a very long prompt and it competes with everything else for that budget.

You're done when: In 'the animal didn't cross the street because it was tired', you can say what 'it' attends to and why.

Common traps:

Check yourself:

  1. In "the animal didn't cross the street because it was tired", what should "it" attend to, and why? Answer: "animal" — being tired is something an animal can be and a street cannot. Swap "tired" for "wide" and the weight should move to "street".
  2. Why can a key instruction get ignored in a 50-page prompt? Answer: Attention is spread across every token in the context. The instruction is a tiny share of it; put it near the end, state it plainly, and cut what isn't needed.

Learning — one ball, one hill

Loss, gradients and learning rate stop being jargon. You have watched a model diverge.

loss surface · gradient descent · learning rate · divergence

Interactive explainer on the live page: gradient descent — Loss, gradients and learning rate stop being jargon. You have watched a model diverge.

In short:

  1. Loss is the score of being wrong. Training measures how far the model's predictions are from the targets. Lower loss means better predictions on the training data.
  2. The gradient points uphill. It tells you which way each weight should move to increase the loss, so training steps the opposite way — gradient descent.
  3. The learning rate is the step size. Too small and training crawls; too large and it overshoots the valley, bounces and diverges. Most failed runs are a step-size problem first.

You're done when: You can say what happens if the learning rate is 10× too high, before running it.

Common traps:

Check yourself:

  1. The learning rate is 10× too high. What does the loss curve look like? Answer: It jumps around or climbs instead of falling, often ending in NaN. Every step overshoots the minimum.
  2. Training loss keeps dropping while validation loss starts rising. What is happening? Answer: Overfitting — the model is memorising the training set. Stop earlier, add data or add regularisation.

Sampling — the same model, different nerve

Temperature and top-p become knobs you reason about instead of copy from a blog post.

logits · softmax · temperature · top-p · greedy decoding

Interactive explainer on the live page: sampling — Temperature and top-p become knobs you reason about instead of copy from a blog post.

In short:

  1. The model outputs a distribution. At each step it scores every token in the vocabulary (logits); softmax turns those scores into probabilities.
  2. Temperature reshapes it. Low temperature sharpens the distribution toward the top choice; high temperature flattens it and lets unlikely tokens through.
  3. Top-p cuts the tail. Sample only from the smallest set of tokens whose probabilities add up to p. It keeps variety without letting in the truly unlikely.

You're done when: You can pick a temperature for an extraction task and defend the number.

Common traps:

Check yourself:

  1. Which temperature for pulling invoice fields into JSON, and why? Answer: 0 or near it. There is one right answer; randomness only adds ways to be wrong.
  2. What does top-p = 0.9 actually do? Answer: It keeps the most likely tokens until their probabilities sum to 0.9, drops the rest, and samples from what remains.

Context — everything competes for the same room

You see why prompt, history and retrieved documents are in a fight, and who should lose.

context window · token budget · eviction · cost per call

Interactive explainer on the live page: context budget — You see why prompt, history and retrieved documents are in a fight, and who should lose.

In short:

  1. One window holds everything. System prompt, tool definitions, retrieved documents, chat history and the answer being written all share the same token budget.
  2. Something has to be evicted. Long sessions overflow. You decide what goes — old turns get summarised, stale tool output gets dropped — or the API decides for you with an error.
  3. Every call re-sends it all. Models are stateless. The full context is billed and processed on every turn, so a chatty agent gets slower and more expensive as it runs.

You're done when: Given a 128k window and a chatty agent, you can say what gets dropped first and why.

Common traps:

Check yourself:

  1. A 128k window, a long-running agent. What do you drop first? Answer: Old, bulky tool outputs whose conclusions are already stated elsewhere. Keep the system prompt, the task and recent turns; summarise the middle.
  2. Why does turn 40 of a conversation cost more than turn 4? Answer: The whole history is sent again on every call, so input tokens grow with each turn. Prompt caching reduces the cost; it does not remove it.

Training vs inference — what is actually frozen

You stop reaching for fine-tuning when the real answer is retrieval or a better prompt.

pretraining · post-training · inference · why weights don't learn from your chat

Interactive explainer on the live page: train vs infer — You stop reaching for fine-tuning when the real answer is retrieval or a better prompt.

In short:

  1. Training changes weights. Pretraining and post-training (SFT, RLHF) are where the model's weights get updated. That happens before you ever call it.
  2. Inference changes nothing. When you call the model, the weights are frozen. Anything it "remembers" within a session is text in the context, not learning.
  3. Fine-tuning shapes behaviour. It is good at style, format and narrow task skill. It is a poor way to add facts that change, and it does not fix bad retrieval.

You're done when: You can name three problems fine-tuning does NOT solve.

Common traps:

Check yourself:

  1. Name three problems fine-tuning does NOT solve. Answer: Knowledge that changes weekly, citing sources for an answer, and bad retrieval. It also won't reliably stop hallucination.
  2. A user corrects the model and it gets it right next turn. Did the model learn? Answer: No. The correction is in the context window. Start a new session and it is gone.

Explore more

GitHub · LinkedIn · Email