How to learn AI, ML and Agentic AI engineering — the path I actually walked

A learning path written by K Laxman, a Senior AI Engineer who ships production Generative AI systems — not a generic link list. It starts from "you can already code" and goes straight at large language models, retrieval and agents. Every stage ends in something you build, and links to the production system it was used to build.

The order is the argument: LLM fundamentals → RAG → Agent loops → Tools / MCP → Evals + tracing → Production architecture → System design.

19 stages across 8 tracks, about 57 hours of study plus the builds. 13 of 14 resources are free.

Choose a path

AI — Understand the model, then ground it in your own data

The most common job going, and the one with the widest gap between a demo and something you would put in front of a customer. Understand what the model actually is, then learn to feed it your data without it inventing things.

Track 1 — LLM fundamentals

Understand the thing everything else sits on (~6 hours)

If you only ever do one item on this page, do the first one. Almost every confused conversation I have about AI systems traces back to someone never having watched this.

The whole pipeline, end to end

You can trace a token from raw internet text all the way to a chat reply, naming every stage.

pretraining · tokenisation · transformers · inference · SFT · RLHF · hallucination · tool use · reasoning models

Build this — Re-implement decoding. Take any provider that returns logprobs and implement greedy, temperature and top-p sampling yourself. Write a README explaining, in your own words, when you'd use each. It is thirty lines of code and it makes §0.5 permanent.

You're done when: You can explain, without notes, why a model hallucinates confidently rather than saying 'I don't know'.

Put to work in: Jarvis — six interchangeable providers behind one loop · CAMSCode — enterprise AI coding assistant

Track 2 — Retrieval-Augmented Generation

The part most people do badly (~10 hours)

Everyone can build a RAG demo in an afternoon. Almost nobody can tell you why theirs returns the wrong chunk. The difference between those two people is this track — and the eval set at the end of it.

RAG properly — all fifteen techniques

You can name, and choose between, the retrieval strategies. Not just 'embed, store, search'.

indexing · retrieval · generation · multi-query · RAG-Fusion · query decomposition · step-back prompting · HyDE · routing · query construction · multi-representation indexing · RAPTOR · ColBERT · CRAG · adaptive RAG

Build this — RAG over your own notes — plus a 30-question eval set. The bot is the easy half. The assignment is the eval set: thirty real questions with known answers, and a score you can quote. Without it you have a demo, and you will not be able to tell whether tomorrow's change helped.

You're done when: Given a wrong answer, you can separate retrieval failure from generation failure in under a minute.

Put to work in: CAMS Lens — SEBI-grounded compliance intelligence · Agentic Research & Decision Intelligence Assistant

Track 4 — Evals, tracing & observability

The stage everyone skips (~5 hours)

This is the one that separates a demo from a product. As a system gets complicated, evals and tracing stop being nice-to-have and become the only way you can change anything without breaking it. If you cannot state your system's accuracy as a number with a dataset behind it, you do not have a system — you have a vibe.

Dataset → evaluator → task → interpretation

You can build an evaluation harness before you build the feature.

datasets · evaluators · LLM-as-judge and its failure modes · tracing · regression detection

Build this — Put a number on the RAG bot. Add the eval set and tracing to what you built in Track 2. Report the score in the README. Then change the chunk size and report it again — that delta is the whole discipline.

You're done when: You can say what your system's accuracy is, with a number and a dataset behind it.

Put to work in: CAMS Lens — grounded answers that have to be right

ML — The classical craft — rows, columns, and not fooling yourself

Quieter than the AI headlines and still what most companies actually need. A table, a column you want to predict, and the discipline to know whether it worked. Boosted trees have not stopped winning just because language models arrived.

Track 4 — Classic ML

Still what wins on the data most companies actually have (~10 hours)

The half of the job the AI headlines skipped, and the half that quietly pays for most ML salaries: a table of rows, a column you want to predict, and the discipline not to fool yourself about how well it worked. Boosted trees did not stop winning on tabular data just because language models arrived — if your data lives in a database rather than in documents, this is the work.

Supervised learning, end to end

You can go from a CSV to a model with an honest score, and say what the score means.

train/validation/test · baselines · choosing the metric · cross-validation · over- and underfitting

Build this — A model with a baseline you have to beat. Pick any tabular dataset. Write down the dumbest possible predictor first — the mean, the majority class — and score it. Then beat it, and report both numbers. If you cannot beat the dumb one, that is the finding.

You're done when: You can say why a 99%-accurate model can be worthless, without looking it up.

Put to work in: Driver Drowsiness Detection

Leakage, imbalance, and the ways you fool yourself

You can spot the mistakes that make a model look brilliant offline and useless in production.

data leakage · time-based splits · class imbalance · distribution shift · error analysis

There is no single video for this one, and that is telling: it is the part of ML that gets learned by being burned. The two habits that matter — split on time whenever the real system will predict the future, and look at fifty actual errors before you touch the model — will save you more than any algorithm choice.

Build this — Break your own model on purpose. Take the model from the previous stage and introduce a leaked feature deliberately. Watch the score jump. That jump is exactly what a subtle leak looks like, and now you know its shape.

You're done when: Given a suspiciously good result, leakage is the first thing you check — and you know where to look.

Trees, boosting — and when to reach for deep learning

You choose a model class on the shape of the data, not on what is fashionable.

decision trees · random forests · gradient boosting · when deep learning wins · when it does not

Build this — Same dataset, three model classes. Boosted trees, a linear model, and a small neural net on identical splits. Report all three, plus training time. The answer on tabular data will usually surprise anyone who only reads AI news.

You're done when: Asked whether to use deep learning, your first question is about the data, not the model.

Put to work in: Table Recognition System

Track 4 — Evals, tracing & observability

The stage everyone skips (~5 hours)

This is the one that separates a demo from a product. As a system gets complicated, evals and tracing stop being nice-to-have and become the only way you can change anything without breaking it. If you cannot state your system's accuracy as a number with a dataset behind it, you do not have a system — you have a vibe.

Dataset → evaluator → task → interpretation

You can build an evaluation harness before you build the feature.

datasets · evaluators · LLM-as-judge and its failure modes · tracing · regression detection

Build this — Put a number on the RAG bot. Add the eval set and tracing to what you built in Track 2. Report the score in the README. Then change the chunk size and report it again — that delta is the whole discipline.

You're done when: You can say what your system's accuracy is, with a number and a dataset behind it.

Put to work in: CAMS Lens — grounded answers that have to be right

Agentic AI Engineering — A model, in a loop, with tools — and everything that keeps it up

Where I spend my own time, and where the marketing is furthest from the engineering. It is a while loop with an approval gate — the hard parts are the tools, the evaluation and the failure isolation, not the magic.

Track 1 — LLM fundamentals

Understand the thing everything else sits on (~6 hours)

If you only ever do one item on this page, do the first one. Almost every confused conversation I have about AI systems traces back to someone never having watched this.

The whole pipeline, end to end

You can trace a token from raw internet text all the way to a chat reply, naming every stage.

pretraining · tokenisation · transformers · inference · SFT · RLHF · hallucination · tool use · reasoning models

Build this — Re-implement decoding. Take any provider that returns logprobs and implement greedy, temperature and top-p sampling yourself. Write a README explaining, in your own words, when you'd use each. It is thirty lines of code and it makes §0.5 permanent.

You're done when: You can explain, without notes, why a model hallucinates confidently rather than saying 'I don't know'.

Put to work in: Jarvis — six interchangeable providers behind one loop · CAMSCode — enterprise AI coding assistant

Track 3 — Agents & tools

A model, in a loop, with tools. That's it. (~8 hours)

This is the track I care most about, because it is the one where the industry's marketing is furthest from the engineering. An agent is not a mysterious autonomous intelligence. It is a while loop. Write one by hand and the mystique never comes back.

The agent loop, by hand

You can write an agent without a framework and explain every line of it.

environment · tool schemas · the while loop · context accumulation · approval gates

Interactive explainer on the live page: agent loop — You can write an agent without a framework and explain every line of it.

Build this — Write the loop. No framework.. A while loop, three tools, JSON-schema arguments, one approval gate before anything destructive. This is the keystone exercise of the whole page — do not substitute a framework tutorial for it.

You're done when: You can draw the loop on a whiteboard and point at exactly where a tool result re-enters the context.

Put to work in: Jarvis — that loop, in production, running live in your browser · OpsPilot — agentic request triage & remediation

Tools and MCP — connecting agents to real systems

You can design a tool an LLM actually uses correctly, and expose it over MCP.

tool design · argument validation · MCP architecture · tool servers · when a tool is too broad

Build this — Wrap your Track 2 RAG bot as an MCP server. One tool, a real schema, connected to a client you didn't write. The moment someone else's agent can call your thing, tool design stops being theoretical.

You're done when: You can say why a tool named `do_stuff(input: string)` will be called wrongly, and rewrite it so it won't be.

Put to work in: Intelligent Document Processing — multi-agent system

Frameworks — deliberately last

You choose a framework because it removes work you understand, not because you're lost.

LangGraph and friends · what they abstract · what they hide

There is no video here on purpose. Read the docs of one framework after you have written the loop yourself. You will find you already know what every abstraction is for — and you will notice the ones that are solving a problem you don't have.

You're done when: You can name one thing your chosen framework does that you would rather do yourself, and say why.

Track 4 — Evals, tracing & observability

The stage everyone skips (~5 hours)

This is the one that separates a demo from a product. As a system gets complicated, evals and tracing stop being nice-to-have and become the only way you can change anything without breaking it. If you cannot state your system's accuracy as a number with a dataset behind it, you do not have a system — you have a vibe.

Dataset → evaluator → task → interpretation

You can build an evaluation harness before you build the feature.

datasets · evaluators · LLM-as-judge and its failure modes · tracing · regression detection

Build this — Put a number on the RAG bot. Add the eval set and tracing to what you built in Track 2. Report the score in the README. Then change the chunk size and report it again — that delta is the whole discipline.

You're done when: You can say what your system's accuracy is, with a number and a dataset behind it.

Put to work in: CAMS Lens — grounded answers that have to be right

Track 5 — Production architecture

Where AI engineering becomes engineering (~6 hours)

The gap between a working prototype and a system real users depend on is not model quality. It is latency, failure isolation, cost, caching and observability — ordinary distributed-systems work, applied to a component that is slow, expensive and occasionally wrong.

A production case study worth reading twice

You can reason about an AI system the way you'd reason about any distributed system.

streaming & stateful inference · full-duplex voice · asynchronous delegation · WebRTC · latency budgets · session affinity · prompt caching · failure isolation · observability

Build this — Instrument something you already built. Take the Track 3 agent. Add a latency budget per turn, a cost counter, provider failover, and a trace you can actually read when it misbehaves at 2am.

You're done when: You can draw your system's fast path and slow path as separate lines, and say what happens when the slow one stalls.

Put to work in: Jarvis — per-turn provider failover and streaming · Dialogflow CX voicebot — production conversational system

Explore more

GitHub · LinkedIn · Email