Retrieval-Augmented Generation — The part most people do badly

Everyone can build a RAG demo in an afternoon. Almost nobody can tell you why theirs returns the wrong chunk. The difference between those two people is this track — and the eval set at the end of it.

Track 2 of the AI engineering roadmap by K Laxman — about 10 hours across 3 stages.

RAG properly — all fifteen techniques

You can name, and choose between, the retrieval strategies. Not just 'embed, store, search'.

indexing · retrieval · generation · multi-query · RAG-Fusion · query decomposition · step-back prompting · HyDE · routing · query construction · multi-representation indexing · RAPTOR · ColBERT · CRAG · adaptive RAG

In short:

  1. Three stages, three failure points. Indexing (split and embed), retrieval (find the right chunks) and generation (answer from them). A wrong answer came from exactly one of them.
  2. Most techniques fix the query or the index. Multi-query, HyDE, step-back and decomposition rewrite the question; multi-representation indexing and RAPTOR change what gets stored. Each fixes a specific miss.
  3. Corrective and adaptive RAG add judgement. Grade the retrieved chunks before answering, and re-retrieve or fall back when they are weak, instead of generating from whatever came back.

Build this — RAG over your own notes — plus a 30-question eval set. The bot is the easy half. The assignment is the eval set: thirty real questions with known answers, and a score you can quote. Without it you have a demo, and you will not be able to tell whether tomorrow's change helped.

You're done when: Given a wrong answer, you can separate retrieval failure from generation failure in under a minute.

Common traps:

Check yourself:

  1. An answer is wrong. What is the one-minute test that tells you where? Answer: Look at the retrieved chunks. If the answer isn't in them, it's a retrieval failure. If it is and the model still got it wrong, it's a generation failure.
  2. When does HyDE help? Answer: When questions are short and phrased very differently from the documents. Embedding a hypothetical answer lands closer to real answers than the bare question does.

Put to work in: CAMS Lens — SEBI-grounded compliance intelligence · Agentic Research & Decision Intelligence Assistant

Chunking — the decision that quietly decides everything

You can explain why your chunk size is what it is, instead of because a tutorial said 512.

fixed vs semantic splitting · overlap · chunk size vs recall · metadata on the chunk · contextual retrieval

Interactive explainer on the live page: context budget — You can explain why your chunk size is what it is, instead of because a tutorial said 512.

In short:

  1. The chunk is the unit of retrieval. You can only retrieve what you stored. If the answer is split across two chunks, or buried in a huge one, retrieval can't deliver it cleanly.
  2. Size is a trade-off. Small chunks match precisely but lose context; large chunks keep context but dilute the embedding. Overlap reduces boundary losses at the cost of duplication.
  3. Give each chunk its context. Prepending the document title, section heading or a short generated summary (contextual retrieval) makes a chunk findable on its own.

Build this — Same questions, four chunkings. Take the eval set from the stage above and run it against four indexes: 256 fixed, 512 fixed, 512 with overlap, and semantic. One table, four scores. You will never again accept a chunk size on faith, and you now own a number you can quote in an interview.

You're done when: You can name the chunk size you use, the one you tried before it, and the score difference between them.

Common traps:

Check yourself:

  1. Your chunks say "the policy above applies" and retrieval returns them useless. What is the fix? Answer: Add context to each chunk — document and section titles, or a generated one-line summary — so it stands on its own when retrieved.
  2. How do you choose between 300-token and 1,000-token chunks? Answer: Index both and run the same questions against each. Compare recall@k and answer quality; pick with numbers, not intuition.

Put to work in: CAMS Lens — SEBI-grounded compliance intelligence

When retrieval is the bug

You can tell a retrieval problem from a generation problem before you touch the prompt.

hybrid search · reranking · recall@k · query rewriting · the missing-document case

In short:

  1. Measure recall@k first. For a set of questions with known source documents, how often is the right one in the top k results? Until you know that, prompt tweaks are guesses.
  2. Hybrid search covers both kinds of miss. Keyword search (BM25) catches exact terms and identifiers; vectors catch paraphrase. Combining them fixes misses either one makes alone.
  3. Rerank a wide net. Retrieve 50 cheaply, then let a cross-encoder reranker pick the best 5. The reranker reads query and chunk together, which is far more precise.

Build this — Measure recall@k before you tune anything. For each question in your eval set, record whether the answer-bearing chunk appeared in the top k at all. If it didn't, no prompt on earth was going to save that answer. Split your failures into 'never retrieved' and 'retrieved and ignored' — they have completely different fixes.

You're done when: You have a recall@k number for your own corpus, and you fixed retrieval before you touched the prompt.

Common traps:

Check yourself:

  1. Users search for error code "E4031" and get nothing useful. Why, and what fixes it? Answer: Embeddings don't represent exact identifiers well. Add keyword search (hybrid) so exact matches get found.
  2. Recall@5 is 60% but recall@50 is 95%. What do you add? Answer: A reranker. The right documents are being found; they just aren't ranked high enough. Rerank the top 50 down to 5.

Put to work in: CAMS Lens — SEBI-grounded compliance intelligence · Agentic Research & Decision Intelligence Assistant

Explore more

GitHub · LinkedIn · Email