Ardan Ultimate AI #05 — The same question with and without RAG

Field notes from working through example 05 of Ardan Labs’ Ultimate AI course by Bill Kennedy and Florin Pățan (Apache 2.0). My fork: PratikDhanave/ai-training. Thank you Bill and Florin for teaching this material — the patterns in this post are derived from the course; the production reflections at the end are mine.

What the example teaches

The motivating “before / after” demo. Pick a question whose answer is in a specific document the LLM wasn’t trained on (a new internal policy, a recent product update, your team’s runbook).

Without RAG: the LLM produces plausible-sounding nonsense.
With RAG: it produces the correct answer with citations.

This is the moment “RAG” stops being marketing and starts being obvious.

What it looks like

// Without RAG
respNoCtx := llm.Generate(ctx, query)
fmt.Println("WITHOUT RAG:", respNoCtx.Text)
// "I believe the policy allows..." — hallucinated

// With RAG
hits := db.Search(query, k=3)
respWithCtx := llm.Generate(ctx, fmt.Sprintf(template, query, hits))
fmt.Println("WITH RAG:", respWithCtx.Text)
// "According to section 4.2 of the document, the policy states..." — correct

What I learned

Hallucination is a confidence problem, not a knowledge problem. The LLM doesn’t say “I don’t know”; it produces something that sounds like an answer. Without grounding, you can’t tell apart “knows” from “guessing.” RAG turns the guess into a cite.

The contrast is the pitch. When explaining RAG to a non-technical stakeholder, this example is the one that lands. Words about embeddings don’t; the side-by-side does.

Production connection

I open every RAG project pitch with the equivalent of this demo, scaled to the client’s domain. The without-RAG answer is always slightly wrong in a way the client recognises; the with-RAG answer is correct. Sale closed.

Credit & reference. This post is field notes on example 05 from Ardan Labs’ Ultimate AI by Bill Kennedy + Florin Pățan, licensed Apache 2.0. The original example: cmd/examples/example05-rag-motivation/. My fork with notes: PratikDhanave/ai-training. Highly recommend the course for anyone building AI applications in Go — the material is rigorous and the Kronk + yzma + llama.cpp pipeline gives you hardware-accelerated local inference end-to-end. Thank you, Bill and Florin.