What is RAG? Why non-tech people care?
There is no one who hasn’t used AI to find an answer lately — and if we’re being honest, it has made our lives significantly easier. Problems that would have taken days now get solved in minutes. But if there’s one limitation worth talking about, it’s this: sometimes AI answers with complete confidence, and it’s still wrong. That’s exactly where RAG comes in.
RAG stands for Retrieval-Augmented Generation — a term that might not mean much at first, but bear with me.
Get Nandini Sah’s stories in your inbox
Join Medium for free to get updates from this writer.
Think about how a team handles a large project. Instead of one person doing everything, the work gets divided. Each person handles their piece, and when they come together at the end, the combined result is faster, more thorough, and more reliable than any one person could manage alone. RAG works on a similar principle.
When you feed documents into a RAG system, it breaks them into small, overlapping pieces called chunks — each one carrying a slice of the information, with a little overlap to ensure nothing gets lost at the edges. These chunks get stored in a way that makes them quick and precise to search through.
When you ask a question, RAG doesn’t just hand it to the AI and hope for the best. It first checks your documents for anything relevant, pulls the most useful pieces, and uses them as grounding before generating an answer. The AI isn’t guessing anymore — it’s reading, then responding.
The result? Far fewer hallucinations. Far fewer confident wrong answers. Just responses that are actually rooted in something real.
