Short answerRAG (retrieval-augmented generation) lets an AI assistant answer from your own documents instead of only its general training. It finds the most relevant passages for each question, gives them to the model, and the model answers with citations. For most businesses it is faster, cheaper and easier to keep up to date than fine-tuning. A focused RAG assistant typically takes 3 to 6 weeks to build.
General AI models know a lot about the world and nothing about your return policy, your product catalogue or the SOP your operations team updated last month. Ask them anyway and they may answer confidently and wrongly. Retrieval-augmented generation, usually shortened to RAG, is the standard way to fix that. This guide explains how it works in plain English, where it is useful, and what it takes to build one properly.
How RAG works, step by step
Think of RAG as an open-book exam: the model looks up the right pages before it answers.
- Collect documents. Policies, manuals, product sheets, FAQs, past support tickets, PDFs, Google Docs and website pages.
- Split them into chunks. Each document is broken into passages of a few paragraphs, with headings and source details kept attached so every chunk knows where it came from.
- Create embeddings. Each chunk is turned into an embedding: a list of numbers that represents its meaning. Passages about similar things end up with similar numbers, even when the wording differs.
- Store them for vector search. The embeddings go into a store built for similarity search. For many projects this is simply pgvector, an extension to the PostgreSQL database you may already run.
- Search at question time. When someone asks a question, it is turned into an embedding too, and the system fetches the closest chunks. Good systems combine this with keyword search, which helps with product codes and names.
- Answer with citations. The model receives the question plus the retrieved passages, with instructions to answer only from them and to cite the source. If the passages do not contain the answer, it should say so.
When a document changes, you re-process that document. The assistant works from the new version within minutes, with no retraining.
Why not fine-tune a model instead?
Fine-tuning means further training a model on your data. It sounds like the obvious way to make AI "know your business", but for most use cases it is the wrong tool:
- It does not reliably teach facts. Fine-tuning is good at teaching style and format, and much less dependable for exact details like prices or policy clauses.
- It is hard to update. Every change to your documents means another training run.
- It cannot cite sources. You cannot easily see where an answer came from.
- It costs more in data preparation, training and testing.
RAG keeps your knowledge separate from the model, so you can update it, check it and control who sees what. Fine-tuning has its place, for example teaching a consistent tone or a specialised output format, and it can be added on top of RAG later. It is rarely where we would start.
Where a RAG assistant earns its keep
Customer support on policies and manuals
An assistant on your website or WhatsApp that answers questions about warranties, returns, installation or plan details, quoting the exact policy section. It handles the repetitive questions and passes the rest to your team with the conversation attached.
Internal knowledge search
New employees ask the same questions about leave rules, expense limits and process SOPs. An internal assistant lets them ask in plain language and get an answer with a link to the source document, instead of searching shared drives or waiting for a colleague to reply.
Sales assistant on the product catalogue
Sales and dealer teams can ask "which pump suits a three-storey building with a one-inch line?" and get a recommendation drawn from your specification sheets, with the right model numbers. This works especially well for manufacturers and distributors with large catalogues.
Multilingual answers
Your documents may be in English while your customers ask in Hindi, Marathi, Tamil or Hinglish. Current models handle this well: retrieval can match a Hindi question to an English passage, and the model replies in the customer's language while citing the English source. Test it carefully with real questions in each language you plan to support.
If the assistant also needs to take actions, such as raising a ticket or booking a service visit, it moves from assistant to agent. We explain that difference in AI agents vs chatbots.
Data preparation and privacy
In our experience, the quality of a RAG assistant depends more on the documents than on the model. Before building, expect to:
- Remove outdated and conflicting documents. If two versions of a policy exist, the assistant may quote the wrong one.
- Fix scanned PDFs and tables. Scans need OCR, and tables need careful handling so rows are not scrambled.
- Add structure. Clear headings, dates and owners on each document help both retrieval and citations.
- Fill obvious gaps. If your team answers a common question from memory, write the answer down.
On privacy, a few rules we follow:
- Apply access control at retrieval time, so a user only gets answers from documents they are allowed to see. HR records should never reach a customer chat.
- Strip or mask personal data that does not need to be in the index, in line with India's Digital Personal Data Protection Act.
- Use model providers and settings that do not train on your data, and hosting regions that match your compliance needs.
- Log questions and answers for quality review, with a sensible retention period.
Measuring accuracy
"It seems to work" is not a test. A RAG assistant should be measured before launch and after every change:
- An eval set. Build a list of real questions with the correct answers and source documents, including the hard ones. A few dozen is a useful start; grow it over time. Run it every time you change the prompts, the model or the documents.
- Retrieval checks. For each question, did the system fetch the right passage? Most wrong answers start with the wrong retrieval.
- Citations. Every answer should point to its source, so users and reviewers can check it in one click.
- "I don't know" behaviour. Include questions the documents cannot answer. A good assistant says it does not know and offers a human, rather than guessing.
- Feedback in production. A simple thumbs up or down on each answer, reviewed weekly, shows where documents or retrieval need work.
Timeline and cost
A focused RAG assistant on a defined set of documents typically takes 3 to 6 weeks: document preparation and a working prototype first, then retrieval tuning, the interface, access control and evaluation. A first prototype answering from a sample of your documents can often be ready in about a week, using the approach in how to build an MVP in 7 days.
As a guide, an AI assistant added to an existing product typically costs ₹1.2 L to ₹3 L, and a standalone AI tool or assistant ₹2 L to ₹4.5 L, with more for large or messy document sets, several languages, or integrations with your CRM or helpdesk. Running costs are mainly model usage per question plus modest hosting; creating embeddings is usually a small cost. Our AI app cost guide for India has the full breakdown.
Next step
If you have a pile of documents and a team answering the same questions every day, a RAG assistant is often the most practical first AI project. Describe your use case in our free Idea Checker for an instant estimate, or send us a brief and we will suggest where to start. You can also see our wider AI and software services.