Building a RAG-Based Product: How to Turn Business Knowledge Into a Useful Feature Using AI
A routine question can send an employee looking through company policies, product guides, and old support tickets. Retrieval-augmented generation (RAG) lets an AI assistant search those materials and use the relevant passages to answer. How useful the answer is depends on whether the assistant finds the right source and uses the version that applies.
What is retrieval-augmented generation (RAG) and how does it work?
RAG (retrieval-augmented generation) is a technical approach in which the system finds information in internal company materials to give an answer to the input query. Then, the gathered data goes to the LLM, where the model processes it and generates the accurate answer. Every query has its own source of truth, so the RAG system operates on different company materials and needs access to them.
For example, an employee asks the corporate assistant how to arrange a business trip. The system finds the instructions in the company’s knowledge base, and the assistant uses them to explain where to submit the request and who approves it. The assistant includes a link to the document in the response so the employee can check the details.
To explain this more clearly, here’s how a RAG-based product processes a query.

Submitting a request for an international business trip may require additional approval. In this case, the assistant locates two documents: the general guidelines for submitting a request and the specific regulations for international travel. Links to both sources help the employee verify that the assistant has taken the additional requirement into account and the answer he got based on official and complete company knowledge sources. In the RAGTruth study, unconfirmed statements were found even in responses for which the model had received search results. Therefore, when testing RAG, it is important to verify whether the sources actually support each key claim in the assistant’s response.
If a company changes its business travel approval process, the assistant can use the updated policy version after the search index is refreshed. Before launch, the team decides who approves changes and how quickly they should appear in search results. They also define where the assistant should direct employees if a document is temporarily unavailable.
Which business challenges can be effectively addressed using RAG?
RAG for business is a helpful assistant in situations with repetitive tasks, where teams search through large amounts of documentation, compare rules, and figure out how they apply to a specific case.
To support a SaaS product, the assistant can find pricing terms, product version updates, and instructions for the required configuration. In the insurance industry, a similar feature can compile policy provisions and endorsements related to a claim and highlight what information the specialist needs to review the case.
Another scenario involves analyzing an incident following a product update. A team can ask the system to compile a timeline based on deployment logs, tickets, and analysis notes. The authors of the EMNLP 2025 study simulated this process using synthetic corporate data, where the response sometimes required combining multiple sources. For a pilot project, this means it’s worth testing such queries as well, even if the first version of the product handles typical help desk questions well.
To proceed with the first scenario, review actual inquiries and note how often they recur, how long it takes to find the information, and what can happen if something goes wrong. Getting the status of a specific payment requires contacting the payment system. RAG can explain which refund terms apply to that payment. A good place to start is a process that already has approved sources, where the team can verify the results in their familiar work interface and compare them with current metrics.
How can corporate knowledge be prepared for AI-powered retrieval?
Prepare corporate knowledge so the RAG system can find the authoritative source, select the version that applies, and enforce access permissions. To organize this process, act as follows:
- Inventory your sources and assign owners. Identify where approved rules are stored and where teams record how those rules were applied. Then, assign an owner for specific materials with full access to them.
- Define metadata and source rules. Decide which fields to capture, including the source link, approval status, version, effective dates, market, product, and access permissions. Specify which sources take precedence and when approved exceptions apply.
- Prepare the documents. Run OCR on scanned PDFs before splitting the text into passages. Keep headings and table headers with the relevant content, and link passages to their parent sections so the system can retrieve related conditions together.
- Index the passages with their metadata. At query time, filter by access, scope, and effective date. For past events, use the version in effect on the event date. For current questions, use the approved version in effect today and apply any relevant exceptions.
- Plan how updates reach search. Set a target for how soon approved changes must appear. Then sync updates and deletions to the index and clear any affected cache entries.
- Test version conflicts and update delays. Include both versions of a company policy in the pilot index. Verify which version the assistant uses for past and current questions, and test how it responds before a newly approved change reaches search.
The HoH benchmark found that outdated content reduced answer accuracy even when current information was available.

How to design a RAG architecture – from data ingestion to generating AI-powered responses?
The first step is to split the RAG architecture into two parts: preparing the search index and processing the user’s query. The first process updates the content based on changes in the sources, while the second uses this index to generate a response based on user permissions and query context. During RAG development, this split helps teams trace an incorrect answer back to outdated content in the index, a search that missed the right document, or the model’s interpretation of the retrieved text.
Visually, index preparation and query processing look like this.

During indexing, connectors receive documents; the system extracts the text and divides it into meaningful fragments. The embedding model converts these fragments into vectors – numeric representations that the search engine uses to find similar content. Along with the fragment, the index stores its identifier, metadata, and a link to the original.
When a query is received, the server limits the search to authorized materials. Hybrid search combines word-based matches (useful for error codes and contract numbers) with content-based search. Next, a re-ranking model can sort the results by relevance to the query.
In Anthropic’s tests, short descriptions were added to each document section, and the search used embeddings, keywords, and re-ranking. The proportion of relevant snippets that did not appear in the top 20 results decreased from 5.7% to 1.9%.
The service generates a query for the LLM containing selected excerpts and their identifiers. After generation, it checks whether the links lead to the sources provided in the context, and the interface shows the answer along with the source materials for review. If the search doesn’t provide sufficient grounds, the system may ask for clarification.
How to choose the right embedding models, vector databases, and large language models?
RAG development starts by choosing components based on quality as specified in your documentation, latency, data requirements, and total cost. Test embeddings using queries in the required languages, the database using access filters, and the LLM using queries with exceptions and without sufficient evidence.
In your budget, account for reindexing and traffic spikes. For the new embedding model, prepare a separate index and test it before switching over.
|
Component |
Capabilities and deployment |
Pricing |
| Embedding model | Converts text into embeddings through a managed API. | $0.02 per million tokens. |
| Embedding model Qwen3-Embedding-0.6B | Open-weight embedding model under Apache 2.0. Can be self-hosted or accessed through a hosted API. | $0 model license fee.
$0.0118 per million input tokens through Cloudflare Workers AI. |
| Vector search | Managed vector database with hybrid search and metadata filters. | $0 for the 1 GB RAM free cluster.
About $120.34/month for an example three-node cluster on AWS N. Virginia with 1 million dense vectors at 1,536 dimensions and a replication factor of 1. |
| Vector search | Open-source PostgreSQL extension that supports vector search with SQL filters. It can be paired with full-text search. | $0 software license fee.
Managed hosting starts at $15/month for a single-node DigitalOcean PostgreSQL cluster with 1 GiB RAM. |
| LLM | Generates answers from retrieved sources through a managed API. | $2 per million input tokens and $10 per million output tokens. |
| LLM | Open-weight LLM under Apache 2.0. Can be self-hosted or accessed through a hosted API. | $0 model license fee.
Groq charges $0.075 per million input tokens and $0.30 per million output tokens. |
Capabilities and pricing were checked in November, 2026. API rates are in USD for standard requests without caching or batch processing.

How can RAG be integrated into existing products and business processes?
Integrate RAG for business into the screen and stage of the process where the user already needs an answer. For a support service, this could be a panel within a ticket that generates a draft, displays sources, and allows the agent to edit the text before sending it. In simple terms, make it clear and visual for users, showing how the system generates its answers.
Pass the user ID, organization, request type, and verified account details from the session and ticket data. For refunds, use the billing API to check payment details and find the right policy, while keeping the usual approval steps.
For the pilot, test past requests with the right user roles and document versions. Then give access to a small group of agents, track their edits, and show the source, effective date, and escalation option in the interface.
Upload the hint separately from the ticket and set a time limit for it. In case the search or model isn’t available, the agent can open a ticket and keep working.
Roll out the feature by inquiry type. Questions about general policy and requests involving contractual exceptions may have different criteria for approval before launch. Before rolling out the feature, check which prompts agents actually use and at which stage they spend time verifying information.
How to keep data secure and compliant in a RAG system?
Protect data during the indexing, search, and generation phases. The server must verify access to a document before its text is fed into the model, and data processing policies must also cover indexes, caches, and logs.
- Check document permissions at query time. Keep each organization’s data separate and check what the user can access before sending text to the model. Recheck access before serving a cached answer or opening a cited document.
- Treat retrieved text as untrusted. A document or support ticket can contain instructions designed to make the model disclose restricted data. Review sources before indexing, limit what the assistant can do, and require approval for sensitive actions.
- Encrypt data and limit what you log. Encrypt data in transit and at rest, and give service accounts only the permissions they need. For audits, record who made the request and which sources the system used. Keep the full text only when needed for a defined purpose.
- Review how providers handle your data. Map where documents, embeddings, prompts, and logs go. Check what data each provider gets, where it’s processed, how long it’s stored, and whether it’s used to train models.
- Account for GDPR where it applies. Determine why you collect personal data, only collect what you need for that purpose, set retention periods, and review international transfer requirements.
- Disclose AI interactions where required. Under Article 50 of the EU AI Act, providers of systems designed to interact directly with people generally need to tell users they’re interacting with AI.
How to test RAG performance and evaluate its business value?
Test search and generation separately, and determine value based on changes in workflow results. Before the pilot, record the metrics your current processes have. Then, prepare a set of real questions with approved answers and excerpts, and indicate when clarification is needed. Separate diagnostics of search and generation form the basis of the RAGChecker methodology.
|
What to evaluate |
Metric |
How to measure it |
| Retrieval | Recall@k | Share of all reference passages for a query that appear in the top k search results. |
| Answer quality | Source-supported claims | Share of factual claims in the answer that the cited sources support. |
| Insufficient evidence | Appropriate clarification or escalation | Share of queries with insufficient evidence that the system handles as expected by asking for clarification or escalating. |
| Workflow | Productivity and service quality | Support tickets resolved per hour and the repeat contact rate. |
| Cost | Cost per resolved ticket | Total spending on support labor, indexing, retrieval, answer generation, and maintenance divided by the number of tickets resolved in the same period. |
In a study published in The Quarterly Journal of Economics in 2025, support agents using a generative assistant resolved, on average, 15% more inquiries per hour. The effect was greater among employees with less experience and lower baseline performance. In your pilot, compare agents with similar experience who handle requests of similar complexity.

What are the best practices for building a RAG-based product?
Start with small steps. Pick one clear process, decide who owns the knowledge base, and test every change on real requests. Before the pilot starts, agree on which mistakes are serious enough to block a release. For example, showing a document from the wrong company is much more serious than a draft that simply needs a wording fix.
Give agents an easy way to report a wrong answer. Save the source, its version, and a short note about the issue. Keep personal data out unless it’s needed. The knowledge owner can then fix the source or add an exception, and the product team can include that case in future tests.
After an update, run the same test questions again, including cases with conflicting sources, missing information, and removed access. Roll updates out gradually and keep a backup version ready in case you need to go back.
Search can also get tricky when old and new information look similar. FRESCO research showcases that re-ranking may sometimes choose a longer, outdated source over a shorter, current one. Test this with a current policy and an older, more detailed FAQ. Include historical questions too, since the older version may sometimes be the correct one.
Also test cases where there isn’t enough information to answer. The system should show what’s missing and send the request with its context to the right team. This way, all the key steps are covered across different scenarios.
Conclusion
RAG-based product transforms corporate materials into a product feature, so the team can follow each request back to the right source and see how it affects the workflow.
At TRIARE, we help design retrieval-augmented generation features and integrate them into a product or internal system. During Discovery, we review the use case, check the available sources and access rules, and agree on how the pilot will be measured. From there, we can test the idea with a prototype, select the right tools, and integrate the solution into the interface.
Do you have a knowledge base, guidelines, or a history of user inquiries that you’d like to incorporate into your product? During our initial conversation, we’ll determine which scenario is worth testing and what’s needed to evaluate it using your data. Contact us to start building the right RAG solution for business with the TRIARE team.