How Much Does a RAG AI Assistant Cost to Build?
A retrieval-augmented generation (RAG) assistant, an AI that answers questions from your documents rather than the model's general training, is the most common AI project businesses actually buy. Here is what it costs to do properly, including the part most quotes omit: what it costs to run.
Build costs
| Tier | Typical range | Includes |
|---|---|---|
| Internal pilot | $15,000–$25,000 | One document set, internal users, basic evaluation, client-owned API keys |
| Standard internal assistant | $25,000–$75,000 | Multiple sources, permissions-aware retrieval, gateway with cost caps, evaluation suite, admin analytics |
| Customer-facing assistant | $100,000+ | Guardrails, brand-safety evaluation, scale testing, abuse controls, monitoring SLOs |
The parts that separate production from demo
- A gateway, not raw API calls. Provider failover (Anthropic to OpenAI and back), caching, retries, rate limits, and per-feature cost caps. Without one, a provider outage is your outage and a runaway loop is your credit card's problem.
- Evaluation. A test set of real questions with graded answers, run on every change. This is how you know the assistant improved rather than merely changed, and it is the single most skipped component in the industry.
- Permissions-aware retrieval. If your documents have access controls, your assistant must respect them, or it becomes a data-leak machine with a friendly interface.
- Honest fallbacks. Production assistants say "I don't know, here's a human" by design. That behavior is engineered, not hoped for.
The monthly bill nobody budgets
AI usage is a metered utility: token costs scale with use and change with model pricing. A modest internal assistant typically runs $100–$1,000 per month in model usage; customer-facing systems more. The governance answer is a hard monthly cap enforced at the gateway with per-feature attribution, so spend is a budget line, not a surprise. Add $1,500–$5,000 monthly for platform operations if you want evaluations, cost monitoring, and model updates handled for you.
When not to build one
If your knowledge base is thirty documents that change yearly, a well-organized FAQ beats a $40,000 assistant. RAG earns its cost when content is large, changing, and searched badly today, when support volume is real, or when the assistant sits inside a workflow (drafting, triage, analysis) rather than replacing a search box. An honest firm will tell you which case you are before quoting.
Frequently asked questions
Which model should we use?
The unsatisfying truth: it should be swappable. Models leapfrog each other every few months; a gateway architecture lets you change providers in configuration, not in a rebuild. Locking your product to one vendor's model is 2024 thinking.
Can we use our confidential documents safely?
Yes, with enterprise API terms (no training on your data), permissions-aware retrieval, and, for regulated data, providers that sign BAAs. What you cannot safely do is paste confidential documents into consumer chat tools.
How do we know it won't make things up?
You reduce hallucination with grounded retrieval, citations back to source documents, and an evaluation suite that measures failure honestly. Anyone promising zero hallucination is selling something; anyone with an eval dashboard is engineering something.
Talk to the people who do this work
Ashton Group provides ai & machine learning engineering for organizations across Maryland, DC, Virginia, and beyond. Tell us about your project and we'll reply with a candid assessment within one business day.
Email hello@ashtongroup.org About this service