A GraphRAG business case, done the evidence-based way

Most AI business cases follow the same recipe: take a benchmark result from a vendor, multiply it by headcount, call the product ROI. It reads well until someone asks where the percentage came from, whether it applies to our data, and what happens if it is half as good.

Evidence-based management, as developed by the Center for Evidence-Based Management and set out by Barends and Rousseau, offers a structure that survives those questions. It has six steps. Here they are applied to one decision: whether to add graph retrieval to an existing RAG system.

The six steps, applied

Step 1. Ask: write the decision, not the technology

"Implement GraphRAG" is a solution looking for a problem. A decision looks like this: can graph retrieval raise the share of correctly answered multi-hop support questions enough to pay for itself within a year?

Write down the baseline (today's accuracy, today's time to answer), who owns the decision and who lives with the result. If you cannot state the baseline, measuring it is the first piece of work, not building the graph.

Step 2. Acquire: four sources of evidence

Evidence-based management looks at four sources, not one:

• Scientific evidence: published studies. For GraphRAG, the three in GraphRAG vs RAG: what the evidence actually says.
• Organisational data: a labelled sample of your real questions (see Five questions before you build a knowledge graph), today's answer accuracy, time spent on questions the system gets wrong.
• Professional expertise: what your support engineers and domain experts know about where answers fail.
• Stakeholder concerns: what users, security and compliance need: permissions, audit, explainability.

Vendor material belongs in the first group but deserves the hardest look in the next step.

Step 3. Appraise: how much is each piece worth?

Here is my appraisal of the main GraphRAG evidence, and where your own data fits:

Step 4. Aggregate: a cost model and a value range

Two simple formulas carry most of the business case:

• Cost per month = questions per month x prompt tokens per question x price per token, plus re-indexing (corpus tokens x indexing ratio x price) spread over the months between rebuilds, plus engineering time.
• Value per month = questions per month x share that are multi-hop x accuracy improvement x cost of a wrong or missing answer.

The ratios do not have to be guessed. GraphRAG-Bench (Xiang et al. 2025) measured them: Microsoft's GraphRAG processed about 12 tokens per token of source text to build its index and HippoRAG2 about 6; per question, prompts averaged 879 tokens for vector RAG, 1,008 for HippoRAG2 and 38,707 for GraphRAG's local search.

A worked example, with illustrative assumptions. A support knowledge base of 50 million tokens, 20,000 questions a month, and a model price of $0.50 per million input tokens (replace all three with your own figures):

Two things stand out. Query cost depends far more on the method than on the idea of a graph: HippoRAG2 costs about the same as vector RAG, GraphRAG local search about 44 times more. And at these volumes, token costs are small next to people: building, evaluating and maintaining extraction will usually be the largest line. Estimate that time explicitly rather than leaving it out.

For value, use a range rather than a number. The evidence gives a prior: in Han et al., GraphRAG's local search added 5.4 accuracy points on multi-hop questions and combining it with vector RAG added 11.9. Your pilot will tell you where you really land. With an assumed $10 cost per wrong answer (ten minutes of an engineer's time):

Presenting the result this way changes the conversation. Instead of defending one ROI figure, you show which inputs matter, which ones the pilot will measure, and the point below which the project is not worth it.

Step 5. Apply: a pilot with the threshold agreed in advance

A pilot that can fail is the most valuable part of the business case. Design it before anyone sees results:

1. A labelled set of a few hundred real questions, split into lookups, multi-hop and whole-corpus questions, with answers checked by a domain expert.
2. Several options, not two: today's system, hybrid retrieval with a reranker, one or two graph methods, and a combination.
3. Ground truth first. Where an LLM judge is unavoidable, swap the order of the answers and keep only verdicts that hold both ways; Han et al. found that order alone can flip the result.
4. Cost and latency recorded for every option, measured on your prompts, which will be longer than a benchmark's.
5. A go or no-go rule, written down: for example, go if the multi-hop gain clears the break-even point from step 4, with no loss on lookups and on questions that have no answer.

Break-even is just the value formula solved for the gain: monthly cost divided by (questions x multi-hop share x cost of a wrong answer). In the example above, token costs alone break even at well under one point; with engineering time included, the threshold is the number your stakeholders should agree on before the pilot starts.

Step 6. Assess: decide again, on a date

After rollout, track the same measures the pilot used and set a date, say 90 days out, to decide whether to keep, extend or roll back:

• Accuracy by question type, including questions with no answer in the corpus.
• Cost per answered question and p95 latency.
• Graph freshness: how long a document change takes to reach the answers.
• Time spent maintaining extraction, compared with the estimate.

A one-page template

The whole case should fit on one page, in this order:

1. Decision in one sentence, with owner and date.
2. Baseline: today's accuracy and cost, by question type.
3. Evidence with its appraisal, in a table like the one above.
4. Options compared, including doing nothing and improving retrieval.
5. Cost model with every assumption visible.
6. Value range and the break-even point.
7. Pilot design and the go or no-go rule.
8. Risks: extraction quality, permissions, freshness.
9. Assessment plan: metrics and the date of the next decision.

The template is also a workbook you can download and fill in: AI business case template (Excel). It has the decision brief, a cost and value model with formulas, and a sensitivity table, and it works for any AI investment, not only GraphRAG.

The cost lines are worked through with current prices in The real cost of GraphRAG. The same one-page structure works for any AI investment, not only graphs: I use it for agent platforms (Agentic AI from demo to production) and for coding assistants (Does AI make developers faster?). Which governance framework the case has to satisfy is covered in AI transformation frameworks compared.

Sources

• Center for Evidence-Based Management (CEBMa).
• Barends, E. and Rousseau, D. M. (2018). Evidence-Based Management: How to Use Evidence to Make Better Organizational Decisions. Kogan Page.
• Edge et al. (2024). From Local to Global: A Graph RAG Approach to Query-Focused Summarization. Microsoft Research.
• Han et al. (2025). RAG vs. GraphRAG: A Systematic Evaluation and Key Insights.
• Xiang et al. (2025). When to use Graphs in RAG (GraphRAG-Bench).
• Microsoft Research (2024). LazyGraphRAG: Setting a new standard for quality and cost.