GraphRAG vs RAG: what the evidence actually says

Every enterprise AI team I talk to asks the same question this year: should we put a knowledge graph behind our RAG system? The pitch for GraphRAG is strong, and the demos are good. The evidence is more interesting than the pitch, and it points to a clear rule.

The short answer: GraphRAG helps when a question needs connections across documents or a view of the whole corpus. For plain lookups, a well-tuned vector or hybrid RAG is as good or better, at a fraction of the cost. In the strongest independent study, the best results came from using both.

This post looks at three studies, says how much weight each deserves, and ends with a decision rule and the numbers to watch after rollout.

The evidence at a glance

1. Questions about the whole corpus: GraphRAG's home ground

The original GraphRAG paper from Microsoft Research (Edge et al. 2024) targets what it calls global sensemaking questions: "What are the main themes across these podcasts?" rather than "What did guest X say about Y?". Vector RAG struggles here by design, because no single chunk contains the answer.

On two corpora of about a million tokens, the global approaches (GraphRAG's community summaries at four levels, plus a map-reduce summary of the raw text) beat vector RAG in most pairwise comparisons:

Two things keep this result in proportion. The questions were generated to need a whole-corpus view, and the judge was an LLM. A later study found that LLM judges flip their verdict when you swap the order of the two answers, which is a reason to treat any LLM-judged win rate as a strong hint, not a measurement.

2. Questions that need several hops: graphs help, by a little

The most useful study for an architect is Han et al. 2025. It runs vector RAG and several GraphRAG variants on the same benchmarks with the same models (Llama 3.1, 8B and 70B), scored against ground truth. With the 70B model:

The margins are smaller than most vendor material suggests: about one F1 point on HotpotQA, five accuracy points on MultiHop-RAG. The split by question type is where it gets instructive:

Three more findings from the same paper matter for design:

• The kind of graph matters. Retrieving knowledge-graph triplets alone scored worst of all methods, far below plain RAG. For NQ and HotpotQA, only about 65% of the answer entities made it into the knowledge graph at all.
• Global search loses detail. It helps with big-picture and comparison questions but hallucinated on questions that had no answer.
• The methods are complementary. A notable share of questions was answered correctly by only one of the two. Routing each question to one method, or giving the model both retrievals, improved overall results; combining costs more compute.

3. Plain lookups: vector RAG holds its own

GraphRAG-Bench (Xiang et al. 2025) grades tasks from simple to hard on a novel corpus and a medical corpus, with GPT-4o-mini generating the answers. On the novel corpus:

The medical corpus is a useful warning. There, Microsoft's GraphRAG scored 38.6% on fact retrieval against 64.7% for vector RAG with a reranker, while HippoRAG2 reached 66.3%. "GraphRAG" is not one thing: the variant you pick can move results by twenty points in either direction.

What it costs per question

Quality is half the decision. The same benchmark reports the average number of prompt tokens each method sends to the model per question, which drives both latency and the bill. Plotting reasoning accuracy against that cost:

Indexing cost is the other line item. Building a full GraphRAG index means running an LLM over every chunk to extract entities and summarise communities. Microsoft's own LazyGraphRAG claims to cut indexing to the cost of vector RAG, 0.1% of full GraphRAG. GraphRAG-Bench indexed one book (about 56,000 tokens) and counted 591,150 tokens for LazyGraphRAG against 654,673 for Microsoft's GraphRAG: about 90%, not 0.1%. The implementations and settings may differ, which is exactly why the disagreement deserves its own post.

How strong is this evidence?

Before acting on any of the numbers above, weigh them:

• Who ran it. The most favourable results come from the team that built GraphRAG. The two independent studies are more mixed.
• How it was scored. LLM-judged comparisons are sensitive to answer order. Ground-truth scores are more trustworthy but only exist for questions with a single right answer.
• On what data. Podcasts, news, Wikipedia-style questions, a novel and medical texts. None of them is your contract archive, ticket history or product catalogue.
• How big the gaps are. Outside whole-corpus questions, differences are often a few points, smaller than the swing between GraphRAG variants.

The fair summary: the direction of the evidence is consistent across studies, the size of the effect is not, and none of it replaces a test on your own questions.

What a multi-hop question looks like in an enterprise

Most enterprise questions are lookups: a policy, a spec, a price. The ones where a graph earns its cost connect facts that sit in different documents:

A decision rule

1. Sample your real questions. Take a few hundred from search logs or support tickets and label each as a lookup, a multi-hop question or a whole-corpus question.
2. If most are lookups, fix retrieval first. Hybrid search (keyword plus vector) with a reranker is the baseline to beat. On the novel corpus in GraphRAG-Bench, vector RAG with a reranker matched or beat every graph method on fact retrieval.
3. If many need connections, test graph methods on a sample. Try more than one variant: the spread between them is larger than the spread between graph and no graph.
4. Score against ground truth. Where you must use an LLM judge, randomise the answer order and check that the result holds both ways.
5. Plan to combine, not replace. Routing or merging both retrievals gave the best overall results in Han et al. Keep the vector index.
6. Count tokens before you scale. Measure prompt tokens per question and the indexing bill on your corpus, not on a benchmark.

Each of these steps is unpacked in Five questions before you build a knowledge graph for RAG. To turn the answer into a budget request, see A GraphRAG business case, done the evidence-based way. The token and indexing bill is broken down in The real cost of GraphRAG.

What to measure after rollout

• Accuracy on a labelled question set, reported separately for lookups, multi-hop and whole-corpus questions.
• Share of "no answer" questions where the system correctly says it does not know.
• Cost per answered question and p95 latency.
• Graph freshness: the delay between a document change and the graph reflecting it.

If the answers feed an agent rather than a person, retrieval is one of five areas to design; the others are in Agentic AI from demo to production.

Sources

• Edge et al. (2024). From Local to Global: A Graph RAG Approach to Query-Focused Summarization. Microsoft Research.
• Han et al. (2025). RAG vs. GraphRAG: A Systematic Evaluation and Key Insights.
• Xiang et al. (2025). When to use Graphs in RAG: A Comprehensive Analysis for Graph Retrieval-Augmented Generation (GraphRAG-Bench).
• Microsoft Research (2024). LazyGraphRAG: Setting a new standard for quality and cost.