--- title: "Experimenting with GraphRAG: Adding Knowledge Graphs to RAG Pipelines" description: "Blending knowledge graphs with RAG pipelines unlocks richer, scalable insights—bridging the gap between granular retrieval and holistic understanding." canonical_url: "https://2389.ai/posts/experimenting-with-rag/" last_updated: "2026-03-17T11:10:28-05:00" doc_version: "1.0" author: "Michael Sugimura" date: 2025-03-06 tags: ["knowledge-graph", "rag", "llm", "semantic-search", "embeddings", "infrastructure"] --- # Experimenting with GraphRAG: Adding Knowledge Graphs to RAG Pipelines > Blending knowledge graphs with RAG pipelines unlocks richer, scalable insights—bridging the gap between granular retrieval and holistic understanding. Recently, my team and I have been experimenting with implementing aspects of Microsoft's GraphRAG and LazyGraphRAG pipelines. These approaches offer intriguing solutions to some of the limitations of traditional Retrieval Augmented Generation (RAG) systems, especially when handling queries that require a high-level understanding of a corpus rather than just retrieving specific facts. ## The RAG Problem Space A colleague of mine framed it well: LLMs are strong at general knowledge and language generation tasks, but they lack contextual knowledge about specific domains. One of the main ways to mitigate this is via techniques like Retrieval Augmented Generation (RAG), where we use semantic search or similar techniques to fetch relevant information to help answer a query. For a long time, when I needed to add a knowledge base for an LLM, I would do so the same way I built semantic search pipelines at various e-commerce companies—using either off-the-shelf or fine-tuned models to search a vector database, sometimes augmented with tool usage for fetching more recent information. These techniques have been helpful in mitigating the most egregious types of hallucination that were common when ChatGPT first became popular. Asking about a specific recipe? We can fetch instances of that recipe from our database to give the LLM the appropriate context it needs to focus its output on domain-specific knowledge. ## The Limitations of Traditional RAG Traditional RAG handles direct, specific queries well, but struggles with broader, more abstract ones. For example, when I queried a corpus of presidential orders: - A specific query like "What executive order addressed immigration enforcement in 2023?" works well and retrieves relevant documents and passages. - But global queries like "What are the major themes across all executive orders?" or "How have policy priorities evolved over time?" fall flat. Here's why: traditional RAG hunts for documents that sound like your query. Let's take our query, "What are the major themes across all executive orders?" as an example. Traditional RAG would conduct a semantic search across the corpus and retrieve entities that mention, "Major changes to ICE" or "Major Sugi is promoted." Then, it would summarize this skewed dataset. There's no amount of post-hoc summarization that can fix this. All because the model never saw the full conceptual landscape in the first place. This limitation becomes especially obvious when the task demands a holistic understanding of a large corpus -- which is exactly where newer techniques like Microsoft's GraphRAG come into play. ## GraphRAG: Microsoft's Graph-Based Approach Microsoft's GraphRAG, introduced in a [recent paper](https://arxiv.org/abs/2404.16130), leverages knowledge graphs for information retrieval to help augment the knowledge base of an LLM. The broad strokes of their approach are: 1. Take a corpus of data (internet text, product catalogs, executive orders, etc.) 2. Use an LLM to extract a knowledge graph structure from this arbitrary data 3. Run community detection algorithms across the graph to infer high-level groupings 4. Summarize each community using the data available about all entities within that community 5. When a user submits a query, use the node-level and community-level information to provide a response The advantage of this approach is that the community-level information captures groups of nodes and their inputs, requiring the LLM to do less aggregation work and allowing it to focus more on text generation while drawing from a mixture of node and community-level information. ## LazyGraphRAG: A More Efficient Approach Following GraphRAG, Microsoft released [LazyGraphRAG](https://www.microsoft.com/en-us/research/blog/lazygraphrag-setting-a-new-standard-for-quality-and-cost/), which tries to minimize the number of LLM calls required—a sensible optimization. The LazyGraphRAG approach: 1. Build a RAG pipeline and perform semantic search across your corpus 2. Once you've identified a relevant subset of the corpus, run that subset through an LLM to construct a knowledge graph 3. Detect communities and generate community summaries for that subset 4. Use those structures for RAG This approach shifts the LLM calls later in the pipeline, which creates additional computational overhead at query time since the knowledge graphs and community summaries aren't constructed until the user makes a query. In some ways, it seems conceptually stronger but also computationally heavier than the approach we've implemented. ## Our Implementation: A Middle Ground For our experimentation, we've created a hybrid approach that sits between the original GraphRAG and LazyGraphRAG methods. Our pipeline works like this: 1. **Preprocessing Stage:** - We build the knowledge graph upfront by extracting entities and relationships from our corpus - We run community detection to identify clusters of related information and create summaries for each cluster 2. **Query-Time Processing:** - When a user asks a question, we use semantic search to find the most relevant nodes and or communities - This focused approach lets us pull context from our precomputed graph without having to process the entire dataset **Why did we choose this approach?** It offers several advantages: - **Efficiency:** By doing the heavy computational work in advance, we reduce query-time delays - **Best of Both Worlds:** We get the rich structure of knowledge graphs with the speed of semantic search - **Cost-Effective:** Unlike GraphRAG (which runs LLMs across the entire dataset) or LazyGraphRAG (which builds graphs on-the-fly), - **Potential Downsides**: The main downside is that we would aim to generate community summaries ahead of time while LazyGraphRAG pushes that off until query time. Drawing from our experience building search systems for e-commerce, we recognized that searching through all communities would be inefficient. Instead, using semantic search to quickly narrow down relevant information ensures we're only processing what matters for each query. ## Experimenting with Knowledge Graph Construction One interesting thing about working with knowledge graphs is how flexible they are. As one paper I read joked, they're almost completely unstructured—just about anything can be framed as a graph. Computer vision, for example, could be seen as a 2D graph of images organized by RGB channels. Language might be a 1D directional graph, where each word links to the next. If you look at things long enough, everything is a graph—but also nothing is a graph. For our GraphRAG pipeline, we're using LLMs for free-form attribute enrichment of data. For an e-commerce catalog, we might extract color, pattern, style, and sizing information about a product. For executive orders, we might look at relevant agencies, policy areas, and mentioned individuals. We've had success with both specific constructions of knowledge graphs per domain and more generic knowledge graphs across different domains. This gives us flexibility to create arbitrary knowledge graphs for different downstream use cases. ## Results and Observations Traditional RAG is good at answering specific queries but struggles to provide higher-level context. With our implementation of the GraphRAG pipeline, we're seeing improved performance on queries that require aggregation and trend analysis. Interestingly, in all these formulations, the actual graph structure isn't used after the community detection and summarization steps. That's largely fine for now, and it's one of the reasons why we've been experimenting with more generic knowledge graphs. They enable us to process an arbitrary corpus for an arbitrary agent who specializes in a given domain. ## Conclusion and Future Work Our experimentation with GraphRAG and LazyGraphRAG shows that integrating knowledge graphs into RAG pipelines can significantly enhance an LLM's ability to provide both specific and high-level, aggregated insights. By building the knowledge graph upfront—using community detection and summarization—and then relying on fast semantic search at query time, we get a system that delivers comprehensive answers without incurring the full cost of exhaustive LLM processing. This approach not only addresses the inherent challenges in traditional RAG systems but also offers a scalable, cost-effective solution for a wide range of use cases. Looking ahead, we see strong potential in expanding this approach: - **Enhanced Graph Formulations:** As we continue to refine our approach, we expect to develop even more sophisticated graph representations that capture not only the static relationships among entities but also their dynamic interactions over time. This will be key in domains where context rapidly evolves. - **Deep Integration of Graph Structures:** There is significant potential in leveraging Graph Neural Networks (GNNs) to more deeply integrate graph structures into the reasoning process of LLMs. Whether by directly incorporating GNN outputs or by drawing on the success of recommendation systems like PinSage, future systems may blend these graph-based insights to further improve accuracy and relevance. - **Automated Graph Updates and Real-Time Analysis:** Future implementations might focus on dynamically updating the knowledge graphs as new data becomes available, enabling real-time analysis and support a continuously evolving understanding of the underlying domain. - **Hybrid Reasoning and Explainability:** By combining the structured nature of graphs with the generative capabilities of LLMs, there is a promising opportunity to create systems that not only generate richer answers but also provide greater transparency—tracing back the reasoning to specific clusters or nodes in the graph, thereby boosting user trust and interpretability. In summary, our work represents an important step toward more intelligent, context-aware systems. We believe that combining structured graph representations with semantic search in RAG pipelines will enable systems to more accurately interpret and represent the complex, interconnected information that defines our world. ## Sitemap Parent: [Blog](https://2389.ai/blog/index.md) Related pages in this section: - [Horton Hears a Whisper](https://2389.ai/posts/horton-hears-a-whisper/index.md) - [Why We Built a Language for AI Pipelines](https://2389.ai/posts/why-we-built-a-language-for-ai-pipelines/index.md) - [Word Compiler, A Context Compiler for Long-Form Fiction](https://2389.ai/posts/word-compiler/index.md) - [We Turned a 3D Printer Into an AI Portrait Artist](https://2389.ai/posts/we-turned-a-3d-printer-into-an-ai-portrait-artist/index.md) - [Simmer: A Self Honing Skill](https://2389.ai/posts/simmer-skill/index.md) - [Cookoff: Same Spec, Different Code](https://2389.ai/posts/cookoff-same-spec-different-code/index.md) - [Omakase: Show Me](https://2389.ai/posts/omakase-show-me/index.md) - [Deliberation: Perspectives, Not Answers](https://2389.ai/posts/deliberation-perspectives-not-answers/index.md) - [The Dark Factory Is a .dot file](https://2389.ai/posts/the-dark-factory-is-a-dot-file/index.md) - [Week 0 Nvidia DGX Spark Experiments](https://2389.ai/posts/week-0-nvidia-dgx-spark-experiments/index.md) Site index: [llms.txt](https://2389.ai/llms.txt) · [sitemap.md](https://2389.ai/sitemap.md) · [HTML](https://2389.ai/posts/experimenting-with-rag/)