Customers require greater precision to deploy generative AI applications into production environments. In a world where decision-making is increasingly data-driven, the integrity and reliability of information is paramount. To address this, customers often integrate vector-based search systems with search augmentation and generation (RAG) architecture patterns that integrate dense embedding to ground AI output in relevant context. Start by improving the accuracy of your AI. If even higher precision and context fidelity are required, the solution evolves to a graph augmented RAG (GraphRAG). Graph Augmented RAG (GraphRAG) provides inference and relational modeling capabilities enhanced by graph structures.
AWS Partner Lettria has demonstrated that integrating graph-based structures into RAG workflows can improve answer accuracy by up to 35% compared to vector-only search methods. This enhancement is achieved by using graphs’ ability to model complex relationships and dependencies between data points, providing a more nuanced and contextually accurate foundation for generative AI output.
In this post, we explore why GraphRAG is more comprehensive and easier to explain than Vector RAG alone, and how you can use this approach using AWS services and Lettria.
How graphs make RAG more accurate
This section describes how to use graphs to make RAG more accurate.
Capturing complex human questions with graphs
Human questions are complex in nature and often require connecting multiple pieces of information. Traditional data representations struggle to accommodate this complexity without losing context. However, charts are designed to reflect the way humans naturally think and ask questions. They represent data in a machine-readable format that preserves rich relationships between entities.
By modeling your data as a graph, you can capture more context and intent. This means that RAG applications can access and interpret data in a way that closely matches human thought processes. The result is more accurate and relevant answers to complex queries.
Avoiding loss of context in data representation
Relying solely on vector similarity for information retrieval misses the subtle relationships that exist within the data. Converting natural language to vectors can reduce the richness of information and reduce the accuracy of answers. Additionally, end-user queries do not always semantically match useful information in the provided documents, and vector searches exclude key data points needed to construct accurate answers. It leads to
Graphs preserve the natural structure of your data, allowing for more accurate mapping between questions and answers. These enable RAG systems to understand and navigate complex connections within the data, leading to increased accuracy.
Lettria demonstrated that using GraphRAG within a hybrid approach increases answer accuracy from 50% over traditional RAG to more than 80%. Testing covers financial (Amazon financial reports), healthcare (scientific research on COVID-19 vaccines), industrial (technical specifications for aviation construction materials), and legal (European Union directives on environmental regulation) datasets It became.
Prove the graph is more accurate
To demonstrate the improved accuracy of graph-augmented RAGs, Lettria conducted a series of benchmarks comparing its GraphRAG solution (a hybrid RAG that uses both vector and graph stores) to a baseline vector-only RAG reference. did.
Lettria’s hybrid approach to RAG
Lettria’s hybrid approach to question answering combines the strengths of vector similarity and graph search to optimize the performance of RAG applications on complex documents. By integrating these two search systems, Lettria handles complex queries with both structured precision and semantic flexibility.
GraphRAG specializes in using fine-grained context data, making it ideal for answering questions that require explicit connections between entities. In contrast, vector RAG is better at capturing semantically related information and provides broader contextual insights. This dual system is further enhanced by a fallback mechanism. This means that if one system has difficulty providing relevant data, the other system will pick it up. For example, GraphRAG identifies explicit relationships when available, while vector RAG fills in relationship gaps or enhances context when structure is missing.
benchmark process
To demonstrate the value of this hybrid approach, Lettria conducted extensive benchmarking across various industry datasets. Using their solution, they compared GraphRAG’s hybrid pipeline to Weaviate’s Verba, a baseline RAG reference that relies solely on a vector store, a leading open source RAG package. The dataset includes Amazon financial reports, scientific literature on COVID-19 vaccines, aeronautics technical specifications, and European environmental directives, providing a diverse and representative test bed. Provide.
This evaluation focused on six different question types, including fact-based, multi-hop, numeric, tabular, temporal, and multi-constraint queries, to address real-world complexity. Questions ranged from simple fact-finding, such as identifying the formula for a vaccine, to multi-layered reasoning tasks, such as comparing revenue numbers over different time periods. An example of a multi-hop query in finance is “Compare the oldest booked Amazon revenue to the latest revenue.”
Lettria’s in-house team manually evaluated responses using a detailed rating grid and categorized results as correct, partially correct (acceptable or not), or incorrect. In this process, we measured how the hybrid GraphRAG approach outperformed the baseline, especially in processing multidimensional queries that require a combination of structured relationships and semantic breadth. By harnessing the best of both vector and graph-based search, Lettria’s system has demonstrated its ability to accurately and flexibly address the nuanced demands of a variety of industries.
Benchmark results
The results were significant and convincing. GraphRAG achieved a correct answer rate of 80%, compared to 50.83% for traditional RAG. When including acceptable answers, GraphRAG’s accuracy increased to nearly 90%, while the vector approach reached 67.5%.
The following graph shows the results for vector RAG and GraphRAG.
In the industrial domain with complex technical specifications, GraphRAG achieved an accuracy rate of 90.63%, almost double the 46.88% of vector RAG. These figures demonstrate that GraphRAG offers significant advantages over vector-only approaches, especially for clients focused on structuring complex data.
GraphRAG’s overall reliability and superior handling of complex queries allows customers to make more informed decisions with confidence. Significantly improve efficiency and reduce time spent sifting through unstructured data by providing up to 35% more accurate answers. These compelling results demonstrate that incorporating graphs into RAG workflows not only improves accuracy but is essential for tackling complex real-world questions.
Extended RAG application using AWS and Lettria
This section describes how to enable enhanced RAG applications using AWS and Lettria.
AWS: A robust foundation for generative AI
AWS provides a comprehensive suite of tools and services for building and deploying generative AI applications. With AWS, you have access to scalable infrastructure and advanced services such as Amazon Neptune, a fully managed graph database service. Neptune allows you to efficiently model and navigate complex relationships in your data, making it an ideal choice for implementing graph-based RAG systems.
Implementing GraphRAG from scratch typically involves a process similar to the following diagram.
This process can be categorized as:
- Large-scale language models (LLMs) identify entities and relationships in unstructured data based on domain definitions and store them in graph databases such as Neptune.
- At query time, user intent is translated into an efficient graph query based on the domain definition to retrieve related entities and relationships.
- The results are then used to enrich the prompts to produce more accurate responses compared to standard vector-based RAGs.
Implementing such processes requires teams to develop specific skills in topics such as graph modeling, graph queries, prompt engineering, and LLM workflow maintenance. AWS has released the open source GraphRAG Toolkit to make it easier for customers who want to build and customize GraphRAG workflows. It is expected that the extraction process and graph search will be iterated to improve accuracy.
Managed GraphRAG implementation
There are two solutions for managed GraphRAG with AWS. Lettria’s solution (soon to be available on AWS Marketplace), and Amazon Bedrock have GraphRAG support integrated with Neptune. Lettria provides an accessible way to integrate GraphRAG into your applications. By combining Lettria’s natural language processing (NLP) and graph technology expertise with a scalable, managed AWS infrastructure, you can develop RAG solutions that deliver more accurate and reliable results.
The main benefits of Lettria on AWS are:
- simple integration – Lettria’s solutions simplify the ingestion and processing of complex datasets
- Improved accuracy – Achieve up to 35% better performance on question answering tasks
- Scalability – Meet growing data volumes and user demands using scalable AWS services
- flexibility – A hybrid approach that combines the strengths of vector and graph representations.
In addition to Lettria’s solution, Amazon Bedrock introduced managed GraphRAG support on December 4, 2024, integrating directly with Neptune. GraphRAG with Neptune is built into the Amazon Bedrock Knowledge Base and provides an integrated experience with no additional setup or additional charges beyond the underlying services. GraphRAG is available in AWS Regions where both Amazon Bedrock Knowledge Bases and Amazon Neptune Analytics are available (see the current list of supported Regions). For more information, see Get Data and Generate AI Responses Using Amazon Bedrock Knowledge Bases.
conclusion
Data accuracy is a critical concern for companies implementing generative AI applications. Incorporating graphs into RAG workflows can significantly improve the accuracy of the system. Graphs capture the complexity of human queries and preserve context while providing a richer, more nuanced representation of data.
GraphRAG is an important option to consider for organizations looking to unlock the full potential of their data. By combining AWS and Lettria, you can build advanced RAG applications that meet the demanding needs of today’s data-driven enterprises and improve accuracy by up to 35%.
Explore how to implement GraphRAG for your generative AI applications on AWS.
About the author
Dennis Gosnell He is a Principal Product Manager at Amazon Neptune, focused on generative AI infrastructure and graph data applications that enable scalable, cutting-edge solutions across industries.
Vivian de Saint Pern is a startup solutions architect working with French AI/ML startups, with a focus on generative AI workloads.
