From Hallucinations to Accuracy: Enhancing Generative Models with Graph RAG

Introduction 

The generative capabilities of LLMs are amazing and provide reasonable answers to user queries. But do those answers always make sense, in terms of truthfulness of the information generated? Is the LLM always right? These are some questions you must be asking yourself while playing with LLMs. While most of the generation might sound realistic, the reality is it might not be.

For example: 

‍

‍

Which is not true! 

‍

While enabling LLM-centered projects, it is crucial to understand their limitations. At the end of the day an LLM is a probabilistic model that is really good (or at least assumed to be) in predicting the next word given some context. And given its large context space (ability to hold information), an LLM can do computations to generate words relevant to that context.

Why do hallucinations happen? 

The operations of LLMs are rooted in probabilistic modeling, which is conceptually linked to Bayes' theorem in mathematics, where the odds of future events happening are calculated given certain conditions. In LLM terms, this translates to predicting the next word based on a given context. Underlying this probabilistic modeling is the neural network architecture of the model, which learns complex patterns and relationships in the data through layers of interconnected neurons. By scaling these concepts, LLMs can generate entire sentences and even predict subsequent ones.

Interestingly, the next word predicted by an LLM can influence the flow of all subsequent generations, making the process dynamic and non-monotonous. As a result, there could be multiple valid continuations for a given context, but the model selects the one deemed most probable. However, despite the seemingly precise mathematics, there is always room for bias. This bias or misrepresentation of patterns when scaled up can lead to hallucinations, where the model produces outputs that deviate from reality. 

Hallucination, in this sense, is not purely a mathematical error but a manifestation of the following factors:

1. Model architecture limitations
2. Biases present in training data
3. Lack of real-world grounding
4. Difficulty in maintaining accuracy over long contexts

It's important to note that these biases don't negate the computational and contextual strengths of LLMs, but do highlight areas for improvement.

From hallucination to accuracy 

To tackle the issue of hallucinations in LLMs, several innovative solutions have been proposed, including fine-tuning, zero-shot learning, and various forms of Retrieval Augmented Generation (RAG). However, each of these approaches comes with trade-offs related to cost, dataset availability, context size, and processing speed. Among these options, RAG techniques are particularly appealing as they offer minimal trade-offs while being data-driven.

RAG provides data-driven context to LLMs, ensuring that the information remains relevant and helping the models generate more accurate and coherent outputs. By leveraging RAG, LLMs can enhance their performance while mitigating the risks associated with hallucination.

‍

                            An example of a simple RAG 

Here the user query triggers relevant context which is used by the LLM to make decisions. Let's look at this with a simple example.  

Assume you are a business owner looking to get some information on international trade laws. When posing a specific legal question about international trade to an LLM, there's a significant risk of receiving an irrelevant or inaccurate response, regardless of how convincing it may sound. But if relevant documents were provided to this LLM, the answer to the user query could be deducted.  This approach exposes the model to more directly applicable information, which it can use as a foundation for its response.

However It's important to note that this method isn't foolproof. There's no guarantee that the vector search will retrieve all of the relevant information as it is dependent on the embedding model, on the ranking of data, and even on the structure of the supplied data. Additionally, the context size might be insufficient to handle all information. 

But conceptually, utilizing RAG techniques is likely to yield more dependable and relevant results compared to relying solely on the LLM's pre-trained knowledge. 

GraphRAG 

We've established that context helps reduce hallucinations in LLMs. The next logical question is: How can we provide even more relevant context? This leads to the use of a more sophisticated data structure called Graphs. Coupled with RAG, this is called GraphRAG.

In a graph structure:

• Nodes represent key subjects or entities 
• Edges represent relationships between these points

These edges are crucial because they provide meaning and context beyond simple data points. By mapping relationships explicitly, GraphRAG allows for a more distinct understanding of complex information. Let's look once again at an example.

Consider the following context which I want to be RAG-enabled: 

‍

It is written in a complicated structure, encapsulating several pieces of information. Such a supply of information is prone to hallucinations. But what if I made this piece of information more structured and graph-enabled? 

As we can see, the Graph approach declutters a lot of information. Here, we've almost reduced the context size by 66% while maintaining the core information. Retrieval of this structure is called Graph Retrieval. The graph objects such as nodes and edges can also carry properties as attributes to make this even more interesting. 

             Example of Relationship properties. 

‍

Furthermore, two or more distinct pieces of information could be studied and analysed by the LLM using the properties of graphs. This is a bit like traversing from one point to another, allowing you to study more complex correlations, causes, and effects, providing better, richer generated content. 

So what steps are done differently here when compared to simple RAG?

‍
Two steps are important: 

1. Constructing graphs from your documents 
2. Graph retrieval

Lettria's tools can help you seamlessly enable these steps and develop a GraphRAG solution.

Benefits of choosing Graph RAG 

‍

1. MultiHop: In a simple RAG, two or more pieces of information may be related but could be in any section of the context. Chunking and ranking may not capture this information. Graph systems capture this information and connect the dots, providing chained reasoning. 

2. Smaller context size: Graphs hold more relevant information in their structure, removing all the unessential clutter.

3. Domain Knowledge Integration: Graphs can be tailored with domain-specific ontologies and taxonomies, improving relevance for niche queries.

All these benefits directly contribute to generating accurate and acceptable answers.

Conclusion

Moving from Vanilla LLMs to GraphRAG represents a significant leap forward in our quest for accurate, trustworthy AI systems. Say goodbye to wild hallucinations and hello to AI you can actually trust - no more wondering if your virtual assistant is moonlighting as a fiction writer.