GraphRAG & Knowledge Graphs

Vector RAG retrieves passages that mention things. It cannot answer "which of our suppliers depends on a company in a sanctioned country?" — because that answer is a path through relationships, not a passage of text. GraphRAG stores entities and their connections in a knowledge graph and retrieves over them, answering the relationship questions vector search cannot. This tutorial covers when and how, building on the RAG pipeline.

The gap GraphRAG fills#

Vector search finds passages by semantic similarity. Some questions have no answering passage — the answer only exists as a connection between facts scattered across documents:

• "Which products are affected if supplier X fails?" — a dependency path.
• "How is person A connected to company B?" — a relationship traversal.
• "What are the second-order effects of deprecating this API?" — a chain of dependencies.

No single chunk states these; they require following relationships. That is what a graph does.

Building the graph#

An LLM extracts entities and relationships from text; a graph database stores them:

record Entity(String name, String type) {}
record Relationship(String from, String type, String to) {}
record Extraction(List<Entity> entities, List<Relationship> relationships) {}
 
public void ingest(String document) {
    // The model extracts a graph from the text.
    // "Company A acquired Company B in 2023" →
    //   entities: A (Company), B (Company)
    //   relationship: A -[ACQUIRED]-> B
    Extraction extracted = extractor.extract(document);   // structured output
 
    // Validate before writing — a wrong relationship silently corrupts the graph.
    Extraction validated = validate(extracted);
    graphStore.merge(validated);   // dedupe entities, add relationships
}

Retrieving over the graph#

Graph retrieval traverses relationships to answer connection questions:

@Tool("""
        Find how two entities are connected. Returns the shortest relationship
        path between them, or NOT_CONNECTED.
        """)
public String findConnection(String entityA, String entityB) {
    // Parameterised Cypher — never concatenate model output into a query.
    var result = neo4j.query("""
            MATCH (a:Entity {name: $a}), (b:Entity {name: $b}),
                  path = shortestPath((a)-[*..5]-(b))
            RETURN path
            """)
            .bind(entityA).to("a")
            .bind(entityB).to("b")
            .fetch().all();
 
    return result.isEmpty() ? "NOT_CONNECTED" : describePath(result);
}

The FinAgentic project uses this pattern for questions about how companies, people and holdings connect.

Hybrid: graph plus vector#

The strongest systems use both, routing by question type:

public Answer answer(String question) {
    return switch (questionClassifier.classify(question)) {
        // Relationship questions traverse the graph.
        case RELATIONSHIP -> graphRag.answer(question);
        // Factual lookups retrieve passages.
        case FACTUAL -> vectorRag.answer(question);
        // Complex questions may need both.
        case COMPLEX -> hybridAnswer(question);
    };
}

This routing is a chain: classify the question, dispatch to the retrieval method that fits. Vector RAG for "what does the policy say", GraphRAG for "how are these connected".

Community summaries#

An advanced GraphRAG technique: cluster the graph into communities of related entities and pre-generate a summary of each. This lets the system answer broad, high-level questions ("what are the main themes across these documents?") that neither passage retrieval nor single-path traversal handles well — the summary of a community captures the gestalt.

Is it worth it?#

Be honest about the cost. Building a knowledge graph adds: entity extraction (imperfect, needs validation), deduplication (genuinely hard), a graph database to operate, and ongoing maintenance to keep it current. That is a lot.

Next#

• Small language models
• Agentic RAG advanced patterns
• FinAgentic project