Skip to content
JavaAgentic

Type at least two characters. Try “RAG”, “pgvector” or “tool calling”.

LangChain4j Retrievers & RAG

Build RAG in LangChain4j with ContentRetriever: attach retrieval to AI Services, transform queries, re-rank results, and assemble an advanced RAG pipeline with the RetrievalAugmentor.

Advanced3 min readUpdated
On this page

LangChain4j makes basic RAG a one-line attachment and advanced RAG a composable pipeline. This tutorial covers both: the ContentRetriever that turns any AI Service into a RAG system, and the RetrievalAugmentor that adds query transformation and re-ranking when basic retrieval is not enough.

Key Takeaways

  • A ContentRetriever attached to an AI Service gives you transparent RAG — no signature change.
  • The RetrievalAugmentor is the full pipeline: transform query → retrieve → re-rank → inject.
  • Query transformation fixes the "user wording ≠ good search query" problem.
  • Re-ranking is usually the biggest quality jump once basic RAG works.

Basic RAG in one attachment

RAG via ContentRetriever
interface Assistant {
    String answer(String question);
}
 
Assistant assistant = AiServices.builder(Assistant.class)
        .chatModel(chatModel)
        .contentRetriever(EmbeddingStoreContentRetriever.builder()
                .embeddingStore(embeddingStore)
                .embeddingModel(embeddingModel)
                .maxResults(5)
                .minScore(0.6)      // drop weak matches
                .build())
        .build();
 
// The retriever runs before every call, injecting relevant passages. The
// method signature never mentions retrieval.
String answer = assistant.answer("What is the refund policy?");

That is a complete RAG system. For most applications with clean, well-chunked content, it is enough.

Dynamic filtering per request

Restrict retrieval per call — essential for multi-tenant systems:

EmbeddingStoreContentRetriever.builder()
        .embeddingStore(embeddingStore)
        .embeddingModel(embeddingModel)
        .maxResults(5)
        // Filter derived from the request context (the authenticated tenant),
        // applied to every retrieval.
        .dynamicFilter(query -> metadataKey("tenantId")
                .isEqualTo(currentTenantId()))
        .build();

Advanced RAG with the RetrievalAugmentor

When basic retrieval underperforms, the RetrievalAugmentor lets you add stages:

An advanced RAG pipeline: transform the query, retrieve, re-rank, then inject.
A RetrievalAugmentor pipeline
RetrievalAugmentor augmentor = DefaultRetrievalAugmentor.builder()
        // 1. Rewrite the question into a better search query.
        .queryTransformer(new CompressingQueryTransformer(chatModel))
        // 2. Retrieve candidates.
        .contentRetriever(EmbeddingStoreContentRetriever.builder()
                .embeddingStore(embeddingStore)
                .embeddingModel(embeddingModel)
                .maxResults(20)   // retrieve more, re-rank down
                .build())
        // 3. Re-rank to the best few.
        .contentAggregator(new ReRankingContentAggregator(scoringModel, 5))
        .build();
 
Assistant assistant = AiServices.builder(Assistant.class)
        .chatModel(chatModel)
        .retrievalAugmentor(augmentor)
        .build();

Query transformation

The user's exact words are often a poor search query — too short, conversational, or missing terms that appear in your documents. Transforming the query first can lift retrieval noticeably:

// Compress a multi-turn conversation into a standalone query, so follow-up
// questions like "what about the annual plan?" become searchable.
QueryTransformer transformer = new CompressingQueryTransformer(chatModel);

For questions that span topics, an expanding transformer splits them into sub-queries and retrieves for each, covering more ground.

Re-ranking: the big quality lever

Vector similarity is fast but coarse. A re-ranker scores each candidate against the query with a cross-encoder, which is far more accurate, and reorders them:

ContentAggregator reRanker = ReRankingContentAggregator.builder()
        .scoringModel(scoringModel)   // e.g. a Cohere rerank model
        .maxResults(5)                // keep the best 5 of the 20 retrieved
        .build();

Debugging retrieval

When answers are wrong, the cause is nearly always retrieval, not generation. Log what was retrieved:

ContentRetriever logging = query -> {
    List<Content> results = delegate.retrieve(query);
    log.info("query='{}' retrieved={} topScore={}",
            query.text(), results.size(),
            results.isEmpty() ? "none" : results.get(0));
    return results;
};

If the correct passage is not in the retrieved set, fix chunking, thresholds or the embedding model — no prompt change will recover it. If it is present and the answer still ignored it, the problem is the prompt. This diagnostic split is the core skill of RAG debugging, covered in building a RAG pipeline.

Next

Frequently Asked Questions

How do I add RAG to a LangChain4j AI Service?
Attach a ContentRetriever when building the AI Service with .contentRetriever(). The retriever runs before every call, fetches relevant passages from your embedding store, and injects them into the prompt automatically. Your interface method signature does not change — retrieval happens transparently.
What is the difference between a ContentRetriever and a RetrievalAugmentor?
A ContentRetriever fetches relevant content for a query — the basic building block. A RetrievalAugmentor is the full pipeline that can transform the query first, route to multiple retrievers, and re-rank or compress results before injecting them. Use a plain retriever for simple RAG and a RetrievalAugmentor when you need advanced steps.
What is query transformation and why does it help?
Query transformation rewrites the user's question before retrieval — expanding it, rephrasing it, or splitting it into sub-questions. It helps because the user's exact wording is often not the best search query. Rewriting can turn a vague or conversational question into one that matches your source documents more closely.
When should I add a re-ranker?
When retrieval returns roughly relevant results but the best passage is often not ranked first. A re-ranker uses a cross-encoder model to score each candidate against the query directly, which is more accurate than vector similarity, and reorders them. It is usually the single biggest quality improvement once basic RAG works.

Related tutorials