Skip to content
JavaAgentic

Type at least two characters. Try “RAG”, “pgvector” or “tool calling”.

LangChain4j Embedding Stores

Store and search vectors in LangChain4j: the in-memory store for tests, PgVector for production, Redis and Elasticsearch, plus metadata filtering and picking the right store.

Intermediate3 min readUpdated
On this page

The embedding store persists your vectors and answers similarity queries. LangChain4j abstracts over many stores behind one interface, so you can develop against an in-memory store and deploy against PgVector without changing your retrieval code. This tutorial covers the common stores and the metadata filtering that makes multi-tenant retrieval safe.

Key Takeaways

  • InMemoryEmbeddingStore for tests; PgVector as the pragmatic production default.
  • The store interface is uniform — swap implementations without touching retrieval code.
  • Metadata filtering enforces tenant isolation; never rely on similarity alone for that.
  • Adopt a dedicated vector database only when you have a concrete reason.

In-memory: for tests and prototypes

InMemoryEmbeddingStore
EmbeddingStore<TextSegment> store = new InMemoryEmbeddingStore<>();
 
// Add and search — the same API every store implements.
store.add(embedding, segment);
EmbeddingSearchResult<TextSegment> result = store.search(EmbeddingSearchRequest.builder()
        .queryEmbedding(queryVector)
        .maxResults(5)
        .minScore(0.7)
        .build());

Fast, zero setup, and lost on restart — exactly right for unit tests and prototypes, wrong for production.

PgVector: the production default

If you already run PostgreSQL, PgVector removes an entire piece of infrastructure:

PgVector store
EmbeddingStore<TextSegment> store = PgVectorEmbeddingStore.builder()
        .host("localhost")
        .port(5432)
        .database("app")
        .user("app")
        .password(System.getenv("DB_PASSWORD"))
        .table("embeddings")
        .dimension(384)   // MUST match your embedding model's output
        .build();

Other stores

Redis
EmbeddingStore<TextSegment> redis = RedisEmbeddingStore.builder()
        .host("localhost").port(6379).dimension(384).build();
Elasticsearch (for hybrid keyword + vector search)
EmbeddingStore<TextSegment> es = ElasticsearchEmbeddingStore.builder()
        .serverUrl("http://localhost:9200").dimension(384).build();

Qdrant, Milvus, Weaviate, Chroma, MongoDB Atlas, Neo4j and others each have a module. They all implement the same EmbeddingStore interface, so your retrieval code is identical across them.

Metadata filtering

Filtering restricts the search before ranking — essential for multi-tenant systems and for narrowing by document version or date:

Filtered search
import static dev.langchain4j.store.embedding.filter.MetadataFilterBuilder.metadataKey;
 
var result = store.search(EmbeddingSearchRequest.builder()
        .queryEmbedding(queryVector)
        .maxResults(5)
        .minScore(0.7)
        // Tenant isolation belongs in the filter, derived from the
        // authenticated principal — never from a request parameter.
        .filter(metadataKey("tenantId").isEqualTo(currentTenantId()))
        .build());

Filters compose:

var filter = metadataKey("tenantId").isEqualTo(tenantId)
        .and(metadataKey("year").isGreaterThanOrEqualTo(2024));

Choosing a store

StoreReach for it whenWatch out for
InMemoryTests, prototypesNo persistence, no scale
PgVectorYou already run PostgresIndex build time on huge tables
RedisYou run Redis, want low latencyMemory cost at scale
ElasticsearchYou need hybrid keyword + vectorHeavier to operate
Qdrant / MilvusHundreds of millions of vectorsReal distributed-systems overhead

The honest advice, same as for Spring AI: start with PgVector. Most teams that adopted a dedicated vector database early would have been fine without one and carry the operational cost anyway.

Making ingestion idempotent

Re-ingesting must replace, not duplicate:

// Remove a source's existing vectors before re-adding, keyed on metadata.
store.removeAll(metadataKey("sourceId").isEqualTo(sourceId));
ingestor.ingest(freshDocuments);

Next

Frequently Asked Questions

Which embedding store should I use in LangChain4j?
Use InMemoryEmbeddingStore for tests and prototypes. For production, PgVector is the pragmatic default if you already run PostgreSQL — it keeps vectors beside your relational data with one operational story. Move to a dedicated store like Qdrant, Redis or Elasticsearch when you outgrow that, for example with very large corpora or a need for hybrid search.
How do I filter vector search by metadata in LangChain4j?
Build a metadata filter and pass it in the search request. LangChain4j's filter API supports equality, comparison and boolean combinations against the metadata you attached at ingestion — so you can restrict a search to a tenant, a document version or a date range. This is the correct way to enforce tenant isolation.
Is the in-memory store usable in production?
No. InMemoryEmbeddingStore holds everything in the JVM heap, is lost on restart, and is not shared across instances. It is perfect for unit tests and small prototypes and wrong for anything that must persist or scale. Move to a persistent store before production.
Do I need a dedicated vector database?
Usually not at first. PgVector handles millions of vectors on ordinary hardware, and keeping vectors in PostgreSQL means one backup, one transaction boundary and one runbook. Adopt a dedicated store when you have a concrete reason — scale, query rate, or features like hybrid search that Postgres does not offer.

Related tutorials