Skip to content
JavaAgentic

Type at least two characters. Try “RAG”, “pgvector” or “tool calling”.

LangChain4j Embedding Models

Configure embedding models in LangChain4j: hosted models like OpenAI and Cohere, free in-process ONNX models, dimension matching, and choosing an embedding model for RAG.

Intermediate3 min readUpdated
On this page

The embedding model turns text into the vectors that make semantic search possible. LangChain4j supports both hosted models and free in-process ones, and the choice affects retrieval quality, cost and privacy. This tutorial covers configuration and the dimension discipline that prevents painful migrations.

Key Takeaways

  • In-process ONNX models (all-MiniLM) are free, fast and offline — an excellent default for development.
  • Hosted models (OpenAI, Cohere) give higher retrieval quality on nuanced queries.
  • Dimensions are a contract — changing embedding model means re-creating the store and re-embedding.
  • Embeddings are cheap; quality and re-ingest time drive the decision, not cost.

In-process models: the easy default

An in-process model runs entirely inside the JVM. No API key, no network, no per-call cost:

all-MiniLM-L6-v2, in-process
// Add the langchain4j-embeddings-all-minilm-l6-v2 module.
EmbeddingModel embeddingModel = new AllMiniLmL6V2EmbeddingModel();
 
Embedding vector = embeddingModel.embed("How do I reset my password?").content();
// 384 dimensions, computed locally in a few milliseconds.

This is the right default for development — it removes cost and latency from your ingestion and test loops entirely, so iterating on chunking is fast.

Hosted models: higher quality

OpenAI embeddings
EmbeddingModel openai = OpenAiEmbeddingModel.builder()
        .apiKey(System.getenv("OPENAI_API_KEY"))
        .modelName("text-embedding-3-small")   // 1536 dimensions
        .build();
Cohere embeddings
EmbeddingModel cohere = CohereEmbeddingModel.builder()
        .apiKey(System.getenv("COHERE_API_KEY"))
        .modelName("embed-english-v3.0")
        .build();

Hosted models generally retrieve better on subtle, paraphrased queries, at the cost of a network call and a small per-token charge.

Dimensions: the contract you cannot break casually

// text-embedding-3-small → 1536 dimensions
// text-embedding-3-large → 3072 dimensions
// all-MiniLM-L6-v2       → 384 dimensions

Your vector store is created with a fixed dimension. The embedding model's output must match it, and must keep matching.

Embedding in the ingestion pipeline

Usually you do not call embed() directly — the ingestor does it for you:

Ingestion with embeddings
EmbeddingStoreIngestor.builder()
        .documentSplitter(DocumentSplitters.recursive(500, 100))
        .embeddingModel(embeddingModel)    // embeds each chunk
        .embeddingStore(embeddingStore)
        .build()
        .ingest(documents);

The same embeddingModel must be used for queries as for ingestion — you cannot embed documents with one model and queries with another, because the vectors would live in different spaces.

Batch embedding for efficiency

When embedding many chunks, batch them — one API call for many texts is far faster than one call per text:

List<TextSegment> segments = splitter.split(document);
// One batched call rather than N individual calls.
List<Embedding> vectors = embeddingModel.embedAll(segments).content();

Choosing an embedding model

NeedChoice
Development / fast iterationIn-process all-MiniLM (free, offline)
Privacy-sensitive / air-gappedIn-process model
General production qualityHosted, e.g. text-embedding-3-small
Highest retrieval qualityLarger hosted model, or add re-ranking
Multilingual contentA multilingual model (Cohere multilingual, etc.)

Measuring embedding quality

Embedding quality is not abstract — measure it as retrieval hit rate on a real question set. If two models are candidates, ingest with each and compare which retrieves the correct passage more often. The better retriever is the better embedding model for your content, regardless of benchmark rankings. See embedding models and semantic search.

Next

Frequently Asked Questions

What embedding model should I use with LangChain4j?
For development and many production cases, an in-process ONNX model like all-MiniLM-L6-v2 is excellent — free, fast and needs no external service. For higher retrieval quality, a hosted model such as OpenAI text-embedding-3-small or a Cohere model performs better on nuanced queries. Match the choice to your quality needs and privacy constraints.
Can I run embedding models locally in Java?
Yes. LangChain4j ships in-process embedding models that run entirely inside the JVM via ONNX Runtime — all-MiniLM-L6-v2 is the common default. They need no API key, no network and no cost per call, which makes them ideal for development and for privacy-sensitive workloads.
Why do embedding dimensions matter?
Each embedding model outputs a fixed number of dimensions, and your vector store column is created with a fixed width. They must match. Changing embedding model to one with a different dimension means re-creating the store and re-embedding every document — it is a migration, not a config tweak. Choose deliberately.
Are embeddings expensive?
Far less than generation. Hosted embedding models are priced per token at a small fraction of chat model rates, and you embed each document once. In-process models are free after the initial download. Cost is rarely the constraint with embeddings; retrieval quality and re-ingest time are the real considerations.

Related tutorials