Introduction to the Spring AI Framework

If you have shipped Spring Boot applications for years and are now being asked to "add AI", the hardest part is not the model. It is working out which of the twenty new nouns in every tutorial actually matter, and which are Python-ecosystem accidents that do not apply to you.

Spring AI exists to shrink that list. It gives you a small set of interfaces that behave like every other Spring abstraction you have used — a port with pluggable adapters, auto-configured from properties, testable with mocks, observable through Micrometer.

What Spring AI actually gives you#

Strip away the vocabulary and Spring AI provides five things:

Everything else in the framework is built from those pieces. Retrieval-augmented generation is an advisor that queries a VectorStore before the model call. Conversational memory is an advisor that prepends previous messages. Structured output is a converter applied to the response.

Architecture at a glance#

The important property of this diagram is the ChatModel port in the middle. Your service depends on ChatClient, ChatClient depends on ChatModel, and only the adapter knows it is talking to OpenAI. Swapping providers does not touch your business logic.

Your first call, end to end#

Dependencies#

Spring AI publishes a BOM, so you declare the version once and let it manage every module.

<dependencyManagement>
  <dependencies>
    <dependency>
      <groupId>org.springframework.ai</groupId>
      <artifactId>spring-ai-bom</artifactId>
      <version>${spring-ai.version}</version>
      <type>pom</type>
      <scope>import</scope>
    </dependency>
  </dependencies>
</dependencyManagement>
 
<dependencies>
  <dependency>
    <groupId>org.springframework.boot</groupId>
    <artifactId>spring-boot-starter-web</artifactId>
  </dependency>
  <dependency>
    <groupId>org.springframework.ai</groupId>
    <artifactId>spring-ai-starter-model-openai</artifactId>
  </dependency>
</dependencies>

Configuration#

spring:
  ai:
    openai:
      api-key: ${OPENAI_API_KEY}
      chat:
        options:
          model: gpt-4o-mini
          temperature: 0.2

Two things worth noticing immediately.

First, the API key comes from an environment variable. A key committed to Git is a key you will be rotating at 2am — treat it exactly like a database password.

Second, temperature: 0.2. The default is usually higher, which is right for creative writing and wrong for almost everything you will build first. Low temperature makes output more repeatable, which makes it testable.

The service#

package com.javaagentic.demo.chat;
 
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.stereotype.Service;
 
@Service
public class ChatService {
 
    private final ChatClient chatClient;
 
    // Spring AI auto-configures a ChatClient.Builder. Injecting the builder
    // rather than a finished ChatClient lets each service apply its own
    // defaults — a different system prompt, its own advisors, its own tools.
    public ChatService(ChatClient.Builder builder) {
        this.chatClient = builder
                .defaultSystem("""
                        You are a senior Java engineer.
                        Answer concisely and prefer code over prose.
                        If you are unsure, say so rather than guessing.
                        """)
                .build();
    }
 
    public String ask(String question) {
        return chatClient.prompt()
                .user(question)
                .call()
                .content();
    }
}

The controller#

package com.javaagentic.demo.chat;
 
import org.springframework.web.bind.annotation.PostMapping;
import org.springframework.web.bind.annotation.RequestBody;
import org.springframework.web.bind.annotation.RestController;
 
@RestController
public class ChatController {
 
    private final ChatService chatService;
 
    public ChatController(ChatService chatService) {
        this.chatService = chatService;
    }
 
    public record AskRequest(String question) {}
    public record AskResponse(String answer) {}
 
    @PostMapping("/api/ask")
    public AskResponse ask(@RequestBody AskRequest request) {
        return new AskResponse(chatService.ask(request.question()));
    }
}

That is a complete, working AI feature: two records, a service and a controller. No vector database, no agent framework, no Python.

The four abstractions in more detail#

ChatClient#

ChatClient is where you will spend most of your time. Its fluent API assembles a prompt, applies advisors, dispatches any tool calls the model requests, and converts the response.

// Blocking call, plain text back
String answer = chatClient.prompt().user(question).call().content();
 
// Streaming, for interactive UIs
Flux<String> tokens = chatClient.prompt().user(question).stream().content();
 
// Typed response, parsed straight into a record
record Summary(String headline, List<String> bulletPoints) {}
Summary summary = chatClient.prompt().user(document).call().entity(Summary.class);

The last form deserves attention. You did not write a parser, a schema or a retry loop. Spring AI derives a JSON schema from the record, instructs the model to conform to it, and deserialises the response. This is covered properly in structured output with Spring AI.

EmbeddingModel#

An embedding turns text into a vector whose position encodes meaning. Similar texts land near each other, which is what makes semantic search work.

@Service
public class SimilarityService {
 
    private final EmbeddingModel embeddingModel;
 
    public SimilarityService(EmbeddingModel embeddingModel) {
        this.embeddingModel = embeddingModel;
    }
 
    public float[] embed(String text) {
        return embeddingModel.embed(text);
    }
}

You rarely call this directly — VectorStore does it for you — but knowing it exists explains why the vector store needs a model and why the embedding model and store dimensions have to match.

VectorStore#

// Ingest
vectorStore.add(List.of(new Document("Spring AI supports many model providers.")));
 
// Retrieve
List<Document> hits = vectorStore.similaritySearch(
        SearchRequest.builder()
                .query("which providers does Spring AI support?")
                .topK(4)
                .similarityThreshold(0.7)
                .build());

The same two calls work against a simple in-memory store during development and against pgvector, Qdrant or Milvus in production. See Spring AI embeddings and vector stores.

Tool calling#

Tools are how a model stops being a text generator and starts being useful against your systems. You annotate a method; the model decides when to call it.

@Component
public class OrderTools {
 
    private final OrderRepository orders;
 
    public OrderTools(OrderRepository orders) {
        this.orders = orders;
    }
 
    @Tool(description = "Look up the current status of a customer order by its ID")
    public String orderStatus(String orderId) {
        return orders.findById(orderId)
                .map(order -> order.status().name())
                .orElse("NOT_FOUND");
    }
}

Full treatment in Spring AI function calling and @Tool.

Spring AI or LangChain4j?#

Both are good. The honest summary:

They are not mutually exclusive, and they are not a religious choice. A common production setup uses Spring AI for transport, configuration and metrics, and LangChain4j for its richer document loaders during ingestion. Start with whichever matches your application shape and revisit later — the models are behind interfaces either way. There is a fuller comparison in LangChain4j introduction and architecture.

What to get right before you ship#

Six things separate a demo from something you can leave running.

1. Timeouts. A model call can hang. Configure a client timeout and a circuit breaker. An unbounded LLM call in a request thread is an outage waiting for traffic.
2. Cost visibility. Token usage is on the response metadata. Record it as a metric per endpoint from day one, or your first invoice will be a surprise.
3. Low temperature for anything structured. Extraction, classification and tool calling want determinism, not creativity.
4. Validate the output. The model returns plausible text, not correct text. If you act on it, check it first.
5. Never interpolate untrusted input into a system prompt. That is prompt injection, and it is the top entry in the OWASP LLM Top 10.
6. Pin your versions. This ecosystem moves fast enough that a floating range will break a build you did not touch.

Where to go next#

• Setting up Spring AI with OpenAI — dependencies, keys, model options and the errors you will hit
• The Spring AI ChatClient API — templating, streaming, advisors
• Building a RAG pipeline with Spring Boot — answering from your own documents

External references worth bookmarking: the Spring AI reference documentation (opens in a new tab) and the Spring AI GitHub repository (opens in a new tab), which is often ahead of the docs.