Skip to content
JavaAgentic

Type at least two characters. Try “RAG”, “pgvector” or “tool calling”.

The Spring AI ChatClient API

Master the Spring AI ChatClient: system messages, prompt templates, streaming with SSE, chat memory, advisors and per-call options — with complete Spring Boot code.

Beginner8 min readUpdated
On this page

ChatClient is the API you will touch most. It looks like a small fluent builder, and the parts people miss are exactly the parts that make the difference between a demo and a feature: templating, streaming, advisors and per-call options.

Key Takeaways

  • Build one ChatClient per use case from the injected ChatClient.Builder, with its own defaults.
  • Use .user(u -> u.text(...).param(...)) templating instead of string concatenation — it is clearer and it keeps user input out of the instruction text.
  • stream() returns a Flux; pair it with SSE for anything a human watches.
  • Advisors are the extension point. RAG, memory, logging and redaction are all advisors.
  • Prefer .entity(Type.class) over parsing text yourself.

Building the client

AssistantConfig.java
package com.javaagentic.demo.config;
 
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
 
@Configuration
public class AssistantConfig {
 
    /**
     * One ChatClient per role, not one per application. A support assistant and
     * a code reviewer want different system prompts, different temperatures and
     * different tools — sharing a single client forces you to re-specify all of
     * that at every call site.
     */
    @Bean
    ChatClient supportAssistant(ChatClient.Builder builder) {
        return builder
                .defaultSystem("""
                        You are a support assistant for the Acme billing product.
                        Answer only questions about Acme billing.
                        If a question is outside that scope, say so and suggest
                        contacting support directly.
                        Never invent account details, prices or policy.
                        """)
                .build();
    }
}

Three properties of that system prompt are worth copying: an explicit scope, an explicit out-of-scope behaviour, and an explicit prohibition. Vague system prompts produce assistants that confidently answer questions about anything.

Prompt templates

String concatenation works until a user pastes something with a brace in it, or writes "ignore your previous instructions". Use parameters.

public String summarise(String document, String audience) {
    return chatClient.prompt()
            .user(u -> u.text("""
                    Summarise the document below for {audience}.
                    Use at most five bullet points.
 
                    <document>
                    {document}
                    </document>
                    """)
                    .param("audience", audience)
                    .param("document", document))
            .call()
            .content();
}

Streaming responses

A ten-second wait for a complete answer feels broken. The same ten seconds with text appearing immediately feels fast. This is the highest-value change you can make to a chat UI.

StreamingChatController.java
package com.javaagentic.demo.chat;
 
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.http.MediaType;
import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RequestParam;
import org.springframework.web.bind.annotation.RestController;
import reactor.core.publisher.Flux;
 
@RestController
public class StreamingChatController {
 
    private final ChatClient chatClient;
 
    public StreamingChatController(ChatClient supportAssistant) {
        this.chatClient = supportAssistant;
    }
 
    @GetMapping(value = "/api/chat/stream", produces = MediaType.TEXT_EVENT_STREAM_VALUE)
    public Flux<String> stream(@RequestParam String question) {
        return chatClient.prompt()
                .user(question)
                .stream()
                .content()
                // A stalled provider must not hold the connection forever.
                .timeout(java.time.Duration.ofSeconds(60))
                .onErrorResume(error -> Flux.just("Sorry — something went wrong. Please try again."));
    }
}
Consuming it in the browser
const source = new EventSource(`/api/chat/stream?question=${encodeURIComponent(question)}`);
 
source.onmessage = (event) => {
  output.textContent += event.data;
};
 
// EventSource retries automatically on network errors, which is usually not
// what you want mid-generation — close it explicitly when the answer ends.
source.onerror = () => source.close();

Advisors

Advisors wrap the model call. They can modify the request before it goes out and the response on the way back. This is where RAG, memory and observability plug in.

Advisors form a bidirectional chain around the model call, ordered by getOrder().

Conversation memory

A model is stateless. Every call is the first call unless you send the history yourself.

ConversationalChatService.java
package com.javaagentic.demo.chat;
 
import org.springframework.ai.chat.client.ChatClient;
import org.springframework.ai.chat.client.advisor.MessageChatMemoryAdvisor;
import org.springframework.ai.chat.memory.ChatMemory;
import org.springframework.ai.chat.memory.InMemoryChatMemoryRepository;
import org.springframework.ai.chat.memory.MessageWindowChatMemory;
import org.springframework.stereotype.Service;
 
@Service
public class ConversationalChatService {
 
    private final ChatClient chatClient;
 
    public ConversationalChatService(ChatClient.Builder builder) {
        // A bounded window. Unbounded history grows the prompt on every turn
        // until the request exceeds the context limit — and you pay for the
        // whole history on every single call, not just the new message.
        ChatMemory memory = MessageWindowChatMemory.builder()
                .chatMemoryRepository(new InMemoryChatMemoryRepository())
                .maxMessages(20)
                .build();
 
        this.chatClient = builder
                .defaultAdvisors(MessageChatMemoryAdvisor.builder(memory).build())
                .build();
    }
 
    public String chat(String conversationId, String message) {
        return chatClient.prompt()
                .user(message)
                // The conversation ID keeps users' histories separate. Deriving
                // it from the authenticated principal — never from a request
                // parameter — is what stops one user reading another's context.
                .advisors(a -> a.param(ChatMemory.CONVERSATION_ID, conversationId))
                .call()
                .content();
    }
}

A custom advisor

Logging every prompt and response, with timing, is about fifteen lines:

LoggingAdvisor.java
package com.javaagentic.demo.advisor;
 
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
import org.springframework.ai.chat.client.ChatClientRequest;
import org.springframework.ai.chat.client.ChatClientResponse;
import org.springframework.ai.chat.client.advisor.api.CallAdvisor;
import org.springframework.ai.chat.client.advisor.api.CallAdvisorChain;
 
public class LoggingAdvisor implements CallAdvisor {
 
    private static final Logger log = LoggerFactory.getLogger(LoggingAdvisor.class);
 
    @Override
    public ChatClientResponse adviseCall(ChatClientRequest request, CallAdvisorChain chain) {
        long startedAt = System.currentTimeMillis();
        ChatClientResponse response = chain.nextCall(request);
        long elapsed = System.currentTimeMillis() - startedAt;
 
        // Log the length, not the content. Prompts routinely contain personal
        // data, and a log aggregator is a much wider audience than the model.
        log.info("model call completed in {}ms, prompt chars={}", elapsed,
                request.prompt().getContents().length());
 
        return response;
    }
 
    @Override
    public String getName() {
        return "logging";
    }
 
    @Override
    public int getOrder() {
        // Low order runs early on the way in and late on the way out, so this
        // measures the full chain including retrieval.
        return 0;
    }
}

Per-call options

Defaults belong on the builder; exceptions belong on the call.

String creative = chatClient.prompt()
        .user("Write three taglines for a Java AI course")
        .options(ChatOptions.builder()
                .temperature(0.9)      // variety is the point here
                .maxTokens(200)
                .build())
        .call()
        .content();

Typed responses

Structured extraction
public record Ticket(
        String summary,
        Priority priority,
        List<String> affectedComponents) {
 
    public enum Priority { LOW, MEDIUM, HIGH, CRITICAL }
}
 
public Ticket triage(String description) {
    return chatClient.prompt()
            .user(u -> u.text("Triage this bug report:\n\n{report}")
                    .param("report", description))
            .options(ChatOptions.builder().temperature(0.0).build())
            .call()
            .entity(Ticket.class);
}

Spring AI derives a JSON schema from the record, including the enum's permitted values, asks the model to conform, and deserialises the result. Temperature zero matters here — you want the same bug report to triage the same way every time. Full detail in structured output with Spring AI.

Five mistakes that show up in review

One shared ChatClient for the whole application. It starts as convenience and ends as a system prompt that tries to serve four unrelated features, contradicts itself, and cannot be changed without regression-testing all of them. One client per use case costs three lines each.

Concatenating user input into the instruction text. "Summarise this: " + userInput puts attacker-controlled text in the same position as your instructions. Templated parameters and explicit delimiters at least make the boundary visible to the model; concatenation removes it entirely.

Streaming everything. Streaming feels modern, and it forfeits your chance to inspect the response. Any output you would have validated — a schema, a moderation check, a business rule — must use call(). The decision is per feature; making it globally is how unvalidated model output reaches production.

Unbounded chat memory. The prompt grows on every turn, you pay for the entire history on every single call, and eventually a long conversation exceeds the context window and starts failing for one user in a way nobody can reproduce. Bound it from the first commit.

Default temperature for structured work. The default suits prose. Extraction, classification and tool calling want near-zero, because you want the same input to produce the same output — and because a test suite over a temperature-0.7 endpoint is a flaky test suite.

A checklist for ChatClient code

  • One client per use case, with its own system prompt and defaults
  • Templated parameters, never concatenation, for anything user-supplied
  • stream() for interactive UIs, call() when you must validate before responding
  • Bounded, persistent chat memory keyed by an authenticated principal
  • Temperature at or near zero for extraction, classification and tool calling
  • A timeout and an error path on every streaming endpoint
  • Log metadata, not prompt content

Next

Frequently Asked Questions

What is the difference between call() and stream() in Spring AI?
call() blocks until the full response is generated and returns it in one piece. stream() returns a Flux that emits chunks as the model produces them, so the user sees text appearing immediately. Use stream() for anything a person watches, and call() for background work, tool-driven flows and anything whose output you need to validate before acting on it.
How do I add conversation memory to Spring AI?
Attach a MessageChatMemoryAdvisor backed by a ChatMemory implementation and pass a conversation ID with each call. The advisor loads prior messages before the request and stores the exchange afterwards. Bound the window — unbounded history grows the prompt until it exceeds the context limit.
Are Spring AI advisors executed in a defined order?
Yes. Advisors form a chain ordered by their getOrder() value, lowest first, and each may modify the request before and the response after the model call. It is the same interceptor pattern as servlet filters.
How do I stream a Spring AI response to a browser?
Return the Flux from chatClient.prompt().stream().content() from a controller method producing text/event-stream. The browser consumes it with EventSource or a fetch ReadableStream. Server-Sent Events is simpler than WebSockets here because only the server pushes.

Related tutorials