Skip to content
JavaAgentic

Type at least two characters. Try “RAG”, “pgvector” or “tool calling”.

WebClient & HTTP Client Integration

Calling other services without taking yourself down: WebClient configuration, the four timeouts that matter, connection pool sizing, retry with backoff, and testing against a real socket.

Intermediate7 min readUpdated
On this page

Every outbound call is a way for another team's incident to become yours. A correctly configured HTTP client is the boundary that stops that happening, and almost all of it is timeouts.

Key Takeaways

  • Configure four timeouts: connect, response, read and write. Defaults are effectively infinite.
  • Size the connection pool deliberately and monitor pending acquisitions.
  • Retry only idempotent operations, with exponential backoff and jitter.
  • A circuit breaker stops you queueing requests for a service that is already down.
  • Test against a real socket — MockWebServer or WireMock — not a mocked client.

A client configured properly

PaymentClientConfig.java
@Configuration
public class PaymentClientConfig {
 
    @Bean
    public WebClient paymentWebClient(WebClient.Builder builder, PaymentProperties props) {
 
        ConnectionProvider pool = ConnectionProvider.builder("payments")
                .maxConnections(50)
                // Bounded: when the pool is exhausted, fail fast rather than
                // queueing an unbounded number of waiters.
                .pendingAcquireMaxCount(100)
                .pendingAcquireTimeout(Duration.ofSeconds(2))
                .maxIdleTime(Duration.ofSeconds(30))
                // Shorter than typical load-balancer idle timeouts, so we close
                // connections before the LB silently drops them.
                .maxLifeTime(Duration.ofMinutes(5))
                .evictInBackground(Duration.ofSeconds(60))
                .metrics(true)
                .build();
 
        HttpClient httpClient = HttpClient.create(pool)
                .option(ChannelOption.CONNECT_TIMEOUT_MILLIS, 2_000)
                .responseTimeout(Duration.ofSeconds(5))
                .doOnConnected(conn -> conn
                        .addHandlerLast(new ReadTimeoutHandler(5, TimeUnit.SECONDS))
                        .addHandlerLast(new WriteTimeoutHandler(5, TimeUnit.SECONDS)));
 
        return builder
                .baseUrl(props.baseUrl())
                .clientConnector(new ReactorClientHttpConnector(httpClient))
                .defaultHeader(HttpHeaders.ACCEPT, MediaType.APPLICATION_JSON_VALUE)
                .filter(correlationIdPropagation())
                .filter(logRequests())
                .build();
    }
 
    private ExchangeFilterFunction correlationIdPropagation() {
        return (request, next) -> next.exchange(
                ClientRequest.from(request)
                        .header("X-Correlation-Id", Objects.toString(MDC.get("correlationId"), ""))
                        .build());
    }
}

The four timeouts do different jobs and you need all of them. Connect bounds establishing the TCP connection. Response bounds the whole exchange and is the one that saves you. Read catches a server that sends headers then stalls mid-body. Write catches a peer that stops reading your request. A server that accepts the connection and then goes silent defeats a connect timeout entirely.

Making the call

PaymentClient.java
@Component
public class PaymentClient {
 
    private final WebClient client;
 
    public Receipt charge(ChargeRequest request) {
        return client.post()
                .uri("/v2/charges")
                .header("Idempotency-Key", request.idempotencyKey())
                .bodyValue(request)
                .retrieve()
                // Map status ranges to domain exceptions before the body is read,
                // so callers never see a raw WebClientResponseException.
                .onStatus(status -> status.value() == 422,
                        response -> response.bodyToMono(ProblemDetail.class)
                                .map(problem -> new PaymentDeclinedException(problem.getDetail())))
                .onStatus(HttpStatusCode::is5xxServerError,
                        response -> Mono.error(new PaymentUnavailableException()))
                .bodyToMono(Receipt.class)
                .timeout(Duration.ofSeconds(6))     // belt and braces above responseTimeout
                .block();
    }
}

For a blocking service, RestClient expresses the same thing without Reactor types:

RestClientVersion.java
Receipt receipt = restClient.post()
        .uri("/v2/charges")
        .body(request)
        .retrieve()
        .onStatus(status -> status.value() == 422, (req, res) -> { throw new PaymentDeclinedException(); })
        .body(Receipt.class);

Retry, correctly

Retry only when the operation is safe to repeat and the failure could plausibly go away.
RetryConfig.java
.retryWhen(Retry.backoff(3, Duration.ofMillis(200))
        .maxBackoff(Duration.ofSeconds(3))
        // Jitter stops every instance retrying in the same millisecond and
        // turning a brief blip into a synchronised thundering herd.
        .jitter(0.5)
        .filter(this::isTransient)
        .onRetryExhaustedThrow((spec, signal) -> signal.failure()))

Jitter matters more than people expect. Without it, a hundred instances that all failed at the same moment retry at the same moment, and the recovering downstream is knocked over again by the retry wave rather than by the original traffic.

Circuit breaking

Retries help with a blip and make an outage worse. When a downstream is genuinely down, every retry is another request queued against a service that cannot answer, consuming your connections and threads. A circuit breaker detects the sustained failure and fails fast:

CircuitBreakerClient.java
@CircuitBreaker(name = "payments", fallbackMethod = "chargeFallback")
@Retry(name = "payments")
public Receipt charge(ChargeRequest request) {
    return client.post().uri("/v2/charges").bodyValue(request)
                 .retrieve().bodyToMono(Receipt.class).block();
}
 
private Receipt chargeFallback(ChargeRequest request, CallNotPermittedException ex) {
    // The breaker is open: queue for later rather than failing the user's order.
    outbox.enqueue(request);
    return Receipt.pending(request.orderId());
}

Order matters here: retry inside the breaker means a burst of retries counts as multiple failures and opens the breaker faster, which is usually what you want. Note also that the fallback signature must match the original method plus the exception type.

Testing against a socket

PaymentClientTest.java
class PaymentClientTest {
 
    private MockWebServer server;
    private PaymentClient client;
 
    @BeforeEach
    void setUp() throws IOException {
        server = new MockWebServer();
        server.start();
        client = new PaymentClient(WebClient.builder()
                .baseUrl(server.url("/").toString()).build());
    }
 
    @Test
    void timesOutRatherThanHanging() {
        server.enqueue(new MockResponse()
                .setBody("{}")
                .setBodyDelay(10, TimeUnit.SECONDS));   // a stalled downstream
 
        assertThatThrownBy(() -> client.charge(request()))
                .isInstanceOf(WebClientRequestException.class);
    }
 
    @Test
    void translates422IntoDomainException() {
        server.enqueue(new MockResponse()
                .setResponseCode(422)
                .setHeader("Content-Type", "application/problem+json")
                .setBody("""
                    {"type":"/errors/card-declined","title":"Declined","status":422,
                     "detail":"insufficient funds"}
                    """));
 
        assertThatThrownBy(() -> client.charge(request()))
                .isInstanceOf(PaymentDeclinedException.class)
                .hasMessageContaining("insufficient funds");
    }
}

A mocked client can never prove a timeout works, because a mock has no notion of time on the wire. Testing against a real socket with an injected delay is the only way to know the configuration is actually applied — and configuration that is silently not applied is the most common failure here.

Sizing the pool

Connection pool size is one of the few numbers worth deriving rather than guessing. Little's Law gives you the shape: the number of concurrent connections you need equals your target throughput multiplied by the average call duration. Two hundred calls per second at an average of 50ms needs about ten concurrent connections; the same throughput at 500ms needs a hundred.

That relationship is why latency regressions downstream cause connection exhaustion upstream. Nothing about your traffic changed, but each call now occupies a connection ten times longer, so the pool that was comfortable is suddenly the bottleneck. This is the mechanism behind most "our service went down but our service was fine" incidents.

Two guards follow. Set pendingAcquireMaxCount so that when the pool is saturated, requests fail quickly rather than queueing without limit — an unbounded wait queue converts a downstream slowdown into an out-of-memory error. And set pendingAcquireTimeout low, because a request that has already waited two seconds for a connection has probably exceeded whatever deadline the original caller had.

The other setting that quietly matters is maxLifeTime. Load balancers and reverse proxies close idle connections on their own schedule, and a connection closed by the peer between your reuse and your write produces a confusing intermittent failure. Setting a maximum lifetime shorter than the infrastructure's idle timeout means you always close first.

Bulkheads

A circuit breaker protects you from a downstream that is failing. A bulkhead protects you from one that is merely slow — which is more common and more damaging, because slow calls do not trip a breaker but do occupy resources.

The idea is to cap how much of your capacity any single dependency can consume. If the payment service is allowed at most twenty concurrent calls, then a payment slowdown degrades checkout while leaving search, browsing and account pages entirely unaffected. Without the cap, every request thread eventually ends up blocked on payments and the whole application stops responding.

Resilience4j offers two implementations. The semaphore bulkhead limits concurrent calls in the calling thread and is nearly free. The thread-pool bulkhead runs calls on a separate bounded pool, which isolates more strongly at the cost of a context switch and losing thread-local context such as the MDC. Start with the semaphore version; reach for the pool only when you need genuine thread isolation.

Observability

Register the client with Micrometer so you get request count, latency percentiles and status distribution per downstream. Two graphs pay for themselves during the first incident: p99 latency per downstream, which tells you whose slowness you are inheriting, and pending connection acquisitions, which tells you the pool is too small before it starts timing out.

Propagate the correlation id and trace context on every outbound call, as the filter above does. Without it, a trace stops at your service boundary and the distributed part of distributed tracing does not happen.

What to take away

Set all four timeouts on every client. Bound the connection pool and its wait queue. Retry only idempotent calls, with jitter, behind a circuit breaker. Translate downstream failures into your own exceptions at the client boundary, and test the whole thing against a real socket.

Frequently Asked Questions

WebClient or the new RestClient?
RestClient for blocking code — it has WebClient fluent API with synchronous semantics and no Reactor types leaking into your service. WebClient when you are genuinely reactive or need streaming. Both share the same underlying infrastructure, so timeouts and connection pooling are configured the same way.
What is the single most important setting?
A response timeout. Without one, a downstream that accepts your connection and then goes silent holds your thread or connection until the OS gives up, which can be minutes. Every other resilience mechanism assumes calls eventually terminate.
Should I retry every failed call?
No. Retry only idempotent operations and only on transient failures — connection errors, 502, 503, 504, and timeouts you are sure did not reach the server. Retrying a non-idempotent POST after a timeout can charge a customer twice. Use an idempotency key if you must retry writes.

Related tutorials