Kubernetes for Java Microservices

Kubernetes will happily run a Spring Boot container with default settings, and it will also restart it during outages, kill it for using memory you told it to use, and drop requests on every deploy. Each of those is a configuration problem with a known fix.

Probes#

spec:
  containers:
    - name: order-service
      # Guards a slow start without forcing a long liveness period. Until this
      # passes, liveness and readiness are not evaluated at all.
      startupProbe:
        httpGet: { path: /actuator/health/readiness, port: 8081 }
        failureThreshold: 30
        periodSeconds: 5          # allows up to 150s to boot
 
      livenessProbe:
        httpGet: { path: /actuator/health/liveness, port: 8081 }
        periodSeconds: 10
        failureThreshold: 3
        timeoutSeconds: 2
 
      readinessProbe:
        httpGet: { path: /actuator/health/readiness, port: 8081 }
        periodSeconds: 5
        failureThreshold: 2
        timeoutSeconds: 2

management:
  server:
    port: 8081                     # not routed by the ingress
  endpoint:
    health:
      probes:
        enabled: true
      group:
        liveness:
          include: livenessState
        readiness:
          include: 'readinessState,db,redis'

Note what is in each group. liveness contains only Spring's internal state — is the application context broken. readiness includes the database and Redis, because a pod that cannot reach them should stop receiving traffic while staying alive to recover.

Getting this backwards produces a specific and memorable incident: the database has a brief blip, every pod fails liveness simultaneously, Kubernetes restarts all of them, and now you have a cold JVM fleet hammering a recovering database.

Memory#

The JVM historically read the host's memory, not the container's limit, and would happily size a heap larger than its cgroup allowed. Java 10+ fixed the detection, but the defaults still need help.

FROM eclipse-temurin:21-jre-alpine
WORKDIR /app
COPY --from=build /app/target/*.jar app.jar
USER 1000
ENTRYPOINT ["java", \
  "-XX:MaxRAMPercentage=75.0", \
  "-XX:InitialRAMPercentage=50.0", \
  "-XX:+UseG1GC", \
  "-XX:MaxGCPauseMillis=200", \
  "-XX:+ExitOnOutOfMemoryError", \
  "-jar", "/app/app.jar"]

resources:
  requests:
    memory: 768Mi
    cpu: 250m
  limits:
    memory: 1Gi
    cpu: 1000m

The percentage matters because the container limit covers far more than the heap. Metaspace, thread stacks (1MB each by default), direct byte buffers, the JIT code cache, GC structures and the JVM itself all live outside it. With a 1Gi limit, a 75% heap leaves 256Mi for everything else — roughly right for a typical service. Setting -Xmx1g against a 1Gi limit guarantees an eventual OOMKill, and the JVM will not have logged an OutOfMemoryError because it never exceeded its heap.

-XX:+ExitOnOutOfMemoryError is worth adding. Without it, a heap exhaustion leaves the JVM alive but useless, failing every request while liveness — which only checks that the context is up — keeps passing.

Requests versus limits: requests are what the scheduler reserves and what guarantees you get; limits are the ceiling. Exceeding a memory limit means immediate kill; exceeding a CPU limit means throttling, which shows up as latency rather than failure. Many teams deliberately omit the CPU limit and set only a request, because CPU throttling on a JVM during garbage collection produces very confusing latency spikes.

Zero-downtime rollouts#

spec:
  strategy:
    type: RollingUpdate
    rollingUpdate:
      maxSurge: 1
      maxUnavailable: 0          # never dip below the desired replica count
  minReadySeconds: 10
  template:
    spec:
      terminationGracePeriodSeconds: 60
      containers:
        - name: order-service
          lifecycle:
            preStop:
              exec:
                command: ["sh", "-c", "sleep 10"]

server:
  shutdown: graceful
spring:
  lifecycle:
    timeout-per-shutdown-phase: 30s

The preStop sleep looks like a hack and is the standard fix. Kubernetes sends SIGTERM and removes the endpoint at the same time, but endpoint removal propagates asynchronously through kube-proxy and the ingress. Without the delay, a pod that shuts down immediately drops requests that were routed to it microseconds earlier.

Add a PodDisruptionBudget so voluntary disruptions — node drains, cluster upgrades — cannot take every replica at once:

apiVersion: policy/v1
kind: PodDisruptionBudget
spec:
  minAvailable: 2
  selector:
    matchLabels: { app: order-service }

Autoscaling#

apiVersion: autoscaling/v2
kind: HorizontalPodAutoscaler
spec:
  scaleTargetRef: { apiVersion: apps/v1, kind: Deployment, name: order-service }
  minReplicas: 3
  maxReplicas: 20
  metrics:
    - type: Resource
      resource: { name: cpu, target: { type: Utilization, averageUtilization: 70 } }
    # For I/O-bound services CPU barely moves under load. A request-rate or
    # queue-depth metric from Prometheus reflects actual pressure.
    - type: Pods
      pods:
        metric: { name: http_server_requests_seconds_count }
        target: { type: AverageValue, averageValue: "100" }
  behavior:
    scaleUp:
      stabilizationWindowSeconds: 60
    scaleDown:
      # Slow down. Aggressive scale-down plus a traffic wobble produces
      # thrashing, and each new pod pays JVM warm-up again.
      stabilizationWindowSeconds: 300
      policies:
        - type: Percent
          value: 25
          periodSeconds: 60

CPU-based autoscaling is a poor fit for a typical Spring Boot service, because most of them spend their time waiting on a database or another service. CPU stays low while latency climbs, so the HPA does nothing until the situation is already bad. Scale on request rate, queue depth or p95 latency instead — whatever actually correlates with the pressure you feel.

Configuration and secrets#

Mount ConfigMap values as environment variables, and remember relaxed binding makes SPRING_DATASOURCE_URL bind to spring.datasource.url without any application change.

Kubernetes Secret objects are base64-encoded, not encrypted — anyone with read access to the namespace can decode them. Enable etcd encryption at rest, and for anything sensitive use the External Secrets Operator to sync from Vault or a cloud secret manager, so the source of truth is a system with rotation and audit.

What to take away#

Split liveness and readiness by what each one can fix. Size the heap as a percentage of the container limit and leave real headroom. Set maxUnavailable: 0, a preStop delay and graceful shutdown for deploys nobody notices. And autoscale on a signal that actually moves when your service is under pressure.