JVM Flags and Container Awareness in Kubernetes

Running a JVM in a container is the default now, and the two runtimes have overlapping ideas about resources. Getting the interaction wrong produces two very recognisable failures: pods killed with no stack trace, and latency spikes with no CPU usage to explain them.

Memory#

resources:
  limits:
    memory: "2Gi"
env:
  - name: JAVA_OPTS
    value: "-Xmx2g"          # the heap alone is the entire container budget

Nothing is left for Metaspace, thread stacks, the code cache, GC structures or direct buffers. The process exceeds 2GiB, the kernel sends SIGKILL, and the pod restarts with exit code 137 — no Java error, no heap dump, nothing in the application log.

resources:
  requests:
    memory: "2Gi"
  limits:
    memory: "2Gi"            # requests == limits: Guaranteed QoS, evicted last
env:
  - name: JAVA_OPTS
    value: >-
      -XX:MaxRAMPercentage=70.0
      -XX:InitialRAMPercentage=70.0
      -XX:MaxMetaspaceSize=256m
      -XX:MaxDirectMemorySize=256m
      -XX:+HeapDumpOnOutOfMemoryError
      -XX:HeapDumpPath=/dumps/
      -XX:NativeMemoryTracking=summary

MaxRAMPercentage is a percentage of the container limit, read from the cgroup. Change the pod's memory and the heap follows automatically. Setting InitialRAMPercentage to the same value avoids heap resizing during warm-up, which sometimes triggers full collections at exactly the wrong time.

Bounding Metaspace and direct memory converts two silent native-memory growth paths into diagnosable OutOfMemoryErrors.

Heap (70%)                      1434 MB
Metaspace                        200 MB
Code cache                       100 MB
100 threads × 1MB (committed)     60 MB
GC structures (~5% of heap)       70 MB
Direct buffers                   100 MB
JVM native, malloc arenas        100 MB
                                --------
                                ~2064 MB  — already at the edge

That table is why 70% is a starting point rather than a rule. A service with many threads or heavy NIO needs less; a compute service with few threads can go to 80%.

CPU#

availableProcessors()
    -> GC thread count           (ParallelGCThreads, ConcGCThreads)
    -> ForkJoinPool.commonPool() parallelism  (parallel streams, CompletableFuture)
    -> JIT compiler threads      (CICompilerCount)
    -> your own pool sizing, if it reads availableProcessors()

Kubernetes CPU limits are expressed in millicores and enforced as a cgroup quota. The JVM computes availableProcessors() as roughly ceil(quota / period), so:

The jump from 1 to 2 matters more than the raw number: at one processor the JVM chooses SerialGC and a minimal compiler configuration, which is correct for that budget but very different from what you get in a test environment on a laptop with eight cores.

Throttling#

This is the failure that wastes the most debugging time.

rate(container_cpu_cfs_throttled_seconds_total[5m]) > 0

Any sustained non-zero value means the container is being frozen. Three responses, in order of preference:

Raise the limit, or remove it entirely and rely on requests for scheduling. Many teams run production with CPU requests set and limits unset, precisely to avoid throttling; the trade-off is weaker isolation between noisy neighbours.

Reduce concurrency inside the container so it cannot burn the quota in a burst — a smaller thread pool means fewer threads racing through the budget.

Give the JVM fewer threads. -XX:ActiveProcessorCount=N overrides the detected value, which is useful when the automatic detection produces a number that does not match how you want the pools sized.

Startup#

Containers make JVM startup cost visible, because pods restart far more often than servers did.

-XX:TieredStopAtLevel=1          # C1 only: faster warm-up, ~30% lower peak throughput
-Xshare:auto                     # class data sharing, on by default
-XX:+UseSerialGC                 # for a small, single-CPU sidecar
 
# Application class data sharing — a real win for large Spring applications
java -XX:ArchiveClassesAtExit=app.jsa -jar app.jar
java -XX:SharedArchiveFile=app.jsa -jar app.jar

AppCDS memory-maps a pre-parsed class archive and typically cuts Spring Boot startup by 20–30%. For a service that scales up and down frequently, that is more valuable than peak throughput.

TieredStopAtLevel=1 is the right trade only for short-lived processes — a batch job or a scale-to-zero function. For a long-running service it leaves 30% of throughput on the table.

Probes#

startupProbe:                    # gives slow JVM startup room without weakening the others
  httpGet: { path: /actuator/health/readiness, port: 8080 }
  failureThreshold: 30
  periodSeconds: 5
 
livenessProbe:                   # restart only when genuinely stuck
  httpGet: { path: /actuator/health/liveness, port: 8080 }
  periodSeconds: 10
  failureThreshold: 3
 
readinessProbe:                  # remove from load balancing when temporarily busy
  httpGet: { path: /actuator/health/readiness, port: 8080 }
  periodSeconds: 5

Two mistakes worth naming. Without a startupProbe, a liveness probe with a short threshold kills the pod during the 40 seconds a large Spring context takes to start — producing a crash loop that looks like an application bug. And a liveness probe that checks the database restarts every pod in the fleet when the database has a hiccup, turning a partial outage into a total one. Liveness should ask "is this process stuck?", readiness should ask "can it serve right now?".

Graceful shutdown#

terminationGracePeriodSeconds: 45
lifecycle:
  preStop:
    exec: { command: ["sh", "-c", "sleep 5"] }   # let endpoints propagate first

server.shutdown=graceful
spring.lifecycle.timeout-per-shutdown-phase=30s

Kubernetes sends SIGTERM and removes the pod from endpoints concurrently, so without the preStop sleep some in-flight traffic still arrives after shutdown has begun. Five seconds of delay before the JVM starts shutting down covers the propagation window.

What gets asked#

The scenario question is the common one: "your Java pod keeps getting OOMKilled — what do you do?" Answer with the accounting: the container limit must cover heap plus Metaspace plus stacks plus code cache plus direct memory, so use MaxRAMPercentage around 70 and bound the native regions.

The follow-up worth being ready for is throttling, because it is less well known: latency spikes with low reported CPU usage almost always mean the cgroup quota is being exhausted in bursts, and container_cpu_cfs_throttled_seconds_total is the metric that proves it.