LoadLab — Break a Spring Boot Service on Purpose

Almost every interview asks about a production issue you have debugged, and most developers have only ever read about memory leaks and CPU spikes. LoadLab exists to fix that: a service that fails on demand, in eight specific documented ways, with the real tooling wired in so you diagnose each one with jcmd, JFR and MAT rather than with a tutorial.

What you build#

A small order service — create, look up, list, enrich — with a Postgres database, a Redis cache and a stubbed downstream dependency whose latency you control. On top of that, a configuration layer that enables one deliberate defect at a time.

git clone https://github.com/javaagentic/loadlab && cd loadlab
docker compose up -d              # postgres, redis, prometheus, grafana
./mvnw spring-boot:run            # the service, with JFR already recording
 
# In another terminal — steady baseline load
k6 run harness/baseline.js
 
# Enable a scenario
curl -X POST localhost:8080/lab/scenario/memory-leak/enable

The eight scenarios#

Each maps directly onto a tutorial in Phase 6, so you can read the theory, reproduce the failure, and then compare what you found against what the page said you would find.

How a session goes#

# 1. Baseline for five minutes, note the heap floor after each Full GC
k6 run harness/baseline.js &
jstat -gcutil $(pgrep -f loadlab) 5000
 
# 2. Enable the leak
curl -X POST localhost:8080/lab/scenario/memory-leak/enable
 
# 3. Watch the floor climb. This is the signature — not "the heap is full",
#    but "occupancy after a full collection keeps rising".
 
# 4. Capture the evidence BEFORE it dies
jcmd $(pgrep -f loadlab) GC.heap_dump /tmp/leak.hprof
 
# 5. Open in Eclipse MAT: Leak Suspects, then the dominator tree,
#    then Path to GC Roots with weak references excluded.
 
# 6. Compare what you found against
#    /learn/jvm-memory-performance/java-memory-leak-patterns

The runbook for each scenario gives you the symptom and the commands but not the cause — that is in a separate answers file, so the exercise is diagnosis rather than reading.

Blind mode#

curl -X POST localhost:8080/lab/scenario/random/enable
# Enables one of the eight, without telling you which.

Knowing which failure you are looking for makes the diagnosis trivial and teaches you very little. Blind mode forces the real skill: start from the symptom, use RED to find the unhealthy component, use USE to find the exhausted resource, and narrow down from there — the sequence in the incident playbook.

Give yourself fifteen minutes per attempt and write down what you concluded before checking. Being wrong is the useful part.

Measuring honestly#

export const options = {
  scenarios: {
    steady: {
      executor: 'constant-arrival-rate',   // NOT constant-vus
      rate: 200, timeUnit: '1s',
      duration: '30m',
      preAllocatedVUs: 200, maxVUs: 2000,
    },
  },
  thresholds: {
    http_req_duration: ['p(99)<500'],
    dropped_iterations: ['count<1'],       // the generator kept the schedule
  },
};

The arrival-rate executor is deliberate. A closed model would slow its own request rate whenever the service slowed, hiding the exact behaviour the lab exists to demonstrate — that is coordinated omission, explained in Latency spikes. The dropped_iterations threshold fails the run if the generator itself could not keep up, which keeps the numbers trustworthy.

Extending it#

Once the eight scenarios are familiar, the useful additions are the ones that mirror what you would actually build at work:

Fix each one properly, then re-run the same load and confirm the metric moved. A fix you have not measured is a hypothesis.

Add the guardrail. For each scenario, add the alert that would have caught it: pool pending count, executor queue depth, cache hit ratio, GC time percentage. Then re-enable the scenario and check the alert fires before the failure does.

Add a scenario of your own from an incident you have actually seen. Reproducing your own war story is the best possible preparation for telling it.

Wire in distributed tracing and add a second service, so the retry-storm scenario propagates the way it does in a real estate.

Why this is worth twenty hours#

The interview answer changes completely. Instead of "I know that a memory leak means objects are retained", you can say: "I have taken a heap dump on a service leaking about 200MB an hour, opened it in MAT, found the accumulation point in the dominator tree, and traced it back through path-to-GC-roots to a static map with no eviction. The tell was that heap occupancy after each Full GC kept climbing rather than returning to the same floor."

The second answer is specific, uses the right vocabulary, and describes a process. That is the difference the lab buys, and it is why this project exists.