Load Testing & Capacity Planning

Capacity planning without measurement is guessing, and the guess is always discovered on the busiest day. Load testing replaces the guess with a number, and the number is usually surprising.

The five test types#

The soak test is the one most often skipped and the one that finds the most expensive problems. Memory leaks, connection pool exhaustion, file handle exhaustion and unbounded cache growth all look fine for ten minutes and fail after eight hours — which means they are discovered in production, at night.

k6#

import http from 'k6/http';
import { check, group } from 'k6';
import { Trend, Rate } from 'k6/metrics';
 
const orderLatency = new Trend('order_creation_duration');
const businessErrors = new Rate('business_errors');
 
export const options = {
  stages: [
    { duration: '2m', target: 100 },   // ramp up
    { duration: '10m', target: 100 },  // steady state — the measurement
    { duration: '2m', target: 300 },   // step up
    { duration: '5m', target: 300 },
    { duration: '2m', target: 0 },     // ramp down
  ],
  thresholds: {
    // The test FAILS if these are breached, which is what makes it a gate
    // rather than a report nobody reads.
    'http_req_duration{expected_response:true}': ['p(95)<500', 'p(99)<1500'],
    'http_req_failed': ['rate<0.01'],
    'business_errors': ['rate<0.005'],
  },
};
 
export default function () {
  group('place order', () => {
    const res = http.post(`${__ENV.BASE_URL}/api/v1/orders`,
      JSON.stringify({ customerId: 'cus_1', lines: [{ sku: 'ABC-1234', quantity: 2 }] }),
      { headers: { 'Content-Type': 'application/json',
                   'Authorization': `Bearer ${__ENV.TOKEN}`,
                   'Idempotency-Key': `${__VU}-${__ITER}` } });
 
    const ok = check(res, {
      'status is 201': (r) => r.status === 201,
      'has an order id': (r) => r.json('id') !== undefined,
    });
 
    orderLatency.add(res.timings.duration);
    businessErrors.add(!ok);
  });
}

Thresholds are what turn a load test into a gate. Without them the test produces a report someone might read; with them it exits non-zero and fails the pipeline.

Note the idempotency key derived from virtual user and iteration — without it, a retry inside the load generator creates duplicate orders and pollutes the results.

Gatling#

class OrderSimulation extends Simulation {
 
  val httpProtocol = http
    .baseUrl(System.getenv("BASE_URL"))
    .acceptHeader("application/json")
    .shareConnections
 
  val feeder = csv("customers.csv").random
 
  val placeOrder = scenario("Place order")
    .feed(feeder)
    .exec(http("POST /orders")
      .post("/api/v1/orders")
      .body(StringBody("""{"customerId":"${customerId}","lines":[{"sku":"ABC-1234","quantity":2}]}"""))
      .check(status.is(201))
      .check(jsonPath("$.id").saveAs("orderId")))
    .pause(1, 3)
    .exec(http("GET /orders/{id}")
      .get("/api/v1/orders/${orderId}")
      .check(status.is(200)))
 
  setUp(placeOrder.inject(
    rampUsersPerSec(1).to(50).during(2.minutes),
    constantUsersPerSec(50).during(10.minutes)
  )).protocols(httpProtocol)
   .assertions(
     global.responseTime.percentile(99).lt(1500),
     global.successfulRequests.percent.gt(99))
}

Gatling's HTML report is the best in class for diagnosis — response time distribution over the run, percentiles per request type, and errors correlated with the point in the ramp where they started.

Reading the results#

The counter-intuitive part is the right-hand branch. Past saturation, throughput does not plateau — it falls, because queued requests consume memory and time out, and retries add load. That is why a service at 95% capacity is not "nearly fine": it is one small spike from a self-reinforcing collapse.

One measurement trap is worth knowing by name: coordinated omission. A generator that waits for each response before sending the next request stops generating load at precisely the moment the service slows down, so the worst latencies never get recorded and the percentiles come out flattering. Open-model injection — k6's arrival-rate executors, Gatling's constantUsersPerSec — sends at a fixed rate regardless of whether responses come back, which is what both examples above use and what makes their tail numbers worth believing.

While the test runs, watch the service's own metrics rather than only the load generator's. The bottleneck is almost never CPU:

• Connection pool pending acquisitions above zero — the pool is the constraint.
• Database CPU or lock waits climbing — you are testing the database, not the service.
• Thread pool queue depth growing — work arriving faster than it completes.
• GC time percentage rising — heap pressure, often from allocation rather than a leak.

Capacity planning#

The arithmetic is simple once you have a measured number:

measured capacity per instance : 400 requests/second at p99 = 400ms
current peak                   : 3,000 requests/second
growth over the next 6 months  : 40%
projected peak                 : 4,200 requests/second
 
instances at 100% utilisation  : 4,200 / 400 = 11
target utilisation 45%         : 11 / 0.45  = 25 instances
survive losing one of 3 zones  : 25 / (2/3) = 38 instances

That last line is the one omitted most often. If capacity is spread across three availability zones and one fails, the remaining two carry everything — so the plan must work at two-thirds of the fleet.

Re-measure after significant changes. A new feature, a library upgrade or a schema change can move per-instance capacity substantially, and a plan built on last quarter's number is a plan built on nothing.

What to take away#

Run load tests to find the saturation point, soak tests to find leaks, and spike tests to verify autoscaling. Set thresholds so the test gates rather than reports. Watch pools and the database rather than CPU. Then size for peak plus growth plus a zone failure, which means running at under half of measured capacity.