Scheduling & Async Processing

Background work is where quiet production incidents come from. A scheduled job overlaps itself, an async task swallows an exception, a thread pool queues without limit until the heap fills. All three are configuration problems with well-known answers.

Scheduling#

@Component
@EnableScheduling
public class ReportScheduler {
 
    private static final Logger log = LoggerFactory.getLogger(ReportScheduler.class);
 
    // Starts every 5 minutes regardless of how long the previous run took.
    // Two runs WILL overlap if one takes longer than the interval.
    @Scheduled(fixedRate = 5, timeUnit = TimeUnit.MINUTES)
    public void pollInbox() { }
 
    // Waits 5 minutes AFTER the previous run finishes. Never overlaps.
    @Scheduled(fixedDelay = 5, timeUnit = TimeUnit.MINUTES, initialDelay = 30)
    public void reconcile() { }
 
    // Spring cron is six fields: second minute hour day-of-month month day-of-week
    @Scheduled(cron = "0 15 3 * * MON-FRI", zone = "Europe/Berlin")
    public void nightlyReport() {
        try {
            reportService.generate();
        } catch (Exception ex) {
            // An uncaught exception from a @Scheduled method cancels all FUTURE
            // executions of that task. Always catch.
            log.error("nightly report failed", ex);
        }
    }
 
    // Interval from configuration, so operations can retune without a rebuild.
    @Scheduled(fixedDelayString = "${app.sync.delay:PT10M}")
    public void syncCatalogue() { }
}

That catch block is not defensive style — it is required. When a @Scheduled method throws, the ScheduledExecutorService cancels the recurring task. The job stops running, silently, until the next deploy. This surprises people roughly once per career.

Size the scheduler pool so slow jobs cannot starve fast ones:

spring:
  task:
    scheduling:
      pool:
        size: 5
      thread-name-prefix: 'sched-'

Async methods#

@Configuration
@EnableAsync
public class AsyncConfig implements AsyncConfigurer {
 
    @Override
    @Bean(name = "taskExecutor")
    public Executor getAsyncExecutor() {
        var executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(8);
        executor.setMaxPoolSize(32);
        // A bounded queue is the point. Integer.MAX_VALUE, the default, means
        // the pool never grows past core size and the queue eats the heap.
        executor.setQueueCapacity(500);
        executor.setThreadNamePrefix("async-");
        // CallerRunsPolicy applies back-pressure: when the queue is full the
        // submitting thread executes the task itself and naturally slows down.
        executor.setRejectedExecutionHandler(new ThreadPoolExecutor.CallerRunsPolicy());
        executor.setWaitForTasksToCompleteOnShutdown(true);
        executor.setAwaitTerminationSeconds(30);
        executor.initialize();
        return executor;
    }
 
    @Override
    public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
        return (ex, method, params) ->
            LoggerFactory.getLogger("async").error("uncaught in {}", method.getName(), ex);
    }
}

There is a counter-intuitive detail in ThreadPoolExecutor that trips up almost everyone: the pool grows past corePoolSize only when the queue is full. With an unbounded queue, maxPoolSize is decorative — the pool never exceeds core size, and work accumulates in memory instead. Bound the queue and the sizing behaves the way the names suggest.

@Service
public class NotificationService {
 
    @Async
    public CompletableFuture<DeliveryReceipt> send(Notification n) {
        return CompletableFuture.completedFuture(gateway.deliver(n));
    }
 
    @Async("reportExecutor")     // route to a specific pool
    public void generateReport(String id) { }
}

Return CompletableFuture whenever the caller might care whether the work succeeded. A void @Async method is fire-and-forget in the strongest sense: the caller cannot know it failed, and neither will you unless the handler above is installed.

Two constraints inherited from the proxy mechanism apply here as much as anywhere: calling an @Async method from within the same class runs it synchronously, and the method must be public.

Virtual threads#

On Java 21 with Spring Boot 3.2 or later:

spring:
  threads:
    virtual:
      enabled: true

Async and scheduled tasks then run on virtual threads. For I/O-bound work — HTTP calls, database queries — this removes the pool-sizing question almost entirely, because blocking a virtual thread is cheap. Two caveats: synchronized blocks still pin the carrier thread in Java 21, and CPU-bound work gains nothing.

Distributed scheduling#

@Component
public class LockedScheduler {
 
    @Scheduled(cron = "0 0 2 * * *")
    @SchedulerLock(name = "nightlyBilling",
                   lockAtLeastFor = "PT5M",     // guard against clock skew
                   lockAtMostFor = "PT30M")     // released even if the JVM dies
    public void runBilling() {
        LockAssert.assertLocked();
        billingService.runNightly();
    }
}

lockAtMostFor is the safety valve. If the instance holding the lock crashes without releasing it, the lock expires after that duration and the next scheduled run proceeds. Set it comfortably longer than the worst realistic runtime, or a slow run will be treated as a dead one and a second instance will start concurrently.

lockAtLeastFor covers the opposite hazard: a job that finishes in milliseconds could otherwise be picked up again by another instance whose clock is a second behind.

Choosing between scheduling, async and a queue#

These three mechanisms look interchangeable and are not. Getting the choice right removes a whole category of later rework.

A scheduled job is right when the work is driven by time rather than by an event: a nightly reconciliation, an hourly export, a poll of an external system that has no webhook. The defining characteristic is that nobody is waiting for the result, and skipping one run is usually survivable because the next run will catch up.

An async method is right when work is triggered by a request but the caller should not wait for it: sending a confirmation email after an order, warming a cache, writing an audit record to a slow sink. The defining characteristic is that the work is short, the failure is tolerable, and the result is not needed to answer the request. The moment any of those stops being true — the work takes minutes, or losing it means losing money — the answer is a queue, not an executor.

A message queue is right when the work must survive a restart. An executor's queue lives in heap; if the process dies, everything in it is gone with no record that it ever existed. A broker persists the message, redelivers it if processing fails, and gives you a dead-letter queue for the ones that keep failing. The cost is an extra piece of infrastructure and a serialisation format, which is a real cost — but it is the correct one for work you cannot afford to drop.

A useful test: ask what happens if the JVM is killed at the worst possible moment. If the answer is "a user gets a duplicate email", an executor is fine. If the answer is "an order is never shipped", you need durability.

Making scheduled jobs observable#

Background work fails silently by nature — there is no user staring at a spinner to tell you something is wrong. Three cheap habits close that gap.

Record a timer per job so you can see runtime drift. A nightly job that has crept from two minutes to fifty is going to collide with the morning traffic peak eventually, and the graph gives you months of warning.

Record a last-success timestamp as a gauge and alert on its age rather than on failures. This catches the case that failure alerting misses entirely: the job that stopped being scheduled at all. An alert on "billing has not succeeded in 26 hours" fires whether the cause was an exception, a cancelled task, a stuck lock or a deployment that removed the annotation.

Log a structured start and finish line with the job name and a run id. When something looks wrong at 3am, the first question is always whether the job ran, and the second is how long it took; both should be answerable with one query.

Graceful shutdown#

server:
  shutdown: graceful
spring:
  lifecycle:
    timeout-per-shutdown-phase: 30s

On SIGTERM the server stops accepting new requests and lets in-flight ones finish. Combined with setWaitForTasksToCompleteOnShutdown(true) on the executor, a rolling deploy no longer drops half-processed background work.

What to take away#

Choose fixedDelay unless you specifically want overlap. Catch every exception inside a scheduled method. Bound your executor queues and pick a rejection policy that applies back-pressure. And the moment you run more than one instance, lock your scheduled jobs.