[Advanced Java] ExecutorService and Thread Pools in Java

A thread pool separates two things that a raw new Thread(task).start() welds together: what work needs doing and which thread does it. Once those are separate, the number of threads stops being a consequence of the number of tasks and becomes a number you choose.

ExecutorService is the interface for that. Almost every implementation you will meet is one class, ThreadPoolExecutor, and almost every bug people hit with it comes from not knowing what its constructor arguments do to each other. This article walks the class from the constructor outward: where a submitted task actually goes, why the maximum pool size is usually dead configuration, why submit can hide a failing task forever, and how to shut the thing down without leaving the JVM running.

Every output block below was produced by compiling and running the code on OpenJDK 21.0.6 (arm64). Where a result varies between runs, it says so and gives more than one observed value.

Why a thread pool exists at all

A platform thread is an operating-system thread. Creating one is a system call, it reserves a stack, and the scheduler has to know about it for as long as it lives. Doing that per task is fine for ten tasks and ruinous for ten thousand.

The structural cost is easy to make visible without timing anything. ThreadMXBean.getTotalStartedThreadCount() counts every thread the JVM has ever started, so it answers the question directly:

import java.util.concurrent.*;
import java.lang.management.*;
 
public class WhyPool {
    static final int TASKS = 10_000;
 
    public static void main(String[] args) throws Exception {
        ThreadMXBean mx = ManagementFactory.getThreadMXBean();
 
        long before = mx.getTotalStartedThreadCount();
        for (int i = 0; i < TASKS; i++) {
            Thread t = new Thread(() -> Math.sqrt(42));
            t.start();
            t.join();
        }
        long threadPerTask = mx.getTotalStartedThreadCount() - before;
 
        before = mx.getTotalStartedThreadCount();
        ExecutorService pool = Executors.newFixedThreadPool(4);
        for (int i = 0; i < TASKS; i++) pool.execute(() -> Math.sqrt(42));
        pool.shutdown();
        pool.awaitTermination(1, TimeUnit.MINUTES);
        long pooled = mx.getTotalStartedThreadCount() - before;
 
        System.out.println("tasks run                       : " + TASKS);
        System.out.println("threads started, thread-per-task: " + threadPerTask);
        System.out.println("threads started, pool of 4      : " + pooled);
    }
}

tasks run                       : 10000
threads started, thread-per-task: 10000
threads started, pool of 4      : 4

Ten thousand tasks, ten thousand threads created and destroyed, against four threads that were created once and reused ten thousand times. That is the entire argument for a pool, and it holds regardless of how fast the machine is.

The second half of the argument is admission control. A thread-per-task program has no upper bound on how much work is in flight — every arriving request immediately becomes a live thread. A pool has a fixed number of workers and a queue, so a burst of arrivals turns into a longer queue rather than into more threads competing for the same cores.

Inside ThreadPoolExecutor: the seven constructor arguments

ThreadPoolExecutor has one full constructor. Everything else in java.util.concurrent that produces a pool is a call to it with some of these already chosen for you.

The interesting part is not what each one means in isolation, it is the order in which execute consults them. That order is fixed, and it is the source of the behaviour that surprises people.

Why an unbounded queue means maximumPoolSize is never reached

Read step 2 and step 3 together. A new thread above the core size is created only after the queue has refused the task. An unbounded queue never refuses anything, so step 3 never runs.

That is easy to state and easy to disbelieve, so here is the pool that proves it: core 1, maximum 10, and a LinkedBlockingQueue constructed with no capacity argument, which means a capacity of Integer.MAX_VALUE.

import java.util.concurrent.*;
import java.util.*;
 
public class Anatomy1 {
    public static void main(String[] args) throws Exception {
        ThreadPoolExecutor pool = new ThreadPoolExecutor(
                1,                              // corePoolSize
                10,                             // maximumPoolSize
                60L, TimeUnit.SECONDS,          // keepAliveTime
                new LinkedBlockingQueue<>());   // UNBOUNDED queue
 
        Set<String> threads = Collections.synchronizedSet(new TreeSet<>());
        for (int i = 0; i < 50; i++) {
            pool.execute(() -> {
                threads.add(Thread.currentThread().getName());
                try { Thread.sleep(20); }
                catch (InterruptedException e) { Thread.currentThread().interrupt(); }
            });
        }
        System.out.println("queued right after submitting 50: " + pool.getQueue().size());
        System.out.println("poolSize right after submitting 50: " + pool.getPoolSize());
        pool.shutdown();
        pool.awaitTermination(1, TimeUnit.MINUTES);
        System.out.println("largestPoolSize ever reached: " + pool.getLargestPoolSize());
        System.out.println("maximumPoolSize configured:   " + pool.getMaximumPoolSize());
        System.out.println("distinct worker threads used: " + threads);
        System.out.println("completed tasks: " + pool.getCompletedTaskCount());
    }
}

queued right after submitting 50: 49
poolSize right after submitting 50: 1
largestPoolSize ever reached: 1
maximumPoolSize configured:   10
distinct worker threads used: [pool-1-thread-1]
completed tasks: 50

Fifty tasks, a pool allowed ten threads, and exactly one thread ever existed. getLargestPoolSize() is the pool's own high-water mark, and it stayed at 1 across all six runs sampled. The 10 in maximumPoolSize was never consulted, because offer on a LinkedBlockingQueue with Integer.MAX_VALUE capacity returns true every time.

You can confirm the capacity directly rather than trusting the Javadoc:

ThreadPoolExecutor fixed = (ThreadPoolExecutor) Executors.newFixedThreadPool(4);
System.out.println("fixed pool queue remainingCapacity = " + fixed.getQueue().remainingCapacity());

fixed pool queue remainingCapacity = 2147483647

This is the single most useful thing to know about ThreadPoolExecutor. A pool declared as "core 1, max 10" reads like a pool that grows under load. It does not. It is a single-threaded pool with an unlimited backlog, and the only symptom of overload is a queue that grows until the heap gives out.

The bounded queue: growth to the maximum, then rejection

Change one argument — the queue — and every other argument comes to life. Here is the same pool with an ArrayBlockingQueue of capacity 5:

import java.util.concurrent.*;
import java.util.*;
 
public class Anatomy2 {
    public static void main(String[] args) throws Exception {
        ThreadPoolExecutor pool = new ThreadPoolExecutor(
                1, 10,
                60L, TimeUnit.SECONDS,
                new ArrayBlockingQueue<>(5));   // BOUNDED: capacity 5
 
        Set<String> threads = Collections.synchronizedSet(new TreeSet<>());
        int accepted = 0, rejected = 0;
        for (int i = 0; i < 50; i++) {
            final int id = i;
            try {
                pool.execute(() -> {
                    threads.add(Thread.currentThread().getName());
                    try { Thread.sleep(50); }
                    catch (InterruptedException e) { Thread.currentThread().interrupt(); }
                });
                accepted++;
            } catch (RejectedExecutionException e) {
                rejected++;
                if (rejected == 1) {
                    System.out.println("first rejection at task " + id);
                    System.out.println(e);
                }
            }
        }
        System.out.println("accepted " + accepted + ", rejected " + rejected);
        System.out.println("poolSize now: " + pool.getPoolSize()
                + ", queued: " + pool.getQueue().size());
        pool.shutdown();
        pool.awaitTermination(1, TimeUnit.MINUTES);
        System.out.println("largestPoolSize: " + pool.getLargestPoolSize()
                + " / max " + pool.getMaximumPoolSize());
        System.out.println("distinct worker threads: " + threads.size());
    }
}

first rejection at task 15
java.util.concurrent.RejectedExecutionException: Task Anatomy2$$Lambda/0x00000070010009f8@60e53b93 rejected from java.util.concurrent.ThreadPoolExecutor@55f96302[Running, pool size = 10, active threads = 10, queued tasks = 5, completed tasks = 0]
accepted 15, rejected 35
poolSize now: 10, queued: 5
largestPoolSize: 10 / max 10
distinct worker threads: 10

Fifteen tasks fit: ten in threads plus five in the queue. The sixteenth has nowhere to go and the pool throws. Task index 15, accepted 15 and rejected 35 were identical across all five runs sampled; the hexadecimal lambda identity inside the message (0x00000070010009f8 above) changes on every run, because it is a class-loader-generated name and an identity hash code.

RejectedExecutionException extends RuntimeException, so nothing forces you to catch it. That is worth remembering: switching a pool from an unbounded to a bounded queue moves the failure from "the heap fills up eventually" to "this call throws right now", and the second one only helps if somebody handles it.

Note the message's own diagnostics — pool size = 10, active threads = 10, queued tasks = 5. When a RejectedExecutionException shows up in production logs, that bracket tells you whether the pool was saturated or already shut down, without any extra instrumentation.

The rejected-execution handler and its four policies

The handler decides what "no room" means. The JDK ships four nested classes in ThreadPoolExecutor, and the default is AbortPolicy. This pool has one thread and a queue of 2, and gets 8 tasks:

static void run(String label, RejectedExecutionHandler h) throws Exception {
    ThreadPoolExecutor p = new ThreadPoolExecutor(1, 1, 0, TimeUnit.SECONDS,
            new ArrayBlockingQueue<>(2), h);
    Set<String> ranOn = Collections.synchronizedSet(new TreeSet<>());
    int rejected = 0;
    for (int i = 0; i < 8; i++) {
        try {
            p.execute(() -> {
                ranOn.add(Thread.currentThread().getName());
                try { Thread.sleep(60); }
                catch (InterruptedException e) { Thread.currentThread().interrupt(); }
            });
        } catch (RejectedExecutionException e) { rejected++; }
    }
    p.shutdown();
    p.awaitTermination(10, TimeUnit.SECONDS);
    System.out.printf("%-18s submitted=8 completed=%d rejectedToCaller=%d ranOn=%s%n",
            label, p.getCompletedTaskCount(), rejected, ranOn);
}

AbortPolicy        submitted=8 completed=3 rejectedToCaller=5 ranOn=[pool-1-thread-1]
CallerRunsPolicy   submitted=8 completed=5 rejectedToCaller=0 ranOn=[main, pool-2-thread-1]
DiscardPolicy      submitted=8 completed=3 rejectedToCaller=0 ranOn=[pool-3-thread-1]
DiscardOldestPolicy submitted=8 completed=3 rejectedToCaller=0 ranOn=[pool-4-thread-1]

The ranOn column shows what CallerRunsPolicy actually does: main appears in the set of threads that ran tasks. Five tasks completed on the worker and three ran on the caller. Note that getCompletedTaskCount() counts only work done by pool threads, which is why its number is 5 rather than 8.

CallerRunsPolicy is the one to reach for by default when a pool feeds a producer you control. It converts an overflow into a slowdown instead of a failure, with no queue growth and no lost work. It also means the producing thread stops producing while it runs the task, which is exactly the throttle you wanted.

Keep-alive: what happens to threads with nothing to do

keepAliveTime applies to threads above the core size. When one has been idle that long, it exits and the pool shrinks. allowCoreThreadTimeOut(true) extends the same rule to core threads, letting an idle pool drop to zero.

ThreadPoolExecutor p = new ThreadPoolExecutor(2, 6, 500, TimeUnit.MILLISECONDS,
        new SynchronousQueue<>());
for (int i = 0; i < 6; i++)
    p.execute(() -> { try { Thread.sleep(200); } catch (InterruptedException e) {} });
Thread.sleep(100);
System.out.println("under load        : poolSize=" + p.getPoolSize());
Thread.sleep(1500);
System.out.println("after idle 1.5s   : poolSize=" + p.getPoolSize()
        + " (core=" + p.getCorePoolSize() + ")");
p.allowCoreThreadTimeOut(true);
Thread.sleep(1500);
System.out.println("allowCoreThreadTimeOut(true), idle again: poolSize=" + p.getPoolSize());

under load        : poolSize=6
after idle 1.5s   : poolSize=2 (core=2)
allowCoreThreadTimeOut(true), idle again: poolSize=0

Note the queue here: SynchronousQueue has zero capacity, so offer succeeds only when a thread is already waiting to take the task. That is the queue that makes a pool grow eagerly, and it is exactly what newCachedThreadPool uses.

The Executors factory methods and their hidden defaults

Executors is a set of shortcuts. Each one is a ThreadPoolExecutor with arguments pre-chosen, and reflection over a live instance tells you which:

static void describe(String name, ExecutorService es) {
    if (es instanceof ThreadPoolExecutor p) {
        System.out.printf("%-32s core=%-4d max=%-11s keepAlive=%-4ds queue=%s%n",
                name, p.getCorePoolSize(),
                p.getMaximumPoolSize() == Integer.MAX_VALUE
                        ? "MAX_VALUE" : p.getMaximumPoolSize(),
                p.getKeepAliveTime(TimeUnit.SECONDS),
                p.getQueue().getClass().getSimpleName());
    } else {
        System.out.printf("%-32s not a ThreadPoolExecutor: %s%n", name, es.getClass().getName());
    }
    es.shutdown();
}

availableProcessors = 10
newFixedThreadPool(4)            core=4    max=4           keepAlive=0   s queue=LinkedBlockingQueue
newCachedThreadPool()            core=0    max=MAX_VALUE   keepAlive=60  s queue=SynchronousQueue
newSingleThreadExecutor()        not a ThreadPoolExecutor: java.util.concurrent.Executors$AutoShutdownDelegatedExecutorService
newScheduledThreadPool(2)        core=2    max=MAX_VALUE   keepAlive=0   s queue=DelayedWorkQueue
newWorkStealingPool()            not a ThreadPoolExecutor: java.util.concurrent.ForkJoinPool
newVirtualThreadPerTaskExecutor() not a ThreadPoolExecutor: java.util.concurrent.ThreadPerTaskExecutor

newFixedThreadPool and newSingleThreadExecutor

newFixedThreadPool(n) is core = max = n with an unbounded LinkedBlockingQueue. Core equals maximum, so the growth path is irrelevant anyway — but the unbounded queue is the real default you are accepting. A fixed pool never rejects work; it accumulates it. Under sustained overload the failure mode is a growing queue and eventually OutOfMemoryError, in a stack trace that will not mention the pool at all.

newSingleThreadExecutor() is a fixed pool of one, wrapped so that the ThreadPoolExecutor methods are not reachable. On JDK 21 the wrapper class is Executors$AutoShutdownDelegatedExecutorService; the cast to ThreadPoolExecutor fails at run time. That wrapper is deliberate — it guarantees the "exactly one thread, tasks run in submission order" contract by making the pool unreconfigurable. If you need the guarantee, use it. If you need to inspect or tune the pool, build a ThreadPoolExecutor with core 1 and max 1 instead.

newCachedThreadPool

core = 0, max = Integer.MAX_VALUE, SynchronousQueue, 60-second keep-alive. Because the queue has zero capacity, every task that arrives when no thread is idle creates a new thread — up to a maximum of about two billion.

cached pool queue remainingCapacity = 0

That is the mirror image of the fixed pool's problem. The fixed pool has a bounded thread count and an unbounded queue; the cached pool has an unbounded thread count and no queue at all. Under a burst it will happily try to create a thread per task, which is exactly the situation a pool was supposed to prevent. It is fine for short-lived, low-volume, bursty work, and dangerous for anything driven by external load.

Those two defaults, together, are why many teams have a rule against Executors entirely and construct ThreadPoolExecutor directly. The constructor is verbose, but nothing about it is hidden.

newScheduledThreadPool

ScheduledExecutorService adds schedule, scheduleAtFixedRate and scheduleWithFixedDelay. Its queue is a DelayedWorkQueue, which is unbounded, so — by the same argument as above — its MAX_VALUE maximum is unreachable and the pool never grows past the core size.

It also has a trap of its own that belongs in the same family as submit:

ScheduledExecutorService s = Executors.newScheduledThreadPool(2);
AtomicInteger ticks = new AtomicInteger();
ScheduledFuture<?> repeat = s.scheduleAtFixedRate(() -> {
    int n = ticks.incrementAndGet();
    if (n == 3) throw new IllegalStateException("tick 3 failed");
}, 0, 100, TimeUnit.MILLISECONDS);
 
Thread.sleep(1200);
System.out.println("ticks after 1.2s of a 100ms fixed rate: " + ticks.get());
System.out.println("repeating future isDone=" + repeat.isDone()
        + " isCancelled=" + repeat.isCancelled());
try { repeat.get(); }
catch (ExecutionException e) { System.out.println("its get() -> " + e.getCause()); }

ticks after 1.2s of a 100ms fixed rate: 3
repeating future isDone=true isCancelled=false
its get() -> java.lang.IllegalStateException: tick 3 failed

Twelve ticks were due and three happened. A repeating task that throws is not retried and not logged — the repetition simply stops, permanently, and the only trace is a Future that has quietly become done. Every periodic task body should be wrapped in its own try/catch around everything it does.

newVirtualThreadPerTaskExecutor

Java 21 finalised virtual threads, and Executors.newVirtualThreadPerTaskExecutor() returns an ExecutorService that is not a pool at all — the class is ThreadPerTaskExecutor and it starts one virtual thread per task, unbounded.

virtual tasks run          : 10000
platform threads started   : 11
distinct carrier threads   : 10
executor class             : java.util.concurrent.ThreadPerTaskExecutor
virtual thread isDaemon    : true

Ten thousand tasks, each sleeping, ran on eleven platform threads. Nothing was pooled: ten thousand virtual threads were created and thrown away, and the ten carrier threads underneath match this machine's ten processors. Pooling virtual threads is pointless because creating one is cheap; the reason to keep a ThreadPoolExecutor in a Java 21 codebase is the thing a virtual-thread executor does not give you — a bound on how much work runs at once. That bound is often the point, especially when the pool is fronting a database or an API with a connection limit.

submit versus execute, and the exception that disappears

Executor.execute(Runnable) returns void. ExecutorService.submit is overloaded three ways and returns a Future:

Future<?> r = es.submit(() -> System.out.println("runnable body"));
System.out.println("submit(Runnable).get() = " + r.get());
Future<String> t = es.submit(() -> System.out.println("with result"), "TOKEN");
System.out.println("submit(Runnable, T).get() = " + t.get());
Future<Integer> c = es.submit(() -> 6 * 7);
System.out.println("submit(Callable).get() = " + c.get());

runnable body
submit(Runnable).get() = null
with result
submit(Runnable, T).get() = TOKEN
submit(Callable).get() = 42

execute only accepts a Runnable, and handing it a Callable does not compile:

Bad.java:7: error: incompatible types: Callable<String> cannot be converted to Runnable
        es.execute(c);
                   ^

That is the visible difference. The invisible one matters far more.

The failure you never see

The same task, the same pool, both methods:

import java.util.concurrent.*;
 
public class SubmitVsExecute {
    static void boom() {
        throw new IllegalStateException("task blew up");
    }
 
    public static void main(String[] args) throws Exception {
        ExecutorService es = Executors.newFixedThreadPool(1);
 
        System.out.println("--- execute(Runnable) ---");
        es.execute(SubmitVsExecute::boom);
        Thread.sleep(300);
 
        System.out.println("--- submit(Runnable), Future ignored ---");
        Future<?> f = es.submit(SubmitVsExecute::boom);
        Thread.sleep(300);
        System.out.println("nothing printed above. isDone=" + f.isDone());
 
        System.out.println("--- the same Future, get() called ---");
        try {
            f.get();
        } catch (ExecutionException e) {
            System.out.println("caught: " + e);
            System.out.println("cause:  " + e.getCause());
            System.out.println("cause class: " + e.getCause().getClass().getName());
        }
        es.shutdown();
    }
}

--- execute(Runnable) ---
Exception in thread "pool-1-thread-1" java.lang.IllegalStateException: task blew up
	at SubmitVsExecute.boom(SubmitVsExecute.java:5)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1144)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:642)
	at java.base/java.lang.Thread.run(Thread.java:1583)
--- submit(Runnable), Future ignored ---
nothing printed above. isDone=true
--- the same Future, get() called ---
caught: java.util.concurrent.ExecutionException: java.lang.IllegalStateException: task blew up
cause:  java.lang.IllegalStateException: task blew up
cause class: java.lang.IllegalStateException

With execute, the throwable propagates out of Runnable.run(), nothing in runWorker catches it, the worker thread dies and its uncaught-exception handler prints the trace. The pool then quietly starts a replacement:

next task ran on: pool-1-thread-2

With submit, the task is wrapped in a FutureTask whose run() catches Throwable and stores it as the task's outcome. The worker thread never sees an exception, so nothing is printed and nothing dies. The failure exists only inside the Future, and Future.get() is the only thing that will ever hand it back — wrapped in an ExecutionException, with the original available through getCause().

⚠️ A task submitted with submit whose Future is discarded can fail on every single execution, forever, without producing one line of output. This is the most common way a background job silently stops working.

Making a submitted failure visible again

Setting an UncaughtExceptionHandler on the ThreadFactory does not help, because with submit nothing is ever uncaught:

AtomicInteger n = new AtomicInteger();
ThreadFactory tf = r -> {
    Thread t = new Thread(r, "ingest-" + n.incrementAndGet());
    t.setUncaughtExceptionHandler((th, e) ->
            System.out.println("  UEH on " + th.getName() + ": " + e));
    return t;
};
ThreadPoolExecutor p = new ThreadPoolExecutor(1, 1, 0, TimeUnit.SECONDS,
        new LinkedBlockingQueue<>(), tf);
 
System.out.println("execute:");
p.execute(() -> { throw new IllegalStateException("via execute"); });
Thread.sleep(300);
System.out.println("submit:");
p.submit(() -> { throw new IllegalStateException("via submit"); });
Thread.sleep(300);
System.out.println("(nothing printed for submit)");

execute:
  UEH on ingest-1: java.lang.IllegalStateException: via execute
submit:
(nothing printed for submit)

There are three things that do work, in increasing order of effort:

1. Catch inside the task. Wrap the whole task body in try/catch (Throwable t) and log there. Boring, explicit, and it works for both methods.
2. Keep the Future and call get(). Correct wherever you have somewhere to wait, and it also gives you the result.
3. Override afterExecute. The hook runs on the worker thread after every task and can dig the throwable back out of the Future:

ThreadPoolExecutor hooked = new ThreadPoolExecutor(1, 1, 0, TimeUnit.SECONDS,
        new LinkedBlockingQueue<>()) {
    @Override protected void afterExecute(Runnable r, Throwable t) {
        if (t == null && r instanceof Future<?> f && f.isDone()) {
            try { f.get(); }
            catch (CancellationException ce) { t = ce; }
            catch (ExecutionException ee) { t = ee.getCause(); }
            catch (InterruptedException ie) { Thread.currentThread().interrupt(); }
        }
        if (t != null) System.out.println("  afterExecute saw: " + t);
    }
};
hooked.submit(() -> { throw new IllegalStateException("via submit, hooked"); });

  afterExecute saw: java.lang.IllegalStateException: via submit, hooked

The rule that falls out of this: use execute for fire-and-forget work so that a failure is loud, and use submit when you are actually going to look at the Future. submit with an ignored return value is the worst of both.

Callable and Future

Runnable.run() returns nothing and may not throw a checked exception. Callable<V>.call() returns a V and may throw anything. That is the whole difference, and it is why a lambda that returns a value or throws a checked exception is a Callable.

Future<V> is the handle to a task that is already running somewhere else. It has five useful methods: get(), get(timeout, unit), cancel(boolean), isCancelled() and isDone().

get, the timed get and TimeoutException

get() blocks until the task finishes, then returns its value, throws ExecutionException if the task threw, or throws CancellationException if it was cancelled. The timed overload adds a fourth outcome:

Future<String> f2 = es.submit(slow("B", 2000));
try {
    f2.get(200, TimeUnit.MILLISECONDS);
} catch (TimeoutException e) {
    System.out.println("get(timeout)     -> threw " + e.getClass().getName()
            + ", message=" + e.getMessage());
    System.out.println("  after timeout: isDone=" + f2.isDone()
            + " isCancelled=" + f2.isCancelled());
}
boolean c = f2.cancel(true);
System.out.println("  cancel(true)   -> " + c + ", isCancelled=" + f2.isCancelled());
try { f2.get(); }
catch (CancellationException e) { System.out.println("  get() after cancel -> "
        + e.getClass().getName()); }

get()            -> A done
get(timeout)     -> threw java.util.concurrent.TimeoutException, message=null
  after timeout: isDone=false isCancelled=false
  cancel(true)   -> true, isCancelled=true
  get() after cancel -> java.util.concurrent.CancellationException

Two details worth keeping. TimeoutException carries a null message, so logging e.getMessage() alone gives you nothing — log the operation name yourself. And the timeout does not cancel anything: after it fires, isDone() is still false and the task is still running. A timed get without a following cancel leaks a running task and a pool thread.

An untimed get() inside a pool task that waits on another task in the same pool is how a thread pool deadlocks itself: the waiter occupies the only thread that could have run the task it is waiting for. That failure mode, and deadlock generally, is the subject of a later article in this series.

cancel(true) versus cancel(false)

cancel marks the Future cancelled either way. The boolean decides only whether the running thread is interrupted:

// A: blocked in sleep, cancelled with true
Future<?> a = es.submit(() -> {
    try {
        Thread.sleep(3000);
        System.out.println("A: ran to completion");
    } catch (InterruptedException e) {
        System.out.println("A: InterruptedException, interrupt flag now = "
                + Thread.currentThread().isInterrupted());
    }
});
// B: blocked in sleep, cancelled with false
Future<?> b = es.submit(() -> {
    try { Thread.sleep(600); System.out.println("B: ran to completion despite cancel(false)"); }
    catch (InterruptedException e) { System.out.println("B: InterruptedException"); }
});
// C: a CPU loop that never checks the interrupt flag
Future<?> c = es.submit(() -> {
    long end = System.nanoTime() + 1_500_000_000L;
    while (System.nanoTime() < end) spins.incrementAndGet();
    System.out.println("C: finished anyway, interrupt flag = "
            + Thread.currentThread().isInterrupted());
});
 
Thread.sleep(200);
System.out.println("cancel A (true)  = " + a.cancel(true));
System.out.println("cancel B (false) = " + b.cancel(false));
System.out.println("cancel C (true)  = " + c.cancel(true));

cancel A (true)  = true
A: InterruptedException, interrupt flag now = false
cancel B (false) = true
cancel C (true)  = true
B: ran to completion despite cancel(false)
C: finished anyway, interrupt flag = true
A isCancelled=true completed=false
B isCancelled=true completed=true
C isCancelled=true

The interleaving of the cancel A line with A's own output varies between runs — in three sampled runs, A: InterruptedException printed after the cancel A line twice and before it once, because two threads are writing to the same stream. Everything else was identical in all three.

Three lessons, all visible above:

• cancel(true) on a thread parked in sleep, wait, join or a blocking queue operation makes that call throw InterruptedException immediately, and the JVM clears the interrupt flag as it throws — which is why A prints interrupt flag now = false.
• cancel(false) does nothing to a running task. B was marked cancelled, ran to completion anyway, and its result was thrown away. cancel(false) is only useful for a task that has not started yet: it removes it from the queue so it never runs.
• Interruption is cooperative. C got its flag set and finished its loop regardless, because nothing in that loop ever checks. isCancelled() returned true for a task that ran to the end. There is no way to force a Java thread to stop.

invokeAll and invokeAny

invokeAll submits a collection and blocks until every task is finished, then returns the Future list in the order of the input collection — not in completion order:

List<Callable<String>> tasks = List.of(
        task("slow-300", 300), task("fast-50", 50), task("mid-150", 150));
List<Future<String>> all = es.invokeAll(tasks);
System.out.println("invokeAll returned " + all.size() + " futures, all done? "
        + all.stream().allMatch(Future::isDone));
for (Future<String> f : all) System.out.println("  " + f.get());
String winner = es.invokeAny(tasks);
System.out.println("invokeAny -> " + winner);

invokeAll returned 3 futures, all done? true
  slow-300
  fast-50
  mid-150
invokeAny -> fast-50

Every returned Future is already done, so the get() calls do not block. A task that failed still gets a Future; the exception surfaces on get(), as usual:

  mixed -> ok
  mixed -> ExecutionException, cause java.lang.IllegalArgumentException: bad input

invokeAny returns the result of the first task to succeed and cancels the rest. It threw ExecutionException only when every task failed, and which failure it reported varies: across five sampled runs the cause was IllegalStateException: first three times and IllegalStateException: second twice.

Both have timed overloads. invokeAll(tasks, timeout, unit) cancels whatever is unfinished when the clock runs out and still returns a Future per task, so the incomplete ones are cancelled rather than missing.

Shutting a pool down

An ExecutorService is a resource. Nothing collects it for you, and by default its threads keep the JVM alive.

The program that will not exit

Executors.defaultThreadFactory() creates non-daemon threads:

ExecutorService es = Executors.newFixedThreadPool(1);
Future<Boolean> f = es.submit(() -> Thread.currentThread().isDaemon());
System.out.println("pool thread isDaemon = " + f.get());

pool thread isDaemon = false

The JVM exits when the last non-daemon thread finishes, so an idle pool with no shutdown holds it open indefinitely. Two programs, identical except for one line:

public class NoShutdown {
    public static void main(String[] args) throws Exception {
        ExecutorService es = Executors.newFixedThreadPool(2);
        es.submit(() -> System.out.println("task ran on " + Thread.currentThread().getName()));
        Thread.sleep(200);
        System.out.println("main() is returning now");
        // no shutdown
    }
}

task ran on pool-1-thread-1
main() is returning now
STILL ALIVE after 5s -> JVM did not exit

The program printed everything it had to print, main returned, and the process was still running when it was killed five seconds later. Adding es.shutdown(); before main returns is the entire fix — the same program then exits immediately after the last line.

This is one of those bugs that never shows up in a test and always shows up in a CI job or a CLI tool that mysteriously never terminates.

shutdown, shutdownNow and awaitTermination

The two shutdown methods do different things and neither one blocks:

=== shutdown() ===
  started task1
  submit after shutdown: RejectedExecutionException
  isShutdown=true isTerminated=false
  finished task1
  started task2
  finished task2
  started task3
  finished task3
  awaitTermination(2s) = true
  isTerminated=true
=== shutdownNow() ===
  started task1
  interrupted task1
  shutdownNow returned 2 never-run tasks
  awaitTermination(2s) = true

isShutdown() becomes true the moment you call either. isTerminated() becomes true only when every task has actually finished, which is why awaitTermination exists — it is the only method here that blocks.

shutdownNow interrupts, and interruption is cooperative, exactly as in the cancellation section. A task that catches InterruptedException and carries on will not stop, and awaitTermination will time out.

The full sequence, which is worth keeping as a utility:

static void shutdownGracefully(ExecutorService es, long timeoutSeconds) {
    es.shutdown();                                  // stop accepting, drain the queue
    try {
        if (!es.awaitTermination(timeoutSeconds, TimeUnit.SECONDS)) {
            List<Runnable> dropped = es.shutdownNow();  // give up, interrupt everything
            System.out.println("  forced: " + dropped.size() + " tasks never started");
            if (!es.awaitTermination(timeoutSeconds, TimeUnit.SECONDS))
                System.out.println("  pool did not terminate");
        }
    } catch (InterruptedException e) {
        es.shutdownNow();
        Thread.currentThread().interrupt();          // restore the flag for the caller
    }
}

well-behaved pool:
  isTerminated=true
pool with a task that outlives the timeout:
  forced: 3 tasks never started
  isTerminated=true

The catch block matters as much as the rest. If the thread calling shutdownGracefully is itself interrupted while waiting, swallowing that interrupt hides a shutdown signal from everything further up the stack.

close() on ExecutorService in Java 19 and later

Java 19 made ExecutorService extend AutoCloseable and gave it a default close(). On JDK 21:

System.out.println("java.version = " + System.getProperty("java.version"));
System.out.println("ExecutorService instanceof AutoCloseable? "
        + AutoCloseable.class.isAssignableFrom(ExecutorService.class));
Method m = ExecutorService.class.getMethod("close");
System.out.println("declared by: " + m.getDeclaringClass().getName()
        + ", default? " + m.isDefault());
 
try (ExecutorService es = Executors.newFixedThreadPool(2)) {
    es.submit(() -> { Thread.sleep(400); System.out.println("  long task finished"); return null; });
    System.out.println("  leaving the try block now");
}
System.out.println("  after close(): the block waited for the task");

java.version = 21.0.6
ExecutorService instanceof AutoCloseable? true
declared by: java.util.concurrent.ExecutorService, default? true
  leaving the try block now
  long task finished
  after close(): the block waited for the task

So it exists, it is a default method on the interface itself, and try-with-resources works. What it does is shutdown() followed by an untimed wait for termination — the try block does not exit until every submitted task has finished. There is no timeout parameter and no way to pass one.

Its escape hatch is interruption. Closing on a pool whose task sleeps for a minute blocks; interrupting the closing thread makes close() fall back to shutdownNow():

  closer thread still waiting? true
  hung task interrupted
  after interrupting the closing thread, closer alive = false, executor isShutdown=true isTerminated=true

close() is the right default for a pool with a bounded lifetime inside one method. It is the wrong tool for a long-lived application pool, where you want an explicit timeout and a decision about what to do when it expires — which is the shutdownGracefully sequence above.

How many threads should a pool have?

A thread occupies a core only while it is computing. The moment it blocks on a socket, a lock or a disk read, the scheduler hands the core to something else. That single fact is the whole of pool sizing.

Start with the number of processors the JVM can see:

System.out.println("availableProcessors = " + Runtime.getRuntime().availableProcessors());

availableProcessors = 10

This value is per-machine, not per-run — it was 10 on every run here — and inside a container it reflects the cgroup CPU limit, not the host's core count.

For CPU-bound work, roughly one thread per processor. Adding more does not add throughput because there is no more CPU to have; it adds context switches and cache pressure. The thread states make that concrete — 40 threads all RUNNABLE on a 10-processor machine means 30 of them are queued in the OS scheduler at any instant:

CPU workload, pool of 40:
  worker thread states: {RUNNABLE=40}
  runnable threads=40 but only 10 cores to run them

For blocking work, one thread per processor leaves the machine idle. Ten threads that each sleep 100 ms are ten threads doing nothing at all:

blocking workload, pool of 10:
  active=10 queued=180 poolSize=10
  worker thread states: {TIMED_WAITING=10}

Ten workers, all in TIMED_WAITING, 180 tasks in the queue, and ten processors doing nothing. The queue depth here is timing-sensitive by nature; it read queued=180 on all four sampled runs, and the thread-state map was {TIMED_WAITING=10} every time.

The standard formula, from Brian Goetz's Java Concurrency in Practice, is the arithmetic of exactly that picture:

threads ≈ cores × targetUtilisation × (1 + waitTime / computeTime)

waitTime / computeTime is the ratio that decides everything. It is 0 for pure computation, giving one thread per core. A task that spends 90% of its time blocked has a ratio of 9, giving ten threads per core. A task that blocks 99% of the time gives ninety-nine.

Be honest about what the formula is: an order-of-magnitude starting point, from two numbers you usually do not know precisely, for a workload whose mix changes with the input. It cannot account for a pool sharing a machine with other pools, for a downstream database that has its own connection limit, or for tasks whose blocking fraction varies by request.

The only way to get a real number is to measure your own workload — and measuring it properly means a benchmark harness such as JMH, with warm-up, forked JVMs and statistics over many iterations. A loop with System.nanoTime() around it will measure JIT warm-up, garbage collection and whatever else the machine happens to be doing, and give you a number confident enough to be misleading. Two rules of thumb that are worth more than a bad measurement: bound the pool at whatever the downstream resource can take, and never let a blocking pool and a CPU pool be the same pool.

FAQ

Why does my thread pool never grow past corePoolSize?

Because the queue never refuses a task. ThreadPoolExecutor creates a thread above the core size only after workQueue.offer(task) has returned false, and an unbounded queue always returns true. With core 1, maximum 10, a LinkedBlockingQueue and 50 tasks, getLargestPoolSize() stayed at 1 — the maximum was never consulted. newFixedThreadPool uses an unbounded queue by default. If you want the pool to grow, give it a bounded queue such as ArrayBlockingQueue, or a SynchronousQueue if you want it to grow immediately.

What is the difference between submit and execute in Java?

execute takes a Runnable and returns void; submit takes a Runnable or a Callable and returns a Future. The consequence that matters is exception handling. An exception from execute propagates out of the worker's run method, kills that worker and reaches its uncaught-exception handler, so it gets printed and the pool starts a replacement thread. An exception from submit is caught by the FutureTask wrapper and stored, so nothing is printed and the worker survives — you see it only when someone calls Future.get(), which throws ExecutionException with the original as its cause. Use execute for fire-and-forget, and submit only when you will actually consume the Future.

How many threads should a thread pool have?

For CPU-bound work, about Runtime.getRuntime().availableProcessors(). For blocking work, more, in proportion to how much of each task is spent waiting: threads ≈ cores × targetUtilisation × (1 + waitTime / computeTime). A task that blocks 90% of the time wants roughly ten threads per core. Those are starting points, not answers — the real number depends on your workload's actual mix and on whatever downstream resource you are talking to, and getting it requires a real benchmark harness such as JMH rather than a timing loop. Also keep CPU-bound and blocking work in separate pools, because no single size is right for both.

Why does my Java program hang after main returns?

Almost certainly an ExecutorService that was never shut down. Executors.defaultThreadFactory() creates non-daemon threads, and the JVM exits only when the last non-daemon thread finishes. A pool whose workers are idle and waiting on the queue counts. A test program that ran a single task and printed its last line was still alive five seconds later and had to be killed; adding es.shutdown() made it exit immediately. Either call shutdown(), use try-with-resources so close() is called, or supply a ThreadFactory that marks the threads daemon — the last only if losing in-flight work at exit is acceptable.

Is newCachedThreadPool safe to use in production?

Only for short, low-volume, bursty work. Its configuration is core = 0, max = Integer.MAX_VALUE and a SynchronousQueue whose capacity is zero, so every task arriving when no thread is idle creates a new one. There is no effective upper bound on threads. Under load driven by anything external — a request rate, a message backlog — it will try to create a thread per task, which is exactly the failure a pool is supposed to prevent. Prefer an explicit ThreadPoolExecutor with a bounded queue and a rejection policy you have chosen deliberately.

Does Future.cancel(true) actually stop a running task?

It interrupts the thread; whether that stops anything depends on the task. A task blocked in Thread.sleep, wait, join or a blocking queue operation throws InterruptedException immediately, so it stops. A task in a computational loop that never checks Thread.currentThread().isInterrupted() runs to completion with the flag set — one such task printed interrupt flag = true from its own last line while isCancelled() already reported true. cancel(false) does not interrupt at all: a sleeping task cancelled that way ran to completion and its result was discarded. cancel(false) is only genuinely useful for a task that has not started, where it removes the task from the queue.

Why did my scheduleAtFixedRate task stop running?

It threw. A repeating task scheduled with scheduleAtFixedRate or scheduleWithFixedDelay stops permanently the first time it throws an exception, and nothing is printed. In a test where twelve ticks were due at a 100 ms rate and the third one threw, the counter stopped at 3 and the repeating Future quietly became done; the exception was retrievable only through repeat.get(). Wrap the entire body of every periodic task in its own try/catch (Throwable t) and log there.

Should I still use thread pools now that Java 21 has virtual threads?

Yes, when you need a limit. Executors.newVirtualThreadPerTaskExecutor() is not a pool: it starts one virtual thread per task with no bound. Ten thousand sleeping tasks ran on eleven platform threads with ten carriers, so pooling virtual threads is pointless — creating one is cheap. But an unbounded executor gives you no admission control, and a bound is often the reason the pool was there: a database with 20 connections, a rate-limited API, a memory budget. Blocking, unbounded-concurrency work suits a virtual-thread executor; CPU-bound work and anything fronting a limited resource still wants a sized ThreadPoolExecutor.

Conclusion

Most of what goes wrong with ExecutorService comes from three things that are invisible in the code as written. The queue decides whether the maximum pool size means anything, and newFixedThreadPool's unbounded default means it usually does not — core 1, max 10 and 50 tasks produced exactly one thread. submit swallows the exception that execute would have printed, so a broken background job can run silently forever. And a pool with no shutdown keeps a non-daemon thread alive and the JVM with it, long after the program has finished everything it meant to do.

None of those are hard once you know the order execute checks things in: core threads, then the queue, then extra threads, then the handler. Build the pool with the explicit constructor so nothing is hidden, choose a bounded queue and a rejection policy on purpose, prefer CallerRunsPolicy when the producer is yours to slow down, and end every pool's life with shutdown then awaitTermination then shutdownNow.

Future is a handle you have to hold and block on. The next article moves to CompletableFuture, which turns that handle into something you can chain, combine and complete from elsewhere, so a sequence of asynchronous steps stops being a sequence of get() calls.