How do thread pools work and how do you size them?

ThreadPoolExecutor has a core pool size, a maximum pool size, a work queue and a rejection policy. Tasks first use core threads, then queue, then grow toward the maximum, and finally hit rejection. Executors offers presets, but the presets hide these parameters.

ExecutorService executor = Executors.newFixedThreadPool(8);
try {
    Future<Integer> result = executor.submit(() -> 40 + 2);
    System.out.println(result.get());
} finally {
    executor.shutdown();
}

Sizing and failure modes

• CPU-bound work: roughly the number of cores, plus a small margin
• Blocking IO: size from concurrency and latency, not just cores
• Use a bounded queue; an unbounded queue can hide overload until memory exhausts
• Choose a rejection policy deliberately: abort, caller-runs, discard or discard-oldest
• Virtual threads (Java 21) make thread-per-task viable for blocking IO

newCachedThreadPool is unbounded and can create a thread per task, which is dangerous under load. Always shut down executors and await termination during application shutdown, or tasks and threads can outlive the request that created them.