Executors, Futures, Concurrent Collections and Virtual Threads

The previous article showed raw threads, which let you see the problems. Real programs seldom use them directly, for three reasons. Creating a thread per task is expensive. A raw thread cannot return a result or report an exception to the code that started it. And nobody is in charge of how many threads exist, so a burst of work can swamp the machine.

Java's java.util.concurrent package solves these. This article tours the parts you will use in practice, and ends with virtual threads, the biggest change to Java concurrency in years, with an honest account of what they are for.

• How executors, Callable and Future replace hand-made threads.
• How to size and shut down a thread pool.
• How a BlockingQueue coordinates producers and consumers.
• Which concurrent collections to use, and what they guarantee.
• How CompletableFuture composes asynchronous work.
• What virtual threads are, when they help and when they do not.

The key idea of the executor framework is to separate what to run (a task) from how to run it (threads). You submit tasks to an ExecutorService, which decides which thread runs each one, typically from a thread pool of reusable threads.

A task is either a Runnable (no result) or a Callable<T> (returns a value and may throw a checked exception). Submitting a task returns a Future<T>: a handle to the result that is not available yet. The documentation describes submit as creating and returning a Future that can be used to cancel the execution or wait for its completion.

// File: Main.javaimport java.util.ArrayList;import java.util.List;import java.util.concurrent.Callable;import java.util.concurrent.ExecutionException;import java.util.concurrent.ExecutorService;import java.util.concurrent.Executors;import java.util.concurrent.Future;import java.util.concurrent.TimeUnit;import java.util.concurrent.TimeoutException;public class Main {    static long sumRange(long from, long to) {        long total = 0;        for (long i = from; i <= to; i++) {            total += i;        }        return total;    }    public static void main(String[] args) throws Exception {        try (ExecutorService pool = Executors.newFixedThreadPool(4)) {     // closed automatically            List<Future<Long>> parts = new ArrayList<>();            for (int i = 0; i < 4; i++) {                long from = i * 25_000_000L + 1;                long to = (i + 1) * 25_000_000L;                parts.add(pool.submit(() -> sumRange(from, to)));           // a Callable<Long>            }            long total = 0;            for (Future<Long> part : parts) {                total += part.get();                                         // waits for that result            }            System.out.println("sum 1..100,000,000 = " + total);            Future<String> failing = pool.submit((Callable<String>) () -> {                throw new IllegalStateException("task failed");            });            try {                failing.get();            } catch (ExecutionException e) {                System.out.println("exception from the task: " + e.getCause().getMessage());            }            Future<String> slow = pool.submit(() -> {                Thread.sleep(2_000);                return "finished";            });            try {                slow.get(100, TimeUnit.MILLISECONDS);            } catch (TimeoutException e) {                System.out.println("timed out, cancelling: " + slow.cancel(true));            }        }        System.out.println("pool closed");    }}

sum 1..100,000,000 = 5000000050000000exception from the task: task failedtimed out, cancelling: truepool closed

Notice what the framework gives you for free:

• Results come back through Future.get(), which blocks until the task is done. The four partial sums ran in parallel on up to four threads.
• Exceptions are captured. An exception thrown inside the task is wrapped in an ExecutionException and rethrown when you call get(), so you can find out that a task failed. With raw threads, such an exception would just print a stack trace and vanish.
• Timeouts and cancellation. get(timeout) lets you stop waiting, and cancel(true) asks the task to stop (by interrupting its thread, which is the cooperative cancellation from the last article).
• Lifecycle. Since Java 19, ExecutorService is AutoCloseable, and, as the documentation notes, close() waits until all tasks have completed and the executor has terminated. Using try-with-resources means the pool is always shut down, so no threads are left alive to keep the JVM from exiting. Before Java 19 you wrote the two-phase pattern by hand: shutdown() to stop accepting tasks, awaitTermination(...) to wait, and shutdownNow() to force it if it takes too long.

How many threads? It depends on the kind of work:

• CPU-bound tasks (calculations) keep a core busy, so more threads than cores just add switching. A pool size close to the number of cores (Runtime.getRuntime().availableProcessors()) is a sensible start.
• I/O-bound tasks spend most of their time waiting (for a database or a remote service), so a platform-thread pool needs many more threads than cores to keep the CPU busy. This is exactly the situation that virtual threads (below) address.

Pools are configured with a core size, a maximum size, a work queue and a rejection policy for when everything is full (ThreadPoolExecutor exposes these). The most important caution: an unbounded queue hides overload. If tasks arrive faster than they finish, an unbounded queue grows until memory runs out, whereas a bounded queue with a rejection policy makes the overload visible so you can respond to it.

A very common pattern is producer and consumer: some threads produce work items, others process them. A BlockingQueue is the standard tool: put waits if the queue is full, and take waits if it is empty, so the producer and consumer never need explicit locks.

// File: Main.javaimport java.util.ArrayList;import java.util.List;import java.util.concurrent.ArrayBlockingQueue;import java.util.concurrent.BlockingQueue;import java.util.concurrent.CountDownLatch;public class Main {    static final String POISON = "STOP";                       // a marker meaning "no more work"    public static void main(String[] args) throws Exception {        BlockingQueue<String> queue = new ArrayBlockingQueue<>(2);   // small, so the producer must wait        List<String> processed = new ArrayList<>();               // touched only by the consumer thread        CountDownLatch finished = new CountDownLatch(1);        Thread consumer = new Thread(() -> {            try {                while (true) {                    String job = queue.take();                     // waits when the queue is empty                    if (job.equals(POISON)) {                        break;                    }                    processed.add(job.toUpperCase());                }            } catch (InterruptedException e) {                Thread.currentThread().interrupt();            } finally {                finished.countDown();            }        });        consumer.start();        for (String job : List.of("email", "report", "backup", "invoice")) {            queue.put(job);                                        // waits when the queue is full        }        queue.put(POISON);        finished.await();                                          // wait for the consumer to finish        System.out.println(processed);    }}

[EMAIL, REPORT, BACKUP, INVOICE]

The producer filled a queue of capacity two with four jobs, so it was blocked whenever the consumer fell behind. That is back-pressure, which keeps a fast producer from overwhelming a slow consumer. The "poison pill" is a simple way to tell the consumer to stop. The list processed is only touched by the consumer thread, and the main thread reads it only after finished.await(), which establishes the happens-before relationship that makes this safe.

Other coordination tools you may meet: CountDownLatch (wait until N things have happened, used above), Semaphore (limit how many threads use a resource at once), CyclicBarrier (threads wait for each other at a point), and Phaser.

The ordinary collections are not thread-safe. You could wrap them (Collections.synchronizedList), but that serialises all access and still does not make compound actions such as "check, then add" atomic. The java.util.concurrent package offers collections designed for concurrent use.

ConcurrentHashMap deserves a closer look. Its documentation says that retrieval operations (including get) generally do not block and may overlap with updates, and that aggregate operations such as putAll and clear are not atomic: concurrent readers may see only some of the entries. Its iterators are weakly consistent: they do not throw ConcurrentModificationException, and they reflect the state at some point at or since the iterator was created. Methods such as size() are estimates while updates are in flight. The most useful feature is that compound operations are atomic per key:

// File: Main.javaimport java.util.List;import java.util.Map;import java.util.TreeMap;import java.util.concurrent.ConcurrentHashMap;import java.util.concurrent.ExecutorService;import java.util.concurrent.Executors;import java.util.concurrent.atomic.LongAdder;public class Main {    public static void main(String[] args) throws Exception {        Map<String, Integer> counts = new ConcurrentHashMap<>();        Map<String, LongAdder> adders = new ConcurrentHashMap<>();        List<String> words = List.of("java", "spring", "java", "sql", "java", "sql");        try (ExecutorService pool = Executors.newFixedThreadPool(8)) {            for (int round = 0; round < 1_000; round++) {                for (String word : words) {                    pool.submit(() -> {                        counts.merge(word, 1, Integer::sum);                         // atomic per key                        adders.computeIfAbsent(word, w -> new LongAdder()).increment();                    });                }            }        }                                                                           // waits for all tasks        System.out.println(new TreeMap<>(counts));        Map<String, Long> fromAdders = new TreeMap<>();        adders.forEach((word, adder) -> fromAdders.put(word, adder.sum()));        System.out.println(fromAdders);    }}

{java=3000, spring=1000, sql=2000}{java=3000, spring=1000, sql=2000}

Six thousand tasks updated the same map from eight threads, and every count is exact: 3,000 for "java" and 2,000 for "sql" and 1,000 for "spring". Replace ConcurrentHashMap with a plain HashMap and the counts become wrong, or the map becomes corrupted. The two idioms shown, merge for numbers and computeIfAbsent(...).increment() with a LongAdder, are the standard ways to count concurrently, and the second is the pattern that the ConcurrentHashMap Javadoc itself shows as its example for frequency maps.

Future.get() blocks, which wastes a thread waiting. CompletableFuture<T> (Java 8) is a future that you can chain: describe what should happen when the result arrives, without blocking to wait for it. This is the style used for calling several remote services and combining the results.

// File: Main.javaimport java.util.concurrent.CompletableFuture;import java.util.concurrent.ExecutorService;import java.util.concurrent.Executors;public class Main {    static String fetchUser(ExecutorService pool, int id) {        sleep(50);        return "user-" + id;    }    static int fetchScore(int id) {        sleep(50);        if (id < 0) {            throw new IllegalArgumentException("bad id " + id);        }        return id * 10;    }    static void sleep(long millis) {        try {            Thread.sleep(millis);        } catch (InterruptedException e) {            Thread.currentThread().interrupt();        }    }    public static void main(String[] args) {        try (ExecutorService pool = Executors.newFixedThreadPool(4)) {            // Two independent calls run at the same time, then are combined            CompletableFuture<String> user = CompletableFuture.supplyAsync(() -> fetchUser(pool, 7), pool);            CompletableFuture<Integer> score = CompletableFuture.supplyAsync(() -> fetchScore(7), pool);            CompletableFuture<String> summary = user.thenCombine(score, (u, s) -> u + " has score " + s);            System.out.println(summary.join());            // A dependent step: the second call needs the first result (thenCompose flattens)            CompletableFuture<String> chained = CompletableFuture                    .supplyAsync(() -> 3, pool)                    .thenApply(id -> id + 1)                    .thenCompose(id -> CompletableFuture.supplyAsync(() -> fetchUser(pool, id), pool))                    .thenApply(String::toUpperCase);            System.out.println(chained.join());            // Failure handling: an exception travels down the chain until something handles it            CompletableFuture<Integer> risky = CompletableFuture                    .supplyAsync(() -> fetchScore(-1), pool)                    .thenApply(s -> s + 1)                    .exceptionally(e -> -1);            System.out.println("recovered value: " + risky.join());            CompletableFuture<Integer> unhandled = CompletableFuture.supplyAsync(() -> fetchScore(-5), pool);            try {                unhandled.join();            } catch (java.util.concurrent.CompletionException e) {                System.out.println("join threw: " + e.getCause().getMessage());            }            // Wait for several at once            CompletableFuture<Void> all = CompletableFuture.allOf(                    CompletableFuture.runAsync(() -> sleep(30), pool),                    CompletableFuture.runAsync(() -> sleep(60), pool));            all.join();            System.out.println("all done: " + all.isDone());        }    }}

user-7 has score 70USER-4recovered value: -1join threw: bad id -5all done: true

The vocabulary maps onto the lambda shapes from article 15:

Three practical cautions. First, always decide which executor runs your stages. If you omit it, the tasks run on a shared common pool, and blocking calls there can starve unrelated work. Second, handle exceptions: an exception in one stage skips the following thenApply stages and arrives wrapped in a CompletionException at join(), and a future whose failure nobody looks at fails silently. Third, do not overbuild. A chain of eight stages with mixed error handling is hard to read, and for many I/O-bound programs, simple blocking code on virtual threads (next section) is clearer than a CompletableFuture pipeline.

A traditional Java thread (a platform thread) is a thin wrapper around an operating system thread. These are comparatively heavy: each reserves stack memory, and the operating system can handle only a limited number. The classic server design is one thread per request, which is simple to write and debug, but when each request spends most of its time waiting (on a database, a remote call), the server runs out of threads long before it runs out of CPU. JEP 444 describes the bottleneck: the JDK implements threads as wrappers around operating system threads, so the number of available threads is limited.

The old escape was asynchronous programming (callbacks, reactive libraries, long CompletableFuture chains), which scales but is harder to write, read and debug.

Virtual threads, delivered in Java 21 (JEP 444, after previews in 19 and 20), are lightweight threads managed by the JVM. Millions can exist at once. The JVM runs many virtual threads on a small pool of platform threads called carrier threads. When a virtual thread performs a blocking operation (waiting for I/O, sleeping), it is unmounted from its carrier, which is then free to run another virtual thread, and the blocked one is resumed later. You write ordinary blocking code, and get the scalability of asynchronous code.

// File: Main.javaimport java.util.concurrent.ExecutorService;import java.util.concurrent.Executors;import java.util.concurrent.atomic.AtomicInteger;public class Main {    public static void main(String[] args) throws Exception {        AtomicInteger completed = new AtomicInteger();        long start = System.nanoTime();        try (ExecutorService executor = Executors.newVirtualThreadPerTaskExecutor()) {            for (int i = 0; i < 10_000; i++) {                executor.submit(() -> {                    Thread.sleep(500);                          // each task "waits" half a second                    completed.incrementAndGet();                    return null;                });            }        }                                                       // waits for all 10,000 tasks        long millis = (System.nanoTime() - start) / 1_000_000;        System.out.println("completed: " + completed.get());        System.out.println("finished in under 5 seconds: " + (millis < 5_000));        Thread virtual = Thread.ofVirtual().name("demo-virtual").start(() ->                System.out.println("is virtual: " + Thread.currentThread().isVirtual()));        virtual.join();        Thread platform = Thread.ofPlatform().start(() ->                System.out.println("is virtual: " + Thread.currentThread().isVirtual()));        platform.join();    }}

completed: 10000finished in under 5 seconds: trueis virtual: trueis virtual: false

Ten thousand tasks each waited half a second. With 10,000 platform threads you would risk running out of resources, and with a pool of 100 platform threads the work would take about fifty seconds (10,000 tasks ÷ 100 threads × 0.5 s). Here, one virtual thread per task finished everything in a few seconds at most. (The program prints a yes-or-no about the duration, not the duration itself, because the exact time varies from machine to machine.) The programming model did not change: there is no callback and no CompletableFuture, just a task that sleeps.

The marketing-style claim "virtual threads make applications faster" is wrong, and it matters to be precise. JEP 444 says it directly: virtual threads are not faster threads. They do not run code any faster than platform threads. They exist to provide scale (higher throughput), not speed (lower latency).

• They help when there are many concurrent tasks (the JEP says more than a few thousand) and the work is not CPU-bound, mostly waiting on I/O.
• They do not help CPU-bound work: a calculation that saturates the cores gains nothing from more threads, virtual or not.
• They do not remove the need for correct synchronisation. Races, visibility and deadlocks apply exactly as before.
• They do not make a slow database faster. If a downstream system can only handle 50 concurrent requests, a million virtual threads will just queue behind it, or overload it. Limit concurrency to protected resources with a Semaphore.

1. Create a new virtual thread for each task, and do not pool them. They are cheap by design, and a pool defeats the purpose. Use newVirtualThreadPerTaskExecutor() and not newFixedThreadPool.
2. Avoid long blocking inside synchronized blocks on Java 21. In JDK 21, a virtual thread that blocks while holding a monitor is pinned to its carrier thread, which reduces scalability. Prefer ReentrantLock where blocking occurs under a lock. Later JDK releases have reduced this limitation, so check the notes for the release you use.
3. Be careful with ThreadLocal. Millions of virtual threads each with a large thread-local value use a lot of memory.
4. Keep CPU-bound work in a pool sized to the number of cores.
5. Measure. Compare throughput and latency under realistic load before and after switching.

Spring Boot supports them with a single property. Its documentation states that when virtual threads are enabled (using Java 21 or later and spring.threads.virtual.enabled set to true), the auto-configured task executor uses virtual threads. We return to this in article 38.

Running several related subtasks and having them succeed, fail or cancel together is called structured concurrency. In Java 25, the StructuredTaskScope API of JEP 505 is still a preview API, disabled by default and enabled only with --enable-preview, which means that it may still change. It is worth knowing about, and it is not used in this course.

• Creating a thread per task by hand instead of using an executor.
• Forgetting to shut the executor down, so the JVM never exits.
• Calling get() right after submit for every task, which serialises everything.
• Unbounded queues and cached pools that hide overload.
• Ignoring Future results and so missing failed tasks.
• Blocking inside a CompletableFuture stage on the common pool.
• Not handling exceptions in a chain.
• Pooling virtual threads, or using them for CPU-bound work and expecting a speed-up.
• Using null in ConcurrentHashMap, which throws.
• Assuming size() is exact during concurrent updates.

try (ExecutorService pool = Executors.newFixedThreadPool(4)) {    List<Future<Integer>> futures = new ArrayList<>();    for (String s : strings) {        futures.add(pool.submit(() -> s.length()));    }    List<Integer> lengths = new ArrayList<>();    for (Future<Integer> f : futures) {        lengths.add(f.get());    }}

• Next: File I/O and Serialization begins the tools and craft module with files and try-with-resources.
• For concurrency in a database: SQL and PostgreSQL Fundamentals and Relationships, Lazy Loading, N+1 and Transactions.
• For the interview version: Interview Concepts, Part 1.