Imperative, Async Blocking, Reactive & Virtual Threads

May 13, 2024

Why do developers love imperative programming?

Imperative programming focuses on describing a sequence of steps that the computer must follow to solve a problem. Imperative programming has always been popular with programmers because:

1) If/else statements, loops, and methods/blocks make the code easy to understand, debug, and dry run/imagine.

2) Exceptions resulting from the code are easy to track down.

Problems in Imperative programming?

Imperative programming is a programming paradigm that uses a sequential execution model and blocking I/O.

1) The problem in Imperative programming arises from the fact that threads in this model are blocked for longer than necessary.

2) Imperative programs will always be vulnerable to data races because they contain mutable variables. (Not the focus of this article)

In blocking application servers like those based on Java Servlet < 3.0, such as Tomcat 7 or earlier, one thread is usually dedicated to a user request. This thread takes care of the entire request and will be released only once the request is complete and the response is returned back to the user.

In the diagram above, you can see a User request thread in execution from top to down vertically, with the green part representing CPU execution and the red part indicating the time the thread is blocked/waiting for a database/service call for data.

Traditional platform (non-virtual) threads are expensive, as by default, these threads use 1 MB of stack memory, setting an upper limit on the number of threads running in the JVM at a time.

So if the application receives a large number of concurrent requests, it requires a large number of user request threads. One solution to this is maintaining a thread pool. Tomcat, by default, maintains a thread pool of 200 threads, and also, we can scale the application horizontally or vertically to handle the large number of concurrent requests.

How to solve this?

1) Asynchronous Blocking design

One way to improve on the previous "Synchronous Blocking design" is to parallelize some of the tasks that were previously running serially. For example, running database queries and service calls in parallel in separate platform (non-virtual) threads of their own.

In this case, the user request thread will start two new platform threads - one to fetch data from the DB and one to fetch data from the service. The user request thread will then block to retrieve data from both and merge them to send it back to the user.

One way to achieve this in Java is to pass a Callable or Runnable to ExecutorService and utilizing Future.

This approach will definitely improve the performance, as two calls are performed in parallel. However, this is now utilizing three instead of one platform thread. Hence, there may be a short period of time during request execution where there will be thread overhead.

2) Async Servlet & CompletableFuture

We can further improve the above-mentioned solutions (where platform threads are blocked most of the time doing nothing) using Servlet 3.0 (Async Servlet).

With Servlet 3.0 (Async Servlet) and and Java 8's introduction of CompletableFuture, it is possible to create Reactive pipelines using one thread (user request thread) and actually execute the task using other threads, while the user request thread can exit just after creating the pipeline and two new threads.

Even this approach partially solves the problem, as calls to the DB and service are still blocking.

3) Reactive - Async Servlet & NIO (Servelt 3.1)

And once again, we can further improve over the above design using Java NIO (from Java 7).

To make the solution completely reactive, even the I/O (DB call and Service call) must be non-blocking. Since Java 7, all the I/O methods and utilities now have non-blocking versions (Sockets, Files, etc.).

In this design, in each I/O call, the thread that makes the request and the one that handles the request are different, and this is done using non-blocking APIs.

In this design, we can observe that threads are only utilized during CPU execution, and there is no blocked thread.

Spring utilizes this model in a completely dedicated reactive model called Spring Webflux. Spring Webflux itself uses Project Reactor internally.

Why Virtual Threads?

As seen above, reactive programming can solve most of the scalability problems but is a complex system. Fortunately, there is an easy way to solve most of the scalability problems using virtual threads.

Virtual threads are launched as part of Java 21.

The platform thread (old Java thread) is an OS thread with a wrapper and hence is expensive. The virtual thread, on the other hand, is an implementation of the existing Thread class and lives/dies within the JVM only, making it lightweight.

Internally, when a virtual thread needs to execute code, it uses platform threads for CPU operations as carrier threads and releases/unmounts these carrier threads when an I/O operation is encountered. Later, once data is available, any virtual thread is scheduled and mounted to the carried thread to receive the data.

This means blocking a Virtual Thread is not an issue, as the underlying platform thread is released during that time.

Is Reactive programming dead?

As seen before, Virtual threads enable us to use system resources/CPU more efficiently by effectively providing non-blocking IO, as no platform/OS thread is blocked when a Virtual thread is blocked.

Does that mean we don't need reactive programming? No. Reactive programming is not only about non-blocking IO but also includes asynchronous communication and back pressure.

So, for asynchronous communication between server and client and back pressure, we still need Reactive programming.

Plus, let's not forget that even if almost all reactive models, i.e., Reactor, RxJava2, Akka Streams, use platform threads for now, after the launch of Java 21, these models can start using virtual threads. If that happens, Reactive programming models can provide us benefits from both Reactive and Virtual threads world.