Asynchronous requests

Spring MVC has extensive integration of asynchronous request processing on Servlet 3.0:

• The DeferredResult and Callable return values in controller methods provide basic support for a single asynchronous return value.

• Controllers can stream multiple values, including SSE and raw data.

• Controllers can use reactive clients and return reactive types to process responses.

DeferredResult

Once the asynchronous request processing feature is enabled in servlet container, controller methods will be able to wrap any supported return value of the controller method using DeferredResult, as shown in the following example:

@GetMapping("/quotes")
@ResponseBody
public DeferredResult<String> quotes() {
    DeferredResult<String> deferredResult = new DeferredResult<String>();
    // From some other thread...
    return deferredResult;
}
// Save deferredResult somewhere...
deferredResult.setResult(result);

@GetMapping("/quotes")
@ResponseBody
fun quotes(): DeferredResult<String> {
    val deferredResult = DeferredResult<String>()
    // From some other thread...
    return deferredResult
}
// Save deferredResult somewhere...
deferredResult.setResult(result)

The controller can receive the return value asynchronously, from another thread - for example, in response to an external event (JMS message), scheduled task, or other event.

Callable

The controller can wrap any supported return value using java.util.concurrent.Callable, as shown in the following example:

@PostMapping
public Callable<String> processUpload(final MultipartFile file) {
    return new Callable<String>() {
        public String call() throws Exception {
            // ...
            return "someView";
        }
    };
}

@PostMapping
fun processUpload(file: MultipartFile) = Callable<String> {
    // ...
    "someView"
}

The return value can be obtained by executing this task through the configured TaskExecutor.

Processing

Asynchronous processing of servlet requests is briefly described below:

• ServletRequest can be set to asynchronous mode by calling request.startAsync(). The main effect is that the servlet (as well as any filters) can complete execution, but the response will remain open so that processing can finish later.

• Call request.startAsync() returns an AsyncContext that can be used to further control asynchronous processing. For example, it provides a dispatch method, which is similar to the dispatch from the Servlet API, except that it allows the application to resume processing the request on the servlet container thread.

• ServletRequest provides access to the current DispatcherType, which can be used to differentiate between initial request processing, asynchronous dispatch, forwarding, and other dispatcher types.

Processing DeferredResult works as follows:

• The controller returns DeferredResult and stores it in some queue or list in memory where it can be accessed.

• Spring MVC calls request.startAsync().

• Meanwhile, DispatcherServlet and all configured filters exit the request processing thread, but the response remains open.

• The application sets DeferredResult from some thread, and Spring MVC sends the request back to the servlet container.

• DispatcherServlet is called again and processing resumes using the asynchronously received return value.

The processing of Callable occurs as follows:

• The controller returns Callable .

• Spring MVC calls request.startAsync() and passes Callable to TaskExecutor to be processed in a separate thread.

• Meanwhile, DispatcherServlet and all filters exit the servlet container thread, but the response remains open.

• Eventually Callable produces a result, and Spring MVC sends the request back to the servlet container to complete processing.

• DispatcherServlet is called again and processing resumes using the asynchronously received return value from Callable.

To receive For more information and context, please also see blog posts that introduced support for asynchronous request handling for Spring MVC 3.2.

Exception Handling

If you are using DeferredResult, you can choose whether to call setResult or setErrorResult with an exception. In both cases, Spring MVC sends the request back to the servlet container to complete processing. It is then treated either as if the controller method had returned the specified value, or as if it had thrown the specified exception. The exception then goes through the normal exception handling mechanism (for example, calling methods annotated with @ExceptionHandler).

When Callable is used, similar processing logic occurs, but the main difference is whether the result is returned from the Callable or whether it throws an exception.

Interception

HandlerInterceptor instances can have type AsyncHandlerInterceptor to receive an afterConcurrentHandlingStarted callback on the initial request that triggers asynchronous processing (instead of postHandle and afterCompletion).

HandlerInterceptor implementations may also register CallableProcessingInterceptor or DeferredResultProcessingInterceptor for deeper integration with the asynchronous request lifecycle (for example, to handle a timeout event). For more information, see the section on AsyncHandlerInterceptor.

DeferredResult provides onTimeout(Runnable) and onCompletion(Runnable) callbacks. See details in javadoc by DeferredResult. Callable can be replaced with WebAsyncTask, which exposes additional methods for timeout and completion callback.

Compared to WebFlux

The Servlet API was originally designed to perform a single pass along the filter-servlet chain. Asynchronous request processing, added in Servlet 3.0, allows applications to break out of the filter-servlet chain but leave the response open for further processing. Spring MVC's async support is built around this mechanism. When the controller returns DeferredResult, the filter-servlet chain completes execution and the servlet container thread is released. Later, when DeferredResult is set, a ASYNC send is performed (to the same URL), during which the controller matches again, but instead of calling the value of DeferredResult (as if the controller had returned it) to resume processing.

In contrast, Spring WebFlux is not built on the Servlet API, and it does not need this asynchronous request processing feature since it is asynchronous by definition. Asynchronous processing is built into all framework contracts and is inherently supported at all stages of request processing.

From a programming model perspective, both Spring MVC and Spring WebFlux support asynchronous and reactive types as return values in controller methods. Spring MVC even supports threading, including reactive backlash. However, individual writes in response remain blocking (and are executed in a separate thread), unlike WebFlux, which relies on non-blocking I/O and does not need an additional thread for each write.

Another fundamental difference is that Spring MVC does not support asynchronous or reactive types in controller method arguments (for example, @RequestBody, @RequestPart and others), and also does not have explicit support for asynchronous and reactive types as model attributes. Spring WebFlux supports all of the above.

HTTP Streaming

You can use DeferredResult and Callable for a single asynchronous return value. What if you need to receive multiple asynchronous values and write them back? This section describes how to do this.

Objects

You can use the return value of the ResponseBodyEmitter to create a stream of objects, where each object is serialized using HttpMessageConverter and is written to the response, as shown in the following example:

@GetMapping("/events")
public ResponseBodyEmitter handle() {
    ResponseBodyEmitter emitter = new ResponseBodyEmitter();
    // Save the sender (emitter) somewhere...
    return emitter;
}
// In another thread
emitter.send("Hello once");
// and then again
emitter.send("Hello again");
// and at some point will be ready
emitter.complete();

@GetMapping( "/events")
fun handle() = ResponseBodyEmitter().apply {
    // Save the sender (emitter) somewhere...
}
// In another thread
emitter.send("Hello once")
// and then emitter again
.send("Hello again")
// and at some point will be ready
emitter.complete()

You can also use ResponseBodyEmitter as the body in the ResponseEntity, which will allow you to customize the response status and headers.

When the emitter throws IOException ( for example, if the remote client has disappeared), applications are not responsible for cleaning up the connection and should not call emitter.complete or emitter.completeWithError. Instead, the servlet container automatically triggers an AsyncListener error notification in which Spring MVC makes a call to completeWithError. This call in turn makes the final ASYNC dispatch to the application, during which Spring MVC calls the configured exception handlers and completes the request.

SSE

SseEmitter (a subclass of ResponseBodyEmitter) provides support for Server-Sent Events if the events sent by the server are formatted according to the W3C SSE specification. To create an SSE stream from a controller, return a SseEmitter as shown in the following example:

@GetMapping(path="/events ", produces=MediaType.TEXT_EVENT_STREAM_VALUE)
public SseEmitter handle() {
    SseEmitter emitter = new SseEmitter();
    // Save the sender (emitter) somewhere...
    return emitter;
}
// In another thread
emitter.send("Hello once");
// and then again
emitter.send("Hello again");
// and at some point will be ready
emitter.complete();

@GetMapping( "/events", produces = [MediaType.TEXT_EVENT_STREAM_VALUE])
fun handle() = SseEmitter().apply {
    // Save the sender (emitter) somewhere...
}
// In another thread
emitter.send("Hello once ")
// and then again
emitter.send("Hello again")
// and at some point will be ready
emitter.complete()

While SSE is the primary option for streaming to browsers, note that Internet Explorer does not support Server-Sent Events. Consider using Spring's WebSocket protocol messaging with fallback mechanisms from the SockJS protocol (including SSE), which are designed for a wide range of browsers.

Raw Data

Sometimes it's useful to bypass the conversion messages and stream directly to the response OutputStream (for example, when downloading a file). To do this, you can use the StreamingResponseBody return type, as shown in the following example:

@GetMapping("/download")
public StreamingResponseBody handle() {
    return new StreamingResponseBody() {
        @Override
        public void writeTo(OutputStream outputStream) throws IOException {
            // write...
        }
    };
}

@GetMapping("/download")
fun handle() = StreamingResponseBody {
    // write ...
}

You can use StreamingResponseBody as the body in ResponseEntity to configure the status and response headers.

Reactive types

Spring MVC supports the use of reactive client libraries in the controller. These libraries include WebClient from spring-webflux and others such as the Spring Data project's reactive data repositories. In such scenarios, it is convenient to be able to return reactive types from a controller method.

Reactive return values are handled as follows:

• A single-value promise is adjusted as if using DeferredResult. Examples include Mono (Reactor) or Single (RxJava).

• Multi-valued stream with streaming media type data (for example, application/x-ndjson or text/event-stream) is adjusted, as when using ResponseBodyEmitter or SseEmitter. Examples are Flux (Reactor) or Observable (RxJava). Applications can also return Flux<ServerSentEvent> or Observable<ServerSentEvent>.

• Multi-valued stream with any other media type data (for example, application/json) is adjusted, as when using DeferredResult<List<?>>.

Reactive backlash is supported for streaming data to the response, but writing to the response is still blocked and executed in a separate thread via a configured TaskExecutor to avoid blocking the upstream thread's source (for example, Flux returned from WebClient). The default for blocking entry is SimpleAsyncTaskExecutor, but this is not suitable for loading. If you plan to use a stream with a reactive type, then you should use MVC configuration to configure the executor.

Connection Lost

The Servlet API does not send any notification when the remote client goes missing. Therefore, when streaming a response, whether through SseEmitter or reactive types, it is important to send data periodically, since writing will not happen if the client has disconnected. The sending could take the form of an empty (comments only) SSE event or any other data that the other party would have to interpret as a heartbeat message and ignore.

You can also use web messaging solutions ( for example, STOMP over WebSocket or WebSocket with SockJS), which have a built-in mechanism for transmitting heartbeat messages.

Configuration

The asynchronous request processing feature must be enabled at the servlet container level. The MVC configuration also opens up several options for asynchronous requests.

Servlet container

Filter and servlet declarations have an asyncSupported flag that must be set to true to enable asynchronous request processing. Additionally, filter mappings must be declared to handle ASYNC javax.servlet.DispatchType.

In Java configuration, when you use AbstractAnnotationConfigDispatcherServletInitializer to initialize the servlet container , this happens automatically.

In the web.xml configuration, you can add <async-supported>true</async-supported> to DispatcherServlet and to Filter declarations, and add <dispatcher>ASYNC</dispatcher> to filter mappings.

Spring MVC

The MVC configuration provides the following options related to asynchronous request processing:

• Java configuration: Use the configureAsyncSupport callback for the WebMvcConfigurer .

• XML namespace: Use the <async-support> element in the <mvc:annotation-driven> section.

The following parameters can be configured:

• The default timeout value for asynchronous requests, which , if not set, depends on the underlying servlet container.

• AsyncTaskExecutor to be used for write blocking purposes when streaming data using reactive types and for executing Callable instances returned from controller methods. We strongly recommend configuring this property if you are working with reactive types or have controller methods that return Callable, since the default is SimpleAsyncTaskExecutor.

• Implementations of DeferredResultProcessingInterceptor and implementations of CallableProcessingInterceptor.

Note that it is also possible to set the timeout value by default for DeferredResult, ResponseBodyEmitter and SseEmitter. For Callable you can use WebAsyncTask to specify the timeout value.