1. Introduction
Calling external web dependencies while maintaining low latency is a critical task when working with distributed systems.
In this tutorial, we’ll use OpenFeign and CompletableFuture to parallelize multiple HTTP requests, handle errors, and set network and thread timeouts.
2. Setting up a Demo Application
To illustrate the usage of parallel requests, we’ll create a capability that allows customers to purchase items on a website. Firstly, the service makes one request to get the available payment methods based on the country where the customer lives. Secondly, it makes a request to generate a report to the customer about the purchase. The purchase report doesn’t include information on the payment method.
So, let’s first add the dependency to work with spring-cloud-starter-openfeign:
<dependency>
<groupId>org.springframework.cloud</groupId>
<artifactId>spring-cloud-starter-openfeign</artifactId>
</dependency>3. Creating the External Dependency Clients
Now, let’s create two clients pointing to localhost:8083 using the @FeignClient annotation:
@FeignClient(name = "paymentMethodClient", url = "http://localhost:8083")
public interface PaymentMethodClient {
@RequestMapping(method = RequestMethod.GET, value = "/payment_methods")
String getAvailablePaymentMethods(@RequestParam(name = "site_id") String siteId);
}Our first client name is paymentMethodClient. It calls GET /payment_methods to get the available payment methods using a site_id request parameter representing the customer country.
Let’s see our second client:
@FeignClient(name = "reportClient", url = "http://localhost:8083")
public interface ReportClient {
@RequestMapping(method = RequestMethod.POST, value = "/reports")
void sendReport(@RequestBody String reportRequest);
}We named it reportClient and it calls POST /reports to generate the purchase report.
4. Creating the Parallel Request Executor
Calling the two clients in sequence would suffice to accomplish the demo application requirements. In that case, the total response time of this API would be at least the sum of the two requests’ response times.
Noticeably, the report doesn’t contain information about the payment method, so the two requests are independent. Thus, we could parallelize the work to reduce the total response time of our API to approximately the same response time of the slowest request.
In the next sections, we’ll see how to create a parallel executor of HTTP calls and handle external errors.
4.1. Creating the Parallel Executor
Therefore, let’s create the service that parallelizes the two requests using CompletableFutures:
@Service
public class PurchaseService {
private final PaymentMethodClient paymentMethodClient;
private final ReportClient reportClient;
// all-arg constructor
public String executePurchase(String siteId) throws ExecutionException, InterruptedException {
CompletableFuture<String> paymentMethodsFuture = CompletableFuture.supplyAsync(() ->
paymentMethodClient.getAvailablePaymentMethods(siteId));
CompletableFuture.runAsync(() -> reportClient.sendReport("Purchase Order Report"));
return String.format("Purchase executed with payment method %s", paymentMethodsFuture.get());
}
}The executePurchase() method first posts a parallel task to get the available payment methods using supplyAsync(). Then, we submit another parallel task to generate the report using runAsync(). Finally, we retrieve the payment method result using get() and return the complete result.
The choice of supplyAsync() and runAsync() for the two tasks is due to the different nature of the two methods. The supplyAsync() method returns the result from the GET call. On the other hand, runAsync() doesn’t return anything and thus, it’s better suited for generating the report.
Another difference is that runAsync() fires up a new thread immediately as soon as we invoke the code without any task scheduling by the thread pool. In contrast, supplyAsync() tasks might be scheduled or delayed depending on whether there are other tasks scheduled by the thread pool.
To validate our code, let’s use an integration test using WireMock:
@BeforeEach
public void startWireMockServer() {
wireMockServer = new WireMockServer(8083);
configureFor("localhost", 8083);
wireMockServer.start();
stubFor(post(urlEqualTo("/reports"))
.willReturn(aResponse().withStatus(HttpStatus.OK.value())));
}
@AfterEach
public void stopWireMockServer() {
wireMockServer.stop();
}
@Test
void givenRestCalls_whenBothReturnsOk_thenReturnCorrectResult() throws ExecutionException, InterruptedException {
stubFor(get(urlEqualTo("/payment_methods?site_id=BR"))
.willReturn(aResponse().withStatus(HttpStatus.OK.value()).withBody("credit_card")));
String result = purchaseService.executePurchase("BR");
assertNotNull(result);
assertEquals("Purchase executed with payment method credit_card", result);
}In the test above, we first configure a WireMockServer to start up at localhost:8083 and to shut down when done using the @BeforeEach and @AfterEach annotations.
Then, in the test scenario method, we used two stubs that respond with a 200 HTTP status when we call both feign clients. Finally, we assert the correct result from the parallel executor using assertEquals().
4.2. Handling External API Errors Using exceptionally()
What if the GET /payment_methods request fails with a 404 HTTP status, suggesting that there are no available payment methods for that country? It’s useful to do something in scenarios like these, like, for example, returning a default value.
To handle errors in CompletableFuture, let’s add the following exceptionally() block to our paymentMethodsFuture:
CompletableFuture <String> paymentMethodsFuture = CompletableFuture.supplyAsync(() -> paymentMethodClient.getAvailablePaymentMethods(siteId))
.exceptionally(ex -> {
if (ex.getCause() instanceof FeignException &&
((FeignException) ex.getCause()).status() == 404) {
return "cash";
});Now, if we get a 404, we return the default payment method named cash:
@Test
void givenRestCalls_whenPurchaseReturns404_thenReturnDefault() throws ExecutionException, InterruptedException {
stubFor(get(urlEqualTo("/payment_methods?site_id=BR"))
.willReturn(aResponse().withStatus(HttpStatus.NOT_FOUND.value())));
String result = purchaseService.executePurchase("BR");
assertNotNull(result);
assertEquals("Purchase executed with payment method cash", result);
}5. Adding Timeouts for Parallel Tasks and Network Requests
When calling external dependencies, we can’t be sure how long the request will take to run. Hence, if a request takes too long, at some point, we should give up on that request. With this in mind, we can add two types: a FeignClient and a CompletableFuture timeout.
5.1. Adding Network Timeouts to Feign Clients
This type of timeout works for single requests over the wire. Hence, it cuts the connection with the external dependency for one request at the network level.
We can configure timeouts for FeignClient using Spring Boot autoconfiguration:
feign.client.config.paymentMethodClient.readTimeout: 200
feign.client.config.paymentMethodClient.connectTimeout: 100In the above application.properties file, we set read and connect timeouts for PaymentMethodClient. The numeric values are measured in milliseconds.
The connect timeout tells the feign client to cut the TCP handshake connection attempt after the threshold value. Similarly, the read timeout interrupts the request when the connection is made properly, but the protocol can’t read the data from the socket.
Then, we can handle that type of error inside the exceptionally() block in our parallel executor:
if (ex.getCause() instanceof RetryableException) {
// handle TCP timeout
throw new RuntimeException("TCP call network timeout!");
}And to verify the correct behavior, we can add another test scenario:
@Test
void givenRestCalls_whenPurchaseRequestWebTimeout_thenReturnDefault() {
stubFor(get(urlEqualTo("/payment_methods?site_id=BR"))
.willReturn(aResponse().withFixedDelay(250)));
Throwable error = assertThrows(ExecutionException.class, () -> purchaseService.executePurchase("BR"));
assertEquals("java.lang.RuntimeException: REST call network timeout!", error.getMessage());
}Here, we’ve used the withFixedDelay() method with 250 milliseconds to simulate a TCP timeout.
5.2. Adding Thread Timeouts
On the other hand, thread timeouts stop the entire CompletableFuture content, not only a single request attempt. For instance, for feign client retries, the times from the original request and the retry attempts also count when evaluating the timeout threshold.
To configure thread timeouts, we can slightly modify our payment method CompletableFuture:
CompletableFuture<String> paymentMethodsFuture = CompletableFuture.supplyAsync(() -> paymentMethodClient.getAvailablePaymentMethods(siteId))
.orTimeout(400, TimeUnit.MILLISECONDS)
.exceptionally(ex -> {
// exception handlers
});Then, we can handle threat timeout errors inside the exceptionally() block:
if (ex instanceof TimeoutException) {
// handle thread timeout
throw new RuntimeException("Thread timeout!", ex);
}Hence, we can verify it works properly:
@Test
void givenRestCalls_whenPurchaseCompletableFutureTimeout_thenThrowNewException() {
stubFor(get(urlEqualTo("/payment_methods?site_id=BR"))
.willReturn(aResponse().withFixedDelay(450)));
Throwable error = assertThrows(ExecutionException.class, () -> purchaseService.executePurchase("BR"));
assertEquals("java.lang.RuntimeException: Thread timeout!", error.getMessage());
}We’ve added a longer delay to /payments_method so it passes the network timeout threshold, but fails on the thread timeout.
6. Conclusion
In this article, we learned how to execute two external dependency requests in parallel using CompletableFuture and FeignClient.
We also saw how to add network and thread timeouts to interrupt the program execution after a time threshold.
Finally, we handled 404 API and timeout errors gracefully using CompletableFuture.exceptionally().
As always, the source code is available over on GitHub.