Soft Assertions with AssertJ

1. Introduction

In this tutorial, we’ll examine AssertJ‘s soft assertions feature, review its motivation, and discuss similar solutions in other testing frameworks.

2. The Motivation

First, we should understand why soft assertions even exist. For that, let’s explore the following example:

@Test
void test_HardAssertions() {
    RequestMapper requestMapper = new RequestMapper();
    DomainModel result = requestMapper.map(new Request().setType("COMMON"));
    Assertions.assertThat(result.getId()).isNull();
    Assertions.assertThat(result.getType()).isEqualTo(1);
    Assertions.assertThat(result.getStatus()).isEqualTo("DRAFT");
}

This code is quite simple. We have a mapper, that maps the Request entity to some DomainModel instance. We want to test the behavior of this mapper. And, of course, we consider the mapper to work correctly only in case all assertions would pass.

Now, let’s say that our mapper is flawed and maps the id and status incorrectly. In this case, if we had launched this test, we would have gotten an AssertionFailedError error:

org.opentest4j.AssertionFailedError: 
expected: null
but was: "73a3f292-8131-4aa9-8d55-f0dba77adfdb"

That’s all good, except for one thing – although the mapping of id is indeed wrong, the mapping of status is also wrong. The test didn’t tell us that the status mapping is incorrect, it only complains about id. That happens because, by default, when we use assertions with assertj, we use them as hard assertions. It means that the very first assertion in the test that doesn’t pass will trigger an AssertionError immediately.

At first glance, it seems not a big deal since we would just launch the test twice – the first time to find out that our ID mapping is wrong, and finally to discover that our status mapping is wrong. However, in case our mapper is more complex, and it maps entities with dozens of fields, which is possible in real-life projects, it would cost us a significant amount of time to hunt down all the problems by continuously rerunning the test. Specifically, to address this inconvenience, assertj offers us soft assertions.

3. Soft Assertions in AssertJ

Soft assertions solve this problem by doing a very simple thing – by collecting all the errors encountered during all assertions and generating a single report. Let’s take a look at how soft assertions help us in assertj by rewriting the test above:

@Test
void test_softAssertions() {
    RequestMapper requestMapper = new RequestMapper();
    DomainModel result = requestMapper.map(new Request().setType("COMMON"));
    SoftAssertions.assertSoftly(softAssertions -> {
        softAssertions.assertThat(result.getId()).isNull();
        softAssertions.assertThat(result.getType()).isEqualTo(1);
        softAssertions.assertThat(result.getStatus()).isEqualTo("DRAFT");
    });
}

Here, we’re asking assertj to perform a bunch of assertions softly. It means that all assertions in the lambda expression above will execute no matter what, and in case some of them generate an error – these errors would be packed into a single report, like the one below:

org.assertj.core.error.AssertJMultipleFailuresError: 
Multiple Failures (3 failures)
-- failure 1 --
expected: null
 but was: "66f8625c-b5e4-4705-9a49-94db3b347f72"
at SoftAssertionsUnitTest.lambda$test_softAssertions$0(SoftAssertionsUnitTest.java:19)
-- failure 2 --
expected: 1
 but was: 0
at SoftAssertionsUnitTest.lambda$test_softAssertions$0(SoftAssertionsUnitTest.java:20)
-- failure 3 --
expected: "DRAFT"
 but was: "NEW"
at SoftAssertionsUnitTest.lambda$test_softAssertions$0(SoftAssertionsUnitTest.java:21)

It becomes very useful during debugging to shorten the time spent on catching all bugs. Furthermore, it’s worth to mention that there is another approach to writing tests with soft assertions in assertj:

@Test
void test_softAssertionsViaInstanceCreation() {
    RequestMapper requestMapper = new RequestMapper();
    DomainModel result = requestMapper.map(new Request().setType("COMMON"));
    SoftAssertions softAssertions = new SoftAssertions();
    softAssertions.assertThat(result.getId()).isNull();
    softAssertions.assertThat(result.getType()).isEqualTo(1);
    softAssertions.assertThat(result.getStatus()).isEqualTo("DRAFT");
    softAssertions.assertAll();
}

In this case, we’re just creating an instance of SoftAssertions directly, as opposed to the previous example, in which the SoftAssertions instance would still be created, but under the framework hood.

Regarding the differences between those two variants, from a functional standpoint, they are absolutely identical. So we can feel free to choose whatever approach we want.

4. Soft Assertions in Other Testing Frameworks

Because soft assertions are very useful as a feature, a lot of testing frameworks have already adopted them. The tricky thing is that in different frameworks, this feature may have different names or might not even have a name at all. For instance, we have an assertAll() in Junit5, that works in a similar manner, but has its own flavors. TestNG also has soft assertions as a feature, which is very similar to one in the assertj.

So, the point is that most famous testing frameworks have a soft assertion as a feature, although they may not have this name.

5. Summary

Let’s finally sum up all the differences and similarities between hard and soft assertions in a single table:

6. Conclusion

In this article, we’ve explored soft assertions and their motivations. Contrary to hard assertions, soft assertions allow us to continue the test execution even if some of our assertions fail, in which case the framework generates a detailed report. This makes soft assertions a useful feature since they make debugging easier and faster.

Finally, soft assertions are not the unique feature of the Assertj. They are also present in other popular frameworks, possibly by different or without names.

As always, the code for the article is available over on GitHub.