Truth vs. AssertJ and Hamcrest

Overview

Truth is similar to AssertJ. An assertion written with either library looks like this:

assertThat(notificationText).contains("testuser@google.com");

Truth differs significantly from Hamcrest. An assertion written with Hamcrest looks like this:

assertThat(notificationText, containsString("testuser@google.com"));

Why create Truth when AssertJ already exists?

The reason is historical: AssertJ didn’t exist when we started Truth. By the time it was created, we’d begun using Truth widely at Google, and we’d made some decisions that would be difficult to retrofit onto AssertJ.

Truth vs. Hamcrest

Because Truth and Hamcrest differ so significantly, we’ll cover only the main points:

Truth vs. AssertJ

Again, the two are very similar. We prefer Truth for its simpler API:

We also usually prefer Truth’s failure messages (though we find AssertJ’s to often be similar and, in some cases we’re still working on, to even be better).

Additionally, Truth works on Android devices by default, without requiring users to use an older version or import a different class than usual.

Truth vs. AssertJ, more details

Number of assertion methods

AssertJ has more: more classes of assertions (AssertJ, Truth) and more methods per class (AssertJ, Truth).

It’s easy to understand how every extra feature can be a good thing. We have found, though, that more is not always better:

Failure messages

We prefer Truth’s (in most cases, at least). Here’s an example:

value of    : projectsByTeam().valuesForKey(corelibs)
missing (1) : truth
───
expected    : [guava, dagger, truth, auto, caliper]
but was     : [guava, auto, dagger, caliper]
multimap was: {corelibs=[guava, auto, dagger, caliper]}
  at com.google.common.truth.example.DemoTest.testTruth(DemoTest.java:71)

This is similar to AssertJ’s message:

java.lang.AssertionError:
Expecting:
  <["guava", "auto", "dagger", "caliper"]>
to contain exactly in any order:
  <["guava", "dagger", "truth", "auto", "caliper"]>
but could not find the following elements:
  <["truth"]>
  at com.google.common.truth.example.DemoTest.testTruth(DemoTest.java:71) <19 internal calls>

But note a few differences:

We can see other differences by making other assertions. For example, compare AssertJ…

org.junit.ComparisonFailure: expected:<...        <version>0.4[5</version>
        <scope>test</scope>
        <exclusions>
          <exclusion>
            <!-- use the guava we're building. -->
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
          </exclusion>
        </exclusions>
      </dependency>
      <dependency>
        <groupId>com.google.truth.extensions</groupId>
        <artifactId>truth-java8-extension</artifactId>
        <version>0.45]</version>
        <...> but was:<...        <version>0.4[4</version>
        <scope>test</scope>
        <exclusions>
          <exclusion>
            <!-- use the guava we're building. -->
            <groupId>com.google.guava</groupId>
            <artifactId>guava</artifactId>
          </exclusion>
        </exclusions>
      </dependency>
      <dependency>
        <groupId>com.google.truth.extensions</groupId>
        <artifactId>truth-java8-extension</artifactId>
        <version>0.44]</version>
        <...>

…to Truth…

diff:
    @@ -7,7 +7,7 @@
           <dependency>
             <groupId>com.google.truth</groupId>
             <artifactId>truth</artifactId>
    -        <version>0.45</version>
    +        <version>0.44</version>
             <scope>test</scope>
             <exclusions>
               <exclusion>
    @@ -20,7 +20,7 @@
           <dependency>
             <groupId>com.google.truth.extensions</groupId>
             <artifactId>truth-java8-extension</artifactId>
    -        <version>0.45</version>
    +        <version>0.44</version>
             <scope>test</scope>
             <exclusions>
               <exclusion>

Or compare AssertJ…

java.lang.AssertionError:
Expecting:
  <[year: 2019
month: 7
day: 15
]>
to contain exactly in any order:
  <[year: 2019
month: 6
day: 30
]>
elements not found:
  <[year: 2019
month: 6
day: 30
]>
and elements not expected:
  <[year: 2019
month: 7
day: 15
]>

…to Truth…

value of:
    iterable.onlyElement()
expected:
    year: 2019
    month: 6
    day: 30

but was:
    year: 2019
    month: 7
    day: 15

Also note that the exception thrown by Truth is a ComparisonFailure (useful for IDEs) in both of the last two cases, not just one of the two as with AssertJ.

Platform support (Android, GWT)

Both libraries support Android. However, to use AssertJ on Android, you must fall back to AssertJ 2.x, and you can’t use “soft assertions.”

Truth supports Android in its main Truth class, and its equivalent to soft assertions (Expect) works under Android. Truth also supports GWT.

Both libraries have third-party extensions for Android types: AssertJ has AssertJ-Android (which is deprecated), and Truth has AndroidX Test.

Puzzlers

In our years developing Truth, we have found that even the most “obvious” APIs can turn out to be misused, especially when they’re used by many tests across many projects by many developers. We’re fortunate to have enough users that we can justify digging deeply into the design of those APIs. (Here’s an example.) As we do so, we’re also fortunate to have the tools to search our codebase and run other teams’ tests.

Based on our experiences, we present some AssertJ puzzlers, which we have designed Truth to avoid:

assertThat(uniqueIdGenerator.next()).isNotSameAs(uniqueIdGenerator.next());
assertThat(primaryColors).containsAll(RED, YELLOW, BLUE);
assertThat(Longs.tryParse(s)).isEqualTo(parsedValues.get(s));
assertThat(event.getText())
    .usingComparator(comparing(Object::toString))
    .contains("2 suggestions");
assertThat(defaults).has(new Condition<>(x -> x instanceof String, "a string"));

Each of these behaves differently than the reader might expect. See if you can figure them out, and then have a look at the puzzler answers.

Writing your own assertion methods

Both support this.

AssertJ is more verbose overall, including (at least by convention) an abstract superclass, verbose generics, and return this; at the end of each method. AssertJ also requires you to format failure messages yourself.

While Truth has some boilerplate of its own, including a method that returns a Subject.Factory (generally implemented as a method reference) and an actual field, it is usually less. Also, Truth supplies convenience methods to format failure messages.

(To be fair, AssertJ reduces the up-front cost of verbosity by offering an assertions generator.)

Puzzler answers

(If you want to try to figure these out on your own, head back up to view the puzzlers without the answers.)

assertThat(uniqueIdGenerator.next()).isNotSameAs(uniqueIdGenerator.next());

This looks like it tests that each call to next() returns a different long. However, it actually tests that each call returns a long that autoboxes to a distinct instance of Long. Under a typical implementation of Java, this test would pass even if next() were implemented as return 12345; because Java will create a new Long instance after each invocation.

Truth reduces the chance of this bug by naming its method “isNotSameInstanceAs.”

assertThat(primaryColors).containsAll(RED, YELLOW, BLUE);

This looks like it tests that the primary colors are defined to be red, yellow, and blue. However, it actually tests that the primary colors include red, yellow, and blue, along with possibly other colors.

Truth reduces the chance of this bug by naming its method “containsAtLeast.”

assertThat(Longs.tryParse(s)).isEqualTo(parsedValues.get(s));

This looks like it tests that the given string parses to the expected value. However, if parsedValues is a Map<String, Integer> (perhaps because it’s shared with the tests of Ints.tryParse), then the test will always fail because a Long is not equal to an Integer.

Truth reduces the chance of type-mismatch bugs by treating a Long as equal to its equivalent Integer.

assertThat(event.getText())
    .usingComparator(comparing(Object::toString))
    .contains("2 suggestions");

This looks like it tests that the List<CharSequence> returned by getText() contains an element with content “2 suggestions.” However, the Comparator passed to usingComparator does not affect the contains call. (It affects only calls like isEqualTo.) To apply a Comparator to contains and other methods that operate on individual elements, AssertJ has a separate method, usingElementComparator.

Truth avoids this problem by not permitting arbitrary assertions with a Comparator.

assertThat(defaults).has(new Condition<>(x -> x instanceof String, "a string"));

This looks like it tests that the defaults array contains a string. However, it actually tests that defaults is itself a string.

Truth avoids this problem by omitting support for Condition-style assertions (except by using Correspondence, which is exposed only for assertions on collection elements).

A case for AssertJ

While we prefer Truth, we acknowledge that others may prefer AssertJ. Here are some cases in which AssertJ offers advantages:

Some more similarities

Failure reporting

In addition to standard, fast-fail assertions, Truth and AssertJ both support:

Truth also supports custom FailureStrategy implementations. This support also underlies its utility for testing user-defined assertion methods.

Note that AssertJ’s soft assertions have some limitations:

These seem fixable in principle, but they demonstrate some of the reasons that we chose to make FailureStrategy a fundamental part of Truth.

On the other hand, AssertJ’s soft assertions let you divide a test into multiple groups of soft assertions. Truth does not support this.

Library support

Truth and AssertJ both support Guava types. Truth includes them in its main artifact and main Truth class; AssertJ is more modularized, offering a separate artifact.