About TwiP

"Tests with Parameters" allows you to simply add parameters to your JUnit test methods. TwiP calls such methods with all possible combinations of their parameters... or at least some reasonable subset of commonly failing values in the case of Integers, etc. You can define your own set of values to use and/or you can reduce the values set with an assume expression in an annotation, e.g. ">= 0". Default values exist for the primitive types (int, etc.), their Class wrappers (Integer, etc.), Enums and Strings. For other types or for a different set of values, you can define a static method or field and annotate the parameters of your test method(s), or you can annotate the field or method to inject the values by type.

By using TwiP you change the semantics of your tests from existence to for-all quantifiers, i.e. you specify "all ravens are black" instead of "Abraxas is black", "Toni is black", etc. This moves your tests closer to an executable specification, so TwiP is a very nice endorsement to BDD.


Note that the number of test cases can grow quickly when using TwiP, but the test code usually turns out to be much cleaner: Traditionally you write two methods to test two fixtures. They often differ in just one or two statement. Insead of duplicating the code - which is never a good idea - you can refactor some fixture setup or validation code into utility methods. But it is generally difficult to even find a good name for such methods, which smells like a bad practice. By in contrast adding a boolean argument to your first test method, you can add the difference of your second fixture with a simple if statement. The same thing sometimes works for enums as well: If you want to check that all possible values are handled correctly, you can pass it as an argument. Frequently even the variations in the validation code can be phrased in a very concise way, so this is a much better option over writing one test method for i.e. every possible enum value.

A stunningly convincing example of how concise validation code can be, is the inverse of some mathematical operation. Say you want to test if your multiplication code is correct. Instead of writing hundreds of test methods all of the same pattern with only the numbers different, you can write one method with two integer arguments. The assertion simply checks that dividing the result of your multiplication by one argument returns the other.

Of course, such a direct inversion of an operation is not allways available. Then you'll have to resort to other means, primarily if and switch statements; mainly for the validation of the tests. But it allows you to directly prove that some business rule holds true for all value combinations you pass in. And you'll find that the main point is not to think about discrete test setups but about the range of values you can establish an assertion for. For example you can test if all your DSL files adhere to the syntax constraints imposed on them. This is very close to what you would write in a specification and more often than not you'll come up with much more concise code than what you'd expected, clearly conceiving the intentions of your tests.

There are other ways to add parameters to your tests, most notably the JUnit Parameterized Runner. Especially the experimental Theories Runner that is built into JUnit since version 4.4 (see here) was a great inspiration for TwiP, but I think TwiP is much easier to grasp and handle than both of them, esp. for primitive types and enums. The theory behind TwiP is the same as that for the Theories runner. And there are even some experimental tools that try to find the optimal set of values to pass to your tests. It would be wonderful if they'd work with TwiP as well.

Note that test runners run TwiP tests by displaying each invocation with their parameters added, separated by dots. But e.g. double clicking the Eclipse runner doesn't work, because it looks for that whole string as a method name; it would instead have to remove the first dot and everything behind to find the test method.

See here for more information.

Other Improvements

As there can only be one runner for any given JUnit test class, TwiP introduces a more flexible extensions mechanism. TwiP brings some extensions. See here for more information.

The Java reflection APIs treat parameters as second class citizens: you can only access them by several methods with an index, and their names are not kept in the class file by default. TwiP tries to fill that gap as good as it goes.

TwiP also enables the native Java assert keyword for your tests... I wonder why nobody uses them... I mean nobody, even JUnit prefers to define its own assertTrue method instead! I think it would help a lot if it simply was on by default, and TwiP does so... for tests at least.

Since version 3.0 TwiP supports a verifyThat method that you can use instead of assertThat. While your test is stoped abruptly when the first negative assertion is reached, all failing verifications are collected and then reported together, so you can see all failures... maybe one indicates the error more directly. It's not about "one assert per test", it's about "one concept per test".