I’ve been using JUnit for about fifteen years. In that time it has become a central part of Java development. While that’s great, unit testing is still problematic. Most people agree that automated testing is necessary, but its exact form is more controversial.
There are two main candidates for a unit testing framework, JUnit and TestNG. These packages have very different aims – in Senior Developer interviews I would sometimes as the difference between them. I like the question as there are a lot of right answers and the chosen response says a lot about the developer.
The simplest answer is that they are broadly similar. They do have a different order of argument in their assertions, which has led to TestNG having a separate class with JUnit assertion order.Some companies I’ve worked for have solved the problem of using JUnit or TestNG by going for both – and then used both argument orders for TestNG.
There’s a good JUnit/TestNG comparison on Mykong.com which points out that the main difference between the two is that JUnit doesn’t have some of the features included in TestNG. But the best answer (which I never received) is that the philosophy of the two are very different. JUnit is a unit testing framework, and as such does not encourage certain practises, for example, having tests that need to run in a particular order. JUnit 5 Alpha was recently announced and the new features don’t appear to violate this goal.
There are often problems with what unit testing actually is. Unit tests should be small, independent, deterministic and low-level. They should not have any direct dependencies on files, database or underlying OS (random number generation, current time, etc). Tests at higher levels are important too, but these should be clearly separate. By not following these rules, developers risk producing slow, brittle unit tests that require complicated set-up. Good unit tests actually force the code to be written in a more testable way, reducing dependencies, and using smaller elements.
Adding tests to the project is simple enough. There is a new class, GreeterTest, and a few lines in the build.gradle file. The changes to the build include a JUnit dependency, which in turn means adding a link to the central maven repository. Gradle is written so that dependencies can be added with little trouble, but this does bear thinking about a little. How does one know that the files being downloaded have not been tampered with? Are we accessing these repositories efficiently? This will be discussed later in detail later, but meanwhile we need to add a note to the TODO page.
Some years back, I gave a talk on unit testing. It was about an hour, and still only scratched the surface. While unit testing is easy, it introduces a lot of issues around how a project will work. I’ve seen company after company get tangled in unit tests. Simply adding JUnit to a project is not enough without some consistency and a real rigour is needed around how tests are used.
An example with this is the use of set-up methods in the tests. These become complicated, and end up with test subclasses and classes, making fixing tests a chore. Better to have the set-up in each individual test method, and if these become unwieldy then examining our object model. This then means that a broken test can be read from start to finish and understood on a single IDE screen. Yes, it produces duplication, but the aims of test code are very different to production code.
Introducing unit testing also requires the processes and infrastructure to support it. We’ve added Jenkins to the project, and broken unit tests will cause the build to ‘break’, to turn red. Continuous integration relies on tests running quickly, to allow a tight feedback loop for developers. Over time, slow builds become slower, and never quite get fixed. Also, the rules about not committing to a broken build need to be taken seriously. Too often, companies have unit test suites that break in specific ways, and developers are forced to understand when a broken build matters and when it is acceptable. This is far too confusing. Using unit testing means following certain rules and methods of development.
Adding JUnit to Jenkins is relatively simple. Make sure that the JUnit plugin is installed, then edit the build for the project to publish the JUnit results. The build will now fail if the tests fail, and full details can be seen within Jenkins.
The latest commit is a3b7fa1. In the next part, we’ll be looking at adding coverage checking to the project.