In the previous post, we’ve spent some time improving a number of “temporary” tests. We introduced an assertion function to compare two real values, and we prevent failing tests from stopping the entire test program.

Moving tests to their own procedures

The approach of adding tests directly in the main test program block doesn’t really scale, as the saying goes. polyline_length may be a simple function, but for more complicated functions with multiple branches and loops, we’d have to write many of these tests. The test program keeps growing, and eventually it becomes a mess. It doesn’t help that all local variables have to be declared at the top. Even if that wasn’t needed, it isn’t very clear where a test starts or ends. Everything happens in the same scope, potentially becoming a memory management issue too. This also makes it hard to delete tests we no longer want. Removing lines from a big test program likely breaks other tests, or we forget to remove things we no longer need.

How can we know that the function we wrote, works as intended? We could run it, and manually verify its correctness. The simplest way to do this is to call the function in the main program block, print the output, and compare it with what we expect. Say our function calculates the length of a polyline, stored as a two-dimensional array of reals, representing (x,y) coordinates:

pure function polyline_length(coordinates) result(length)
   real(kind=real64), dimension(:, :), intent(in) :: coordinates

   real(kind=real64) :: length

   real(kind=real64) :: distance
   integer :: index

   length = 0.0_real64

   do index = 1, size(coordinates, 1) - 1
      distance = sqrt((coordinates(index, 1) - &
                       coordinates(index + 1, 1))**2 + &
                      (coordinates(index, 2) - &
                       coordinates(index + 1, 2))**2)
      length = length + distance
   end do
end function polyline_length

Temporary test programs

We could modify the main program block of our actual program, but it’s a lot simpler and safer to create a separate “throw-away” test program, with only the code we need. We’d write a short test program that sets up some coordinates, calls the function, then prints the result:

Unbalanced test suites

Having spoken to many developers and development teams so far, I’ve recognized several patterns when it comes to software testing. For example:

1. When the developers use a framework that encourages or sometimes even forces them to marry their code to the framework code, they only write functional tests - with a real database, making (pseudo) HTTP requests, etc. using the test tools bundled with the framework (if you’re lucky, anyway). For these teams it’s often too hard to write proper unit tests. It takes too much time, or is too difficult to set up.
2. When the developers use a framework that enables or encourages them to write code that is decoupled from the framework, they have all these nice, often pretty abstract, units of code. Those units are easy to test. What often happens is that these teams end up writing only unit tests, and don’t supply any tests or “executable specifications” proving the correctness of the behavior of the application at large.
3. Almost nobody writes proper acceptance tests. That is, most tests that use a BDD framework, are still solely focusing on the technical aspects (verifying URLs, HTML elements, rows in a database, etc.), while to be truly beneficial they should be concerned about application behavior, defined in a ubiquitous language that is used by both the technical staff and the project’s stakeholders.

Please note that these are just some very rough conclusions, there’s nothing scientific about them. It would be interesting to do some actual research though. And probably someone has already done this.

Fernando Arconada interviewed me about the subject of testing. He is writing a book about testing Symfony2 applications: Testing para Aplicaciones Symfony2. Fernando will translate this interview to Spanish and and add it to his book, together with the articles in my A better PHP testing experience series.

Who is Matthias Noback?

I’m a PHP developer, writer and speaker. I live in Zeist, The Netherlands, with my girlfriend, a son of 9 and our newborn daughter. Currently I have my own business, called Noback’s Office. This really gives me a lot of freedom: I work as a developer on one project for about half of the week and in the remaining time I can either spend some time with my family or write blog posts, or finish my second book.

Apparently not everyone agrees on how to unit test their Symfony2 controllers. Some treat controller code as the application’s “glue”: a controller does the real job of transforming a request to a response. Thus it should be tested by making a request and check the received response for the right contents. Others treat controller code just like any other code - which means that every path the interpreter may take, should be tested.