In the previous post we improved the output handling of the test runner, making output optional by using an abstract progress printer. Only our assert_equals function, which by the way should also be moved to the test_framework module, still prints directly to stdout:

function assert_equals(expected, actual, epsilon) result(assertion_failed)
   ! ... 
   assertion_failed = .false.

   if (abs(actual - expected) > epsilon) then
      print *, 'Actual: ', actual
      print *, 'Expected: ', expected
      print *, 'Epsilon: ', epsilon
      print *, 'Reals are not equal'
      assertion_failed = .true.
   end if
end function assert_equals

To fix this, we should upgrade the return type. One idea would be to return a message (variable-length string) instead of a logical, but a more flexible, future-proof alternative is to define a custom type for an error. This would allow us to pass more information back to the caller than just the message. A suitable name would be assertion_failed_t:

In the previous post we’ve successfully split the running of the tests and collecting the test results from handling those results by printing the counters and error stop-ping the test program. We still ask each test procedure to print a small description. We do this to assist the programmer when there is some kind of crash during the test run: they need to be able to find out which test caused it.

Maybe we’ve been writing more “generic” code for our custom test program than expected. We have been gradually, and quite safely, extracting code from a simple “check program”. But now that we’re looking at a pretty generic test runner that we’re going to use for all our future unit tests, we inevitably have to address the concern that we’re writing code that isn’t tested itself. Who tests the test framework? Well, the test framework itself has all the ingredients. However, testing the framework requires that we can run its code in isolation: that we can call one of its functions, and assert that it behaves correctly. Unfortunately, that isn’t possible yet. The code still lives in the main block of the tester program, and it doesn’t allow in-memory inspection of the test results, because it only prints them to the terminal, which we can’t unit-test.

In the previous post, we’ve spent some time improving a number of “temporary” tests. We introduced an assertion function to compare two real values, and we prevent failing tests from stopping the entire test program.

Moving tests to their own procedures

The approach of adding tests directly in the main test program block doesn’t really scale, as the saying goes. polyline_length may be a simple function, but for more complicated functions with multiple branches and loops, we’d have to write many of these tests. The test program keeps growing, and eventually it becomes a mess. It doesn’t help that all local variables have to be declared at the top. Even if that wasn’t needed, it isn’t very clear where a test starts or ends. Everything happens in the same scope, potentially becoming a memory management issue too. This also makes it hard to delete tests we no longer want. Removing lines from a big test program likely breaks other tests, or we forget to remove things we no longer need.

How can we know that the function we wrote, works as intended? We could run it, and manually verify its correctness. The simplest way to do this is to call the function in the main program block, print the output, and compare it with what we expect. Say our function calculates the length of a polyline, stored as a two-dimensional array of reals, representing (x,y) coordinates:

pure function polyline_length(coordinates) result(length)
   real(kind=real64), dimension(:, :), intent(in) :: coordinates

   real(kind=real64) :: length

   real(kind=real64) :: distance
   integer :: index

   length = 0.0_real64

   do index = 1, size(coordinates, 1) - 1
      distance = sqrt((coordinates(index, 1) - &
                       coordinates(index + 1, 1))**2 + &
                      (coordinates(index, 2) - &
                       coordinates(index + 1, 2))**2)
      length = length + distance
   end do
end function polyline_length

Temporary test programs

We could modify the main program block of our actual program, but it’s a lot simpler and safer to create a separate “throw-away” test program, with only the code we need. We’d write a short test program that sets up some coordinates, calls the function, then prints the result:

We’ve encountered several ways of designing functions in a way that allows them to fail for some reason, without stopping the program, or making it otherwise risky or awkward to use the function. We introduced the error_t type which is very flexible. It can be used to provide some information to the caller, helping them understand what went wrong and how it can be fixed. By allowing errors to be wrapped inside others, we can create chains of errors that describe the problem at various abstraction levels. It gives back control to the user: how do they want to deal with an error? Would they like to try something else? Or, in the end, should we just stop trying and quit te program?

Parsing an array of strings to a polyline should fail if one of the strings could not be parsed to a string. But it should also fail if the resulting array of points is empty. Or at least, if that makes sense in the application’s domain. You could say that a polyline with no points is still a polyline, just like an empty set is still a set. Similarly, in some scenarios it may be okay for a polyline to have just a single point. For now, let’s skip the mathematical discussion and take this as a practical rule that we want to enforce: a polyline has at least 2 points. The type we currently have can’t give us such a guarantee:

Parsing a string to a point may not be successful, and we were able to communicate that to users of parse_point() by returning a custom type called point_or_error_t.

What if we want to parse not just one point but a set of points, forming a polyline, from a list of strings? The result of parsing each string separately is a point or an error. Only if the list of those point_or_error_t values contains no error at all should we consider turning the points into a polyline. If any of the values is an error, we shouldn’t make a polyline of the other valid points, but instead return an error: “Could not create polyline from strings. Previous error: Could not extract two decimal numbers from …” With this error we communicate what we were trying to do at the highest conceptual level (create a polyline), and what lower-level error caused this to fail.

We looked at optionally returning a value from a function. In the case of average it was an average value, or no value at all. In some more elaborate functions, the “no result” case needs some more explanation to the caller of the function. We may feel the need to communicate what went wrong, or for what reason. This gives the user the opportunity to pass better input next time, or fix some external issue that the program can’t fix itself (e.g. a file not being readable).

In the previous post we introduced an optional_real_t type to be the return value of an average function, because depending on the size of the array of numbers passed to it this function may or may not return a real value.

What if we could just prevent people from calling the average function when they don’t have any numbers? Maybe we’re lagging too far behind if we have to verify the size of the array inside the average function.