The PHP testing experience: Interview by Fernando Arconada

Matthias Noback

Testing para Aplicaciones Symfony2 Fernando Arconada interviewed me about the subject of testing. He is writing a book about testing Symfony2 applications: Testing para Aplicaciones Symfony2. Fernando will translate this interview to Spanish and and add it to his book, together with the articles in my A better PHP testing experience series.

Who is Matthias Noback?

I’m a PHP developer, writer and speaker. I live in Zeist, The Netherlands, with my girlfriend, a son of 9 and our newborn daughter. Currently I have my own business, called Noback’s Office. This really gives me a lot of freedom: I work as a developer on one project for about half of the week and in the remaining time I can either spend some time with my family or write blog posts, or finish my second book.

What’s your experience with BDD/TDD testing?

As I wrote in the introduction to a series of articles about testing on my blog: I’ve been writing unit tests for my code for a couple of years now. Unit testing really changed the way I work and it has also made the code I deliver much better. Classes that are well-designed are also easily testable (and vice versa). Equally important: unit testing helped me reduce the bug count and the amount of rework a lot.

I don’t always write a test first, though I’m gradually moving towards the test-first practice, since I believe it’s the only way I can guarantee that what I’m building serves the purpose which I or my customer had in mind.

Recently I also started writing scenario’s (for a private project), running them with Behat. I did this after writing the code, which made the process somewhat harder. Making existing code testable is almost always harder than writing testable code in te first place. Still, seeing all those successful steps in each scenario gave me a lot of confidence that the application provides what I need in terms of my own business (it’s a time-tracking, auto-invoicing tool - it’s simple but there’s money involved, so it must always work…).

Any open source project that you are proud of?

I haven’t really worked on any big projects yet, just small libraries. I’m somewhat proud of the Leanpub sampler. You can use it to loop over Leanpub book manuscript files and extract sample text from them. It contains some nice iterators and uses the Aura.Cli command line library.

Should I write tests even for small projects?

The reason you’re asking is - I think - that automated tests are presumed to be a replacement for manual checks. When your project is small, why bother with writing those automated tests? The time you need to invest for them really outweighs the time you would need for quickly checking the behavior of your application using the UI (be it a browser or a command prompt).

In my experience small projects become big projects. If you want to introduce tests for the usual reasons, then you will have a hard time adding tests for existing code (like I mentioned above: making untested production code testable probably takes quite some time). On the other hand writing tests for small projects is particularly easy, much easier than writing tests for larger projects. And at such small cost, if you do write tests for a small project it will give you all the usual advantages of an automated test suite:

  • You can be sure that one change here, doesn’t break the application there.
  • You don’t need to rely on the consistency of your own behavior when you do your manual checks. You can leave it to the computer.

What’s your opinion about PHP projects in general?

Looking at the PHP community today and comparing it to a couple of years ago, I’ve come to the conclusion that PHP developers nowadays care more about open source. The good thing about open source software is: people work really hard to get it right. In particular because the open source code you write may be judged by anybody who looks at your GitHub page. So you make it as good as you can (you may even add some tests!). In reality there is still lots of undocumented, untested rubbish code on GitHub, but in general (although I have no numbers to support my claim), things have been really getting better over time.

Do we need 100% coverage?

No, almost nobody has a 100% unit test coverage. Using a strict TDD workflow, you would automatically have full coverage. But you probably don’t do that. Even if you do believe that TDD is the ideal way of developing code (like I do), still you “won’t feel like it” all the time. So there goes your 100% coverage.

Do you test from the beginning of the project?

No, not always. Though I always think hard about it before I choose not to. Open source code needs tests because people are going to rely on it. My own code needs tests if I don’t trust myself with the subject matter or if (part of) my business depends on it. In other circumstances I may choose not to write tests for my code. Please note however that I almost always regret this decision because not writing tests initially made me go faster, but really slowed me down in the end.

PHPSpec, Behat, Codeception, PHPUnit, which ones do you use?

I mostly use PHPUnit, sometimes also for functional tests. As I mentioned earlier I also use Behat now. I tried Codeception for some time but it “didn’t feel good” (I’ve got no better reason than just this gut feeling). I’ve experimented with PHPSpec a bit, as well as Prophecy, the test double generation tool it uses. In my experience you can easily use PHPUnit like you use PHPSpec, but if you are new to testing, I definitely recommend PHPSpec since it will more easily guide you in the right direction when it comes to unit testing (or “object spec-ing”). Prophecy is actually really nice and I feel that it makes your tests better if you use it. I personally use PHPUnit and the PHPUnit - Prophecy bridge package, which makes it easy to define test doubles with Prophecy in PHPUnit test cases.

Is there any other quality tool that you use?

I use tools to verify the coding standard of a project (i.e. PHP_CodeSniffer). For open source libraries I use Scrutinizer CI which is a continuous integration service for PHP projects. For each commit it gives you some figures about the code quality and ways you can improve it. Over time you can track the state of the project. It offers some good insight in your code and if you follow (most of) its advice, your project ends up being a lot better than before.

Is TDD dead? And BDD?

No, I don’t think that TDD is dead. I think “do everything test-first” is dead (at least when it comes to unit testing), or maybe it was never alive, except in coding dojos.

What started the “TDD is dead” discussion is a post by David Hansson. Though I’m not sure he abandons TDD for all the right reasons, he says some interesting and honest things about his experience with TDD:

[…] At times I got sucked into that fundamentalist vortex, feeling bad about not following the true gospel. […] It was yoyo cycle of pride, when I was able to adhere to the literal letter of the teachings, and a crash of despair, when I wasn’t. It felt like falling off the wagon. Something to keep quiet about. Certainly not something to admit in public.

At first I didn’t even get this. I myself was a pretty strong teacher of the “right way”, which is test-first or strict TDD. After reading just the title of this post I was blinded already by a feeling of strong opposition towards David’s opinions. When my initial revolt was gone, this changed. I got into a Twitter discussion (or “fight”) with Manuel Lemos, foreman of the PHPClasses community, after he wrote the article Why TDD failed to become mainstream. I think the article itself is very dangerous for the PHP community and the community of PHPClasses in particular. However, I agree on one thing. He says (in the spirit of David):

Many people followed the preachers of TDD to later realize that it is not right for them. Still some were afraid to admit it and never contested openly.

I now understand both David and Manuel better. Preaching really is a dangerous thing. Yelling things like “What, you don’t write tests?!” is even more dangerous. For newcomers to testing this can be very intimidating. They will think that they are doing something very wrong. They won’t dare to admit that they don’t write tests themselves, because: who would be such a sinner as to not write any test at all? This is actually one of the shortcomings of PHP’s most famous proponent of unit testing, Chris Hartjes. As the “Grumpy Programmer” he depicts an angry alter-ego on Twitter. The grumpy programmer, or grumpy tester, will be angry at you for not testing your code.

In real life, Chris is a very friendly person (I never met him, but I know this from witness reports) and he has written some useful books about unit testing that sell well, which is great for world-wide test coverage. Still, there is a bit of a dangerous side on the approach of Chris: someone being angry at you for not testing is definitely not a good reason for you to start testing. In fact, you should not be taught to feel ashamed of your lack of testing experience. Instead you should be encouraged to talk about testing, to try it and be mentored by someone. I know Chris would agree with me, which is why I really like a recent tweet by Chris:

Why am I telling you all of this? Well, for several reasons. First, “TDD” does not equal “testing” (did you notice I already made the huge conceptual jump between these two words somewhere in the preceding paragraphs?). This means that TDD can be dead, but at the same time testing can be very much alive (which I think is the case). Anyway, TDD is a technique you need to master as a developer. And more generally, testing is something you definitely need to do as a developer, and you should learn all about it. But you can’t learn about testing from other developers who make you feel ashamed for something you did not yet learn to do (or to do well). The solution? All experienced testers in the world need to become a mentor to someone who is a beginning tester! So here is my call to action:

If you know someone who doesn’t know how to write tests, or doesn’t do it will, teach them.

And if you are that someone:

Find someone else who knows how to write tests better than you and ask them to teach you how they do it.

Do you have any advise for testing newcomers?

We are all just highly accustomed to quickly slapping something together. We somehow prefer the kick of making big jumps without securing our ropes. If we are moderately good at programming, maybe no accidents happen. But only after you learn to write tests first you will recognize all the little accidents you have caused in the past by writing code while constantly refreshing a page in a browser, or adding var_dumps after each line of code. By following this workflow you introduced countless bugs, and invested lots and lots of hours figuring out why something didn’t work. All this debugging was happening at the wrong kind of level (i.e. at the application level), when in fact simply writing some unit tests would have given you smaller amounts of code (“units”) to debug, and you would have been able to fix the problem a lot sooner.

Besides writing unit tests, I highly recommend installing XDebug on your machine. This will immediately stop you from being slowed down by tracing down futile, meaningless mistakes. And then, as I just mentioned: find a mentor to help you become a better tester.

Is there a book, post, article about testing that opened your eyes?

Well, I’ve learned quite a lot from Growing Object-Oriented Software, Guided by Tests. Bridging the Communication Gap: Specification by Example and Agile Acceptance Testing was a very interesting read too. Some of the articles by Robert Martin contain lots of interesting insights about testing.

Do you write tests for your tests? :-P

No, however, depending on the complexity of your fake objects (which is a special kind of test double, it may be wise to add a simple test for them too. They sometimes contain some logic which by itself introduces bugs in test cases. In order to prevent the need for writing tests for tests, don’t use any form of even moderately complex code (like for-, or if-statements) in your test cases. Programming logic is likely to introduce problems (even though I do believe you are quite an advanced programmer!), so a test case is no place for it. Instead, move your code out of the test cases and write some tests for it too. This will allow you to then use that code with confidence in your test cases.

Why do all tests evolve to spaghetti code?

There are two situations in which people tend to write less clean code than they usually do: inside test classes or when they first use a micro-framework. Although this happens for different reasons: people are probably more skilled in writing production code than in writing test code. Tests need structure and refactoring of that structure too, but in my experience you only dare to change the structure of code if you know exactly how something works, in other words: when you feel free to move things around a bit. In general I think people feel more free and at ease when writing production code than writing test code. So this would explain why test code often evolves to spaghetti code.

The remedy is to force yourself to clean up test code, just like you clean up production code, following the same little refactoring steps. It also helps to do some kind of peer review for test code. If the reviewer isn’t able to quickly read your test code and understand what it does, it’s time to refactor it. By the way, this reviewer can also be you, a couple of weeks later.

What do you think about writing tests after writing production code?

Well, this happens all the time and is quite okay. I usually do this myself when I’ve tested the units of code, but not the bigger system of which they are a part. Such tests however are more like integration tests, or functional tests. Writing unit tests after the fact doesn’t really help in my opinion. You will likely feel slowed down because the code may not have been testable in the first place. If you have no unit tests for a particular unit of code, the best time to write unit tests for it is whenever you need to refactor the unit of code (or when you want to add features). Writing a unit test first will give you the confidence required for refactoring the production code. It also ensures that newly added features don’t have any unforeseen influence on existing features.