We’ve seen how to define a derived type to represent a point in 2D space. We were able to add a type-bound procedure to it, and finally to make its components private and define a custom constructor for it.

In essence there are two categories of derived types:

1. Derived types that hold some data. Their type-bound procedures are solely dealing with this data, and may produce valuable information with it. The point is an example of such a derived type: we can use its data to make geometrical calculations, or we can combine it together with other points into lines, polygons, etc. This all happens in memory. In typical business applications you would consider entities, value objects and data transfer objects (DTOs) to be in this first category.
2. Derived types that can perform some task that deals with things external to the application, like a file, the terminal, a network connection, etc. Such a type doesn’t hold data inside. It is able to fetch data, or send/store data. In business applications such a type is often called a service.

In object-oriented programming, service types usually offer an abstraction. This allows the client to decouple from its specific implementation details. For example, take a logger service; it’s often beneficial for the user that they don’t have to be concerned about where the logs are written to (file, screen, log aggregation server, etc.). The user only wants to call a simple log() function to log something to whatever has been configured at some higher level. Introducing an logger abstraction will take a few posts from here, but it’s good to know that service abstraction is the goal.

In the previous post we have worked on the point_t derived type, turning the module procedure distance into a type-bound procedure:

module geometry
   ! ...

   type :: point_t
      real :: x
      real :: y
   contains
      procedure :: distance => point_distance
   end type point_t

contains

   pure function point_distance(point_1, point_2) result(the_distance)
      class(point_t), intent(in) :: point_1
      
      ! ...
   end function point_distance

end module geometry

Private data components

Type-bound procedures can help us tie relevant behaviors (procedures) to derived types. It also allows us to let the derived type keep its data to itself. This is often called data hiding, or encapsulation of state. Nothing outside the module where the derived type is defined will have (read or write) access to its data components. This can be accomplished by adding the attribute private to each data component:

In the previous post we defined a derived type point_t with real data components x and y, to represent a point in 2D space. We also wrote a function to calculate the distance between two such points:

module geometry
   implicit none(type, external)

   private
   public :: point_t
   public :: distance

   type :: point_t
      real :: x
      real :: y
   end type point_t

contains

   pure function distance(point_1, point_2) result(the_distance)
      type(point_t), intent(in) :: point_1
      type(point_t), intent(in) :: point_2
      real :: the_distance

      the_distance = sqrt((point_2%x - point_1%x)**2 + &
                          (point_2%y - point_1%y)**2)
   end function distance

end module geometry

This allowed us to pass point_t instances as function arguments to distance, in another module or in the program:

We’ve seen types and variables and functions and subroutines. Now we can get to the next level and combine both variables and procedures in a so-called derived type.

Declaring a derived type and its data components

Object-oriented programming languages have classes. It took me some time to understand that classes can be seen as “extensions of the type system”. There are often primitive types like strings, integers, etc. and we can group them as properties of a class. Thereby, the class becomes some kind of advanced, composite type.

So far we’ve put code inside the main program block of our application. Soon the need arises to move blocks of code to a better place: a procedure with a proper name living inside a module which groups related procedures by topic.

Fortran has two types of procedures:

• Functions, with arguments and a return value
• Subroutines, which are functions without a return value

Subroutines

Let’s define a new subroutine in a module:

We’ve looked at the program and module blocks that can hold our code. Now we’ll find out where to put our data.

Declaration comes first

In Fortran, every time we want to store some value in a variable, we have to explicitly declare the type and the name of the variable first. This has to happen before any other executable statement, but after imports and other declarations like the implicit none statement:

We have a basic Fortran application and a working environment to edit, compile and run it, thanks to FPM, IFX and Visual Studio Code. Let’s take a look at the structure of our very basic application.

The program keyword

The minimum amount of code for a Fortran executable is this:

program the_program
    ! Do something here
end program the_program

Comment lines start with !. To write a multi-line comment, just repeat ! at the beginning of every line.

program hello_world
   implicit none(type, external)

   print *, 'Hello, world!'
end program hello_world

Since January 2024 I’m working with a smart group of programmers at Deltares. They create and maintain software in the complex “business” domains of hydrodynamics, morphology, water quality and ecology. Their software is used to understand and predict all kinds of phenomena related to water, by which they “Enable Delta life”. And that’s not just for the Netherlands (many of us here live below sea level or close to rivers); the software is used around the world, by governments and businesses.

Get this book on Leanpub

A couple of weeks ago, Tomas Votruba emailed me saying that he just realized that we hadn’t published an update of the book we wrote together since December 2021. The book I’m talking about is “Rector - The Power of Automated Refactoring”. Two years have passed since we published the current version. Of course, we’re all very busy, but no time for excuses - this is a book about keeping projects up-to-date with almost no effort… We are meant to set an example here!

All of a sudden it became book-writing season. It began in August when I started revising my second book, “Principles of Package Design”. Apress had contacted me about adopting it, and they didn’t want to change a lot about it. However, the book was from 2015 and although I had aimed for it to be “timeless”, some parts needed an update. Furthermore, I had happily pressed the “Release” button back then, but it’s the same as with software development: the code you wrote last year, you wouldn’t approve of today.

Today I’m happy to release my latest book, “Microservices for everyone”! 90% of it was done in July but you know what happens with almost-finished projects: they remain almost-finished for a long time. I did some useful things though, like preparing a print edition of the book.

Foreword

Finally, and I’m very proud of that, I received Vaughn Vernon’s foreword for this book. You may know Vaughn as the author of “Implementing Domain-Driven Design”. It made a lot of sense to ask him to write the foreword, since many of the ideas behind microservice architecture can be traced back to messaging and integration patterns, and Domain-Driven Design. Vaughn is an expert in all of these topics, and it was only to be expected of him to write an excellent foreword.

During the last few days I’ve been finishing my latest book, Microservices for everyone. I’ve used the Leanpub self-publishing platform again. I wrote about the process before and still like it very much. Every now and then Leanpub adds a useful feature to their platform too, one of which is an export functionality for downloading a print-ready PDF.

Previously I had to cut up the normal book PDF manually, so this tool saved me a lot of work. Though it’s a relatively smart tool, the resulting PDF isn’t completely print-ready for all circumstances (to be honest, that would be a bit too much to ask from any tool of course!). For example, I wanted to use this PDF file to publish the book using Amazon’s print-on-demand self-publishing service CreateSpace, but I also wanted to order some copies at a local print shop (preferably using the same source files). In this post I’d like to share some of the details of making the print-ready PDF even more print-ready, for whomever may be interested in this.

I’m happy to share with you the first chapter of my new book, Microservices for everyone. I’m still in the process of writing it and intend to release parts of it during the next weeks. If you’re interested, sign up on the book’s landing page and receive a 25% discount when the first chapter gets released.

If you’re curious about the table of contents so far: you’ll find it in the PDF version of the introduction.

TLDR;

• I won’t update A Year With Symfony anymore.
• The e-book is now freely available.

My first book, A Year With Symfony, was well received when I first published it on September 4th. At the time it seemed to be a welcome addition to the official Symfony documentation.

A lot of water has passed under the bridge since then. Several new Symfony components were released and we have now arrived at the next major version, 3.0. In the meantime I published one update of the book: a new chapter about Annotations.

During the great PHP Benelux Conference 2015 edition I published the final version of Principles of Package Design. Let me tell you a little bit about its history and some of the things I had to deal with.

A bit of history

My first book, A Year With Symfony, was released on September 4th, 2013. Just 6 days later, I started working on Principles of PHP Package Design. As it turned out, it was quite hard to keep the process going. There were several months in which I wrote nothing at all. Sometimes I picked up the work, but then I had completely lost my track and tried to get back on it by revising existing chapters over and over again. Meanwhile all kinds of other projects begged for attention as well, including the release of the backend project of the new nos.nl website, the preparations for the Hexagonal architecture training and the birth of our daughter Julia ;)

As a follow-up on a previous article I have some announcements to make.

Feedback

A wonderful number of 67 of you have provided very valuable feedback about “A Year With Symfony” - you sure wrote some nice things about my book! Just a small selection of highlights:

Reading “A Year With Symfony” helped to clarify and validate the ideas and structures that I had been developing with my team, along with explaining some of the internals of Symfony that the documentation did not.

Almost a year ago, on September 4th, in front of a live audience, I published my first book “A Year With Symfony”. Since the first release, well over a 1000 people have read it. In February of this year, I added a new chapter to the book (about annotations).

A third edition?

Flipping through the pages and thinking about how I could best celebrate 1 year and 1000+ readers, I had the following ideas:

Seven months, two presentations and three blog posts later, I’ve published the first installment of my new book, Principles of PHP Package Design.

From the introduction of the book:

Naturally the biggest part of this book covers package design principles. But before we come to that, we first take a close look at what constitutes packages: classes and interfaces. The way you design them has great consequences for the characteristics of the package in which they will eventually reside. So before considering package design principles themselves, we first need to take a look at the principles that govern class design. These are the so-called SOLID principles. Each letter of this acronym stands for a different principle, each of which we will briefly (re)visit in the first part of this book.

On several occasions I have tried to explain my opinion about “optional dependencies” (also known as “suggested dependencies” or “dev requirements”) and I’m doing it again:

There’s no such thing as an optional dependency.

I’m talking about PHP packages here and specifically those defined by a composer.json file.

What is a dependency?

First let’s make sure we all agree about what a dependency is. Take a look at the following piece of code:

Last week was a particularly interesting week. I organised three coding dojos, on three different locations. The first one was at the IPPZ office (one of the companies I’ve worked for), the second one was at the SweetlakePHP meetup in Zoetermeer. The last one was for a group of developers from NOS, a large Dutch news organization. Although that would have been enough, I finished the week with my revised talk “Principles of PHP Package Design” at a meeting of the Dutch Symfony Usergroup in Utrecht…

My first book, A Year With Symfony, has been available since September 4th last year. That’s not even six months ago, but right now it has almost 800 readers already (digital and printed edition combined).

During the past few days I’ve taken the time to write an extra chapter for the book. Consider it as a big thank you to everybody who bought the book! I feel very much supported by the Symfony/PHP community and this really keeps me going.

About the nature of PHP and its misuse in package design

This text will be part of the introduction of my upcoming book Principles of PHP Package Design. If you’d like to be notified when the book is released, you can subscribe on the book’s page. This will entitle you to a 20% discount once the book is available for download.

Before we continue with the “main matter” of this book, I’d like to introduce you to the mindset I had while writing it. This particular mindset amounts to having a constant awareness that PHP has somewhat of a dark side, as well as keeping our hopes high that it is very well possible to expose this “dark side” to a warm sun of programming principles, coding standards and generally good ideas, collected over the years, carefully nurtured and enhanced by great programmers of different languages and cultures.

Last year in September Len Epp from Leanpub.com interviewed me about my book A Year With Symfony. They have fully transcribed the interview as well as published the recording of the interview.

This is the first time somebody interviewed me about my career and my book writing. I remember back then it made me quite nervous. But I must say that talking with Len was a very good way to shed some light on my own thought process and my personal goals. For instance, who did I have in mind as the reader of my book?

Recently I tweeted about phpclasses.org. It was not such a friendly statement:

Why does phpclasses.org still exist? Most of the “packages” contain dangerous, stupid or useless code.

Manuel Lemos, the man behind PHP Classes, made me pull back a bit by pointing out that I was generalizing and that they do what they can to encourage people to do a good job. I recognize their effort. And of course, there is also good code on phpclasses.org.

A couple of weeks ago the reader count of my first book A Year With Symfony reached the number 365. It seemed to me an appropriate moment to write something about my experiences. In this post, I will not be too humble, and just cite some people who wrote some very nice things about my book on Twitter.

The book was highly anticipated ever since I first wrote something about it on Twitter. Leanpub book pages have a nice feature where users can subscribe to news about the book release.

After working on my book “A Year With Symfony” for four months, I published it yesterday at the Dutch Symfony Usergroup meetup in Amsterdam. It was really great to be able to do this real-time while everybody in the room was looking at the screen waiting for the build process to finish.

The book is hosted on Leanpub, which is a great platform for writing and publishing books. They provide the entire infrastructure. You get a good-looking landing page and they handle all things related to payment.

As you may have heard, I’m working on a book for Symfony2 developers. Besides The Book, which is the user documentation for building Symfony applications, some book-like websites and just a few e-books, there are currently no books for intermediate or advanced Symfony developers. My new book, called A Year With Symfony is exactly that.

If you have been reading the documentation and maybe some blog posts, you will want to know how you can get one step further. You wonder how you could structure your application, for it to be maintainable for more than a month. You ask yourself what it would take to write good bundles, which guidelines you and your team should define and follow. You may also question the built-in security measurements and you or your manager would like to know what needs to built on top of them to make a real secure application. A Year With Symfony will help you find an answer to all these questions.

Get this book on Leanpub

A couple of weeks ago, Tomas Votruba emailed me saying that he just realized that we hadn’t published an update of the book we wrote together since December 2021. The book I’m talking about is “Rector - The Power of Automated Refactoring”. Two years have passed since we published the current version. Of course, we’re all very busy, but no time for excuses - this is a book about keeping projects up-to-date with almost no effort… We are meant to set an example here!

It’s quite ironic that my most “popular” tweet has been posted while Twitter itself is in such a chaotic phase. It’s also quite ironic that I try to provide helpful suggestions for doing a better job as a programmer, yet such a bitter tweet ends up to be so popular.

Twitter and Mastodon are micro-blogging platforms. The problem with micro-blogs, and with short interactions in general, is that everybody can proceed to project onto your words whatever they like. So at some point I often feel the need to explain myself with more words, in an “actual” blog like this one.

Refactoring without tests should be fine. Why is it not? When could it be safe?

From the cover of “Refactoring” by Martin Fowler:

Refactoring is a controlled technique for improving the design of an existing code base. Its essence is applying a series of small behavior-preserving transformations, each of which “too small to be worth doing”. However the cumulative effect of each of these transformations is quite significant.

Software development seems to be about change: the business changes and we need to reflect those changes, so the requirements or specifications change, frameworks and libraries change, so we have to change our integrations with them, etc. Changing the code base accordingly is often quite painful, because we made it resistant to change in many ways.

Code that resists change

I find that not every developer notices the “pain level” of a change. As an example, I consider it very painful if I can’t rename a class, or change its namespace. One reason could be that some classes aren’t auto-loaded with Composer, but are still manually loaded with require statements. Another reason could be that the framework expects the class to have a certain name, be in a certain namespace, and so on. This may be something you personally don’t consider painful, since you can avert the pain by simply not considering to rename or move classes.

During a workshop we were discussing the concept of a Data Transfer Object (DTO). The main characteristic of a DTO is that it holds only primitive-type values (strings, integers, booleans), lists or maps of these values including “nested” DTOs. Not sure who came up with this idea, but I’m using it because it ensures that the DTO becomes a data structure that only enforces a schema (field names, the expected types, required fields, and optional fields), but doesn’t enforce semantics for any value put into it. That way it can be created from any data source, like submitted form values, CLI arguments, JSON, XML, Yaml, and so on. Using primitive values in a DTO makes it clear that the values are not validated. The DTO is just used to transfer or carry data from one layer to the next. A question that popped up during the workshop: can we consider DateTimeImmutable a primitive-type value too? If so, can we use this type inside DTOs?

A common misunderstanding in my workshops (well, whose fault is it then? ;)), is about the distinction between a DTO and a value object. And so I’ve been looking for a way to categorize these objects without mistake.

What’s a DTO and how do you recognize it?

A DTO is an object that holds primitive data (strings, booleans, floats, nulls, arrays of these things). It defines the schema of this data by explicitly declaring the names of the fields and their types. It can only guarantee that all the data is there, simply by relying on the strictness of the programming language: if a constructor has a required parameter of type string, you have to pass a string, or you can’t even instantiate the object. However, a DTO does not provide any guarantee that the values actually make sense from a business perspective. Strings could be empty, integers could be negative, etc.

When you’re learning to write custom rules for PHPStan or Rector, you’ll have to learn more about the PHP programming language as well. To be more precise, about the way the interpreter parses PHP code. The result of parsing PHP code is a tree of nodes which represents the structure of the code, e.g. you’ll have a Class definition node, a Method definition node, and within those method Statement nodes, and so on. Each node can be checked for errors (with PHPStan), or automatically refactored in some way (with Rector).

Having discussed different aspects of simplicity in programming solutions, let’s start with the first topic that should be scrutinized regarding their simplicity: persisting model objects. As you may know, we have competing solutions which fall into two categories: they will follow either the Active Record (AR) or the Data Mapper pattern (DM) (as described in Martin Fowler’s “Patterns of Enterprise Application Architecture”, abbrev. PoEAA).

Active record

How do we recognize the AR pattern? It’s when you instantiate a model object and then call save() on it:

“As I’m becoming a more experienced programmer, I tend to prefer simple solutions.” Or something similar. As is the case with many programming-related quotes, this is somewhat of a blanket statement because who doesn’t prefer simple solutions? To make it a powerful statement again, you’d have to explain what a simple solution is, and how you distinguish it from not-so-simple solutions. So the million-dollar question is “What is a simple solution?”, and I’ll answer it now.

By request: what’s my workflow for writing books? Steps, tools, etc.

Writing with the Leanpub platform

A long time ago I noticed that testing-advocate Chris Hartjes published his books on Leanpub. When I had the idea of writing a book about the Symfony framework, I tried this platform and it was a good match. You can write your book in Markdown, commit the manuscript to a GitHub repository, and push the button to publish a new version. Leanpub will generate EPUB and PDF versions for you. Readers can be updated about releases via email or a notification on the website.

When to use a trait? Never.

Well, a trait could be considered to have a few benefits:

Benefits

1. If you want to reuse some code between multiple classes, using a trait is an alternative for extending the class. In that case the trait may be the better option because it doesn’t become part of the type hierarchy, i.e. a class that uses a trait isn’t “an instance of that trait”.
2. A trait can save you some manual copy/paste-ing by offering compile-time copy/paste-ing instead.

Downsides

On the other hand, there are several problems with traits. For instance:

This article shows an example of framework decoupling. You’ll find a more elaborate discussion in my latest book, Recipes for Decoupling.

Why would it be nice to decouple your user model from the framework’s security user or authentication model?

Reason 1: Hexagonal architecture

I like to use hexagonal architecture in my applications, which means among other things that the entities from my domain model stay behind a port. They are never exposed to, for instance, a controller, or a template. Whenever I want to show anything to the user, I create a dedicated view model for it.

This article is about a topic that didn’t make the cut for my latest book, Recipes for Decoupling. It still contains some useful ideas I think, so here it is anyway!

DateTimeImmutable is mutable

I don’t know where I first heard it, but PHP’s DateTimeImmutable is not immutable:

<?php

$dt = new DateTimeImmutable('now');
echo $dt->getTimestamp() . "\n";

$dt->__construct('tomorrow');
echo $dt->getTimestamp() . "\n";

The result:

1656927919
1656972000

Indeed, DateTimeImmutable is not really immutable because its internal state can be modified after instantiation. After calling __construct() again, any existing object reference will have the modified state as well. But an even bigger surprise might be that if a constructor is public, it’s just another method that you can call. You’re not supposed to, but you can. Which is another reason to make the constructor of non-service objects private and add a public static constructor to a class that you want to be immutable:

My new book Recipes for Decoupling is 100% complete and available now!

And now some other news related to this book.

A little background information

My new book is based on two things: 20 years of experience with (mostly framework) coupling issues in legacy code, and the hypothesis that PHPStan, the automated static analysis tool for PHP, can help us keep our code decoupled. Decoupling often means we want to use a dependency, but don’t want to couple our code too tightly to it. The process of decoupling often involves some kind of rule, like “don’t use Container::get()”, but we don’t want to focus on this rule all the time, or explain it to new members of the team. Instead, we want a tool that shows you an error when you don’t follow the rule. Such an error will break the build of the project and prevent you from ever coupling to a dependency in this specific way again.

I was tweeting something about having separate “DDD” and “ORM” entities in a project in a project, and that I don’t understand this. There were some great comments and questions, thanks a lot for that! To be honest, I understand more about it now. In this article I’ll try to provide some more information about this.

I’m glad many developers want to use input from the book “Domain-Driven Design” by Eric Evans to improve their domain model. I recommend reading this book and getting your information from the source, because unfortunately the internet, tweets, e-books, including my own books, aren’t able to reflect a full, nor a correct view of everything there is to find out about this topic. All too often DDD is completely misinterpreted to be “an elitist, dogmatic approach to programming, where we use DTOs, layers, and CQRS”…

Years ago my co-worker Maurits introduced me to the term “magic” in programming. He also provided the valuable dichotomy of convention and configuration (or in fact, he’d choose configuration over convention…). I think this distinction could be very helpful in psychological research, figuring out why some people prefer framework X over framework Y. One requires the developer to spell out everything they want in elaborate configuration files, the other relies on convention: placing certain files with certain names and certain methods in certain places will make everything work “magically”.

Last year I had a video call with Tomas Votruba, creator of Rector, who kindly took the time to explain a lot of things about this project. We finished the call and I couldn’t wait to tell my partner how nice it was. I said to her: we should have recorded it, I’m sure it would be useful for other people too. She replied: this is so typical; millennials having a nice conversation and then they want to let the world know about it.

The one simple trick to do a better job as a programmer is to git commit as if your commit could be accidentally deployed (and it wouldn’t break the production environment…)

Why would this improve your work? Because it pushes for improvements in several areas.

If a commit needs to leave the project in a working state, you need to:

1. Be able to prove that after making your changes, the project is in a working state. This forces you to put automated tests in place.
2. Be able to make changes that don’t break existing code, or that only add new features. This forces you think about first improving the existing code design to allow for your change to happen.

To practice with the latter, I recommend learning about the Mikado method. It makes it easier to recognize prerequisites for the change you want to make, and then forces you to enable the real goals by implementing the prerequisites first.

Paradigm; a nice word that means “a theory or a group of ideas about how something should be done, made, or thought about” (Merriam-Webster). In software development we have them too. From the philosophy and history of science courses I’ve followed, I remember that scientists working with different paradigms have great difficulty understanding each other, and appreciating each other’s work.

Paradigm Shifts

An example of a paradigm is the theory that the sun revolves around the earth. To a certain extent this is a fruitful theory, and it has been used for thousands of years. There’s of course another paradigm: the theory that the earth revolves around the sun. This is also a fruitful theory, and it can be used to explain a lot of observations, more than the previous theory. Still, people got angry with each other for moving the earth out of the center of the universe. Paradigm changes, or shifts, occur when the old theory has been stretched too much. It becomes impossible to hold on to it. Then some people start to experiment with a completely different paradigm, one that sounds totally weird, but in the end proves to have more power.

It’s an extremely common problem in legacy code bases: a new way of doing things was introduced before the team decided on a way to get the old thing out.

Famous examples are:

• Introducing Doctrine ORM next to Propel
• Introducing Symfony FrameworkBundle while still using Zend controllers
• Introducing Twig for the new templates, while using Smarty for the old ones
• Introducing a Makefile while the rest of the project still uses Phing

And so on… I’m sure you also have plenty examples to add here!

I’ve mentioned this several times without explaining: the rule that every class should have a test, or that every class method should have a test, does not make sense at all. Still, it’s a rule that many teams follow. Why? Maybe they used to have a #NoTest culture and they never want to go back to it, so they establish a rule that is easy to enforce. When reviewing you only have to check: does the class have a test? Okay, great. It’s a bad test? No problem, it is a test. I already explained why I think you need to make an effort to not just write any test, but to write good tests (see also: Testing Anything; Better Than Testing Nothing?). In this article I’d like to look closer at the numbers: one class - one test.

I’m a big fan of the BDD Books by Gáspár Nagy and Seb Rose, and I’ve read a lot about writing and improving scenarios, like Specification by Example by Gojko Adzic and Writing Great Specifications by Kamil Nicieja. I can recommend reading anything from Liz Keogh as well. Trying to apply their suggestions in my development work, I realized: specifications benefit from good writing. Writing benefits from good thinking. And so does design. Better writing, thinking, designing: this will make us do a better job at programming. Any effort put into these activities has a positive impact on the other areas, even on the code itself.

In essence, everything is a string.

Well, you can always go one layer deeper and find out what a string really is, but for web apps I work on, both input data and output data are strings. The input is an HTTP request, which is a plain-text message that gets passed to the web server, the PHP server, the framework, and finally a user-land controller. The output is an HTTP response, which is also a plain-text message that gets passed to the client. If my app needs the database to load or store some data, that data too is in its initial form a string. It needs to be deserialized into objects to do something and later be serialized into strings so we can store the results.

“Yes, I know. Our tests aren’t perfect, but it’s better to test anything than to test nothing at all, right?”

Let’s look into that for a bit. We’ll try the “Fowler Heuristic” first:

One of my favourite (of the many) things I learned from consulting with Martin Fowler is that he would often ask “Compared to what?”

• Agile helps you ship faster!
• Compared to what?

[…]

Often there is no baseline.

Whenever I read a test method I want to understand it without having to jump around in the test class (or worse, in dependencies). If I want to know more, I should be able to “click” on one of the method calls and find out more.

I’ll explain later why I want this, but first I’ll show you how to get to this point.

As an example, here is a test I encountered recently:

Several years ago, when the PHP-FIG (PHP Framework Interop Group) created its first PSRs (PHP Standard Recommendations) they started some big changes in the PHP ecosystem. The standard for class auto-loading was created to go hand-in-hand with the then new package manager Composer. PSRs for coding standards were defined, which I’m sure helped a lot of teams to leave coding standard discussions behind. The old tabs versus spaces debate was forever settled and the jokes about it now feel quite outdated.

TLDR;

Rector - The Power of Automated Refactoring is now 100% completed

Tomas Votruba and I first met a couple of years ago at one of my favorite conferences; the Dutch PHP Conference in Amsterdam (so actually, we’re very close to our anniversary, Tomas!). He presented Rector there and it was really inspiring. A year later I was working on a legacy migration problem: our team wanted to migrate from Doctrine ORM to “ORM-less”, with handwritten mapping code, etc. I first tried Laminas Code, a code generation tool, but it lacked many features, and also the precision that I needed. Suddenly I recalled Rector, and decided to give it a try. After some experimenting, everything worked and I learned that this tool really is amazingly powerful!

@ediar asked me on Twitter if I still think a constructor should not be tested. It depends on the type of object you’re working with, so I think it’ll be useful to elaborate here.

Would you test the constructor of a service that just gets some dependencies injected? No. You’ll test the behavior of the service by calling one of its public methods. The injected dependencies are collaborating services and the service as a whole won’t work if anything went wrong in the constructor.

In October 2020 I asked Tomáš Votruba, the mastermind behind Rector, if we could have a little chat about this tool. I wanted to learn more about it and had spent a couple of days experimenting with it. Tomáš answered all my questions, which was tremendously valuable to me personally. When this happens I normally feel the need to share: there should be some kind of artefact that can be published, so others can also learn about Rector and how to extend it based on your own refactoring needs.

I recently heard this interesting question: if your project uses a static analysis tool like PHPStan or Psalm (as it should), should the tests by analysed too?

The first thing to consider: what are potential reasons for not analysing your test code?

Why not?

1. Tests are messy in terms of types

Tests may use mocks, which can be confusing for the static analyser:

$object = $this->createMock(MockedInterface::class);

The actual type of $object is an intersection type, i.e. $object is both a MockObject and a MockedInterface, but the analyser only recognizes MockObject. You may not like all those warnings about “unknown method” calls on MockObject $object so you exclude test code from the analysis.

This article is an excerpt from my book Advanced Web Application Architecture. It contains a couple of sections from the conclusion of Part I: Decoupling from infrastructure.

This chapter covers:

• A deeper discussion on the distinction between core and infrastructure code
• A summary of the strategy for pushing infrastructure to the sides
• A recommendation for using a domain- and test-first approach to software development
• A closer look at the concept of “pure” object-oriented programming

Core code and infrastructure code

In Chapter 1 we’ve looked at definitions for the terms core code and infrastructure code. What I personally find useful about these definitions is that you can look at a piece of code and find out if the definitions apply to it. You can then decide if it’s either core or infrastructure code. But there are other ways of applying these terms to software. One way is to consider the bigger picture of the application and its interactions with actors. You’ll find the term actor in books about user stories and use cases by authors like Ivar Jacobson and Alistair Cockburn, who make a distinction between:

A lot can happen in 9 years. Back then I was still advocating that you should unit-test your controllers and that setter injection is very helpful when replacing controller dependencies with test doubles. I’ve changed my mind: constructor injection is the right way for any service object, including controllers. And controllers shouldn’t be unit tested, because:

• Those unit tests tend to be a one-to-one copy of the controller code itself. There is no healthy distance between the test and the implementation.
• Controllers need some form of integrated testing, because by zooming in on the class-level, you don’t know if the controller will behave well when the application is actually used. Is the routing configuration correct? Can the framework resolve all of the controller’s arguments? Will dependencies be injected properly? And so on.

The alternative I mentioned in 2012 is to write functional tests for your controller. But this is not preferable in the end. These tests are slow and fragile, because you end up invoking much more code than just the domain logic.

Where should it go?

If you’re one of those people who make a separation between an application and a domain layer in their code base (like I do), then a question you’ll often have is: does this service go in the application or in the domain layer? It sometimes makes you wonder if the distinction between these layers is superficial after all. I’m not going to write again about what the layers mean, but here is how I decide if a service goes into Application or Domain:

This is an excerpt from my book PHP for the Web. It’s a book for people who want to learn to build web applications with PHP. It doesn’t focus on PHP programming, but shows how PHP can be used to serve dynamic web pages. HTTP requests and responses, forms, cookies, and sessions. Use it all to build a CRUD interface and an authentication system for your very first web application.

Chapter 11: Error handling

As a rather unusual pastime for the Saturday night I attended the third Domain-Driven Design Africa online meetup. Thomas Pierrain a.k.a. use case driven spoke about his adaptation of Hexagonal architecture. “It’s not by the book,” as he said, but it solves a lot of the issues he encountered over the years. I’ll try to summarize his approach here, but I recommend watching the full talk as well.

Hexagonal architecture

Hexagonal architecture makes a distinction between the use cases of an application, and how they are connected to their surrounding infrastructure. Domain logic is represented by pure code (in the FP sense of the word), surrounded by a set of adapters that expose the use cases of the application to actual users and connect the application to databases, messages queues, and so on.

You’ve picked a good refactoring goal. You are prepared to stop the project at anytime. Now how to determine the steps that lead to the goal?

Bottom-up development

There is an interesting similarity between refactoring projects, and regular projects, where the goal is to add some new feature to the application. When working on a feature, I’m always happy to jump right in and think about what value objects, entities, controllers, etc. I need to build. Once I’ve written all that code and I’m ready to connect the dots, I often realize that I have created building blocks that I don’t even need, or that don’t offer a convenient API. This is the downside of what’s commonly called “bottom-up development”. Starting to build the low-level stuff, you can’t be certain if you’re contributing to the higher-level goal you have in mind.

Refactoring is often mentioned in the context of working with legacy code. Maybe you like to define legacy code as code without tests, or code you don’t understand, or even as code you didn’t write. Very often, legacy code is code you just don’t like, whether you wrote it, or someone else did. Since the code was written the team has introduced new and better ways of doing things. Unfortunately, half of the code base still uses the old and deprecated way…

Refactoring projects

A common case of refactoring-gone-wrong is when refactoring becomes a large project in a branch that can never be merged because the refactoring project is never completed. The refactoring project is considered a separate project, and soon starts to feel like “The Big Rewrite That Always Fails” from programming literature.

The work happens in a branch because people actually fear the change. They want to see it before they believe it, and review every single part of it before it can be merged. This process may take months. Meanwhile, other developers keep making changes to the main branch, so merging the refactoring branch is going to be a very tedious, if not dangerous thing to do. A task that, on its own, can cause the failure of the refactoring project itself.

Since I’ve been writing a lot about decoupled application development it made sense that one of my readers asked the following question: “Why should we use a framework?” The quick answer is: because you need it. A summary of the reasons:

• It would be too much work to replace all the work that the framework does for you with code written by yourself. Software development is too costly for this.
• Framework maintainers have fixed many issues before you even encountered them. They have done everything to make the code secure, and when a new security issue pops up, they fix it so you can just pull the latest version of the framework.
• By not using a framework you will be decoupled from Symfony, Laravel, etc. but you will be coupled to Your Own Framework, which is a bigger problem since you’re the maintainer and it’s likely that you won’t actually maintain it (in my experience, this is what often happens to projects that use their own home-grown framework).

So, yes, you/we need a framework. At the same time you may want to write framework-decoupled code whenever possible.

This is just a quick post sharing something I was able to figure out after doing some research.

The situation: our application throws exceptions by means of “talking back to the user”. As developer we don’t want to be notified about all these exceptions. They aren’t as important as any other exception that should be considered “critical”. Still, we do want to find these exceptions in the logs, because they can sometimes provide valuable feedback about the usability of the system.

If you want to write applications that are maintainable in the long run, you have to decouple from your framework, ORM, HTTP client, etc. because your application will outlive all of them.

Three simple rules

To accomplish framework decoupling you only have to follow these simple rules:

1. All services should get all their dependencies and configuration values injected as constructor arguments. When a dependency uses IO, you have to introduce an abstraction for it.
2. Other types of objects shouldn’t have service responsibilities.
3. Contextual information should always be passed as method arguments.

Explanations

Rule 1

Following rule 1 ensures that you’ll never fetch a service ad hoc, e.g. by using Container::get(UserRepository::class). This is needed for framework decoupling because the global static facility that returns the service for you is by definition framework-specific. The same is true for fetching configuration values (e.g. Config::get('email.default_sender')).

I write about design rules a lot, but I sometimes forget to:

1. Mention that these rules can’t always be applied,
2. Describe when that would be the case, and
3. Add examples of situations where the rule really doesn’t matter.

The rules should work in most cases, but sometimes need to be “violated”. Which is too strong a word anyway. When someone points out to me that I violated a rule, I’m like: Wow! I violated the rule? I’m so sorry! Let’s fix this immediately. Whereas in practice it should be more like: Yeah, I know that rule, but it makes more sense to follow that other rule here, because […]. In other words, pointing out that a certain rule has been violated should not be a sufficient reason to adhere to that rule. My favorite example is “But that violates SRP!” (Single Responsibility Principle). Whoops, I wouldn’t want to do that! Or would I?

One of my pet peeves is using the database schema to validate data.

Several ways in which this normally happens:

• Specifying a column as “required”, e.g. email VARCHAR(255) NOT NULL
• Adding an index to force column values to be unique (e.g. CREATE UNIQUE INDEX email_idx ON users(email))
• Adding an index for foreign key integrity, including cascading deletes, etc.

Yes, I want data integrity too. No, I don’t want to rely on the database for that.

I wanted to share with you a free chapter from my latest book, “Advanced Web Application Architecture”. I’ve picked Chapter 11, which gives a compact overview of all the design patterns that are useful for structuring your web application in a way that will (almost) automatically make it independent of surrounding infrastructure, including the web framework you use.

Chapter 11 is the first chapter of Part II of the book. In Part I we’ve been discovering these design patterns by refactoring different areas of a simple web application. Part II provides some higher-level concepts that can help you structure your application. Besides design patterns, it covers architectural layering, and hexagonal architecture (ports & adapters). It also includes a chapter on testing decoupled applications.

100% done

I’m happy to announce that my latest book “Advanced Web Application Architecture” is now complete. With ~390 pages it has become a well-rounded book full of useful design patterns and architectural principles built around the notion of object-pure code, that will help you create decoupled applications that are testable by definition, and support a domain-first approach to software development.

Use this link to get 10% off the price: https://leanpub.com/web-application-architecture/c/RELEASE_DAY

Recently I received a question; if I could explain these four lines:

/**
 * @test
 */
public function it_works_with_a_standard_use_case_for_command_objects(): void

The author of the email had some great points.

1. For each, my test I should write +3 new line of code instead write, public function testItWorksWithAStandardUseCaseForCommandObjects(): void

2. PSR-12 say that “Method names MUST be declared in camelCase”. [The source of this is actually PSR-1].

3. In PHPUnit documentation author say “The tests are public methods that are named test*” and left example below

While Advanced Web Application Architecture is still a work in progress, I decided to release another project in the meantime: a book for beginning PHP developers called PHP for the Web.

Of course, there are PHP books for beginners, but most of them aren’t very concise. They cover many related topics like relational databases, CLI applications, and sending emails. I wanted to write a book that only shows what makes PHP so special when it comes to web development.

The introduction of Domain-Driven Design (DDD) to a larger audience has led to a few really damaging ideas among developers, like this one (maybe it’s more a sentiment than an idea):

Data is bad, behavior is good. The domain model is great, the database awful.

(We’re not even discussing CRUD in this article, which apparently is the worst of the worst.)

By now many of us feel ashamed of using an ORM alongside a “DDD domain model”, putting some mapping configuration in it, doing things inside your entities (or do you call them aggregates?) just to make them easily serializable to the database.

The problem: if you publish any document as PDF, in print, etc. and the text contains URLs, there is a chance that one day those URLs won’t work anymore. There’s nothing to do about that, it happens.

Luckily, this is a solved problem. The solution is to link to a stable and trustworthy website, that is, one that you maintain and host (of course, you’re trustworthy!). Then in the document you link to that website, and the website redirects visitors to the actual location.

In the Epilogue of the Object Design Style Guide, I started happily outlining some of the architectural patterns I’ve been using for several years now. I wanted to give some kind of an overview of how the overall design of your application would improve if you apply the object-design rules in that book. I soon realized that an Epilogue was not enough to cover all the details, or to communicate the ideas in such a way that they would be applicable in everyday projects. And so a new book project began…

When Symfony2 was created I first learned about the functional test, which is an interesting type of test where everything about your application is as real as possible. Just like with an integration or end-to-end test. One big difference: the test runner exercises the application’s front controller programmatically, instead of through a web server. This means that the input for the test is a Request object, not an actual HTTP message.

It’s very easy not to write a book.

Mostly since it’s such an awful lot of work. You’ll first need to figure out what you’re going to write about, and find an angle that makes it interesting enough for potential readers to buy the book and spend the time to finish it. The writing itself is hard work as well, and may take several years. Along the way you’ll be telling yourself things that will keep you from finishing the task: I’m not doing anything original here. Nobody will be interested. Somebody else wrote a better book about this than I could ever do. If I don’t cover topic B as well, it’s not going to be a useful book at all.

During a recent run of my Advanced Web Application Architecture training, we discussed the distinction between infrastructure code and non-infrastructure code, which I usually call core code. One of the participants summarized the difference between the two as: “everything in your vendor directory is infrastructure code”. I don’t agree with that, and I will explain why in this article.

Not all code in vendor is infrastructure code

Admittedly, it’s easy for anyone to not agree with a statement like this, because you can simply make up your own definitions of “infrastructure” that turn the statement false. As a matter of fact, I’m currently working on my next book (which has the same title as the training), and I’m working on a memorable definition that covers all the cases. I’ll share with you the current version of that definition, which consists of two rules defining core code. Any code that doesn’t follow both these rules at the same time, should be considered infrastructure code.

Here are some rules I use for working with dynamic arrays. It’s pretty much a Style Guide for Array Design, but it didn’t feel right to add it to the Object Design Style Guide, because not every object-oriented language has dynamic arrays. The examples in this post are written in PHP, because PHP is pretty much Java (which might be familiar), but with dynamic arrays instead of built-in collection classes and interfaces.

Today Manning released my latest book! It’s called “Object Design Style Guide”.

In November 2018 I started working on this book. The idea for it came from a conversation I had with the friendly folks at Akeneo (Nantes) earlier that year. It turned out that, after days of high level training on web application architecture and Domain-Driven Design, there was a need for some kind of manual for low level object-oriented programming. Not as low level as the kind of programming advice people usually refer to as clean code, but general programming rules for different kinds of objects. For instance:

This blog runs on Sculpin, a static site generator. The generator itself runs on Symfony, which for me makes it easy to extend. However, I find that if you want something special, it can usually be done, but it may take several hours to get it right. In the end though, the solution is often quite elegant.

A custom content type for events

One custom feature I wanted for this website was a list of events (conference talks, trainings, etc.). Sculpin’s documentation suggests using a custom content type for that. This allows you to create a directory with files, each of which will be considered an “event”.

I wanted to be able to deploy MailComments to my Digital Ocean droplet (VPS) easily and without thinking. Due to a lack of maintenance, some more “operations” work had piled up as well:

• The Digital Ocean monitoring agent had to be upgraded, but apt didn’t have enough memory to do that on this old, small droplet.
• The Ubuntu version running on that droplet was also a bit old by now.
• The easiest thing to do was to just create a new droplet and prepare it for deploying my personal websites.
• Unfortunately, my DNS setup was completely tied to the IP address of the droplet, so I couldn’t really create a new droplet, and quickly switch. I’d have to wait for the new DNS information to propagate.

These issues were in the way of progress, so I decided to take some more time to rearrange things.

Many people use Disqus as a commenting system for their (static) blog. It’s a free service, easy to get started with, and it has everything you’d expect from a commenting system. It’s free up to the point where Disqus decides to show advertisements alongside the comments, and these advertisements are so bad, you will look for better options very quickly. The way out of advertisements, of course, is to start paying for their services. Actually, I would have paid already for their service, if only they would have reduced the tremendous amount of stuff they load on your page. And the cookies they need for tracking you across the internet of course.

Triggered by Marco Pivetta who apparently said during his talk at Symfony Live Berlin: “If you still don’t have tests, this is unprofessional”, I thought I’d tweet about that too: “It’s good for someone to point this out from time to time”.

I don’t like it when a blog post is about tweets, just as I don’t like it when a news organization quotes tweets; as if they are some important source of wisdom. But since this kind of tweet tends to invoke many reactions (and often emotionally charged ones too), I thought it would be smart to get the discussion off Twitter and write something more nuanced instead.

In part 1 of this series we discussed the size and complexity metrics calculated by phploc. We continue with a discussion about dependencies and structure.

Structure

This section gives us an idea of how many things there are of a certain type. These can be useful indicators too. For example, if the number of Namespaces is low, there may be a lack of grouping, which is bad for discoverability. It’ll be hard to find out where a certain piece of logic can be found in the code. Too many namespaces, relative to the number of Classes/Interfaces/Traits, is not a good sign either. I would expect every namespace to have a couple of classes that naturally belong together.

When I want to get a very rough idea of the quality of both code and structure of a PHP code base, I like to run phploc on it. This is a tool created by Sebastian Bergmann for (I assume) exactly this purpose. It produces the following kind of output:

Directories                                          3
Files                                               10

Size
  Lines of Code (LOC)                             1882
  Comment Lines of Code (CLOC)                     255 (13.55%)
  Non-Comment Lines of Code (NCLOC)               1627 (86.45%)
  Logical Lines of Code (LLOC)                     377 (20.03%)
    Classes                                        351 (93.10%)
      Average Class Length                          35
        Minimum Class Length                         0
        Maximum Class Length                       172
      Average Method Length                          2
        Minimum Method Length                        1
        Maximum Method Length                      117
    Functions                                        0 (0.00%)
      Average Function Length                        0
    Not in classes or functions                     26 (6.90%)

Cyclomatic Complexity
  Average Complexity per LLOC                     0.49
  Average Complexity per Class                   19.60
    Minimum Class Complexity                      1.00
    Maximum Class Complexity                    139.00
  Average Complexity per Method                   2.43
    Minimum Method Complexity                     1.00
    Maximum Method Complexity                    96.00

Dependencies
  Global Accesses                                    0
    Global Constants                                 0 (0.00%)
    Global Variables                                 0 (0.00%)
    Super-Global Variables                           0 (0.00%)
  Attribute Accesses                                85
    Non-Static                                      85 (100.00%)
    Static                                           0 (0.00%)
  Method Calls                                     280
    Non-Static                                     276 (98.57%)
    Static                                           4 (1.43%)

Structure
  Namespaces                                         3
  Interfaces                                         1
  Traits                                             0
  Classes                                            9
    Abstract Classes                                 0 (0.00%)
    Concrete Classes                                 9 (100.00%)
  Methods                                          130
    Scope
      Non-Static Methods                           130 (100.00%)
      Static Methods                                 0 (0.00%)
    Visibility
      Public Methods                               103 (79.23%)
      Non-Public Methods                            27 (20.77%)
  Functions                                          0
    Named Functions                                  0 (0.00%)
    Anonymous Functions                              0 (0.00%)
  Constants                                          0
    Global Constants                                 0 (0.00%)
    Class Constants                                  0 (0.00%)

These numbers are statistics, and as you know, they can be used to tell the biggest lies. Without the context of the actual code, you can interpret them in any way you like. So you need to be careful making conclusions based on them. However, in my experience these numbers are often quite good indicators of the overall code quality as well as the structural quality of a project.

This is another excerpt from my latest book, which is currently part of Manning’s Early Access Program. Take 37% off Object Design Style Guide by entering fccnoback into the discount code box at checkout at manning.com.

Make sure to read part 1 first.

Create read models directly from their data source

Instead of creating a StockReport model from PurchaseOrderForStock objects, we could go directly to the source of the data, that is, the database where the application stores its purchase orders. If this is a relational database, there might be a table called purchase_orders, with columns for purchase_order_id, product_id, ordered_quantity, and was_received. If that’s the case, then StockReportRepository wouldn’t have to load any other object before it could build a StockReport object; it could make a single SQL query and use it to create the StockReport, as shown in Listing 11).

I’m happy to share with you an excerpt of my latest book, which is currently part of Manning’s Early Access Program. Take 37% off Object Design Style Guide by entering fccnoback into the discount code box at checkout at manning.com.

Chapter 7: Dividing responsibilities

We’ve looked at how objects can be used to retrieve information, or perform tasks. The methods for retrieving information are called query methods, the ones that perform tasks are command methods. Service objects may combine both of these responsibilities. For instance, a repository (like the one in Listing 1) could perform the task of saving an entity to the database, and at the same time it would also be capable of retrieving an entity from the database.

“Can you make a query bus with SimpleBus?” The question has been asked many times. I’ve always said no. Basically, because I didn’t build in the option to return anything from a command handler. So a handler could never become a query handler, since a query will of course have to return something.

I’ve always thought that the demand for a query bus is just a sign of the need for symmetry. If you have command and query methods, then why not have command and query buses too? A desire for symmetry isn’t a bad thing per se. Symmetry is attractive because it feels natural, and that feeling can serve as design feedback. For instance, you can use lack of symmetry to find out what aspect of a design is still missing, or to find alternative solutions.

With excerpts from the documentation

I have worked with symfony 1, Symfony 2+, Zend Framework 1, Zend Expressive, but never with Laravel. Until now. My partner was looking for a way to learn PHP and building web applications with it. Most of my own framework knowledge is related to Symfony, so my initial plan was to find a Symfony course for her. However, Jeffrey Way’s Laracasts also came to mind, and I thought it would be an interesting learning experience for us both if Laravel would be the framework of choice in this matter. It turned out to be a good idea. My partner is making good progress, and I get to see what Laravel is all about. We have a lot of fun together, finding out how it works, or what you’re supposed to be doing with it.

This week I attended and spoke at the Newcrafts conference in Paris. The following is a mix of notes and personal observations I wanted to share, centered around some of the talks I saw there.

Romeu Romera: Bourdieu’s Social theory and our work in tech

I had never attended a talk by Romeu before. I really enjoyed this one. Somehow I already knew that he uses a mindmap to support his talk. I thought he would use an existing mind map to navigate through the talk, but it turned out he was creating one during the talk. For me personally, a slide deck helps to keep track of the story, and it helps me remember all the different topics I need to talk about. Not so much for Romeu, who knew exactly what he was going to talk about, and didn’t seem to forget to mention any important part, or make important connections.

With a foreword by Ross Tuck

Today I’ve released the revised edition of my latest book “Style Guide for Object Design”. Over the past few weeks I’ve fixed many issues, smaller and larger ones. Ross Tuck and Laura Cody have read the manuscript and provided me with excellent feedback, which has helped me a lot to get the book in a better shape. I added many sections, asides, and even an extra chapter, because some topics deserve more detail and background than just a few paragraphs. Oh, and Ross wrote the kindest of forewords to kick off the book. It’s available in the free sample of the book.

You say "convention over configuration;" I hear "ambient information stuck in someone's head." You say "configuration over hardcoding;" I hear "information in a different language that must be parsed, can be malformed, or not exist."

— Paul Snively (@paul_snively) March 2, 2019

Dependency injection is very important. Dependency injection containers are too. The trouble is with the tools, that let us define services in a meta-language, and rely on conventions to work well. This extra layer requires the “ambient information” Paul speaks about in his tweet, and easily lets us make mistakes that we wouldn’t make if we’d just write out the code for instantiating our services.

Cyrille Martraire: Domain modeling towards First Principles

This was a talk from the first day, but it required some more processing before writing about it. Cyrille is one of my favorite speakers. He’s fast, funny and brings a lot of interesting topics to the table. So many that it’s sometimes hard to keep following his train of thought, and writing down some notes at the same time.

A central concept from his talk was what he called the waterline between IT and the business. In a traditional scenario, developers get provided with “work” on a case-by-case basis. They don’t learn about the general idea or plan, or even the vision, goal or need that’s behind the “work item”. They just have to “implement” it. It leads to badly designed code. But it also leads to the wrong solutions being delivered. If only developers could have talked with the people who actually have the problem for which they build the solution. Maybe there’s another problem behind it, or maybe the business has provided the developer with a solution, instead of a problem. To higher the waterline means to get more involved with the customers and users, to understand their problems, and work together on a solution. Make sure you get involved.

Eric Evans: Keynote (“Language in Context”)

Starting out with the basics (words have meaning within a context; when we make the boundary of this context explicit we end up with a bounded context), Eric discussed two main topics: the Big Ball of Mud, and bounded contexts in the context (no pun intended) of microservices.

Legacy applications are always framed in a negative way, as if it’s something to get away from. Personally, I’ve come to enjoy them a lot. However, there will always be the urge to work around legacy code. The Bubble Context (PDF) can be a great way of creating a new model that works well next to the already existing models. To keep a safe buffer between the new and the old model, you could build an Anti-Corruption Layer (ACL). A fun thing Eric mentioned is that the buffer works in two directions. The ACL also allows the old model to keep functioning without being disturbed by all the new things that are going on in the Bubble Context.

All of a sudden it became book-writing season. It began in August when I started revising my second book, “Principles of Package Design”. Apress had contacted me about adopting it, and they didn’t want to change a lot about it. However, the book was from 2015 and although I had aimed for it to be “timeless”, some parts needed an update. Furthermore, I had happily pressed the “Release” button back then, but it’s the same as with software development: the code you wrote last year, you wouldn’t approve of today.

In part 1 of this short series (it’s going to end with this article) we covered how you can test-drive the queries in a repository class. Returning query results is only part of the job of a repository though. The other part is to store objects (entities), retrieve them using something like a save() and a getById() method, and possibly delete them. Some people will implement these two jobs in one repository class, some like to use two or even many repositories for this. When you have a separate write and read model (CQRS), the read model repositories will have the querying functionality (e.g. find me all the active products), the write model repositories will have the store/retrieve/delete functionality.

There’s something I’ve only been doing since a year or so, and I’ve come to like it a lot. My previous testing experiences were mostly at the level of unit tests or functional/system tests. What was left out of the equation was integration tests. The perfect example of which is a test that proves that your custom repository class works well with the type of database that you use for the project, and possibly the ORM or database abstraction library you use to talk with that database. A test for a repository can’t be a unit test; that wouldn’t make sense. You’d leave a lot of assumptions untested. So, no mocking is allowed.

When you’re looking at a function (an actual function or a method), you can usually identify several blocks of code in there. There are pre-conditions, there’s the function body, and there may be post-conditions. The pre-conditions are there to verify that the function can safely proceed to do its real job. Post-conditions may be there to verify that you’re going to give something back to the caller that will make sense to them.

I recently wrote about when to add an interface to a class. After explaining good reasons for adding an interface, I claim that if none of those reasons apply in your situation, you should just use a class and declare it “final”.

PHP 5 introduces the final keyword, which prevents child classes from overriding a method by prefixing the definition with final. If the class itself is being defined final then it cannot be extended.

Last week I wrote about when to add an interface to a class. The article finishes with the claim that classes from the application’s domain don’t usually need an interface. The reason is that domain code isn’t going to be swapped out with something else. This code is the result of careful modelling work that’s done based on the business domain that you work with. And even if you’d work on, say, two financial software projects in a row, you’ll find that the models you produce for each of them will be different in many subtle (if not radical) ways. Paradoxically you’ll find that in practice a domain model can sometimes be reused after all. There are some great examples out there. In this article I explain different scenarios of where and how reuse could work.

I’m currently revising my book “Principles of Package Design”. It covers lots of design principles, like the SOLID principles and the lesser known Package (or Component) Design Principles. When discussing these principles in the book, I regularly encourage the reader to add more interfaces to their classes, to make the overall design of the package or application more flexible. However, not every class needs an interface, and not every interface makes sense. I thought it would be useful to enumerate some good reasons for adding an interface to a class. At the end of this post I’ll make sure to mention a few good reasons for not adding an interface too.

During the holiday I read a book mentioned to me by Pim Elshoff: “Skin in the game”, by Nassim Nicholas Taleb. Discussing this concept of “skin in the game” with Pim had made me curious about the book. It’s not such a fat book as one of Taleb’s other books, which is possibly more famous, called “Antifragile”. “Skin in the game” is a much lighter book, and also quite polemic, making it an often uncomfortable, but fun reading experience. I can easily see how people could get mad about this book or aggressive towards its author (not that I’m encouraging or approving of that aggression!). While reading it, it reminded me of Nietzsche, and his despise of the “common man”, who Taleb calls “the intellectual” - someone who has no “skin in the game”, let alone “soul in the game”. Taleb’s ideas are interesting, just like Nietzsche’s are, but they could easily be abused, and probably so by misinterpreting them.

Recently I read a comment on Twitter by Nikola Poša:

I guess the discussion on my thread is going in the wrong direction because I left out a crucial hashtag: #NoCommentsInCode - avoid comments in code, write descriptive classes, methods, variables.https://t.co/MuHoOFXCvV

— Nikola Poša (@nikolaposa) July 13, 2018

He was providing us with a useful suggestion, one that I myself have been following ever since reading “Clean Code” by Robert Martin. The paraphrased suggestion in that book, as well as in the tweet, is to consider a comment to be a naming issue in disguise, and to solve that issue, instead of keeping the comment. By the way, the book has some very nice examples of how comments should and should not be used.

In “Negative Architecture”, Michael Feathers speaks about certain aspects of software architecture leading to some kind of negative architecture. Feathers mentions the IO Monad from Haskell (functional programming) as an example, but there are parallel examples in object-oriented programming. For example, by using layers and the dependency inversion principle you can “guarantee” that a certain class, in a certain layer (e.g. domain, application) won’t do any IO - no talking to a database, no HTTP requests to some remote service, etc.

Consider the following rule:

When you create an object, it should be complete, consistent and valid in one go.

It is derived from the more general principle that it should not be possible for an object to exist in an inconsistent state. I think this is a very important rule, one that will gradually lead everyone from the swamps of those dreaded “anemic” domain models. However, the question still remains: what does all of this mean?

System and integration tests need database fixtures. These fixtures should be representative and diverse enough to “fake” normal usage of the application, so that the tests using them will catch any issues that might occur once you deploy the application to the production environment. There are many different options for dealing with fixtures; let’s explore some of them.

Generate fixtures the natural way

The first option, which I assume not many people are choosing, is to start up the application at the beginning of a test, then navigate to specific pages, submit forms, click buttons, etc. until finally the database has been populated with the right data. At that point, the application is in a useful state and you can continue with the act/when and assert/then phases. (See the recent article “Pickled State” by Robert Martin on the topic of tests as specifications of a finite state machine).

A very important “trick” in finding the flow in life is: do what you like most. Of course, you have to do things you don’t like (and then you need different life hacks), but when you can do something you like, you’ll find that you’ll be more successful at it.

When it comes to blogging, I find that it helps to follow my instincts, to write about whatever I like to write about at the moment. I can think of a list of things that need blogging about, but I end up not writing about them because they don’t light the fire inside me (anymore).

The downsides of starting with the domain model

All the architectural focus on having a clean and infrastructure-free domain model is great. It’s awesome to be able to develop your domain model in complete isolation; just a bunch of unit tests helping you design the most beautiful objects. And all the “impure” stuff comes later (like the database, UI interactions, etc.).

However, there’s a big downside to starting with the domain model: it leads to inside-out development. The first negative effect of this is that when you start with designing your aggregates (entities and value objects), you will definitely need to revise them when you end up actually using them from the UI. Some aspects may turn out to be not so well-designed at all, and will make no sense from the user’s perspective. Some functionality may have been designed well, but only theoretically, since it will never actually be used by any real client, except for the unit test you wrote for it.

I’d like to start this article with a quote from Ross Tuck’s article “Persisting Value Objects in Doctrine”. He describes different ways of persisting value objects when using Doctrine ORM. At the end of the page he gives us the following option - the “nuclear” one:

[…] Doctrine is great for the vast majority of applications but if you’ve got edge cases that are making your entity code messy, don’t be afraid to toss Doctrine out. Setup an interface for your repositories and create an alternate implementation where you do the querying or mapping by hand. It might be a PITA but it might also be less frustration in the long run.

Statically fetching dependencies

I’ve worked with several code bases that were littered with calls to Zend_Registry::get(), sfContext::getInstance(), etc. to fetch a dependency when needed. I’m a little afraid to mention façades here, but they also belong in this list. The point of this article is not to bash a certain framework (they are all lovely), but to show how to get rid of these “centralized dependency managers” when you need to. The characteristics of these things are:

I wanted to share an important lesson I learned from my colleague Ramon de la Fuente. I was explaining to him how I made a big effort to preserve some existing behavior, when he said something along the lines of: the people who wrote this code, may or may not have known what they were doing. So don’t worry too much about preserving old stuff.

These wise words eventually connected to other things I’ve learned about programming, and I wanted to combine them under the umbrella of a blog post titled “Deliberate coding”. Doing something “deliberately” means to do it consciously and intentionally. It turns out that not everyone writes code deliberately, and at the very least, not everyone does it all the time.

This is a question that always pops up during my workshops: when and where to determine the ID of an entity? There are different answers, no best answer. Well, there are two best answers, but they apply to two different situations.

Auto-incrementing IDs, by the database

Traditionally, all you need for an entity to have an ID is to designate one integer column in the database as the primary key, and mark it as “auto-incrementing”. So, once a new entity gets persisted as a record in the database (using your favorite ORM), it will get an ID. That is, the entity has no identity until it has been persisted. Even though this happens everywhere, and almost always; it’s a bit weird, because:

This article is part 2 of my review of the book “Fifty quick ideas to improve your tests”. I’ll continue to share some of my personal highlights with you.

Replace multiple steps with a higher-level step

If a test executes multiple tasks in sequence that form a higher-level action, often the language and the concepts used in the test explain the mechanics of test execution rather than the purpose of the test, and in this case the entire block can often be replaced with a single higher-level concept.

Review

After reading “Discovery - Explore behaviour using examples” by Gáspár Nagy and Seb Rose, I picked up another book, which I bought a long time ago: “Fifty Quick Ideas to Improve Your Tests” by Gojko Adzic, David Evans, Tom Roden and Nikola Korac. Like with so many books, I find there’s often a “right” time for them. When I tried to read this book for the first time, I was totally not interested and soon stopped trying to read it. But ever since Julien Janvier asked me if I knew any good resources on how to write good acceptance test scenarios, I kept looking around for more valuable pointers, and so I revisited this book too. After all, one of the author’s of this book - Gojko Adzic - also wrote “Bridging the communication gap - Specification by example and agile acceptance testing”, which made a lasting impression on me. If I remember correctly, the latter doesn’t have too much practical advice on writing goods tests (or scenarios), and it was my hope that “Fifty quick ideas” would.

I’ve just finished reading “Discovery - Explore behaviour using examples” by Gáspár Nagy and Seb Rose. It’s the first in a series of books about BDD (Behavior-Driven Development). The next parts are yet to be written/published. Part of the reason to pick up this book was that I’d seen it on Twitter (that alone would not be a sufficient reason of course). The biggest reason was that after delivering a testing and aggregate design workshop, I noticed that my acceptance test skills aren’t what they should be. After several years of not working as a developer on a project for a client, I realized again that (a quote from the book):

Recently I read Ouarzy’s review of Jason Fried and David Heinemeier Hansson’s “Remote - Office Not Required”. I’d read their previous books, “Getting Real” and “Rework”. They’re all a joy to read. Short chapters, nice little cartoons. Just a lot of fun, and inspiring too. Not many authors make as much of an effort as they do to condense their message and reward the reader for taking the time to read an actual book. It’s insulting how much work some authors expect you to put in before they will reveal their secrets!

I’m working on another “multi-tenant” PHP web application project and I noticed an interesting series of events. It felt like a natural progression and by means of a bit of dangerous induction, I’m posing the hypothesis that this is how things are just bound to happen in such projects.

In the beginning we start out with a framework that has some authentication functionality built-in. We can get the “current user” from the session, or from some other session-based object. We’ll also need the “current company” (or the “current organization”) of which the current user is a member.

I find it very curious that legacy (PHP) code often has the following characteristics:

1. Classes with the name of a central domain concept have grown too large.
2. Methods in these classes have become very generic.

Classes grow too large

I think the following happened:

The original developers tried to capture the domain logic in these classes. They implemented it based on what they knew at the time. Other developers, who worked on the code later, had to implement new features, or modify domain logic, because, well, things change. Also, because we need more things.

Exceptions - for exceptional situations?

From the Domain-Driven Design movement we’ve learned to go somewhat back to the roots of object-oriented design. Designing domain objects is all about offering meaningful behavior and insights through a carefully designed API. We know now that domain objects with setters for every attribute will allow for the objects to be in an inconsistent state. By removing setters and replacing them with methods which only modify the object’s state in valid ways, we can protect an object from violating domain invariants.

In this series, we’ve discussed several topics already. We talked about persistence and time, the filesystem and randomness. The conclusion for all these areas: whenever you want to “mock” these things, you may look for a solution at the level of programming tools used (use database or filesystem abstraction library, replace built-in PHP functions, etc.). But the better solution is always: add your own abstraction. Start by introducing your own interface (including custom return types), which describes exactly what you need. Then mock this interface freely in your application. But also provide an implementation for it, which uses “the real thing”, and write an integration test for just this class.

I recently came across two interesting methods that were part of a bigger class that I had to redesign:

class OrderLine
{
    /**
     * @var float
     */
    private $quantityOrdered;
    
    // ...

    /**
     * @param float $quantity
     */
    public function processDelivery($quantity)
    {
        $this->quantityDelivered += $quantity;
        
        $this->quantityOpen = $this->quantityOrdered - $quantity;
        if ($this->quantityOpen < 0) {
            $this->quantityOpen = 0;
        }
    }

    /**
     * @param float $quantity
     */
    public function undoDelivery($quantity)
    {
        $this->quantityDelivered -= $quantity;
        if ($this->quantityDelivered < 0) {
            $this->quantityDelivered = 0;
        }
        
        $this->quantityOpen += $quantity;
        if ($this->quantityOpen > $this->quantityOrdered) {
            $this->quantityOpen = $this->quantityOrdered;
        }
    }
}

Of course, I’ve already cleaned up the code here to allow you to better understand it.

Something that always bothers me: persistence (the user interface too, but that’s a different topic ;)). Having objects in memory is nice, but when the application shuts down (and for PHP this is after every request-response cycle), you have to persist them somehow. By the way, I think we’ve all forever been annoyed by persistence, since there’s an awful lot of software solutions related to object persistence: different types of databases, different types of ORMs, etc.

In a previous article, we discussed “persistence” and “time” as boundary concepts that need mocking by means of dependency inversion: define your own interface, then provide an implementation for it. There were three other topics left to cover: the filesystem, the network and randomness.

Mocking the filesystem

We already covered “persistence”, but only in the sense that we sometimes need a way to make in-memory objects persistent. After a restart of the application we should be able to bring back those objects and continue to use them as if nothing happened.

I read this tweet:

"The object-oriented version of spaghetti code is, of course, 'lasagna code'. Too many layers." - Roberto Waltman

— Programming Wisdom (@CodeWisdom) February 24, 2018

Jokes taken as advice

It’s not the first time I’d heard of this quote. Somehow it annoys me, not just this one joke, but many jokes like this one. I know I should be laughing, but I’m always worried about jokes like this going to be interpreted as advice, in its most extreme form. E.g. the advice distilled from this tweet could be: “Layers? Don’t go there. Before you know it, you have lasagna code…”

More and more I’ve come to realize that I’ve been mocking less and less.

The thing is, creating test doubles is a very dangerous activity. For example, what I often see is something like this:

$entityManager = $this->createMock(EntityManager::class);

$entityManager->expects($this->once())
    ->method('persist')
    ->with($object);
$entityManager->expects($this->once())
    ->method('flush')
    ->with($object);

Or, what appears to be better, since we’d be mocking an interface instead of a concrete class:

$entityManager = $this->createMock(ObjectManagerInterface::class);
// ...

To be very honest, there isn’t a big different between these two examples. If this code is in, for example, a unit test for a repository class, we’re not testing many of the aspects of the code that should have been tested instead.

Why code coverage for Behat?

PHPUnit has built-in several options for generating code coverage data and reports. Behat doesn’t. As Konstantin Kudryashov (@everzet) points out in an issue asking for code coverage options in Behat:

Code coverage is controversial idea and code coverage for StoryBDD framework is just nonsense. If you’re doing code testing with StoryBDD - you’re doing it wrong.

He’s right I guess. The main issue is that StoryBDD isn’t about code, so it doesn’t make sense to calculate code coverage for it. Furthermore, the danger of collecting code coverage data and generating coverage reports is that people will start using it as a metric for code quality. And maybe they’ll even set management targets based on coverage percentage. Anyway, that’s not what this article is about…

This article continues where Unary call sites and intention-revealing interfaces ended.

While reading David West’s excellent book “Object Thinking”, I stumbled across an interesting quote from David Parnas on the programming method that most of us use by default:

The easiest way to describe the programming method used in most projects today was given to me by a teacher who was explaining how he teaches programming. “Think like a computer,” he said. He instructed his students to begin by thinking about what the computer had to do first and to write that down. They would then think about what the computer had to do next and continue in that way until they had described the last thing the computer would do… […]

Call sites

One of the features I love most about my IDE is the button “Find Usages”. It is invaluable when improving a legacy code base. When used on a class it will show you where this class is used (as a parameter type, in an import statement, etc.). When used on a method, it will show you where this method gets called. Users of a method are often called “clients”, but when we use “Find Usages”, we might as well use the more generic term “call sites”.

Setting the stage: Code complexity

Code complexity often gets measured by calculating the Cyclomatic Complexity per unit of code. The number can be calculated by taking all the branches of the code into consideration.

Code complexity is an indicator for several things:

• How hard it is to understand a piece of code; a high number indicates many branches in the code. When reading the code, a programmer has to keep track of all those branches, in order to understand all the different ways in which the code can work.
• How hard it is to test that piece of code; a high number indicates many branches in the code, and in order to fully test the piece of code, all those branches need to be covered separately.

In both cases, high code complexity is a really bad thing. So, in general, we always strive for low code complexity. Unfortunately, many projects that you’ll inherit (“legacy projects”), will contain code that has high code complexity, and no tests. A common hypothesis is that a high code complexity arises from a lack of tests. At the same time, it’s really hard to write tests for code with high complexity, so this is a situation that is really hard to get out.

CQRS - not a complicated thing

CQRS has some reputation issues. Mainly, people will feel that it’s too complicated to apply in their current projects. It will often be considered over-engineering. I think CQRS is simply misunderstood, which is the reason many people will not choose it as a design technique. One of the common misconceptions is that CQRS always goes together with event sourcing, which is indeed more costly and risky to implement.

In the previous article we discussed a sensible layer system, consisting of three layers:

• Domain
• Application
• Infrastructure

Infrastructure

The infrastructure layer, containing everything that connects the application’s use cases to “the world outside” (like users, hardware, other applications), can become quite large. As I already remarked, a lot of our software consists of infrastructure code, since that’s the realm of things complicated and prone to break. Infrastructure code connects our precious clean code to:

The first key concept of what I think is a very simple, at the very least “clean” architecture, is the concept of a layer. A layer itself is actually nothing, if you think about it. It’s simply determined by how it’s used. Let’s stay a bit philosophical, before we dive into some concrete architectural advice.

Qualities of layers

A layer in software serves the following (more or less abstract) purposes:

Looking back at my old blog posts, I think it’s good to write down a more balanced view on application architecture than the one that speaks from some of the older posts from 2013 and 2014. Before I do, I allow myself a quick self-centered trip down memory lane.

Why Symfony? Seven facts

The archive tells an interesting story about how my thoughts about software development changed over time. It all started with my first post in October 2011, How to make your service use tags. Back in those days, version 2 of the Symfony framework was still quite young. I had been working with symfony (version 1, small caps) for a couple of years and all of a sudden I was using Symfony2, and I absolutely loved this new shiny framework. I learned as much as I could about it (just as I had done with version 1), and eventually I became a Certified Symfony Developer, at the first round of exams held in Paris, during a Symfony Live conference. During those years I blogged and spoke a lot about Symfony, contributed to the documentation, and produced several open source bundles. I also wrote a book about it: A Year With Symfony.

Workshop utilities

For the workshops that I organize, I often need some “utilities” that will do the job, but are as simple as possible. Examples of such utilities are:

• Something to dispatch events with.
• Something to serialize and deserialize objects with.
• Something to store and load objects with.
• Something to use with event sourcing (an event store, an event-sourced repository).

I put several of these tools in a code base called php-workshop-tools. These utilities should be small, and very easy to understand and use. They will be different from the frameworks and libraries most workshop participants usually use, but they should offer more or less the same functionality. While designing these tools, I was constantly looking for the golden mean:

Whenever I run the Sculpin generate command to generate a new version of the static website that is this blog, I notice there are a lot of useless files that get copied from the project’s source/ directory to the project’s output/ directory. All the files in the output/ directory will eventually get copied into a Docker image based on nginx (see also my blog series on Containerizing a static website with Docker). And since I’m on a hotel wifi now, I realized that now was the time to shave off any unnecessary weight from this Docker image.

Last year I took several Scala courses on Coursera. It was an interesting experience and it has brought me a lot of new ideas. One of these is the idea of a singleton object (as opposed to a class). It has the following characteristics:

• There is only one instance of it (hence it’s called a “singleton”, but isn’t an implementation of the Singleton design pattern).
• It doesn’t need to be explicitly instantiated (it doesn’t have the traditional static getInstance() method). In fact, an instance already exists when you first want to use it.
• There doesn’t have to be built-in protection against multiple instantiations (as there is and can only be one instance, by definition).

Converting this notion to PHP is impossible, but if it was possible, you could do something like this:

So, here’s a bit of a personal blog post for once.

Symfony trademark policy

I saw this tweet:

Suite à une mise en demeure, j’ai retiré les tutoriels qui concernent Symfony du Site. Il n’y aura pas de futures vidéos sur le Framework.

— Grafikart (@grafikart_fr) March 11, 2017

Meaning: “Following a formal notice, I removed the tutorials that are related to Symfony from the Site. There will be no future videos on the Framework.”

For quite some time now the PHP community has becoming more and more professional. “More professional” in part means that we use more types in our PHP code. Though it took years to introduce more or less decent types in the programming language itself, it took some more time to really appreciate the fact that by adding parameter and return types to our code, we can verify its correctness in better ways than we could before. And although all the type checks still happen at runtime, it feels as if those type checks already happen at compile time, because our editor validates most of our code before actually running it.

Unbalanced test suites

Having spoken to many developers and development teams so far, I’ve recognized several patterns when it comes to software testing. For example:

1. When the developers use a framework that encourages or sometimes even forces them to marry their code to the framework code, they only write functional tests - with a real database, making (pseudo) HTTP requests, etc. using the test tools bundled with the framework (if you’re lucky, anyway). For these teams it’s often too hard to write proper unit tests. It takes too much time, or is too difficult to set up.
2. When the developers use a framework that enables or encourages them to write code that is decoupled from the framework, they have all these nice, often pretty abstract, units of code. Those units are easy to test. What often happens is that these teams end up writing only unit tests, and don’t supply any tests or “executable specifications” proving the correctness of the behavior of the application at large.
3. Almost nobody writes proper acceptance tests. That is, most tests that use a BDD framework, are still solely focusing on the technical aspects (verifying URLs, HTML elements, rows in a database, etc.), while to be truly beneficial they should be concerned about application behavior, defined in a ubiquitous language that is used by both the technical staff and the project’s stakeholders.

Please note that these are just some very rough conclusions, there’s nothing scientific about them. It would be interesting to do some actual research though. And probably someone has already done this.

As was the case with the SymfonyConsoleForm library, I realized I had not taken the time to properly introduce another library I created some time ago: ConvenientImmutability. This one is a lot simpler, but from a technical standpoint I think it might be interesting to at least talk a bit about the considerations that went into it.

For starters, it’s important to keep in mind that I don’t think you’ll actually need to use this library. Of course, I’ll explain why. But first: what is ConvenientImmutability about?

About 2 years ago I created a package that combines the power of two famous Symfony components: the Form component and the Console component. In short: this package allows you to interactively fill in a form by typing in the answers at the CLI. When I started working on it, this seemed like a pretty far-fetched idea. However, it made a lot of sense to me in terms of a ports & adapters architecture that I was looking for back then (and still always am, by the way). We could (and often should) write the code in our application layer in such a way that it doesn’t make a big difference whether we call applications services from a web controller or from a CLI “controller”.

In the previous posts we looked at creating a build container, and after that we created a blog container, serving our generated static website.

It’s quite surprising to me how simple the current setup is — admittedly, it’s a simple application too. It takes about 50 lines of configuration to get everything up and running.

The idea of the blog container, which has nginx as its main process, is to deploy it to a production server whenever we feel like it, in just “one click”. There should be no need to configure a server to host our website, and it should not be necessary to build the application on the server too. This is in fact the promise, and the true power of Docker.

In the previous post we looked at the process of designing a build container, consisting of all the required build tools for generating a static website from source files. In order to see the result of the build process, we still need to design another container, which runs a simple web server, serving the static website (mainly .html, .css, .js and .jpg files).

Designing the blog container

We’ll use a light-weight install of Nginx as the base image and simply copy the website files to the default document root (/usr/share/nginx/html) (only after removing any placeholder files that are currently inside that directory). The complete file docker/blog/Dockerfile looks like this:

Recently a former colleague of mine, Lucas van Lierop, showed me his new website, which he created using Spress. Lucas took two bold moves: he started freelancing, and he open-sourced his website code. This to me was very inspiring. I’ve been getting up to speed with Docker recently and am planning to do a lot more with it over the coming months, and being able to take a look at the source code of up-to-date projects that use Docker is certainly invaluable.

Previously we looked at how to use the static site generator Sculpin to generate in-project documentation. When Sculpin builds the HTML files and assets it looks at what files are in the source/ directory and processes them based on certain rules (e.g. “parse Markdown files”, “format HTML files with Twig”, etc.). The purpose for my “living documentation” project is to also dynamically generate documentation based on PHP and configuration files from the main project. For example, consider extracting a Composer dependency diagram. I don’t want to manually copy the diagram and an HTML template to the source/ directory. I want Sculpin to generate a current diagram on-demand and wrap it inside a template.

Recently I’ve been reading the book Living documentation by Cyrille Martraire. I can warmly recommend this book to you. It basically provides a solution for the age-old problem of creating and maintaining accurate, up-to-date and useful project documentation. One of the key ideas is to generate documentation instead of writing it. This should help prevent duplication and outdated information that is not trust-worthy and would therefore be neglected. I’m currently looking for ways to technically accomplish such a thing with PHP projects. This should result in reusable tools which will make it easier and more fun to document future projects while writing the code.

TLDR;

• I won’t update A Year With Symfony anymore.
• The e-book is now freely available.

My first book, A Year With Symfony, was well received when I first published it on September 4th. At the time it seemed to be a welcome addition to the official Symfony documentation.

A lot of water has passed under the bridge since then. Several new Symfony components were released and we have now arrived at the next major version, 3.0. In the meantime I published one update of the book: a new chapter about Annotations.

Everybody is organizing their own PHP conference these days. It’s impossible to “catch ’em all”. There’s conferences clashing, conferences too far away, conferences too much like other conferences, etc.

Then there’s conferences that have never been there before, are quite original and are therefore spark everyone’s imagination. One such a conference is the Symfony Catalunya conference. This “international Symfony event” is being organized by the generally awesome and friend of the community Marc Morera. The concept seems to be pretty simple:

Leaving Qandidate, off to Coolblue

After I had a very interesting conversation with the developers behind the Broadway framework for CQRS and event sourcing the day wasn’t over for me yet. I walked about one kilometer to the north to meet Paul de Raaij, who is a senior developer at Coolblue, a company which sells and delivers all kinds of - mostly - electrical consumer devices. Their headquarters are very close to the new and shiny Rotterdam Central station. The company itself is doing quite well. With 1000+ employees they keep needing more office space.

Visiting Qandidate in Rotterdam

Last Thursday I had the honor of meeting (part of the) Broadway team: three very smart developers working for Qandidate in central Rotterdam: Gediminas Šedbaras, Willem-Jan Zijderveld and Fritsjan. A short walk from the train station brought me to their office, where we talked about CQRS, event sourcing and some of the practical aspects of using the Broadway framework.

As you may have read I’ve been experimenting with Broadway extensively during these last weeks. Knowing a lot about the theory behind it, it was really nice to see how everything worked so smoothly. I had some questions however, which have now been answered (thank you guys for taking the time to do this!).

In the first and second part of this series we’ve been doing quite a bit of work to separate concerns that were once all combined into one function.

The major “players” in the field have been identified: there is an HttpClient and a Cache, used by different implementations of CatApi to form a testable, performing client of The Cat Api.

Representing data

We have been looking at behavior, and the overall structure of the code. But we didn’t yet look at the data that is being passed around. Currently everything is a string, including the return value of CatApi::getRandomImage(). When calling the method on an instance we are “guaranteed” to retrieve a string. I say “guaranteed” since PHP would allow anything to be returned, an object, a resource, an array, etc. Though in the case of RealCatApi::getRandomImage() we can be sure that it is a string, because we explicitly cast the return value to a string, we can’t be sure it will be useful to the caller of this function. It might be an empty string, or a string that doesn’t contain a URL, like 'I am not a URL'.

The world is not a safe thing to depend upon

When you’re running unit tests, you don’t want the world itself to be involved. Executing actual database queries, making actual HTTP requests, writing actual files, none of this is desirable: it will make your tests very slow as well as unpredictable. If the server you’re sending requests to is down, or responds in unexpected ways, your unit test will fail for the wrong reasons. A unit test should only fail if your code doesn’t do what it’s supposed to do.

Some time ago I tweeted this:

I didn't mention this yet, but I'm working on a series of videos about the subject matter of my Principles of Package Design book.

— Matthias Noback (@matthiasnoback) May 14, 2015

It turned out, creating a video tutorial isn’t working well for me. I really like writing, and speaking in public, but I’m not very happy about recording videos. I almost never watch videos myself as well, so… the video tutorial I was talking about won’t be there. Sorry!

Event sourcing with Broadway

At the Dutch PHP Conference I attended a workshop by Beau Simensen and Willem-Jan Zijderveld. They showed us some examples of how to work with Broadway, a framework for event sourcing, with full Symfony integration, created by the smart people at Qandidate.

During my two weeks of funemployment, before starting my new job at Ibuildings, I decided to recreate one of my previous projects using Broadway. As it turns out, it’s a great framework that’s quite easy to use and is very powerful at the same time. Even though it’s not a stable package (as in, it’s still in the 0.x version range), I think it’s safe to depend on it.

Bundles, not extensible

A (Symfony) bundle usually contains just some service definition files as well as some configuration options which allow the user of that bundle to configure its behavior (e.g. provide server credentials, etc.). Bundle maintainers usually end up combining lots of these options in one bundle, because they all belong to the same general concept. For example, the NelmioSecurityBundle contains several features which are all related to security, but are code-wise completely unrelated. This means that this one package could have easily been split into several packages. Please note that I’m not making fun of this bundle, nor criticizing it, I’m just taking it as a nice example here. Besides, I’ve created many bundles like this myself.

The PHP community

I’m a proud member of the PHP community. Recently I’ve come to realize though that “the PHP community” is a very complicated thing, and in fact, not even one thing.

I consider it part of my “job” to keep track of what’s going on in the community and I consider Twitter to be the proper place to look for news from the community, as well as a great place to check in on the pulse of the community. What are people happy about, what are people looking at, who are they trolling? This is all fascinating to me.

During the great PHP Benelux Conference 2015 edition I published the final version of Principles of Package Design. Let me tell you a little bit about its history and some of the things I had to deal with.

A bit of history

My first book, A Year With Symfony, was released on September 4th, 2013. Just 6 days later, I started working on Principles of PHP Package Design. As it turned out, it was quite hard to keep the process going. There were several months in which I wrote nothing at all. Sometimes I picked up the work, but then I had completely lost my track and tried to get back on it by revising existing chapters over and over again. Meanwhile all kinds of other projects begged for attention as well, including the release of the backend project of the new nos.nl website, the preparations for the Hexagonal architecture training and the birth of our daughter Julia ;)

It was quite a ride so far. We have seen commands, command buses, events and event buses. We distilled some more knowledge about them while formulating answers to some interesting questions from readers.

Why you should not dispatch events while handling a command

In a previous post we discussed a sample event (the UserSignedUp event):

class UserSignedUp implements Event
{
    public function name()
    {
        return 'user_signed_up';
    }

    public function __construct($userId)
    {
        $this->userId = $userId;
    }

    public function userId()
    {
        return $this->userId;
    }
}

An instance of such an event can be handed over to the event bus. It will look for any number of event handlers that wants to be notified about the event. In the case of the UserSignedUp event, one of the interested event handlers is the SendWelcomeMailWhenUserSignedUp handler:

So far we’ve had three posts in this series about commands, events and their corresponding buses and handlers:

• A wave of command buses
• Responsibilities of the command bus
• From commands to events

Now I’d like to take the time to answer some of the very interesting questions that by readers.

The difference between commands and events

Robert asked:

[…], could you possibly explain what are the main differences between a command bus and an even dispatcher?

In the previous posts we looked at commands and the command bus. Commands are simple objects which express a user’s intention to change something. Internally, the command object is handed over to the command bus, which performs the change that has been requested. While it eventually delegates this task to a dedicated command handler, it also takes care of several other things, like wrapping the command execution in a database transaction and protecting the original order of commands.

In the previous post we looked at commands and how you can use them to separate technical aspects of the input, from the actual behavior of your application. Commands are simple objects, handed over to the command bus, which performs the change that is needed.

As we learned, the command bus eventually calls the command handler which corresponds to the given command object. For example when a SignUp command is provided, the SignUpHandler will be asked to handle the command. So the command bus contains some kind of a lookup mechanism to match commands with their handlers. Some command bus libraries use a naming convention here (e.g. handler name = command name + “Handler”), some use a kind of service locator, etc.

Recently many people in the PHP community have been discussing a thing called the “command bus”. The Laravel framework nowadays contains an implementation of a command bus and people have been talking about it in several vodcasts.

My interest was sparked too. Last year I experimented with LiteCQRS but in the end I developed a collection of PHP packages known as SimpleBus which supports the use of commands and events in any kind of PHP application (there is a Symfony bridge too, if you like that framework). I also cover the subject of commands, events and their corresponding buses extensively during my Hexagonal Architecture workshop.

An ever recurring pattern in my life is this one:

1. I stumble upon some interesting piece of code, an intriguing book chapter, a fascinating concept, etc.
2. Slowly, over the next couple of weeks, my brain realises that, yes, this is some very interesting stuff.
3. Then I want to become all productive about it - writing things on my blog, speaking about it in public, maybe even writing a book about it.

This time it was domain-driven design (DDD), command-query responsibility segregation (CQRS) and in particular its architectural and technical aspects. While playing with existing libraries I soon recognized the huge benefits of applying hexagonal architecture and some of the tactical DDD patterns to a (Symfony) codebase.

Don’t reinvent the wheel

There is this ongoing discussion in the PHP community (and I guess in every software-related community) about reinventing wheels. A refreshing angle in this debate came from an article by Phil Sturgeon pointing to the high number of “duplicate” packages available on Packagist. I agree with Phil:

Sometimes these are carbon copies of other packages, but often they are feature-weak versions of established packages.

It doesn’t make sense to do the same thing over and over again. At least I personally don’t try to make this mistake. If I want to write code that “already exists”, at least I don’t publish it on Packagist.

Decoupling from a service locator - shouldn’t that be: don’t use a service locator?

Well, not really, since there are lots of valid use cases for using a service locator. The main use case is for making things lazy-loading (yes, you can also use some kind of proxy mechanism for that, but let’s assume you need something simpler). Say we have this EventDispatcher class:

class EventDispatcher
{
    public function __construct(array $listeners)
    {
        $this->listeners = $listeners;
    }

    public function dispatch($event)
    {
        foreach ($this->listeners[$event] as $listener) {
            $listener->notify();
        }
    }
}

Now it appears that some event listeners are rather expensive to instantiate. And, even though it may never be notified (because it listens to a rare event), in order to register any event listener, we need to instantiate it:

I just ate a nice pizza at my hotel room in Barcelona. The funny thing is (at least, to a Dutch guy that is): they wouldn’t be able to give me a pizza before 20:00h. At that time in my home country we have long forgotten our desserts, cleaned the dishes and are starting to think about sleeping (just kidding). Anyway, I got the pizza, it was good and now I’m here to write a little report on the things that happened during the last three days.

Of course I’m going to write something about the new Symfony Best Practices book that was written by Fabien Potencier, Ryan Weaver and Javier Eguiluz. It got a lot of negative responses, at least in my Twitter feed, and I think it’s a good idea to read Richard Miller’s post for some suggestions on how to deal with an “early preview release” like this and how we can be a bit more positive about it.

This is a response to Peter Petermann’s article Composer and virtual packages. First, let’s make this totally clear: I don’t want to start an Internet war about this, I’m just pointing out some design issues that may arise from using Composer’s provide option in your package’s composer.json file. This means it’s also nothing personal. To Peter: you wrote a very nice article and shed light on an underused feature of Composer. Thank you for that!

As a follow-up on a previous article I have some announcements to make.

Feedback

A wonderful number of 67 of you have provided very valuable feedback about “A Year With Symfony” - you sure wrote some nice things about my book! Just a small selection of highlights:

Reading “A Year With Symfony” helped to clarify and validate the ideas and structures that I had been developing with my team, along with explaining some of the internals of Symfony that the documentation did not.

The problem

With a new bundle release you may want to rename services or parameters, make a service private, change some constructor arguments, change the structure of the bundle configuration, etc. Some of these changes may acually be backwards incompatible changes for the users of that bundle. Luckily, the Symfony DependenyInjection component and Config component both provide you with some options to prevent such backwards compatibility (BC) breaks. If you want to know more about backwards compatibility and bundle versioning, please read my previous article on this subject.

A short introduction to semantic versioning

Semantic versioning is an agreement between the user of a package and its maintainer. The maintainer should be able to fix bugs, add new features or completely change the API of the software they provide. At the same time, the user of the package should not be forced to make changes to their own project whenever a package maintainer decides to release a new version.

Symfony bundles: providing services and exposing resources

When you look at the source code of the Symfony framework, it becomes clear that bundles play two distinct and very different roles: in the first place a bundle is a service container extension: it offers ways to add, modify or remove service definitions and parameters, optionally by means of bundle configuration. This role is represented by the following methods of BundleInterface:

namespace Symfony\Component\HttpKernel\Bundle;

interface BundleInterface extends ContainerAwareInterface
{
    /** Boots the Bundle. */
    public function boot();

    /** Shutdowns the Bundle. */
    public function shutdown();

    /** Builds the bundle. */
    public function build(ContainerBuilder $container);

    /** Returns the container extension that should be implicitly loaded. */
    public function getContainerExtension();

    ...
}

The second role of a bundle is that of a resource provider. When a bundle is registered in the application kernel, it automatically starts to expose all kinds of resources to the application. Think of routing files, controllers, entities, templates, translation files, etc.

About interface segregation, dependency inversion and package stability

You are creating a nice reusable package. Inside the package you want to use events to allow others to hook into your own code. You look at several event managers that are available. Since you are somewhat familiar with the Symfony EventDispatcher component already, you decide to add it to your package’s composer.json:

{
    "name": "my/package"
    "require": {
        "symfony/event-dispatcher": "~2.5"
    }
}

Your dependency graph now looks like this:

Recently I realized that some of the problems I encountered in the past could have been easily solved by what I’m about to explain in this post.

The problem: an event listener introduces a circular reference

The problem is: having a complicated graph of service definitions and their dependencies, which causes a ServiceCircularReferenceException, saying ‘Circular reference detected for service “…”, path: “… -> … -> …”.’ Somewhere in the path of services that form the circle you then find the event_dispatcher service. For example: event_dispatcher -> your_event_listener -> some_service -> event_dispatcher.

Almost a year ago, on September 4th, in front of a live audience, I published my first book “A Year With Symfony”. Since the first release, well over a 1000 people have read it. In February of this year, I added a new chapter to the book (about annotations).

A third edition?

Flipping through the pages and thinking about how I could best celebrate 1 year and 1000+ readers, I had the following ideas:

In the introduction to this series I mentioned that testing object interactions can be really hard. Most unit testing tutorials cover this subject by introducing the PHPUnit mocking sub-framework. The word “mock” in the context of PHPUnit is given the meaning of the general concept of a “test double”. In reality, a mock is a very particular kind of test double. I can say after writing lots of unit tests for a couple of years now that my testing experience would have definitely been much better if I had known about the different kinds of test doubles that you can use in unit tests. Each type of test double has its own merits and it is vital to the quality of your test suite that you know when to use which one.

Fernando Arconada interviewed me about the subject of testing. He is writing a book about testing Symfony2 applications: Testing para Aplicaciones Symfony2. Fernando will translate this interview to Spanish and and add it to his book, together with the articles in my A better PHP testing experience series.

Who is Matthias Noback?

I’m a PHP developer, writer and speaker. I live in Zeist, The Netherlands, with my girlfriend, a son of 9 and our newborn daughter. Currently I have my own business, called Noback’s Office. This really gives me a lot of freedom: I work as a developer on one project for about half of the week and in the remaining time I can either spend some time with my family or write blog posts, or finish my second book.

In the introduction article of this series I quickly mentioned that I think unit testing often focuses too much on assertions. The historic reason for this is that in introductory articles and workshops it is often said that:

1. You are supposed to pick the most specific assertion the testing framework offers.
2. You are supposed to have only one assertion in each test method.
3. You are supposed to write the assertion first, since that is the goal you are working towards.

I used to preach these things myself too (yes, “development with tests” often comes with a lot of preaching). But now I don’t follow these rules anymore. I will shortly explain my reasons. But before I do, let’s take a step back and consider something that is known as the Test framework in a tweet, by Mathias Verraes. It looks like this:

This is the introduction to a series of articles related to what I call: the “PHP testing experience”. I must say I’m not really happy with it. And so are many others I think. In the last couple of years I’ve met many developers who experienced a lot of trouble while trying to make testing a serious part of their development workflow. It is, I admit, a hard thing to accomplish. I see many people fail at it. Either the learning curve is too steep for them or they are lacking some insight into the concepts and reasoning behind testing. This has unfortunately led many of them to stop trying.

Thanks! Let me explain myself

First of all: thanks everybody for reading the previous parts of this series. Many people posted some interesting comments. I realized quickly that I had to explain myself a bit: why am I writing all of this? Why would you ever want to decouple controllers from the (Symfony2) framework? You probably don’t need to bother anyway, because

The chances of you needing to move controllers to other frameworks is next to none. — Rafael Dohms

In the previous part of this series we decreased coupling of a Symfony controller to the Symfony2 framework by removing its dependency on the standard Controller class from the FrameworkBundle.

Now we take a look at annotations. They were initially introduced for rapid development (no need to create/modify some configuration file, just solve the issues inline!):

namespace Matthias\ClientBundle\Controller;

use Sensio\Bundle\FrameworkExtraBundle\Configuration\Route;
use Sensio\Bundle\FrameworkExtraBundle\Configuration\Method;
use Sensio\Bundle\FrameworkExtraBundle\Configuration\Template;
use Sensio\Bundle\FrameworkExtraBundle\Configuration\ParamConverter;

/**
 * @Route("/client")
 */
class ClientController
{
    /**
     * @Route('/{id}')
     * @Method("GET")
     * @ParamConverter(name="client")
     * @Template
     */
    public function detailsAction(Client $client)
    {
        return array(
            'client' => $client
        );
    }
}

When you use these annotations, the details action will be executed when the URL matches /client/{id}. A param converter will be used to fetch a Client entity from the database based on the id parameter which is extracted from the URL by the router. The return value of the action is an array. These are used as template variables for rendering the Resources/views/Client/Details.html.twig template.

Part I: Don’t use the standard controller

The general belief is that controllers are the most tightly coupled classes in every application. Most of the time based on the request data, they fetch and/or store persistent data from/in some place, then turn the data into HTML, which serves as the response to the client who initially made the request.

So controllers are “all over the place”, they glue parts of the application together which normally lie very far from each other. This would make them highly coupled: they depend on many different things, like the Doctrine entity manager, the Twig templating engine, the base controller from the FrameworkBundle, etc.

It appears that I didn’t make myself clear while writing about entity managers and manager registries yesterday. People were quick to reply that instead you should inject entity repositories. However, I wasn’t talking about entity repositories here. I was talking about classes that get an EntityManager injected because they want to call persist() or flush(). The point of my previous post was that in those cases you should inject the manager registry, because you don’t know beforehand which entity manager manages the entities you are trying to persist. By injecting a manager registry you also make your code useful in contexts where another Doctrine persistence library is used.

Or: how to make your persistence-related code more reusable

You probably know that you should trim your controllers as much as possible. This usually means extracting some classes from the semi-procedural code inside your controller class. You create services for the extracted classes and inject the necessary dependencies into them. You can no longer use the convenient $this->getDoctrine()->getManager(). Instead you inject the doctrine.orm.default_entity_manager service into every service that needs the EntityManager to persist data in your relational database:

Legacy code

I’m happy that I’ve discovered the work of P.M. Jones recently. His and mine interests seem to align at several interesting points. Though I don’t personally enjoy “putting .308 holes in targets at 400 yards” (as quoted by Phil Sturgeon), I do care a great deal about package coupling (and cohesion for that matter) and I’m also lightly inflammable when it comes to the use of service locators. It also appears that Paul, just like myself and many others in this business, has felt the pain of working on a legacy PHP application, trying to add features to it, or change existing behavior. This is a particularly hard thing to do and because so many incompetent developers have been creating PHP applications since the dawn of PHP, chances are that a big part of the job of competent PHP developers nowadays consists of maintaining these dumpsites of include statements and superglobals.

Seven months, two presentations and three blog posts later, I’ve published the first installment of my new book, Principles of PHP Package Design.

From the introduction of the book:

Naturally the biggest part of this book covers package design principles. But before we come to that, we first take a close look at what constitutes packages: classes and interfaces. The way you design them has great consequences for the characteristics of the package in which they will eventually reside. So before considering package design principles themselves, we first need to take a look at the principles that govern class design. These are the so-called SOLID principles. Each letter of this acronym stands for a different principle, each of which we will briefly (re)visit in the first part of this book.

On several occasions I have tried to explain my opinion about “optional dependencies” (also known as “suggested dependencies” or “dev requirements”) and I’m doing it again:

There’s no such thing as an optional dependency.

I’m talking about PHP packages here and specifically those defined by a composer.json file.

What is a dependency?

First let’s make sure we all agree about what a dependency is. Take a look at the following piece of code:

A couple of months ago I wrote about console commands that create a PID file containing the process id (PID) of the PHP process that runs the script. It is very usual to create such a PID file when the process forks itself and thereby creates a daemon which could in theory run forever in the background, with no access to terminal input or output. The process that started the command can take the PID from the PID file and use it to determine whether or not the daemon is still running.

Last week was a particularly interesting week. I organised three coding dojos, on three different locations. The first one was at the IPPZ office (one of the companies I’ve worked for), the second one was at the SweetlakePHP meetup in Zoetermeer. The last one was for a group of developers from NOS, a large Dutch news organization. Although that would have been enough, I finished the week with my revised talk “Principles of PHP Package Design” at a meeting of the Dutch Symfony Usergroup in Utrecht…

My first book, A Year With Symfony, has been available since September 4th last year. That’s not even six months ago, but right now it has almost 800 readers already (digital and printed edition combined).

During the past few days I’ve taken the time to write an extra chapter for the book. Consider it as a big thank you to everybody who bought the book! I feel very much supported by the Symfony/PHP community and this really keeps me going.

About the nature of PHP and its misuse in package design

This text will be part of the introduction of my upcoming book Principles of PHP Package Design. If you’d like to be notified when the book is released, you can subscribe on the book’s page. This will entitle you to a 20% discount once the book is available for download.

Before we continue with the “main matter” of this book, I’d like to introduce you to the mindset I had while writing it. This particular mindset amounts to having a constant awareness that PHP has somewhat of a dark side, as well as keeping our hopes high that it is very well possible to expose this “dark side” to a warm sun of programming principles, coding standards and generally good ideas, collected over the years, carefully nurtured and enhanced by great programmers of different languages and cultures.

Last year in September Len Epp from Leanpub.com interviewed me about my book A Year With Symfony. They have fully transcribed the interview as well as published the recording of the interview.

This is the first time somebody interviewed me about my career and my book writing. I remember back then it made me quite nervous. But I must say that talking with Len was a very good way to shed some light on my own thought process and my personal goals. For instance, who did I have in mind as the reader of my book?

Recently I tweeted about phpclasses.org. It was not such a friendly statement:

Why does phpclasses.org still exist? Most of the “packages” contain dangerous, stupid or useless code.

Manuel Lemos, the man behind PHP Classes, made me pull back a bit by pointing out that I was generalizing and that they do what they can to encourage people to do a good job. I recognize their effort. And of course, there is also good code on phpclasses.org.

Recently I read the book Signaling PHP by Cal Evans. It’s a short book, yet very affordable and it learned me a couple of things. First of all it explains about how you can “capture” a Ctrl+C on your long-running command and do some necessary cleanup work before actually terminating the application. In the appendix it also mentioned the interesting concept of a PID file. Such a file is used to store a single process identifier. The file is created by the process itself, to allow other scripts or applications to terminate it. This is especially useful in the context of daemon applications. They can be started by some wrapper script, run in the background, then be monitored and eventually interrupted by an administrator using the SIGINT signal.

This article could have been titled “Ten things I hate about bundles”, but that would be too harsh. The things I am going to describe, are not things I hate, but things I don’t like. Besides, when I count them, I don’t come to ten things…

Bundle extension discovery

A Symfony2 bundle can have an Extension class, which allows you to load service definitions from configuration files or define services for the application in a more dynamic way. From the Symfony documentation:

A couple of weeks ago the reader count of my first book A Year With Symfony reached the number 365. It seemed to me an appropriate moment to write something about my experiences. In this post, I will not be too humble, and just cite some people who wrote some very nice things about my book on Twitter.

The book was highly anticipated ever since I first wrote something about it on Twitter. Leanpub book pages have a nice feature where users can subscribe to news about the book release.

As you may have read on the Symfony blog: as of Symfony 2.4 you can register console commands using the service tag console.command.

What is the big deal, you would say. Well, let’s discover it now. This is how the documentation tells you to write your console commands:

namespace Matthias\ConsoleBundle\Command;

use Symfony\Bundle\FrameworkBundle\Command\ContainerAwareCommand;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;

class MyCommand extends ContainerAwareCommand
{
    protected function configure()
    {
        $this->setName('my:action');
    }

    protected function execute(InputInterface $input, OutputInterface $output)
    {
        // do something
    }
}

How to make sure your command will be noticed?

Until Symfony 2.4 there were two requirements for your command to be picked up by the console application itself:

Yesterday I presented “Principles of PHP Package Design” at the monthly meetup of the AmsterdamPHP usergroup. This was quite an experimental presentation. I had prepared it on the same day actually, though I had been gathering the contents of it for several weeks.

Package design is a subject I’m very much interested in and you can be sure to hear more from me about it. Actually, I started writing a book about it! If you’re interested too, sign up on leanpub.com/principles-of-php-package-design.

Some of you may already know: I am a big fan of TDD - the real TDD, where you start with a failing test, and only write production code to make that specific test pass (there are three rules concerning this actually).

The TDD approach usually works pretty well for most classes in a project. But there are also classes which are somewhat difficult to test, like your bundle’s extension class, or its configuration class. The usual approach is to skip these classes, or to test them only using integration tests, when they are used in the context of a running application. This means you won’t have all the advantages of unit tests for these “units of code”. Some of the execution paths of the code in these classes may never be executed in a test environment, so errors will be reported when the code is running in production already.

After working on my book “A Year With Symfony” for four months, I published it yesterday at the Dutch Symfony Usergroup meetup in Amsterdam. It was really great to be able to do this real-time while everybody in the room was looking at the screen waiting for the build process to finish.

The book is hosted on Leanpub, which is a great platform for writing and publishing books. They provide the entire infrastructure. You get a good-looking landing page and they handle all things related to payment.

A couple of days ago I was asked to explain myself: “Why Symfony?”. I was reminded of this video - an interview with Fabien Potencier:

Fabien mentions some facts about Symfony that are key to understanding it, as an outsider. These same facts (and some more) will also help you to “sell” Symfony to others.

As you may have heard, I’m working on a book for Symfony2 developers. Besides The Book, which is the user documentation for building Symfony applications, some book-like websites and just a few e-books, there are currently no books for intermediate or advanced Symfony developers. My new book, called A Year With Symfony is exactly that.

If you have been reading the documentation and maybe some blog posts, you will want to know how you can get one step further. You wonder how you could structure your application, for it to be maintainable for more than a month. You ask yourself what it would take to write good bundles, which guidelines you and your team should define and follow. You may also question the built-in security measurements and you or your manager would like to know what needs to built on top of them to make a real secure application. A Year With Symfony will help you find an answer to all these questions.

I really like to write console commands for Symfony2 applications. There is something very cool about the Symfony Console Component which always makes me look for new things that I could do from the command line. A very simple command might look like this:

namespace Matthias\BatchProcessBundle\Command;

use Symfony\Bundle\FrameworkBundle\Command\ContainerAwareCommand;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;

class BatchProcessCommand extends ContainerAwareCommand
{
    protected function configure()
    {
        $this->setName('matthias:batch-process');
    }

    protected function execute(InputInterface $input, OutputInterface $output)
    {
        // do something here

        ...
    }
}

Writing good, clean command classes can very problematic: you want to provide detailed output to the user, and tell him what is currently going on, what the result of some action was, or why something failed. You need the console output for this. It is an instance of OutputInterface and it is provided as the second argument of the command’s execute() method. The body of this method will usually look like this:

Unit tests are tests on a micro-scale. When unit testing, you are testing little units of your code to make sure that, given a certain input, they produce the output you expected. When your unit of code makes calls to other objects, you can “mock” or “stub” these objects and verify that a method is called a specific number of times, or to make sure the unit of code you’re testing will receive the correct data from the other objects.

Below you will find the slides for my presentation about Dependency Injection Smells (see also one of my previous posts from which this idea originated). If you have any questions, or feedback, please let me know by posting a comment below (or on joind.in).

Dependency Injection Smells from matthiasnoback

This week is Dutch PHP Conference week! It is something I’m very excited about. Three years ago I attended this conference for the first time. It was also my first time to visit any (PHP) conference at all, and this is a big conference, with lots of speakers, lots of attendees and multiple tracks. The Dutch PHP Conference (DPC) has a certain grandeur - many large rooms, many famous developers speaking. It is an inspiring event. Visiting the conference back in 2010 lit a fire in me to start investigating anything there was to know about writing good code: reusable, clean code, with well named, short and testable methods, readable and maintainable by others. In other words, I started being critical of the code I produced.

The Symfony Form Component has an architecture that keeps inspiring me. It supports horizontal and vertical inheritance and it has a solid event system on a very fine-grained level. In this post I would like to explain how you can expand the event system with your own events. As an example., I will create a new event type used for indicating that the bound form is valid.

Using event listeners and event subscribers with forms

As you can read in a Symfony Cookbook article you can already hook into special form events, defined in Symfony\Component\Form\FormEvents:

Part II of this series is all about validating the user’s session. You can find Part I here, if you missed it.

Collect Failed Authentication Attempts

Now and then a user will forget his password and try a few times before going to the “reset password” page. However, when a “user” keeps trying to authenticate with bad credentials, you may be subject to a brute-force attack. Therefore, you should collect failed authentication attempts. Your strategy may then be to block the account until further notice, while providing the user with a way to re-activate his account. When authentication fails, an event is fired, which you may intercept by registering an event listener or subscriber:

When working with Symfony2, you already have many of the finest tools for securing your web application. There are cases however that require you to add that extra bit. In this post I will point you to the right extension points within a Symfony2 project (or any other project which uses the Security Component for that matter).

Install NelmioSecurityBundle

See the README of the NelmioSecurityBundle. It contains many add-ons for your project, to sign/encrypt cookies, force SSL, prevent clickjacking and prevent untrusted redirects.

The Symfony2 DependencyInjection Component has made my life and work as a developer a lot easier. Choosing the right way to use it however can be a bit difficult sometimes. Knowing what a/the service container can do, helps a lot, and also thinking about how you would do it with just PHP can put you back on track. To be able to recognize some problems related to dependency injection in your own code, I will describe a few “dependency injection smells” below (a term derived from “code smells”, used by Kent Beck, Martin Fowler and the likes).

Writing PHP code with PHP is not very easy. You are constantly switching between the context of the code that generates the code and the code that is to be generated (see, you lost it already!). Some variables are available in the first context, some in the second, and you will have to pass the right values in the right way. One of the areas in Symfony-land where you will have to do these things is when you extend Twig by hooking into the parser and defining your own tags. A tag for example is the “for” in

Symfony2 provides multiple ways of blocking, providing or modifying the response. You can:

1. Intercept each request by listening to the kernel.request event and set the response directly on the event (which will effectively skip execution of a controller)

2. Modify the controller or its arguments by listening to the kernel.controller event, then calling setController on the event object and modifying the attributes of the Request object.

3. Change the response rendered by a controller, by listening to the kernel.response event.

A Symfony2 application provides developers with several ways to manipulate its behavior. First of all, it is possible to define different environments for different scenarios:

• A “prod” environment to be used when the web application is on the live server

• A “dev” environment used while developing the application. Generated parts of the application are regenerated when one of the files the generation was based on has changed

• A “test” environment used when running functional unit tests/li>

These days, good PHP object-oriented libraries are all around and easily available. To me, it is actually thrilling to be part of this flourishing community, while working with Symfony2 and blogging about the Framework, the Components and their neighbors (like Silex). It seems like everything is made for contributing to this nice and friendly environment, with tools like GitHub (online collaboration), Composer (dependency management), Packagist (package archive) and Travis CI (continuous integration).

Still, to me, contributing felt like too big a step to take right now. Until a few weeks ago, when I was looking for something I needed (a PHP client for the Microsoft Translator API) and could not find a decent solution. I decided to make it myself, and share it online. Below I’ve written down my steps. As you can see, they are very easy and would require just a bit of extra time. So, take from it what you need, and start contributing!

In my previous post I wrote about uploading files to GridFS. Therefor I created a MongoDB Document with a $file property annotated with @MongoDB\File. Because I am using ORM entities more often then ODM documents, I was looking for a seamless way to access a Document from an Entity.

Because it isn’t possible to define a direct relationship between an Entity and a Document I thought it would be a solid solution to create a custom field type. By defining a custom field type I can control the way the reference to the Document will be stored and at the same time I will be able to restore the reference when retrieving the field. The steps needed to create a custom field type for ORM entities are very similar to the post of Matthias on how to create custom field types for ODM documents.

Almost at the same time, I silently celebrate the first birthday of my blog. My first article appeared a little over a year ago. It is great to see how Symfony2 has become more and more popular during these twelve months. Your comments and visits encourage me to keep posting articles. So, thank you all! And thanks, Dennis, for contributing.

GridFS is a specification for storing large files in MongoDB. In this post I will explain how you can easily upload a file to GridFS and then retrieve it from the database to serve it to the browser.

Just a quick post…

What seems to be missing from the DoctrineMongoDBBundle is a ParamConverter service which resolves request attributes to controller arguments by fetching a Document using the MongoDB DocumentManager. For entities, this would work:

/**
 * @Route("/blog/{id}
 */
public function showAction(Post $post)
{
    // $post will be the entity Post with the "id" taken from the route pattern
}

This works because of the DoctrineParamConverter, which is registered by default by the SensioFrameworkExtraBundle. But only for Doctrine ORM, and not for Doctrine MongoDB ODM. As Christophe Coevoet mentioned when someone tried to implement this missing feature, it can be added easily by yourself, without writing any PHP code, though it might not be so clear how to accomplish this. Still, the only thing you have to do is add a service to your services.xml file:

With Doctrine MongoDB ODM it is possible to add custom field types and define how its values should be converted from and to the database. But type management in the MongoDB ODM currently suffers from several design flaws. This makes it non-trivial to create a custom type, especially when your type conversion has any dependencies. This is how it works:

use Doctrine\ODM\MongoDB\Mapping\Types\Type;

Type::registerType('custom', 'Matthias\CustomTypeBundle\MongoDB\Type\CustomType');

$type = Type::getType('custom');
// $type is an instance of Matthias\CustomTypeBundle\MongoDB\Type\CustomType

As you will understand, the Type class is both a Registry and a Factory. Yet, it only allows you to define a type as a class, not as an object. This means: no constructor arguments can be passed. When using Symfony2, this implies that types can not be services, and you can use neither constructor nor setter injection.

Authorization

When any of the authentication providers has verified the still unauthenticated token, an authenticated token will be returned. The authentication listener should set this token directly in the SecurityContext using its setToken() method.

From then on, the user is authenticated, i.e. means identified. Now, other parts of the application can use the token to decide whether or not the user may request a certain URI, or modify a certain object. This decision will be made by an instance of AccessDecisionManagerInterface.

Authentication

When a request points to a secured area, and one of the listeners from the firewall map is able to extract the user’s credentials from the current Request object, it should create a token, containing these credentials. The next thing the listener should do is ask the authentication manager to validate the given token, and return an authenticated token when the supplied credentials were found to be valid. The listener should then store the authenticated token in the security context:

The Security Context

Central to the Security Component is the security context, which is an instance of SecurityContext. When all steps in the process of authenticating the user have been taken successfully, the security context may be asked if the authenticated user has access to a certain action or resource of the application.

use Symfony\Component\Security\SecurityContext;
use Symfony\Component\Security\Core\Exception\AccessDeniedException;

$context = new SecurityContext();

// authenticate the user...

if (!$context->isGranted('ROLE_ADMIN')) {
    throw new AccessDeniedException();
}

A firewall for HTTP requests

Authenticating a user is done by the firewall. An application may have multiple secured areas, so the firewall is configured using a map of these secured areas. For each of these areas, the map contains a request matcher and a collection of listeners. The request matcher gives the firewall the ability to find out if the current request points to a secured area. The listeners are then asked if the current request can be used to authenticate the user.

In the Symfony2 security documentation both the firewalls and the access control rules are demonstrated using the “path” option, which is used to determine if a firewall or rule is applicable to the current URL. Also the “ip” option is demonstrated. The fact of the matter is, the string based configuration options in security.yml are transformed into objects of class RequestMatcher. This is a curious class in the HttpFoundation component which allows you to match a given Request object. The Security component uses it to determine if it should activate a certain firewall for the current request (usually only by checking the request’s path info).

The Symfony Security Component has an AccessDecisionManager which decides whether or not the currently authenticated user has a right to be in some place (or for that matter, use a certain service from the service container, or even call a certain method). The decision manager looks at the current user’s roles, and compares them to the attributes that are required. It relies on dedicated voters to make it’s verdict.

The component itself ships with an AuthenticatedVoter. It supports the “IS_AUTHENTICATED_FULLY”, “IS_AUTHENTICATED_REMEMBERED” and “IS_AUTHENTICATED_ANONYMOUSLY” attributes, which allow you to differentiate between users who are authenticated in the normal way, via a “remember me” cookie, or anonymously (which means: no credentials were supplied, but the user still gets a security context).

The Symfony Security Component provides a two-layer security system: first it authenticates a user, then is authorizes him for the current request. Authentication means “identify yourself”. Authorization means: “let’s see if you have the right to be here”.

The deciding authority in this case will be the AccessDecisionManager. It has a number of voters (which you may create yourself too). Each voter will be asked if the authenticated user has the right “roles” for the current URL.

In my previous post, I wrote about wrapping a legacy application in Silex, using output buffering and Twig. Finally, to allow for better decoupling as well as lazy loading of services, we passed the actual Silex\Application instance as the first argument of legacy controllers.

The first and quite easy way we can enhance our legacy application, is to make use of the request service (which contains all the details about the current request, wrapped inside the Symfony HttpFoundation’s Request class). So, instead of reading directly from $_GET and $_POST, we can change the edit_category() controller into the following:

Ever since I am using the Symfony Framework (be it version 1 or 2), I tend to describe every other project I’ve done (including those that were built on top of some third party “framework” like Joomla or WordPress) as a “legacy project”. Though this has sometimes felt like treason, I still keep doing it: the quality of applications written using Symfony is usually so much higher in terms of maintainability, security and code cleanliness, that even a project done last year using “only PHP” looks like a mess and seems to be no good software at all. So I feel the strong urge to rebuild everything I have in portfolio (as do many other developers), but “this time, I will do it the right way”.

Apparently not everyone agrees on how to unit test their Symfony2 controllers. Some treat controller code as the application’s “glue”: a controller does the real job of transforming a request to a response. Thus it should be tested by making a request and check the received response for the right contents. Others treat controller code just like any other code - which means that every path the interpreter may take, should be tested.

My previous post was about finding and loading configuration files. I now continue my Symfony2 Config Component quest with a post about the way you can “semantically expose your configuration” (using the TreeBuilder). I wrote this piece to later contribute it to the Symfony documentation so feedback is very welcome!

Validate configuration values

After loading configuration values from all kinds of resources, the values and their structure can be validated using the Definition part of the Symfony2 Config Component. Configuration values are usually expected to show some kind of hierarchy. Also, values should be of a certain type, be restricted in number or be one of a given set of values. For example, the following configuration (in Yaml) shows a clear hierarchy and some validation rules that should be applied to it (like: “the value for ‘auto_connect’ must be a boolean”):

The Symfony2 Config Component provides several classes to help you find, load, combine, autofill and validate configuration values of any kind, whatever their source may be (Yaml, XML, INI files, or for instance a database).

Locating resources

Loading the configuration normally starts with a search for resources - in most cases: files. This can be done with FileLocator:

use Symfony\Component\Config\FileLocator;

$configDirectories = array(__DIR__ . DIRECTORY_SEPARATOR . 'app' . DIRECTORY_SEPARATOR . 'config');

$locator = new FileLocator($configDirectories);
$yamlUserFiles = $locator->locate('users.yml', null, false);

The locator receives a collection of locations where it should look for files. The first argument of locate() is the name of the file to look for. The second argument may be the current path and when supplied, the locator will look in this directory first. The third argument indicates whether or not the locator should return the first file it has found, or an array containing all matches.

The JMSSerializerBundle has a VersionExclusionStrategy, which allows you to serialize/deserialize objects for a specific version of your API. You can mark the properties that are available for different versions using the @Since and @Until annotations:

use JMS\SerializerBundle\Annotation\Type;
use JMS\SerializerBundle\Annotation\Since;
use JMS\SerializerBundle\Annotation\Until;

class Comment
{
    /**
     * @Type("DateTime")
     * @Since("1.2.0")
     */
    private $createdAt;

    /**
     * @Type("DateTime")
     * @Until("2.1.3")
     */
    private $updatedAt;
}

The only thing you have to do is tell the serializer which version to use, before you start using it:

In my previous post about the experimental DOM stream wrapper, I discussed the issues that came forth when opening the stream. Now, let’s take a look at another very important thing you want to do with a stream: read from and to it, and maybe move the pointer back and forth.

Reading from the stream

The DOM stream wrapper is already able to open an XML file and look for a specified node in the DOM:

With all the fancy XML parsers/serializers/deserializers around, you would almost forget there are built-in PHP functions for defining highly customized XML parsers. You can use the xml_* functions to assemble an XML parser step by step.

Setting up an XML parser - the procedural way

First, you create a new parser resource using

$encoding = 'UTF-8';
$parser = xml_parser_create($encoding);

The first and only argument of this function is the target encoding of the parser.

I recently felt a strong urge to write something about implementing a stream wrapper for PHP. A stream wrapper is a way to handle file interaction of any kind. PHP has built-in stream wrappers for HTTP, FTP, the filesystem, etc. But you are also allowed to implement custom protocols using your own stream wrapper. Stream wrappers are used by all file functions, like fopen() and fgets(). Creating a custom stream wrapper begins with creating a class (it does not have to extend anything) and then making a call to stream_wrapper_register. In this post (and future posts on the same subject) I will develop a stream wrapper for manipulating a DOMNode’s value using traditional file manipulation functions. The stream wrapper class is called “DOMStreamWrapper” and we register it for the protocol “dom”:

I recently took a quick look at Composer and I have come to like it very much. I want to demonstrate the incredible ease of use. My example is also not trivial, it means to elaborate on the concept of a Console Application. The Symfony2 documentation does not tell us how to use the Console Component stand-alone, but only shows us how to add new commands to an already existing application. Therefore, let’s use Composer to set up a very light-weight Console Application, a good starting point from where you can add your own commands.

After creating a metadata factory and metadata drivers, we now need a way to cache the metadata, since collecting them at each request is way too inefficient. Luckily, the Metadata library provides us with a FileCache (though it may be any kind of a cache, as long as it implements the same CacheInterface). First, we make the FileCache available as a service, and add a call to setCache on the metadata factory:

<parameter key="matthias_annotation.metadata.cache_class">Metadata\Cache\FileCache</parameter>

<!-- ... -->

<service id="matthias_annotation.metadata.cache" class="%matthias_annotation.metadata.cache_class%" public="false">
    <argument /><!-- the cache directory (to be set later) -->
</service>

<service id="matthias_annotation.metadata_factory" class="%matthias_annotation.metadata_factory.class%" public="false">
    <argument type="service" id="matthias_annotation.driver_chain" />
    <!-- call setCache with the new cache service: -->
    <call method="setCache">
        <argument type="service" id="matthias_annotation.metadata.cache" />
    </call>
</service>

This provides the metadata factory with the file cache.

In my previous post, I wrote about annotations and how to gather them using a metadata factory and an annotation driver. I also mentioned a future post in which I explain other ways of collecting metadata, namely by locating and parsing files in several different formats (distinguished by their extension). This is the post. In the case of our @DefaultValue annotation, I’m thinking of a Yaml flavoured alternative. This allows us to store all default values as an array of property “name” - “value” pairs, like

Earlier I wrote about how to create a custom annotation class. I used the annotation reader from Doctrine Common to check for the existence of the custom annotation inside the DocComment block. It is also possible to process the annotations on beforehand, and collect the processed data in ClassMetadata and PropertyMetadata objects. These objects are created by a MetadataFactory. The factory uses Drivers to collect the metadata.

My purpose in this article is to create a custom annotation @DefaultValue which allows me to define default values for properties of a class. It should work like this:

In my previous post I wrote about a listener that deserializes the request content and replaces controller arguments with the result. Johannes Schmitt made the suggestion to use a ParamConverter for this. This of course makes sense: currently there is only a ParamConverter for converting some “id” argument into an entity with the same id (see the documentation for @ParamConverter.

In this post I will show you how to create your own ParamConverter and how we can specialize it in deserializing request content.

Based on the list of most popular search results leading to my blog, using Symfony2 for building some kind of webservice seems to be quite “hot”. It also seems many people are struggling with the question how they should serialize their content to and from XML and/or JSON. Two beautiful bundles are already available for this, FOSRestBundle and JMSSerializerBundle. Both will help you a lot with this issue, by providing transparent conversion between formats for both the request content and the response body. Nevertheless, a few things are missing. For example, I want to really serialize and deserialize objects, for example, when I send a request with a comment (or part of it) like this:

The class Silex\Application extends \Pimple, a very lean dependency injection container, which itself implements \ArrayAccess. This allows you to define parameters and services and retrieve them from the Application like keys in an array. The first time I defined a service in a Silex application, in my case the Buzz Browser, I did it “inline”, i.e. inside my app.php file. This is how I did it:

use Buzz\Browser;

$app = new Silex\Application();

$app['autoloader']->registerNamespace('Buzz', __DIR__.'/vendor/buzz/lib');

$app['buzz.client_class'] = 'Buzz\\Client\\Curl';

$app['browser'] = $app->share(function() use ($app) {
    $clientClass = $app['buzz.client_class'];
    return new Browser(new $clientClass);
});

// $app['browser'] is ready for use

But this is not very reusable; every Silex application that needs Buzz\Browser as a service, needs to take care of the autoloading, configure and define the service. The Browser service is not such a very complex service, but think about everything that needs to be done to define and configure a service like the Doctrine DBAL (of course, Silex has a DoctrineServiceProvider for that already…).

So, as easy as Fabien makes this look like, in my case it wasn’t that easy to get Sismo (his small yet very nice personal continuous integration “server”) up and running on my local machine. These were the steps I had to take:

1. Create a directory for Sismo, e.g. /Users/matthiasnoback/Sites/sismo

2. Download sismo.php and copy the file to the directory you have just created

3. Create a VirtualHost for Sismo (for example, use sismo.local as a server name

When you see yourself repeating a number of assertions in your unit tests, or you have to think hard each time you make some kind of assertion, it’s time to create your own assertions, which wraps these complicated assertions into one single method call on your TestCase class. In the example below, I will create a custom assertion which would recognize the following JSON string as a “successful response”:

{"success":true}

Inside a TestCase we could run the following lines of code to verify the successfulness of the given JSON response string:

Recently I created my first WordPress template (the one you are looking at right now) and I was left both in shock and in great awe. This piece of software written in PHP has many users (like myself), yet it is so very outdated. The filtering mechanism by which you can modify HTML after it is generated by WordPress feels very dirty. But because a lot of the HTML snippets that WordPress generates are not suitable for decorating with Twitter Bootstrap’s beautiful CSS rules, I used those dirty filters extensively.

In my previous post I wrote down a set of requirements for Silex applications. There were a few things left for another post: first of all, I want to have unit tests, nicely organized in directories that correspond to the namespaces of my project’s classes. This means that the tests for Acme\SomeNamespace\SomeClass should be found in Acme\Tests\SomeNamespace\SomeClass.

Organizing your unit tests

I want to write my tests for the PHPUnit framework. This allows me to use some PHPUnit best practices: first of all we define our PHPUnit configuration file in /app/phpunit.xml. First of all we point PHPUnit to a bootstrap file (of which we will later define it’s contents). We also set an environment variable called “env” whose value is “test”. Finally we set the location of our app.php file inside the server variable “env”.

When I created my first Silex project, I almost felt encouraged to let go of my high standards in programming. So things were starting to look very much like my “legacy” PHP projects, in which everything was put in functions with lengthy parameter lists and those functions were called from within a single index.php file. I ignored many of the things about high quality software development I had learned in previous years. The result of this, was a project much less maintainable than my other recent projects.

I was looking for a way to serialize domain objects into JSON strings and back again into objects of the original class. This is essential for webservices, especially those built around the JSON-RPC specification that allow clients to make calls to their server and, for example add new personal data to their database of contacts. It makes life a lot easier when this data matches the field names and inner structure of your own domain objects. For example, a JSON string like {"name":"Matthias Noback"} should be easily translatable to an object of class Acme\Webservice\Entity\Person with a private property name. But how do you know which class to instantiate, when you only have the simple JSON string above?

Earlier I was looking for an alternative to Symfony1’s routing.load_configuration event, so I could add some extra routes on-the-fly. This may be useful, when routes change in more ways than only variable request parameters as part of routes do (you know, like /blog/{id}). I got it completely wrong in my previous post about this subject. Of course, adding extra routes is a matter of creating a custom routing loader, and tell the framework about it using the service container. So, there we go.

I was looking into the Doctrine Common library; it seems to me that especially the AnnotationReader is quite interesting. Several Symfony2 bundles use annotation for quick configuration. For example adding an @Route annotation to your actions allows you to add them “automatically” to the route collection. The bundles that leverage the possibilities of annotation all use the Doctrine Common AnnotationReader (in fact, the cached version) for retrieving all the annotations from your classes and methods. The annotation reader works like this: it looks for annotations, for which it tries to instantiate a class. This may be a class made available by adding a use statement to your file. That is why you have to add use Sensio\Bundle\FrameworkExtraBundle\Configuration\Route to any PHP file in which you use the @Route annotation.

With Symfony2, many things are managed through dependency injection. Except for forms. Oh, wait, forms can be services too of course! Remember? Any class instance can be a service… Now, as in many other cases, Symfony2’s FrameworkBundle adds some magic to the Form component by creating services that link several parts of the Form component together. For example, depending on the value of the framework.csrf_protection parameter in config.yml file a hidden field “_token” will be added to all forms, site-wide. To create this kind of very general “form extension” in your own project, you need to do just a few things (I am going to use the example of a “Captcha” form extension, but I don’t give a full implementation of this idea).

One of the ugly things about Symfony 1 validators was that their dependencies were generally fetched from far away, mainly by calling sfContext::getInstance(). With Symfony2 and it’s service container, this isn’t necessary anymore. Validators can be services. The example below is quite simple, but given you can inject anything you want, you can make it quite complicated. I show you how to create a validator that checks with the router of the value-under-validation is a route.

Symfony 1 had the lovely routing.load_configuration event. Listening to this event enabled the developer to add some routes dynamically, “on-the-fly”. Plugins used to do this most of the time. I was looking for a way to accomplish the same in Symfony 2. I’ve used the following as a solution.

We are going to listen to the kernel.request event, but we make sure we get notified in an early stage, so that the router hasn’t done it’s magic yet. Then we quickly add some extra routes to the RouteCollection.

PHPUnit 3.6 allows us to create our own so-called ResultPrinters. Using such a printer is quite necessary in the case of running your unit tests from within the browser (see my previous post), since we don’t print to a console, but to a screen. You can make this all as nice as you like, but here is the basic version of it.

Create the HtmlResultPrinter

First create the file containing your (for example) HtmlResultPrinter, for example in /src/Acme/DemoBundle/PHPUnit/HtmlResultPrinter.

When you don’t have access to the command-line of your webserver, it may be nice to still run all your unit tests; so you need a way to execute the phpunit command from within a controller. This way, you can call your test suite by browsing to a URL of your site. To do things right, we start with a “test” controller /web/app_test.php containing these lines of code:

if (!in_array(@$_SERVER['REMOTE_ADDR'], array(
    '127.0.0.1',
    '::1',
))) {
    header('HTTP/1.0 403 Forbidden');
    exit('You are not allowed to access this file. Check '.basename(__FILE__).' for more information.');
}

require_once __DIR__.'/../app/bootstrap.php.cache';
require_once __DIR__.'/../app/AppKernel.php';

use Symfony\Component\HttpFoundation\Request;

$kernel = new AppKernel('test', true);
$kernel->loadClassCache();
$kernel->handle(Request::createFromGlobals())->send();

In fact, just copy everything from /web/app_dev.php but replace the string “dev” by “test”.

After reading the Cookbook article How to expose a Semantic Configuration for a Bundle I became interested in the possibilities for defining a configuration tree using the TreeBuilder from the Symfony component Config. This TreeBuilderallows you to build a schema against which the configuration values (or short: config) from/app/config.yml` and the like are to be validated.

Create an Extension

First make sure that your bundle has a /DependencyInjection/[BundleNameWithoutBundle]Extension.php containing the extension class:

One of the beautiful things about PHPUnit is the way it can be easily configured: for example you can alter the paths in which PHPUnit will look for TestCase classes. In your Symfony2 project this can be done in the file /app/phpunit.xml (if it is called phpunit.xml.dist, rename it to phpunit.xml first!). This file contains a default configuration which suffices in most situations, but in case your TestCase classes are housed somewhere else than in your *Bundle/Test, add an extra “directory” to the “testsuite” element in phpunit.xml:

In some cases you want to return a very specific Response in case the Request has some properties like a certain response format.

This situation calls for an event listener, which listens to the kernel.request event. The listener itself should be a service with some special characteristics: it has the tag “kernel.event_listener” and some extra attributes, i.e. the name of the event to which it listens and the method of the service that should be called when this event occurs. When you want to return a very specific Response, you should listen to the kernel.request event. We want the kernel to call the method onKernelRequest(), which we will define later.

I was looking for a way to automatically generate some documentation, and because all the required information was already in the DocComment blocks. I downloaded the great documentation generator DocBlox (which is a PHP application by itself) and looked in it’s source code. Soon I found DocBlox’s Reflection classes which are able to parse a DocComment block. So I wanted to use these classes! Here is how you can do it:

Sometimes you need to make changes to the Response object, after it is returned by your controller, but before it is rendered as output to the client (e.g. the browser). You may want to set some extra response headers, or “completely mess up the content” of the response. You can accomplish this by creating an event listener that listens to the kernel.response event. I give you a sample event listener which changes the Content-Type header in case the requested format is “json” and the browser’s accepted response format contains “text/html”; in that case, at least in my experience, the browser doesn’t render the JSON string as plain text when the status code is 4xx or 5xx. So in these situations, the event listener changes the “Content-Type” to “text/plain”, to be sure you always get decent output in the browser.

Symfony2 firewalls depend for their authentication on UserProviders. These providers are requested by the authentication layer to provide a User object, for a given username. Symfony will check whether the password of this User is correct (i.e. verify it’s password) and will then generate a security token, so the user may stay authenticated during the current session. Out of the box, Symfony has a “in_memory” user provider and an “entity” user provider. In this post I’ll show you how to create your own UserProvider. The UserProvider in this example, tries to load a Yaml file containing information about users in the following format:

First of all: dependency injection is GREAT!

Several of Symfony2’s core services depend on tags to recognize which user-defined services should be loaded, notified of events, etc. For example, Twig uses the tag twig.extension to load extra extensions. It would also be great to use tags in case your service implements some kind of “chain”, in which several alternative strategies are tried until one of them is successful. In this post I use the example of a so-called “TransportChain”. This chain consists of a set of classes which implement the Swift_Transport interface. Using the chain, the mailer may try several ways of transport, until one succeeds. This post focuses on the “dependency injection” part of the story. The implementation of the real TransportChain is left to the reader (as an exercise ;)).

Last week I was one of the several hundreds of PHP developers who traveled “all te way” to Amsterdam for the Dutch PHP Conference 2010. Though I had the impression that many PHP developers are using the symfony framework for their web applications, the speakers at this conference were friendlier towards the “father of the framework”, Fabien Potencier, than towards the framework itself. Fabien is a much admired developer himself, who is very conscious about software architecture, latest trends in PHP development and also is aware of the importance of fully covering the framework’s code with unit tests. In this article I will try to apply to the symfony framework, several comments on PHP programming and frameworks that I heard during this conference.

For my new workshop - “Building Autonomous Services” - I needed to define several Docker containers/services with more or less the same setup:

1. A PHP-FPM process for running the service’s PHP code.
2. An Nginx process for serving static and dynamic requests (using the PHP-FPM process as backend).

To route requests properly, every Nginx service would have its own hostvcname. I didn’t want to do complicated things with ports though - the Nginx services should all listen to port 80. However, on the host machine, only one service can listen on port 80. This is where reverse HTTP proxy Traefik did a good job: it is the only service listening on the host on port 80, and it forwards requests to the right service based on the host name from the request.

Here’s what I wanted to do:

• Run the official redis image as a service in a cluster set up with Docker Swarm.
• Configure a password to be used by connecting clients. See also Redis’s AUTH command. The relevant command-line option when starting the redis server would be --requirepass.

This is just a quick post, sharing what I’ve figured out while trying to accomplish all of this. I hope it’s useful in case you’re looking for a way to make a container image (an official one or your own) ready to be used with Docker Secrets.

The “builder pattern”

As a programmer you may know the Gang-of-Four Builder design pattern. The Docker builder pattern has nothing to do with that. The builder pattern describes the setup that many people use to build a container. It involves two Docker images:

1. a “build” image with all the build tools installed, capable of creating production-ready application files.
2. a “service” image capable of running the application.

Sharing files between containers

At some point the production-ready application files need to be copied from the build container to the host machine. There are several options for that.

In the previous posts we looked at creating a build container, and after that we created a blog container, serving our generated static website.

It’s quite surprising to me how simple the current setup is — admittedly, it’s a simple application too. It takes about 50 lines of configuration to get everything up and running.

The idea of the blog container, which has nginx as its main process, is to deploy it to a production server whenever we feel like it, in just “one click”. There should be no need to configure a server to host our website, and it should not be necessary to build the application on the server too. This is in fact the promise, and the true power of Docker.

In the previous post we looked at the process of designing a build container, consisting of all the required build tools for generating a static website from source files. In order to see the result of the build process, we still need to design another container, which runs a simple web server, serving the static website (mainly .html, .css, .js and .jpg files).

Designing the blog container

We’ll use a light-weight install of Nginx as the base image and simply copy the website files to the default document root (/usr/share/nginx/html) (only after removing any placeholder files that are currently inside that directory). The complete file docker/blog/Dockerfile looks like this:

Recently a former colleague of mine, Lucas van Lierop, showed me his new website, which he created using Spress. Lucas took two bold moves: he started freelancing, and he open-sourced his website code. This to me was very inspiring. I’ve been getting up to speed with Docker recently and am planning to do a lot more with it over the coming months, and being able to take a look at the source code of up-to-date projects that use Docker is certainly invaluable.

A little background: speakers don’t get paid

Speakers like myself don’t get paid for doing a talk at a tech conference. That’s why I call this work “open source”. People will get a video or audio recording of the talk, including separately viewable or downloadable slides for free. The idea is, a conference costs a lot of money to organise. It is quite expensive to fly in all those speakers. So there’s no money to pay the speakers for all their work (for me personally it’s about 80 hours of preparation, plus time spent travelling, usually half a day before and after the conference). Speakers get their travel costs reimbursed, they often get two nights at a hotel, and a ticket to the conference. Plus, they get advertising for their personal brand (increasing their reputation as an expert, a funny person, or just a person with more Google results for their name).

Last year I took several Scala courses on Coursera. It was an interesting experience and it has brought me a lot of new ideas. One of these is the idea of a singleton object (as opposed to a class). It has the following characteristics:

• There is only one instance of it (hence it’s called a “singleton”, but isn’t an implementation of the Singleton design pattern).
• It doesn’t need to be explicitly instantiated (it doesn’t have the traditional static getInstance() method). In fact, an instance already exists when you first want to use it.
• There doesn’t have to be built-in protection against multiple instantiations (as there is and can only be one instance, by definition).

Converting this notion to PHP is impossible, but if it was possible, you could do something like this:

So, here’s a bit of a personal blog post for once.

Symfony trademark policy

I saw this tweet:

Suite à une mise en demeure, j’ai retiré les tutoriels qui concernent Symfony du Site. Il n’y aura pas de futures vidéos sur le Framework.

— Grafikart (@grafikart_fr) March 11, 2017

Meaning: “Following a formal notice, I removed the tutorials that are related to Symfony from the Site. There will be no future videos on the Framework.”

About 2 years ago I created a package that combines the power of two famous Symfony components: the Form component and the Console component. In short: this package allows you to interactively fill in a form by typing in the answers at the CLI. When I started working on it, this seemed like a pretty far-fetched idea. However, it made a lot of sense to me in terms of a ports & adapters architecture that I was looking for back then (and still always am, by the way). We could (and often should) write the code in our application layer in such a way that it doesn’t make a big difference whether we call applications services from a web controller or from a CLI “controller”.

Event sourcing with Broadway

At the Dutch PHP Conference I attended a workshop by Beau Simensen and Willem-Jan Zijderveld. They showed us some examples of how to work with Broadway, a framework for event sourcing, with full Symfony integration, created by the smart people at Qandidate.

During my two weeks of funemployment, before starting my new job at Ibuildings, I decided to recreate one of my previous projects using Broadway. As it turns out, it’s a great framework that’s quite easy to use and is very powerful at the same time. Even though it’s not a stable package (as in, it’s still in the 0.x version range), I think it’s safe to depend on it.

Bundles, not extensible

A (Symfony) bundle usually contains just some service definition files as well as some configuration options which allow the user of that bundle to configure its behavior (e.g. provide server credentials, etc.). Bundle maintainers usually end up combining lots of these options in one bundle, because they all belong to the same general concept. For example, the NelmioSecurityBundle contains several features which are all related to security, but are code-wise completely unrelated. This means that this one package could have easily been split into several packages. Please note that I’m not making fun of this bundle, nor criticizing it, I’m just taking it as a nice example here. Besides, I’ve created many bundles like this myself.

An ever recurring pattern in my life is this one:

1. I stumble upon some interesting piece of code, an intriguing book chapter, a fascinating concept, etc.
2. Slowly, over the next couple of weeks, my brain realises that, yes, this is some very interesting stuff.
3. Then I want to become all productive about it - writing things on my blog, speaking about it in public, maybe even writing a book about it.

This time it was domain-driven design (DDD), command-query responsibility segregation (CQRS) and in particular its architectural and technical aspects. While playing with existing libraries I soon recognized the huge benefits of applying hexagonal architecture and some of the tactical DDD patterns to a (Symfony) codebase.

Thanks! Let me explain myself

First of all: thanks everybody for reading the previous parts of this series. Many people posted some interesting comments. I realized quickly that I had to explain myself a bit: why am I writing all of this? Why would you ever want to decouple controllers from the (Symfony2) framework? You probably don’t need to bother anyway, because

The chances of you needing to move controllers to other frameworks is next to none. — Rafael Dohms

In the previous part of this series we decreased coupling of a Symfony controller to the Symfony2 framework by removing its dependency on the standard Controller class from the FrameworkBundle.

Now we take a look at annotations. They were initially introduced for rapid development (no need to create/modify some configuration file, just solve the issues inline!):

namespace Matthias\ClientBundle\Controller;

use Sensio\Bundle\FrameworkExtraBundle\Configuration\Route;
use Sensio\Bundle\FrameworkExtraBundle\Configuration\Method;
use Sensio\Bundle\FrameworkExtraBundle\Configuration\Template;
use Sensio\Bundle\FrameworkExtraBundle\Configuration\ParamConverter;

/**
 * @Route("/client")
 */
class ClientController
{
    /**
     * @Route('/{id}')
     * @Method("GET")
     * @ParamConverter(name="client")
     * @Template
     */
    public function detailsAction(Client $client)
    {
        return array(
            'client' => $client
        );
    }
}

When you use these annotations, the details action will be executed when the URL matches /client/{id}. A param converter will be used to fetch a Client entity from the database based on the id parameter which is extracted from the URL by the router. The return value of the action is an array. These are used as template variables for rendering the Resources/views/Client/Details.html.twig template.

Part I: Don’t use the standard controller

The general belief is that controllers are the most tightly coupled classes in every application. Most of the time based on the request data, they fetch and/or store persistent data from/in some place, then turn the data into HTML, which serves as the response to the client who initially made the request.

So controllers are “all over the place”, they glue parts of the application together which normally lie very far from each other. This would make them highly coupled: they depend on many different things, like the Doctrine entity manager, the Twig templating engine, the base controller from the FrameworkBundle, etc.

It appears that I didn’t make myself clear while writing about entity managers and manager registries yesterday. People were quick to reply that instead you should inject entity repositories. However, I wasn’t talking about entity repositories here. I was talking about classes that get an EntityManager injected because they want to call persist() or flush(). The point of my previous post was that in those cases you should inject the manager registry, because you don’t know beforehand which entity manager manages the entities you are trying to persist. By injecting a manager registry you also make your code useful in contexts where another Doctrine persistence library is used.

Or: how to make your persistence-related code more reusable

You probably know that you should trim your controllers as much as possible. This usually means extracting some classes from the semi-procedural code inside your controller class. You create services for the extracted classes and inject the necessary dependencies into them. You can no longer use the convenient $this->getDoctrine()->getManager(). Instead you inject the doctrine.orm.default_entity_manager service into every service that needs the EntityManager to persist data in your relational database:

As I promised in my previous “Bash practices” post, I would discuss a query function in this article. Here you have it:

function create_temporary_directory() {
    directory=$(mktemp -d "$1/XXXXX")
}

create_temporary_directory

echo "$directory"

That’s a bad query function!

This function is supposed to return the path of a temporary directory. It accepts one argument: an existing directory we want to create the temporary directory in. It uses the mktemp utility to do the “heavy” lifting. It accepts as an argument a kind of a template for directory names (XXXXX will be replaced by 5 random characters). After creating the directory, mktemp echos the full path of the directory to stdout, meaning we can copy it into a variable by using the $(...) syntax.

Forgive me for my bad pun. As I mentioned in my previous Bash post I’m going to show you some ways in which you can improve the design of Bash scripts. Again, it’s a weird language, and a lot of what’s below probably won’t feel natural to you. Anyway, there we go.

I started out with a piece of code that looked like this:

BUILD_DIR="build"

function clean_up() {
    rm -r "$BUILD_DIR"
}

clean_up

Function arguments

Inside a function you can use all global and environment variables, which easily leads to smelly code like this: clean_up will behave differently based on what’s in the global variable BUILD_DIR. This makes the function itself quite unpredictable, but also error-prone, as the value of BUILD_DIR may at one point not contain the name of a directory, or even be an empty string. Usually we would fix this by providing the path to the directory we’d like to remove as an argument of the function call, like this:

A bit of reminiscing

When I was a kid, MS Windows was still a program called WIN.COM which you needed to start from the MS-DOS command prompt. You could also add it to AUTOEXEC.BAT which was a so-called batch file. You could write these .BAT files yourself. They were basically just command-line scripts. You could make them execute commands, print things, collect input, and make simple decisions. It wasn’t much, and I remember that you often needed some helper .COM or .EXE programs to accomplish anything useful. The most advanced thing I ever wrote was a nice little ASCII-art menu program, spread across multiple .BAT files (with GOTOs and all), which allowed me to easily start my favorite games, like Super Tetris, known to me as SUPERTET.EXE, or Prince of Persia.

TLDR;

• I won’t update A Year With Symfony anymore.
• The e-book is now freely available.

My first book, A Year With Symfony, was well received when I first published it on September 4th. At the time it seemed to be a welcome addition to the official Symfony documentation.

A lot of water has passed under the bridge since then. Several new Symfony components were released and we have now arrived at the next major version, 3.0. In the meantime I published one update of the book: a new chapter about Annotations.

Everybody is organizing their own PHP conference these days. It’s impossible to “catch ’em all”. There’s conferences clashing, conferences too far away, conferences too much like other conferences, etc.

Then there’s conferences that have never been there before, are quite original and are therefore spark everyone’s imagination. One such a conference is the Symfony Catalunya conference. This “international Symfony event” is being organized by the generally awesome and friend of the community Marc Morera. The concept seems to be pretty simple:

Decoupling from a service locator - shouldn’t that be: don’t use a service locator?

Well, not really, since there are lots of valid use cases for using a service locator. The main use case is for making things lazy-loading (yes, you can also use some kind of proxy mechanism for that, but let’s assume you need something simpler). Say we have this EventDispatcher class:

class EventDispatcher
{
    public function __construct(array $listeners)
    {
        $this->listeners = $listeners;
    }

    public function dispatch($event)
    {
        foreach ($this->listeners[$event] as $listener) {
            $listener->notify();
        }
    }
}

Now it appears that some event listeners are rather expensive to instantiate. And, even though it may never be notified (because it listens to a rare event), in order to register any event listener, we need to instantiate it:

I just ate a nice pizza at my hotel room in Barcelona. The funny thing is (at least, to a Dutch guy that is): they wouldn’t be able to give me a pizza before 20:00h. At that time in my home country we have long forgotten our desserts, cleaned the dishes and are starting to think about sleeping (just kidding). Anyway, I got the pizza, it was good and now I’m here to write a little report on the things that happened during the last three days.

Of course I’m going to write something about the new Symfony Best Practices book that was written by Fabien Potencier, Ryan Weaver and Javier Eguiluz. It got a lot of negative responses, at least in my Twitter feed, and I think it’s a good idea to read Richard Miller’s post for some suggestions on how to deal with an “early preview release” like this and how we can be a bit more positive about it.

As a follow-up on a previous article I have some announcements to make.

Feedback

A wonderful number of 67 of you have provided very valuable feedback about “A Year With Symfony” - you sure wrote some nice things about my book! Just a small selection of highlights:

Reading “A Year With Symfony” helped to clarify and validate the ideas and structures that I had been developing with my team, along with explaining some of the internals of Symfony that the documentation did not.

The problem

With a new bundle release you may want to rename services or parameters, make a service private, change some constructor arguments, change the structure of the bundle configuration, etc. Some of these changes may acually be backwards incompatible changes for the users of that bundle. Luckily, the Symfony DependenyInjection component and Config component both provide you with some options to prevent such backwards compatibility (BC) breaks. If you want to know more about backwards compatibility and bundle versioning, please read my previous article on this subject.

A short introduction to semantic versioning

Semantic versioning is an agreement between the user of a package and its maintainer. The maintainer should be able to fix bugs, add new features or completely change the API of the software they provide. At the same time, the user of the package should not be forced to make changes to their own project whenever a package maintainer decides to release a new version.

Symfony bundles: providing services and exposing resources

When you look at the source code of the Symfony framework, it becomes clear that bundles play two distinct and very different roles: in the first place a bundle is a service container extension: it offers ways to add, modify or remove service definitions and parameters, optionally by means of bundle configuration. This role is represented by the following methods of BundleInterface:

namespace Symfony\Component\HttpKernel\Bundle;

interface BundleInterface extends ContainerAwareInterface
{
    /** Boots the Bundle. */
    public function boot();

    /** Shutdowns the Bundle. */
    public function shutdown();

    /** Builds the bundle. */
    public function build(ContainerBuilder $container);

    /** Returns the container extension that should be implicitly loaded. */
    public function getContainerExtension();

    ...
}

The second role of a bundle is that of a resource provider. When a bundle is registered in the application kernel, it automatically starts to expose all kinds of resources to the application. Think of routing files, controllers, entities, templates, translation files, etc.

About interface segregation, dependency inversion and package stability

You are creating a nice reusable package. Inside the package you want to use events to allow others to hook into your own code. You look at several event managers that are available. Since you are somewhat familiar with the Symfony EventDispatcher component already, you decide to add it to your package’s composer.json:

{
    "name": "my/package"
    "require": {
        "symfony/event-dispatcher": "~2.5"
    }
}

Your dependency graph now looks like this:

Recently I realized that some of the problems I encountered in the past could have been easily solved by what I’m about to explain in this post.

The problem: an event listener introduces a circular reference

The problem is: having a complicated graph of service definitions and their dependencies, which causes a ServiceCircularReferenceException, saying ‘Circular reference detected for service “…”, path: “… -> … -> …”.’ Somewhere in the path of services that form the circle you then find the event_dispatcher service. For example: event_dispatcher -> your_event_listener -> some_service -> event_dispatcher.

Almost a year ago, on September 4th, in front of a live audience, I published my first book “A Year With Symfony”. Since the first release, well over a 1000 people have read it. In February of this year, I added a new chapter to the book (about annotations).

A third edition?

Flipping through the pages and thinking about how I could best celebrate 1 year and 1000+ readers, I had the following ideas:

A couple of months ago I wrote about console commands that create a PID file containing the process id (PID) of the PHP process that runs the script. It is very usual to create such a PID file when the process forks itself and thereby creates a daemon which could in theory run forever in the background, with no access to terminal input or output. The process that started the command can take the PID from the PID file and use it to determine whether or not the daemon is still running.

Last week was a particularly interesting week. I organised three coding dojos, on three different locations. The first one was at the IPPZ office (one of the companies I’ve worked for), the second one was at the SweetlakePHP meetup in Zoetermeer. The last one was for a group of developers from NOS, a large Dutch news organization. Although that would have been enough, I finished the week with my revised talk “Principles of PHP Package Design” at a meeting of the Dutch Symfony Usergroup in Utrecht…

My first book, A Year With Symfony, has been available since September 4th last year. That’s not even six months ago, but right now it has almost 800 readers already (digital and printed edition combined).

During the past few days I’ve taken the time to write an extra chapter for the book. Consider it as a big thank you to everybody who bought the book! I feel very much supported by the Symfony/PHP community and this really keeps me going.

Recently I read the book Signaling PHP by Cal Evans. It’s a short book, yet very affordable and it learned me a couple of things. First of all it explains about how you can “capture” a Ctrl+C on your long-running command and do some necessary cleanup work before actually terminating the application. In the appendix it also mentioned the interesting concept of a PID file. Such a file is used to store a single process identifier. The file is created by the process itself, to allow other scripts or applications to terminate it. This is especially useful in the context of daemon applications. They can be started by some wrapper script, run in the background, then be monitored and eventually interrupted by an administrator using the SIGINT signal.

This article could have been titled “Ten things I hate about bundles”, but that would be too harsh. The things I am going to describe, are not things I hate, but things I don’t like. Besides, when I count them, I don’t come to ten things…

Bundle extension discovery

A Symfony2 bundle can have an Extension class, which allows you to load service definitions from configuration files or define services for the application in a more dynamic way. From the Symfony documentation:

A couple of weeks ago the reader count of my first book A Year With Symfony reached the number 365. It seemed to me an appropriate moment to write something about my experiences. In this post, I will not be too humble, and just cite some people who wrote some very nice things about my book on Twitter.

The book was highly anticipated ever since I first wrote something about it on Twitter. Leanpub book pages have a nice feature where users can subscribe to news about the book release.

As you may have read on the Symfony blog: as of Symfony 2.4 you can register console commands using the service tag console.command.

What is the big deal, you would say. Well, let’s discover it now. This is how the documentation tells you to write your console commands:

namespace Matthias\ConsoleBundle\Command;

use Symfony\Bundle\FrameworkBundle\Command\ContainerAwareCommand;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;

class MyCommand extends ContainerAwareCommand
{
    protected function configure()
    {
        $this->setName('my:action');
    }

    protected function execute(InputInterface $input, OutputInterface $output)
    {
        // do something
    }
}

How to make sure your command will be noticed?

Until Symfony 2.4 there were two requirements for your command to be picked up by the console application itself:

Some of you may already know: I am a big fan of TDD - the real TDD, where you start with a failing test, and only write production code to make that specific test pass (there are three rules concerning this actually).

The TDD approach usually works pretty well for most classes in a project. But there are also classes which are somewhat difficult to test, like your bundle’s extension class, or its configuration class. The usual approach is to skip these classes, or to test them only using integration tests, when they are used in the context of a running application. This means you won’t have all the advantages of unit tests for these “units of code”. Some of the execution paths of the code in these classes may never be executed in a test environment, so errors will be reported when the code is running in production already.

After working on my book “A Year With Symfony” for four months, I published it yesterday at the Dutch Symfony Usergroup meetup in Amsterdam. It was really great to be able to do this real-time while everybody in the room was looking at the screen waiting for the build process to finish.

The book is hosted on Leanpub, which is a great platform for writing and publishing books. They provide the entire infrastructure. You get a good-looking landing page and they handle all things related to payment.

A couple of days ago I was asked to explain myself: “Why Symfony?”. I was reminded of this video - an interview with Fabien Potencier:

Fabien mentions some facts about Symfony that are key to understanding it, as an outsider. These same facts (and some more) will also help you to “sell” Symfony to others.

As you may have heard, I’m working on a book for Symfony2 developers. Besides The Book, which is the user documentation for building Symfony applications, some book-like websites and just a few e-books, there are currently no books for intermediate or advanced Symfony developers. My new book, called A Year With Symfony is exactly that.

If you have been reading the documentation and maybe some blog posts, you will want to know how you can get one step further. You wonder how you could structure your application, for it to be maintainable for more than a month. You ask yourself what it would take to write good bundles, which guidelines you and your team should define and follow. You may also question the built-in security measurements and you or your manager would like to know what needs to built on top of them to make a real secure application. A Year With Symfony will help you find an answer to all these questions.

I really like to write console commands for Symfony2 applications. There is something very cool about the Symfony Console Component which always makes me look for new things that I could do from the command line. A very simple command might look like this:

namespace Matthias\BatchProcessBundle\Command;

use Symfony\Bundle\FrameworkBundle\Command\ContainerAwareCommand;
use Symfony\Component\Console\Input\InputInterface;
use Symfony\Component\Console\Output\OutputInterface;

class BatchProcessCommand extends ContainerAwareCommand
{
    protected function configure()
    {
        $this->setName('matthias:batch-process');
    }

    protected function execute(InputInterface $input, OutputInterface $output)
    {
        // do something here

        ...
    }
}

Writing good, clean command classes can very problematic: you want to provide detailed output to the user, and tell him what is currently going on, what the result of some action was, or why something failed. You need the console output for this. It is an instance of OutputInterface and it is provided as the second argument of the command’s execute() method. The body of this method will usually look like this:

The Symfony Form Component has an architecture that keeps inspiring me. It supports horizontal and vertical inheritance and it has a solid event system on a very fine-grained level. In this post I would like to explain how you can expand the event system with your own events. As an example., I will create a new event type used for indicating that the bound form is valid.

Using event listeners and event subscribers with forms

As you can read in a Symfony Cookbook article you can already hook into special form events, defined in Symfony\Component\Form\FormEvents:

Part II of this series is all about validating the user’s session. You can find Part I here, if you missed it.

Collect Failed Authentication Attempts

Now and then a user will forget his password and try a few times before going to the “reset password” page. However, when a “user” keeps trying to authenticate with bad credentials, you may be subject to a brute-force attack. Therefore, you should collect failed authentication attempts. Your strategy may then be to block the account until further notice, while providing the user with a way to re-activate his account. When authentication fails, an event is fired, which you may intercept by registering an event listener or subscriber:

When working with Symfony2, you already have many of the finest tools for securing your web application. There are cases however that require you to add that extra bit. In this post I will point you to the right extension points within a Symfony2 project (or any other project which uses the Security Component for that matter).

Install NelmioSecurityBundle

See the README of the NelmioSecurityBundle. It contains many add-ons for your project, to sign/encrypt cookies, force SSL, prevent clickjacking and prevent untrusted redirects.

The Symfony2 DependencyInjection Component has made my life and work as a developer a lot easier. Choosing the right way to use it however can be a bit difficult sometimes. Knowing what a/the service container can do, helps a lot, and also thinking about how you would do it with just PHP can put you back on track. To be able to recognize some problems related to dependency injection in your own code, I will describe a few “dependency injection smells” below (a term derived from “code smells”, used by Kent Beck, Martin Fowler and the likes).

Writing PHP code with PHP is not very easy. You are constantly switching between the context of the code that generates the code and the code that is to be generated (see, you lost it already!). Some variables are available in the first context, some in the second, and you will have to pass the right values in the right way. One of the areas in Symfony-land where you will have to do these things is when you extend Twig by hooking into the parser and defining your own tags. A tag for example is the “for” in

Symfony2 provides multiple ways of blocking, providing or modifying the response. You can:

1. Intercept each request by listening to the kernel.request event and set the response directly on the event (which will effectively skip execution of a controller)

2. Modify the controller or its arguments by listening to the kernel.controller event, then calling setController on the event object and modifying the attributes of the Request object.

3. Change the response rendered by a controller, by listening to the kernel.response event.

A Symfony2 application provides developers with several ways to manipulate its behavior. First of all, it is possible to define different environments for different scenarios:

• A “prod” environment to be used when the web application is on the live server

• A “dev” environment used while developing the application. Generated parts of the application are regenerated when one of the files the generation was based on has changed

• A “test” environment used when running functional unit tests/li>

In my previous post I wrote about uploading files to GridFS. Therefor I created a MongoDB Document with a $file property annotated with @MongoDB\File. Because I am using ORM entities more often then ODM documents, I was looking for a seamless way to access a Document from an Entity.

Because it isn’t possible to define a direct relationship between an Entity and a Document I thought it would be a solid solution to create a custom field type. By defining a custom field type I can control the way the reference to the Document will be stored and at the same time I will be able to restore the reference when retrieving the field. The steps needed to create a custom field type for ORM entities are very similar to the post of Matthias on how to create custom field types for ODM documents.

Almost at the same time, I silently celebrate the first birthday of my blog. My first article appeared a little over a year ago. It is great to see how Symfony2 has become more and more popular during these twelve months. Your comments and visits encourage me to keep posting articles. So, thank you all! And thanks, Dennis, for contributing.

GridFS is a specification for storing large files in MongoDB. In this post I will explain how you can easily upload a file to GridFS and then retrieve it from the database to serve it to the browser.

Just a quick post…

What seems to be missing from the DoctrineMongoDBBundle is a ParamConverter service which resolves request attributes to controller arguments by fetching a Document using the MongoDB DocumentManager. For entities, this would work:

/**
 * @Route("/blog/{id}
 */
public function showAction(Post $post)
{
    // $post will be the entity Post with the "id" taken from the route pattern
}

This works because of the DoctrineParamConverter, which is registered by default by the SensioFrameworkExtraBundle. But only for Doctrine ORM, and not for Doctrine MongoDB ODM. As Christophe Coevoet mentioned when someone tried to implement this missing feature, it can be added easily by yourself, without writing any PHP code, though it might not be so clear how to accomplish this. Still, the only thing you have to do is add a service to your services.xml file:

With Doctrine MongoDB ODM it is possible to add custom field types and define how its values should be converted from and to the database. But type management in the MongoDB ODM currently suffers from several design flaws. This makes it non-trivial to create a custom type, especially when your type conversion has any dependencies. This is how it works:

use Doctrine\ODM\MongoDB\Mapping\Types\Type;

Type::registerType('custom', 'Matthias\CustomTypeBundle\MongoDB\Type\CustomType');

$type = Type::getType('custom');
// $type is an instance of Matthias\CustomTypeBundle\MongoDB\Type\CustomType

As you will understand, the Type class is both a Registry and a Factory. Yet, it only allows you to define a type as a class, not as an object. This means: no constructor arguments can be passed. When using Symfony2, this implies that types can not be services, and you can use neither constructor nor setter injection.

Authorization

When any of the authentication providers has verified the still unauthenticated token, an authenticated token will be returned. The authentication listener should set this token directly in the SecurityContext using its setToken() method.

From then on, the user is authenticated, i.e. means identified. Now, other parts of the application can use the token to decide whether or not the user may request a certain URI, or modify a certain object. This decision will be made by an instance of AccessDecisionManagerInterface.

Authentication

When a request points to a secured area, and one of the listeners from the firewall map is able to extract the user’s credentials from the current Request object, it should create a token, containing these credentials. The next thing the listener should do is ask the authentication manager to validate the given token, and return an authenticated token when the supplied credentials were found to be valid. The listener should then store the authenticated token in the security context:

The Security Context

Central to the Security Component is the security context, which is an instance of SecurityContext. When all steps in the process of authenticating the user have been taken successfully, the security context may be asked if the authenticated user has access to a certain action or resource of the application.

use Symfony\Component\Security\SecurityContext;
use Symfony\Component\Security\Core\Exception\AccessDeniedException;

$context = new SecurityContext();

// authenticate the user...

if (!$context->isGranted('ROLE_ADMIN')) {
    throw new AccessDeniedException();
}

A firewall for HTTP requests

Authenticating a user is done by the firewall. An application may have multiple secured areas, so the firewall is configured using a map of these secured areas. For each of these areas, the map contains a request matcher and a collection of listeners. The request matcher gives the firewall the ability to find out if the current request points to a secured area. The listeners are then asked if the current request can be used to authenticate the user.

In the Symfony2 security documentation both the firewalls and the access control rules are demonstrated using the “path” option, which is used to determine if a firewall or rule is applicable to the current URL. Also the “ip” option is demonstrated. The fact of the matter is, the string based configuration options in security.yml are transformed into objects of class RequestMatcher. This is a curious class in the HttpFoundation component which allows you to match a given Request object. The Security component uses it to determine if it should activate a certain firewall for the current request (usually only by checking the request’s path info).

The Symfony Security Component has an AccessDecisionManager which decides whether or not the currently authenticated user has a right to be in some place (or for that matter, use a certain service from the service container, or even call a certain method). The decision manager looks at the current user’s roles, and compares them to the attributes that are required. It relies on dedicated voters to make it’s verdict.

The component itself ships with an AuthenticatedVoter. It supports the “IS_AUTHENTICATED_FULLY”, “IS_AUTHENTICATED_REMEMBERED” and “IS_AUTHENTICATED_ANONYMOUSLY” attributes, which allow you to differentiate between users who are authenticated in the normal way, via a “remember me” cookie, or anonymously (which means: no credentials were supplied, but the user still gets a security context).

The Symfony Security Component provides a two-layer security system: first it authenticates a user, then is authorizes him for the current request. Authentication means “identify yourself”. Authorization means: “let’s see if you have the right to be here”.

The deciding authority in this case will be the AccessDecisionManager. It has a number of voters (which you may create yourself too). Each voter will be asked if the authenticated user has the right “roles” for the current URL.

Apparently not everyone agrees on how to unit test their Symfony2 controllers. Some treat controller code as the application’s “glue”: a controller does the real job of transforming a request to a response. Thus it should be tested by making a request and check the received response for the right contents. Others treat controller code just like any other code - which means that every path the interpreter may take, should be tested.

My previous post was about finding and loading configuration files. I now continue my Symfony2 Config Component quest with a post about the way you can “semantically expose your configuration” (using the TreeBuilder). I wrote this piece to later contribute it to the Symfony documentation so feedback is very welcome!

Validate configuration values

After loading configuration values from all kinds of resources, the values and their structure can be validated using the Definition part of the Symfony2 Config Component. Configuration values are usually expected to show some kind of hierarchy. Also, values should be of a certain type, be restricted in number or be one of a given set of values. For example, the following configuration (in Yaml) shows a clear hierarchy and some validation rules that should be applied to it (like: “the value for ‘auto_connect’ must be a boolean”):

The Symfony2 Config Component provides several classes to help you find, load, combine, autofill and validate configuration values of any kind, whatever their source may be (Yaml, XML, INI files, or for instance a database).

Locating resources

Loading the configuration normally starts with a search for resources - in most cases: files. This can be done with FileLocator:

use Symfony\Component\Config\FileLocator;

$configDirectories = array(__DIR__ . DIRECTORY_SEPARATOR . 'app' . DIRECTORY_SEPARATOR . 'config');

$locator = new FileLocator($configDirectories);
$yamlUserFiles = $locator->locate('users.yml', null, false);

The locator receives a collection of locations where it should look for files. The first argument of locate() is the name of the file to look for. The second argument may be the current path and when supplied, the locator will look in this directory first. The third argument indicates whether or not the locator should return the first file it has found, or an array containing all matches.

The JMSSerializerBundle has a VersionExclusionStrategy, which allows you to serialize/deserialize objects for a specific version of your API. You can mark the properties that are available for different versions using the @Since and @Until annotations:

use JMS\SerializerBundle\Annotation\Type;
use JMS\SerializerBundle\Annotation\Since;
use JMS\SerializerBundle\Annotation\Until;

class Comment
{
    /**
     * @Type("DateTime")
     * @Since("1.2.0")
     */
    private $createdAt;

    /**
     * @Type("DateTime")
     * @Until("2.1.3")
     */
    private $updatedAt;
}

The only thing you have to do is tell the serializer which version to use, before you start using it:

I recently took a quick look at Composer and I have come to like it very much. I want to demonstrate the incredible ease of use. My example is also not trivial, it means to elaborate on the concept of a Console Application. The Symfony2 documentation does not tell us how to use the Console Component stand-alone, but only shows us how to add new commands to an already existing application. Therefore, let’s use Composer to set up a very light-weight Console Application, a good starting point from where you can add your own commands.

After creating a metadata factory and metadata drivers, we now need a way to cache the metadata, since collecting them at each request is way too inefficient. Luckily, the Metadata library provides us with a FileCache (though it may be any kind of a cache, as long as it implements the same CacheInterface). First, we make the FileCache available as a service, and add a call to setCache on the metadata factory:

<parameter key="matthias_annotation.metadata.cache_class">Metadata\Cache\FileCache</parameter>

<!-- ... -->

<service id="matthias_annotation.metadata.cache" class="%matthias_annotation.metadata.cache_class%" public="false">
    <argument /><!-- the cache directory (to be set later) -->
</service>

<service id="matthias_annotation.metadata_factory" class="%matthias_annotation.metadata_factory.class%" public="false">
    <argument type="service" id="matthias_annotation.driver_chain" />
    <!-- call setCache with the new cache service: -->
    <call method="setCache">
        <argument type="service" id="matthias_annotation.metadata.cache" />
    </call>
</service>

This provides the metadata factory with the file cache.

In my previous post, I wrote about annotations and how to gather them using a metadata factory and an annotation driver. I also mentioned a future post in which I explain other ways of collecting metadata, namely by locating and parsing files in several different formats (distinguished by their extension). This is the post. In the case of our @DefaultValue annotation, I’m thinking of a Yaml flavoured alternative. This allows us to store all default values as an array of property “name” - “value” pairs, like

Earlier I wrote about how to create a custom annotation class. I used the annotation reader from Doctrine Common to check for the existence of the custom annotation inside the DocComment block. It is also possible to process the annotations on beforehand, and collect the processed data in ClassMetadata and PropertyMetadata objects. These objects are created by a MetadataFactory. The factory uses Drivers to collect the metadata.

My purpose in this article is to create a custom annotation @DefaultValue which allows me to define default values for properties of a class. It should work like this:

In my previous post I wrote about a listener that deserializes the request content and replaces controller arguments with the result. Johannes Schmitt made the suggestion to use a ParamConverter for this. This of course makes sense: currently there is only a ParamConverter for converting some “id” argument into an entity with the same id (see the documentation for @ParamConverter.

In this post I will show you how to create your own ParamConverter and how we can specialize it in deserializing request content.

Based on the list of most popular search results leading to my blog, using Symfony2 for building some kind of webservice seems to be quite “hot”. It also seems many people are struggling with the question how they should serialize their content to and from XML and/or JSON. Two beautiful bundles are already available for this, FOSRestBundle and JMSSerializerBundle. Both will help you a lot with this issue, by providing transparent conversion between formats for both the request content and the response body. Nevertheless, a few things are missing. For example, I want to really serialize and deserialize objects, for example, when I send a request with a comment (or part of it) like this:

Recently I created my first WordPress template (the one you are looking at right now) and I was left both in shock and in great awe. This piece of software written in PHP has many users (like myself), yet it is so very outdated. The filtering mechanism by which you can modify HTML after it is generated by WordPress feels very dirty. But because a lot of the HTML snippets that WordPress generates are not suitable for decorating with Twitter Bootstrap’s beautiful CSS rules, I used those dirty filters extensively.

I was looking for a way to serialize domain objects into JSON strings and back again into objects of the original class. This is essential for webservices, especially those built around the JSON-RPC specification that allow clients to make calls to their server and, for example add new personal data to their database of contacts. It makes life a lot easier when this data matches the field names and inner structure of your own domain objects. For example, a JSON string like {"name":"Matthias Noback"} should be easily translatable to an object of class Acme\Webservice\Entity\Person with a private property name. But how do you know which class to instantiate, when you only have the simple JSON string above?

Earlier I was looking for an alternative to Symfony1’s routing.load_configuration event, so I could add some extra routes on-the-fly. This may be useful, when routes change in more ways than only variable request parameters as part of routes do (you know, like /blog/{id}). I got it completely wrong in my previous post about this subject. Of course, adding extra routes is a matter of creating a custom routing loader, and tell the framework about it using the service container. So, there we go.

With Symfony2, many things are managed through dependency injection. Except for forms. Oh, wait, forms can be services too of course! Remember? Any class instance can be a service… Now, as in many other cases, Symfony2’s FrameworkBundle adds some magic to the Form component by creating services that link several parts of the Form component together. For example, depending on the value of the framework.csrf_protection parameter in config.yml file a hidden field “_token” will be added to all forms, site-wide. To create this kind of very general “form extension” in your own project, you need to do just a few things (I am going to use the example of a “Captcha” form extension, but I don’t give a full implementation of this idea).

One of the ugly things about Symfony 1 validators was that their dependencies were generally fetched from far away, mainly by calling sfContext::getInstance(). With Symfony2 and it’s service container, this isn’t necessary anymore. Validators can be services. The example below is quite simple, but given you can inject anything you want, you can make it quite complicated. I show you how to create a validator that checks with the router of the value-under-validation is a route.

Symfony 1 had the lovely routing.load_configuration event. Listening to this event enabled the developer to add some routes dynamically, “on-the-fly”. Plugins used to do this most of the time. I was looking for a way to accomplish the same in Symfony 2. I’ve used the following as a solution.

We are going to listen to the kernel.request event, but we make sure we get notified in an early stage, so that the router hasn’t done it’s magic yet. Then we quickly add some extra routes to the RouteCollection.

When you don’t have access to the command-line of your webserver, it may be nice to still run all your unit tests; so you need a way to execute the phpunit command from within a controller. This way, you can call your test suite by browsing to a URL of your site. To do things right, we start with a “test” controller /web/app_test.php containing these lines of code:

if (!in_array(@$_SERVER['REMOTE_ADDR'], array(
    '127.0.0.1',
    '::1',
))) {
    header('HTTP/1.0 403 Forbidden');
    exit('You are not allowed to access this file. Check '.basename(__FILE__).' for more information.');
}

require_once __DIR__.'/../app/bootstrap.php.cache';
require_once __DIR__.'/../app/AppKernel.php';

use Symfony\Component\HttpFoundation\Request;

$kernel = new AppKernel('test', true);
$kernel->loadClassCache();
$kernel->handle(Request::createFromGlobals())->send();

In fact, just copy everything from /web/app_dev.php but replace the string “dev” by “test”.

After reading the Cookbook article How to expose a Semantic Configuration for a Bundle I became interested in the possibilities for defining a configuration tree using the TreeBuilder from the Symfony component Config. This TreeBuilderallows you to build a schema against which the configuration values (or short: config) from/app/config.yml` and the like are to be validated.

Create an Extension

First make sure that your bundle has a /DependencyInjection/[BundleNameWithoutBundle]Extension.php containing the extension class:

One of the beautiful things about PHPUnit is the way it can be easily configured: for example you can alter the paths in which PHPUnit will look for TestCase classes. In your Symfony2 project this can be done in the file /app/phpunit.xml (if it is called phpunit.xml.dist, rename it to phpunit.xml first!). This file contains a default configuration which suffices in most situations, but in case your TestCase classes are housed somewhere else than in your *Bundle/Test, add an extra “directory” to the “testsuite” element in phpunit.xml:

In some cases you want to return a very specific Response in case the Request has some properties like a certain response format.

This situation calls for an event listener, which listens to the kernel.request event. The listener itself should be a service with some special characteristics: it has the tag “kernel.event_listener” and some extra attributes, i.e. the name of the event to which it listens and the method of the service that should be called when this event occurs. When you want to return a very specific Response, you should listen to the kernel.request event. We want the kernel to call the method onKernelRequest(), which we will define later.

I was looking for a way to automatically generate some documentation, and because all the required information was already in the DocComment blocks. I downloaded the great documentation generator DocBlox (which is a PHP application by itself) and looked in it’s source code. Soon I found DocBlox’s Reflection classes which are able to parse a DocComment block. So I wanted to use these classes! Here is how you can do it:

Sometimes you need to make changes to the Response object, after it is returned by your controller, but before it is rendered as output to the client (e.g. the browser). You may want to set some extra response headers, or “completely mess up the content” of the response. You can accomplish this by creating an event listener that listens to the kernel.response event. I give you a sample event listener which changes the Content-Type header in case the requested format is “json” and the browser’s accepted response format contains “text/html”; in that case, at least in my experience, the browser doesn’t render the JSON string as plain text when the status code is 4xx or 5xx. So in these situations, the event listener changes the “Content-Type” to “text/plain”, to be sure you always get decent output in the browser.

Symfony2 firewalls depend for their authentication on UserProviders. These providers are requested by the authentication layer to provide a User object, for a given username. Symfony will check whether the password of this User is correct (i.e. verify it’s password) and will then generate a security token, so the user may stay authenticated during the current session. Out of the box, Symfony has a “in_memory” user provider and an “entity” user provider. In this post I’ll show you how to create your own UserProvider. The UserProvider in this example, tries to load a Yaml file containing information about users in the following format:

First of all: dependency injection is GREAT!

Several of Symfony2’s core services depend on tags to recognize which user-defined services should be loaded, notified of events, etc. For example, Twig uses the tag twig.extension to load extra extensions. It would also be great to use tags in case your service implements some kind of “chain”, in which several alternative strategies are tried until one of them is successful. In this post I use the example of a so-called “TransportChain”. This chain consists of a set of classes which implement the Swift_Transport interface. Using the chain, the mailer may try several ways of transport, until one succeeds. This post focuses on the “dependency injection” part of the story. The implementation of the real TransportChain is left to the reader (as an exercise ;)).

Last week I was one of the several hundreds of PHP developers who traveled “all te way” to Amsterdam for the Dutch PHP Conference 2010. Though I had the impression that many PHP developers are using the symfony framework for their web applications, the speakers at this conference were friendlier towards the “father of the framework”, Fabien Potencier, than towards the framework itself. Fabien is a much admired developer himself, who is very conscious about software architecture, latest trends in PHP development and also is aware of the importance of fully covering the framework’s code with unit tests. In this article I will try to apply to the symfony framework, several comments on PHP programming and frameworks that I heard during this conference.

In the introduction to this series I mentioned that testing object interactions can be really hard. Most unit testing tutorials cover this subject by introducing the PHPUnit mocking sub-framework. The word “mock” in the context of PHPUnit is given the meaning of the general concept of a “test double”. In reality, a mock is a very particular kind of test double. I can say after writing lots of unit tests for a couple of years now that my testing experience would have definitely been much better if I had known about the different kinds of test doubles that you can use in unit tests. Each type of test double has its own merits and it is vital to the quality of your test suite that you know when to use which one.

Fernando Arconada interviewed me about the subject of testing. He is writing a book about testing Symfony2 applications: Testing para Aplicaciones Symfony2. Fernando will translate this interview to Spanish and and add it to his book, together with the articles in my A better PHP testing experience series.

Who is Matthias Noback?

I’m a PHP developer, writer and speaker. I live in Zeist, The Netherlands, with my girlfriend, a son of 9 and our newborn daughter. Currently I have my own business, called Noback’s Office. This really gives me a lot of freedom: I work as a developer on one project for about half of the week and in the remaining time I can either spend some time with my family or write blog posts, or finish my second book.

In the introduction article of this series I quickly mentioned that I think unit testing often focuses too much on assertions. The historic reason for this is that in introductory articles and workshops it is often said that:

1. You are supposed to pick the most specific assertion the testing framework offers.
2. You are supposed to have only one assertion in each test method.
3. You are supposed to write the assertion first, since that is the goal you are working towards.

I used to preach these things myself too (yes, “development with tests” often comes with a lot of preaching). But now I don’t follow these rules anymore. I will shortly explain my reasons. But before I do, let’s take a step back and consider something that is known as the Test framework in a tweet, by Mathias Verraes. It looks like this:

This is the introduction to a series of articles related to what I call: the “PHP testing experience”. I must say I’m not really happy with it. And so are many others I think. In the last couple of years I’ve met many developers who experienced a lot of trouble while trying to make testing a serious part of their development workflow. It is, I admit, a hard thing to accomplish. I see many people fail at it. Either the learning curve is too steep for them or they are lacking some insight into the concepts and reasoning behind testing. This has unfortunately led many of them to stop trying.

A couple of months ago I wrote about console commands that create a PID file containing the process id (PID) of the PHP process that runs the script. It is very usual to create such a PID file when the process forks itself and thereby creates a daemon which could in theory run forever in the background, with no access to terminal input or output. The process that started the command can take the PID from the PID file and use it to determine whether or not the daemon is still running.

Some of you may already know: I am a big fan of TDD - the real TDD, where you start with a failing test, and only write production code to make that specific test pass (there are three rules concerning this actually).

The TDD approach usually works pretty well for most classes in a project. But there are also classes which are somewhat difficult to test, like your bundle’s extension class, or its configuration class. The usual approach is to skip these classes, or to test them only using integration tests, when they are used in the context of a running application. This means you won’t have all the advantages of unit tests for these “units of code”. Some of the execution paths of the code in these classes may never be executed in a test environment, so errors will be reported when the code is running in production already.

Unit tests are tests on a micro-scale. When unit testing, you are testing little units of your code to make sure that, given a certain input, they produce the output you expected. When your unit of code makes calls to other objects, you can “mock” or “stub” these objects and verify that a method is called a specific number of times, or to make sure the unit of code you’re testing will receive the correct data from the other objects.

Apparently not everyone agrees on how to unit test their Symfony2 controllers. Some treat controller code as the application’s “glue”: a controller does the real job of transforming a request to a response. Thus it should be tested by making a request and check the received response for the right contents. Others treat controller code just like any other code - which means that every path the interpreter may take, should be tested.

So, as easy as Fabien makes this look like, in my case it wasn’t that easy to get Sismo (his small yet very nice personal continuous integration “server”) up and running on my local machine. These were the steps I had to take:

1. Create a directory for Sismo, e.g. /Users/matthiasnoback/Sites/sismo

2. Download sismo.php and copy the file to the directory you have just created

3. Create a VirtualHost for Sismo (for example, use sismo.local as a server name

When you see yourself repeating a number of assertions in your unit tests, or you have to think hard each time you make some kind of assertion, it’s time to create your own assertions, which wraps these complicated assertions into one single method call on your TestCase class. In the example below, I will create a custom assertion which would recognize the following JSON string as a “successful response”:

{"success":true}

Inside a TestCase we could run the following lines of code to verify the successfulness of the given JSON response string:

In my previous post I wrote down a set of requirements for Silex applications. There were a few things left for another post: first of all, I want to have unit tests, nicely organized in directories that correspond to the namespaces of my project’s classes. This means that the tests for Acme\SomeNamespace\SomeClass should be found in Acme\Tests\SomeNamespace\SomeClass.

Organizing your unit tests

I want to write my tests for the PHPUnit framework. This allows me to use some PHPUnit best practices: first of all we define our PHPUnit configuration file in /app/phpunit.xml. First of all we point PHPUnit to a bootstrap file (of which we will later define it’s contents). We also set an environment variable called “env” whose value is “test”. Finally we set the location of our app.php file inside the server variable “env”.

PHPUnit 3.6 allows us to create our own so-called ResultPrinters. Using such a printer is quite necessary in the case of running your unit tests from within the browser (see my previous post), since we don’t print to a console, but to a screen. You can make this all as nice as you like, but here is the basic version of it.

Create the HtmlResultPrinter

First create the file containing your (for example) HtmlResultPrinter, for example in /src/Acme/DemoBundle/PHPUnit/HtmlResultPrinter.

When you don’t have access to the command-line of your webserver, it may be nice to still run all your unit tests; so you need a way to execute the phpunit command from within a controller. This way, you can call your test suite by browsing to a URL of your site. To do things right, we start with a “test” controller /web/app_test.php containing these lines of code:

if (!in_array(@$_SERVER['REMOTE_ADDR'], array(
    '127.0.0.1',
    '::1',
))) {
    header('HTTP/1.0 403 Forbidden');
    exit('You are not allowed to access this file. Check '.basename(__FILE__).' for more information.');
}

require_once __DIR__.'/../app/bootstrap.php.cache';
require_once __DIR__.'/../app/AppKernel.php';

use Symfony\Component\HttpFoundation\Request;

$kernel = new AppKernel('test', true);
$kernel->loadClassCache();
$kernel->handle(Request::createFromGlobals())->send();

In fact, just copy everything from /web/app_dev.php but replace the string “dev” by “test”.

One of the beautiful things about PHPUnit is the way it can be easily configured: for example you can alter the paths in which PHPUnit will look for TestCase classes. In your Symfony2 project this can be done in the file /app/phpunit.xml (if it is called phpunit.xml.dist, rename it to phpunit.xml first!). This file contains a default configuration which suffices in most situations, but in case your TestCase classes are housed somewhere else than in your *Bundle/Test, add an extra “directory” to the “testsuite” element in phpunit.xml:

Legacy code

I’m happy that I’ve discovered the work of P.M. Jones recently. His and mine interests seem to align at several interesting points. Though I don’t personally enjoy “putting .308 holes in targets at 400 yards” (as quoted by Phil Sturgeon), I do care a great deal about package coupling (and cohesion for that matter) and I’m also lightly inflammable when it comes to the use of service locators. It also appears that Paul, just like myself and many others in this business, has felt the pain of working on a legacy PHP application, trying to add features to it, or change existing behavior. This is a particularly hard thing to do and because so many incompetent developers have been creating PHP applications since the dawn of PHP, chances are that a big part of the job of competent PHP developers nowadays consists of maintaining these dumpsites of include statements and superglobals.

Last week was a particularly interesting week. I organised three coding dojos, on three different locations. The first one was at the IPPZ office (one of the companies I’ve worked for), the second one was at the SweetlakePHP meetup in Zoetermeer. The last one was for a group of developers from NOS, a large Dutch news organization. Although that would have been enough, I finished the week with my revised talk “Principles of PHP Package Design” at a meeting of the Dutch Symfony Usergroup in Utrecht…

Yesterday I presented “Principles of PHP Package Design” at the monthly meetup of the AmsterdamPHP usergroup. This was quite an experimental presentation. I had prepared it on the same day actually, though I had been gathering the contents of it for several weeks.

Package design is a subject I’m very much interested in and you can be sure to hear more from me about it. Actually, I started writing a book about it! If you’re interested too, sign up on leanpub.com/principles-of-php-package-design.

After working on my book “A Year With Symfony” for four months, I published it yesterday at the Dutch Symfony Usergroup meetup in Amsterdam. It was really great to be able to do this real-time while everybody in the room was looking at the screen waiting for the build process to finish.

The book is hosted on Leanpub, which is a great platform for writing and publishing books. They provide the entire infrastructure. You get a good-looking landing page and they handle all things related to payment.

Below you will find the slides for my presentation about Dependency Injection Smells (see also one of my previous posts from which this idea originated). If you have any questions, or feedback, please let me know by posting a comment below (or on joind.in).

Dependency Injection Smells from matthiasnoback

This week is Dutch PHP Conference week! It is something I’m very excited about. Three years ago I attended this conference for the first time. It was also my first time to visit any (PHP) conference at all, and this is a big conference, with lots of speakers, lots of attendees and multiple tracks. The Dutch PHP Conference (DPC) has a certain grandeur - many large rooms, many famous developers speaking. It is an inspiring event. Visiting the conference back in 2010 lit a fire in me to start investigating anything there was to know about writing good code: reusable, clean code, with well named, short and testable methods, readable and maintainable by others. In other words, I started being critical of the code I produced.

Part II of this series is all about validating the user’s session. You can find Part I here, if you missed it.

Collect Failed Authentication Attempts

Now and then a user will forget his password and try a few times before going to the “reset password” page. However, when a “user” keeps trying to authenticate with bad credentials, you may be subject to a brute-force attack. Therefore, you should collect failed authentication attempts. Your strategy may then be to block the account until further notice, while providing the user with a way to re-activate his account. When authentication fails, an event is fired, which you may intercept by registering an event listener or subscriber:

When working with Symfony2, you already have many of the finest tools for securing your web application. There are cases however that require you to add that extra bit. In this post I will point you to the right extension points within a Symfony2 project (or any other project which uses the Security Component for that matter).

Install NelmioSecurityBundle

See the README of the NelmioSecurityBundle. It contains many add-ons for your project, to sign/encrypt cookies, force SSL, prevent clickjacking and prevent untrusted redirects.

Symfony2 provides multiple ways of blocking, providing or modifying the response. You can:

1. Intercept each request by listening to the kernel.request event and set the response directly on the event (which will effectively skip execution of a controller)

2. Modify the controller or its arguments by listening to the kernel.controller event, then calling setController on the event object and modifying the attributes of the Request object.

3. Change the response rendered by a controller, by listening to the kernel.response event.

Authorization

When any of the authentication providers has verified the still unauthenticated token, an authenticated token will be returned. The authentication listener should set this token directly in the SecurityContext using its setToken() method.

From then on, the user is authenticated, i.e. means identified. Now, other parts of the application can use the token to decide whether or not the user may request a certain URI, or modify a certain object. This decision will be made by an instance of AccessDecisionManagerInterface.

Authentication

When a request points to a secured area, and one of the listeners from the firewall map is able to extract the user’s credentials from the current Request object, it should create a token, containing these credentials. The next thing the listener should do is ask the authentication manager to validate the given token, and return an authenticated token when the supplied credentials were found to be valid. The listener should then store the authenticated token in the security context:

The Security Context

Central to the Security Component is the security context, which is an instance of SecurityContext. When all steps in the process of authenticating the user have been taken successfully, the security context may be asked if the authenticated user has access to a certain action or resource of the application.

use Symfony\Component\Security\SecurityContext;
use Symfony\Component\Security\Core\Exception\AccessDeniedException;

$context = new SecurityContext();

// authenticate the user...

if (!$context->isGranted('ROLE_ADMIN')) {
    throw new AccessDeniedException();
}

A firewall for HTTP requests

Authenticating a user is done by the firewall. An application may have multiple secured areas, so the firewall is configured using a map of these secured areas. For each of these areas, the map contains a request matcher and a collection of listeners. The request matcher gives the firewall the ability to find out if the current request points to a secured area. The listeners are then asked if the current request can be used to authenticate the user.

In the Symfony2 security documentation both the firewalls and the access control rules are demonstrated using the “path” option, which is used to determine if a firewall or rule is applicable to the current URL. Also the “ip” option is demonstrated. The fact of the matter is, the string based configuration options in security.yml are transformed into objects of class RequestMatcher. This is a curious class in the HttpFoundation component which allows you to match a given Request object. The Security component uses it to determine if it should activate a certain firewall for the current request (usually only by checking the request’s path info).

The Symfony Security Component has an AccessDecisionManager which decides whether or not the currently authenticated user has a right to be in some place (or for that matter, use a certain service from the service container, or even call a certain method). The decision manager looks at the current user’s roles, and compares them to the attributes that are required. It relies on dedicated voters to make it’s verdict.

The component itself ships with an AuthenticatedVoter. It supports the “IS_AUTHENTICATED_FULLY”, “IS_AUTHENTICATED_REMEMBERED” and “IS_AUTHENTICATED_ANONYMOUSLY” attributes, which allow you to differentiate between users who are authenticated in the normal way, via a “remember me” cookie, or anonymously (which means: no credentials were supplied, but the user still gets a security context).

The Symfony Security Component provides a two-layer security system: first it authenticates a user, then is authorizes him for the current request. Authentication means “identify yourself”. Authorization means: “let’s see if you have the right to be here”.

The deciding authority in this case will be the AccessDecisionManager. It has a number of voters (which you may create yourself too). Each voter will be asked if the authenticated user has the right “roles” for the current URL.

Symfony2 firewalls depend for their authentication on UserProviders. These providers are requested by the authentication layer to provide a User object, for a given username. Symfony will check whether the password of this User is correct (i.e. verify it’s password) and will then generate a security token, so the user may stay authenticated during the current session. Out of the box, Symfony has a “in_memory” user provider and an “entity” user provider. In this post I’ll show you how to create your own UserProvider. The UserProvider in this example, tries to load a Yaml file containing information about users in the following format:

Writing PHP code with PHP is not very easy. You are constantly switching between the context of the code that generates the code and the code that is to be generated (see, you lost it already!). Some variables are available in the first context, some in the second, and you will have to pass the right values in the right way. One of the areas in Symfony-land where you will have to do these things is when you extend Twig by hooking into the parser and defining your own tags. A tag for example is the “for” in

In my previous post I wrote about uploading files to GridFS. Therefor I created a MongoDB Document with a $file property annotated with @MongoDB\File. Because I am using ORM entities more often then ODM documents, I was looking for a seamless way to access a Document from an Entity.

Because it isn’t possible to define a direct relationship between an Entity and a Document I thought it would be a solid solution to create a custom field type. By defining a custom field type I can control the way the reference to the Document will be stored and at the same time I will be able to restore the reference when retrieving the field. The steps needed to create a custom field type for ORM entities are very similar to the post of Matthias on how to create custom field types for ODM documents.

Almost at the same time, I silently celebrate the first birthday of my blog. My first article appeared a little over a year ago. It is great to see how Symfony2 has become more and more popular during these twelve months. Your comments and visits encourage me to keep posting articles. So, thank you all! And thanks, Dennis, for contributing.

GridFS is a specification for storing large files in MongoDB. In this post I will explain how you can easily upload a file to GridFS and then retrieve it from the database to serve it to the browser.

The Symfony Security Component has an AccessDecisionManager which decides whether or not the currently authenticated user has a right to be in some place (or for that matter, use a certain service from the service container, or even call a certain method). The decision manager looks at the current user’s roles, and compares them to the attributes that are required. It relies on dedicated voters to make it’s verdict.

The component itself ships with an AuthenticatedVoter. It supports the “IS_AUTHENTICATED_FULLY”, “IS_AUTHENTICATED_REMEMBERED” and “IS_AUTHENTICATED_ANONYMOUSLY” attributes, which allow you to differentiate between users who are authenticated in the normal way, via a “remember me” cookie, or anonymously (which means: no credentials were supplied, but the user still gets a security context).

In my previous post, I wrote about wrapping a legacy application in Silex, using output buffering and Twig. Finally, to allow for better decoupling as well as lazy loading of services, we passed the actual Silex\Application instance as the first argument of legacy controllers.

The first and quite easy way we can enhance our legacy application, is to make use of the request service (which contains all the details about the current request, wrapped inside the Symfony HttpFoundation’s Request class). So, instead of reading directly from $_GET and $_POST, we can change the edit_category() controller into the following:

Ever since I am using the Symfony Framework (be it version 1 or 2), I tend to describe every other project I’ve done (including those that were built on top of some third party “framework” like Joomla or WordPress) as a “legacy project”. Though this has sometimes felt like treason, I still keep doing it: the quality of applications written using Symfony is usually so much higher in terms of maintainability, security and code cleanliness, that even a project done last year using “only PHP” looks like a mess and seems to be no good software at all. So I feel the strong urge to rebuild everything I have in portfolio (as do many other developers), but “this time, I will do it the right way”.

The class Silex\Application extends \Pimple, a very lean dependency injection container, which itself implements \ArrayAccess. This allows you to define parameters and services and retrieve them from the Application like keys in an array. The first time I defined a service in a Silex application, in my case the Buzz Browser, I did it “inline”, i.e. inside my app.php file. This is how I did it:

use Buzz\Browser;

$app = new Silex\Application();

$app['autoloader']->registerNamespace('Buzz', __DIR__.'/vendor/buzz/lib');

$app['buzz.client_class'] = 'Buzz\\Client\\Curl';

$app['browser'] = $app->share(function() use ($app) {
    $clientClass = $app['buzz.client_class'];
    return new Browser(new $clientClass);
});

// $app['browser'] is ready for use

But this is not very reusable; every Silex application that needs Buzz\Browser as a service, needs to take care of the autoloading, configure and define the service. The Browser service is not such a very complex service, but think about everything that needs to be done to define and configure a service like the Doctrine DBAL (of course, Silex has a DoctrineServiceProvider for that already…).

In my previous post I wrote down a set of requirements for Silex applications. There were a few things left for another post: first of all, I want to have unit tests, nicely organized in directories that correspond to the namespaces of my project’s classes. This means that the tests for Acme\SomeNamespace\SomeClass should be found in Acme\Tests\SomeNamespace\SomeClass.

Organizing your unit tests

I want to write my tests for the PHPUnit framework. This allows me to use some PHPUnit best practices: first of all we define our PHPUnit configuration file in /app/phpunit.xml. First of all we point PHPUnit to a bootstrap file (of which we will later define it’s contents). We also set an environment variable called “env” whose value is “test”. Finally we set the location of our app.php file inside the server variable “env”.

When I created my first Silex project, I almost felt encouraged to let go of my high standards in programming. So things were starting to look very much like my “legacy” PHP projects, in which everything was put in functions with lengthy parameter lists and those functions were called from within a single index.php file. I ignored many of the things about high quality software development I had learned in previous years. The result of this, was a project much less maintainable than my other recent projects.

Recently I created my first WordPress template (the one you are looking at right now) and I was left both in shock and in great awe. This piece of software written in PHP has many users (like myself), yet it is so very outdated. The filtering mechanism by which you can modify HTML after it is generated by WordPress feels very dirty. But because a lot of the HTML snippets that WordPress generates are not suitable for decorating with Twitter Bootstrap’s beautiful CSS rules, I used those dirty filters extensively.