Hand-written service containers

Posted on by Matthias Noback

Dependency injection is very important. Dependency injection containers are too. The trouble is with the tools, that let us define services in a meta-language, and rely on conventions to work well. This extra layer requires the "ambient information" Paul speaks about in his tweet, and easily lets us make mistakes that we wouldn't make if we'd just write out the code for instantiating our services.

Please consider this article to be a thought experiment. If its conclusions are convincing to you, decide for yourself if you want to make it a coding experiment as well.

The alternative: a hand-written service container

I've been using hand-written service containers for workshop projects, and it turns out that it's very nice to work with them. A hand-written service container would look like this:

final class ServiceContainer
{
    public function finalizeInvoiceController(): FinalizeInvoiceController
    {
        return new FinalizeInvoiceController(
            new InvoiceService(
                new InvoiceRepository(
                    $this->dbConnection()
                )
            )
        );
    }

    private function dbConnection(): Connection
    {
        static $connection;

        return $connection ?: $connection = new Connection(/* ... */);
    }
}

The router/dispatcher/controller listener, or any kind of middleware you have for processing an incoming web request, could retrieve a controller from the service container, and call its main method. Simplified, the code would look this:

$serviceContainer = new ServiceContainer();

if ($request->getUri() === '/finalize-invoice') {
    return $serviceContainer->finalizeInvoiceController()->__invoke($request);
}
// and so on

We see the power of dependency injection here: the service won't have to fetch its dependencies, it will get them injected. The controller here is a so-called "entry point" for the service container, because it's a public service that can be requested from it. All the dependencies of an entry-point service (and the dependencies of its dependencies, and so on), will be private services, which can't be fetched directly from the container.

There are many things that I like about a hand-written dependency injection container. Every one of these advantages can show how many modern service containers have to reinvent features that you already have in the programming language itself.

No service ID naming conventions

For starters, service containers usually allow you to request services using a method like get(string $id). The hand-written container doesn't have such a generic service getter. This means, you don't have to think about what the ID should be of every service you want to define. You don't have to come up with arbitrary naming conventions, and you don't have to deal with inconsistent naming schemes in a legacy single project.

The name of a service is just the name of its factory method. Choosing a service name is therefore the same as choosing a method name. But since every method in your service container is going to create and return an object of a given type, why not use that type's name as the name of the method? In fact, this is what most service containers have also started doing at some point: they recommend using the name of the class you want to instantiate.

Type-safe, with full support for static analysis

Several years ago I was looking for a way to check the quality of the Symfony service definitions that I wrote in Yaml. So I created a tool for validating service definitions created with the Symfony Dependency Injection Component. It would inspect the service definitions and find out if they had the right number constructor arguments, if the class name it referenced actually existed, etc. This tool helped me catch several issues that I would only have been able to find out by clicking through the entire web application.

Instead of doing complicated and incomplete analysis after writing service definitions in Yaml (or any other meta-language for that matter), if I write them in PHP, I get all the support from static analysis tools. Even if I don't use a separate tool like PHPStan or Psalm, PhpStorm will point out any issues early on. Missing classes, missing import statements, too few or too many arguments, everything will be pointed out to me when I'm editing the ServiceContainer class in my IDE. This is a huge advantage.

Easy to refactor

Because analysis is easy, we can also expect all the help there is when refactoring our code. If we change production code, opening the ServiceContainer in your IDE will show you any issues you've produced. Furthermore, because there's no special service definition format, your IDE doesn't need a special extension to deal with it. It's just plain PHP code. So any refactoring tool that you use (e.g. rename method, move to different namespace, etc.) will also deal with any existing usages inside the ServiceContainer class.

Easy to make a distinction between public entry points and private dependencies

I like how the Symfony service container allows users to make a distinction between private and public services. Public ones can be fetched using a call to get($id), private ones can only be used as dependencies for other (public or private) services. Some services indeed deserve to be public (mostly the services we earlier called "entry points"), most should remain private. Of course, the distinction between public and private services reminds us of the way we can have public and private methods too, and in fact, if you write your own service container, you will use these method scopes to accomplish the same thing.

If you hand-write the service container you can do some optimizations too, just like the Symfony container does them. For instance, if you have a private service that's only used in one place, you can inline its instantiation. As an example, consider the private invoiceService() method:

public function finalizeInvoiceController(): FinalizeInvoiceController
{
    return new FinalizeInvoiceController(
        $this->invoiceService()
    );
}

private function invoiceService(): InvoiceService()
{
    return new InvoiceService(
        new InvoiceRepository(
            $this->dbConnection()
        )
    );
}

This method is only used by finalizeInvoiceController(), so we can safely inline it:

public function finalizeInvoiceController(): FinalizeInvoiceController
{
    return new FinalizeInvoiceController(
        new InvoiceService(
            new InvoiceRepository(
                $this->dbConnection()
            )
        )
    );
}

If, due to refactoring efforts, a private service is no longer needed, PhpStorm will tell you about it.

No need to define partial service definitions to assist auto-wiring

Auto-wiring has become quite popular, but I'm not convinced it's the way to go. I'm sure most scenarios have been covered by now, so I'm not afraid that things won't work out between me and auto-wiring. However, we'll always have to do tricks to make it work. We have to give the wirer hints about which implementations to use. This means that we may be able to delete many service definitions, but we also have to keep some around, since there are some things that won't work without them. You need to have in-depth knowledge about the dependency injection tool you use, and you need to learn the syntax for helping the auto-wirer. Worse, you may decide to adopt your production code so that the wirer can understand it.

Needless to say: if you write your service definitions in your own PHP class you'll never need custom syntax. In fact, you don't need to look up specific documentation at all, because you don't have to worry about failures to resolve dependencies; you make all the decisions yourself when you write your ServiceContainer class.

No need to inject primitive-type values by their parameter name

A downside of auto-wiring containers is that they need special instructions when the injected constructor arguments aren't objects, but primitive-type configuration values.

namespace App\Db;

final class Connection
{
    private $dsn;

    public function __construct(string $dsn)
    {
        // ...

        $this->dsn = $dsn;    
    }
}

Object-type constructor arguments can usually be resolved, but the service definition needs an extra hint for the $dsn argument:

    App\Db\Connection:
        arguments:
            $dsn: 'mysql:host=localhost;dbname=testdb'

This exposes an implementation aspect of the service class itself, which would normally remain hidden behind its public interface. To a client that instantiates an instance of Connection, only the parameter types should be relevant, not their names. In fact, a developer should be able to rename the parameter $dsn to something else, without breaking the application. Of course, a smart container builder will warn you about it when you rename a parameter, but this comes with some extra indirection that wouldn't be needed at all if we'd just write the instantiation logic inside a manual ServiceContainer class, where parameter names are irrelevant (as they should be).

No magic effects on the service container

Talking about auto-wiring, I have to say I dislike the fact that the location of a file containing a class has an influence on it being defined as a service in the container. I'd want to be able to create a new class anywhere in the project, and decide for myself whether or not it gets defined as a service. Needless to say, you won't have any magical effects like this if you write a custom service container, but personally I won't miss them.

Easier to distinguish between services and singletons

Most service containers will automatically share instances of a service; if a service has been instantiated once, the next time you ask for it, you'll get the exact same object instance. This is important for things like a database connection; you want to reuse the connection, instead of connecting to the database every time you need something from it. However, most services should be stateless anyway, and in that case it isn't really necessary to retrieve the exact same instance.

A service that's instantiated once and then shared between different clients is traditionally called a "singleton" service. Singleton services were usually implemented using the Singleton design pattern, which actually protects them from being instantiated more than once. Nowadays a service container manages service instances, and although it doesn't use the Singleton design pattern, it still makes every service effectively a singleton service: there's at most one instance of every service.

What I like about using a hand-written service container is that you can make a clear distinction between services for which it's actually important to have only one instance, and services for which it doesn't matter. Using the same example as earlier, note that the controller service can be re-instantiated every time a client needs it, and the connection service will be stored in a static variable, so that it can be reused:

final class ServiceContainer
{
    public function finalizeInvoiceController(): FinalizeInvoiceController
    {
        return new FinalizeInvoiceController(/* ... */);
    }

    private function dbConnection(): Connection
    {
        static $connection;

        return $connection ?: $connection = new Connection(/* ... */);
    }
} 

What about performance? Well, if it starts to hurt, you can always add some more shared services. But in most cases, I don't think it'll be needed. In part because of the fire-and-forget nature of the PHP server, but also because most services will be instantiated and used only once or twice anyway.

Easier to override parts for testing

With a service container based on a meta-language, like Yaml service definitions, you have to build in a mechanism for modifying the service definitions for use in different environments. With Yaml you can load in multiple service definition files and override service definitions and parameters. But just like "public" and "private" services, the concept of overriding services is also built into the programming language itself, namely by overriding methods. For example, if you want to use a fake/catch-all mailer or something while in development, you can do something like this:

abstract class ServiceContainer
{
    abstract protected function mailer(): Mailer;
}

final class DevelopmentServiceContainer extends ServiceContainer
{
    protected function mailer(): Mailer
    {
        return new FakeMailer();
    }
}

final class ProductionServiceContainer extends ServiceContainer
{
    protected function mailer(): Mailer
    {
        return new SmtpMailer(/* ... */);
    }
}

Optionally testable

If you want, you can even write an integration test for your hand-written service container. Then you could prove that the DevelopmentServiceContainer actually uses a FakeMailer. Or you can verify that all the public entry-points can be instantiated without producing any runtime errors (although static analysis will catch most of the issues already).

To be honest, this is also possible when you use a meta-language for defining services; you can always run tests against the compiled service container. However, I don't see this happening often, so I just wanted to mention the possibility here.

Composing containers? Separating containers?

A big issue with maintaining a hand-written container is that you wouldn't want to rewrite all of the framework's service definitions yourself. In fact, this isn't what I'm suggesting here. If you use a framework which ships with a service container, you just don't have to use it for your own service definitions. For instance, you can define your service container as a service in your framework's service container, and have access to it in the usual ways the framework supports.

You can even have multiple containers, for each of the modules/contexts you distinguish in your application. This could help keeping them actually separated (however, there's no technical way to enforce a context to protect the integrity of a context, it'll always be a people issue too).

Note that composition of containers is something that service containers don't usually offer, but the programming language itself is capable of. You only have to inject containers as constructor arguments of other containers.

Conclusion

In an educational setting I found that one of the biggest advantages of having your own hand-written service container for your Application and Domain layer services is that it allows you to write acceptance tests using these services. You can freely instantiate the container in your Behat FeatureContext. You can then write tests which talk to the Application layer (instead of the Infrastructure layer as they usually do). These tests will run in a very short time, but most importantly: they will be less brittle, because they don't (and can't) rely on all kinds of infrastructure-level peculiarities.

In a project setting, I haven't been fortunate enough to be able to use a hand-written service container. I'll just wait for the next opportunity to do so. If in the meantime you find yourself agreeing with the thought experiment which is this article, and have even applied the idea in practice, let me know how it worked out for you!

Finally, some suggestions for further reading:

  • Mathias Verraes has a nice explanation-in-a-gist where he argues that "We don't need no DIC libs", written after a discussion on Twitter about this topic. The gist also includes some implementation examples.
  • Marijn Huizendveld has an interesting blog post about how you can deal in a better way with environment variables (which works well with hand-written service containers too).
PHP dependency injection Comments

DDD Europe notes - Day 2

Posted on by Matthias Noback

Cyrille Martraire: Domain modeling towards First Principles

This was a talk from the first day, but it required some more processing before writing about it. Cyrille is one of my favorite speakers. He's fast, funny and brings a lot of interesting topics to the table. So many that it's sometimes hard to keep following his train of thought, and writing down some notes at the same time.

A central concept from his talk was what he called the waterline between IT and the business. In a traditional scenario, developers get provided with "work" on a case-by-case basis. They don't learn about the general idea or plan, or even the vision, goal or need that's behind the "work item". They just have to "implement" it. It leads to badly designed code. But it also leads to the wrong solutions being delivered. If only developers could have talked with the people who actually have the problem for which they build the solution. Maybe there's another problem behind it, or maybe the business has provided the developer with a solution, instead of a problem. To higher the waterline means to get more involved with the customers and users, to understand their problems, and work together on a solution. Make sure you get involved.

When looking for the right solutions, investige the problems and use the following heuristic: "Consider the intensional alternative". Cyrille considers "intensional" a pedantic word and likes it. It's opposed to "extensional". Extensional corresponds to enumarating the results. Intensional means defining the predicate which will produce those results. Looking for intensional solutions means you'll end up with a better understanding of the problem. And also an automatable one.

While collecting these intensional aspects of existing business problems, as a developer you will be building a theory about that business. Cyrille warns against the illusion that a domain is simply a lot of information withs if-s on top. Business is messy, so we shoudn't be obsessed with rules. Some parts of a domain allow theorizing, some don't.

This nicely aligns with my own experience, trying to understand a business domain. Domain experts often aren't aware of the inconsistencies, or impossible rules they come up with. Once you bring in some logic, some rules seem reasonable but just aren't feasible. These may be areas where there's manual intervention in an otherwise automated system.

Another interesting concept Cyrille brought up was that of "skeuomorphism". He noticed that software solutions seem to continue building on technology from the past. Software systems often look like they're rebuilding a paper bureaucracy, but now it's digital. You can't really be "disruptive" if you don't think about radically different ways of solving some problem for the user. This was a bit of shock, because I realized that I often use "paper metaphors" to find out a possible solution for a design issue. Then again, maybe there's room for both - a paper metaphor as a design tool, yet a non-paper bureaucracy inspired software solution.

Maaret Pyhäjärvi: Breaking Illusions with Testing

Identifying herself as a "feedback fairy", Maaret talks about testing, in a very broad sense. It doesn't only cover automated testing, but also exploratory testing, and what you're doing with them. Tests don't break code, they break the illusions you have about it. The way Maaret explained testing, I immediately became scared of all the illusions I have about my code and that I should start breaking them.

I like how Maaret used the term "heuristics", like more DDD-ers do, to share with us how she and other testers do their work. It seems it's very common to share heuristics in the testing community. Very interesting! I don't think programmers share them as freely as they should.

Testing an application starts with realizing: you don't know much. So you have to create some sort of a map of what's there (functions, data, the platform, etc.). Start to discover, and never be bored. Always look for the interesting things, and "poke it until it pops".

In general, what you'll find is all kinds of illusions (they could be called "assumptions", I think). In testing an application ("it has high code coverage, so it's tested") but also in the general activity of developing software ("if we have this new feature, we'll earn more money").

Thinking in terms of illusions, I think most development teams are very good at them. We have all kinds of ideas about our code, our way of working, the application, our teammates, the company, etc. But they aren't really tested. It'll be smart to challenge them.

Also, looking at how others approach their problems is very useful. There will be a different perspective that you didn't think of. Learning is not linear, we are not higher or lower on the scale of "learnedness" or something. We learn different things. So, we have to look for different perspectives deliberately, thereby increasing our ability to shatter illlusions and build good things.

One last idea Maaret shared, one that I wanted to remember is: sharing the pain means more knowledge is going to be shared. I found this very relatable, but it's easy to forget. In every team or company there will be someone who "suffers" from the way they (have to) work, but it's not until they share this pain with someone else, that something is going to be changed about it. And that change is only enabled by the fact that people start sharing knowledge, and will come up with solutions, because they have a different perspective.

PHP DDD conference Comments

DDD Europe notes - Day 1

Posted on by Matthias Noback

Eric Evans: Keynote ("Language in Context")

Starting out with the basics (words have meaning within a context; when we make the boundary of this context explicit we end up with a bounded context), Eric discussed two main topics: the Big Ball of Mud, and bounded contexts in the context (no pun intended) of microservices.

Legacy applications are always framed in a negative way, as if it's something to get away from. Personally, I've come to enjoy them a lot. However, there will always be the urge to work around legacy code. The Bubble Context (PDF) can be a great way of creating a new model that works well next to the already existing models. To keep a safe buffer between the new and the old model, you could build an Anti-Corruption Layer (ACL). A fun thing Eric mentioned is that the buffer works in two directions. The ACL also allows the old model to keep functioning without being disturbed by all the new things that are going on in the Bubble Context.

Given that a bounded context may or may not align with an actual subdomain, it may very well be that a legacy context is actually a Big Ball of Mud, with unified models, and simply just a lot of domain knowledge written between the many spaghetti lines that are in there. However, even though it is a mess, and it's getting harder to work with it every day, it may still be what you could call a "Mature Productive Context". The question is: is it still aligned with business views? If it is, we could improve at least maintainability and the cost of change by performing local refactorings. If it isn't, it'll be very hard to change anything. If the basic assumptions of the model change, rework will be very costly.

As a matter of fact, for a project I'm currently working on, we're looking into a module (or context), which requires some heavy refactoring, because it has become technically very hard to work with it. However, the business is still quite happy about it, and it's quite central to its processes.

An important Domain-Driven approach which can be used in the area of legacy code is where you analyze the different subdomains, and find out which ones are generic, and which ones are "core" to the business. As an example, in the aforementioned project there are actually two candidates for context-level improvements. One is related to Sales (which is the heart of this financial application), and one is related to Addressbook records (which is very much supportive to the Sales part). One could say it's even generic, in the sense that an off the shelf solution might be preferable. We wouldn't want to spend a lot of design or development effort there either.

Eric mentioned the term "Quaint Context" as a suitable name for a context that one would consider "legacy". It uses outdated technology probably, and has become hard to maintain. It won't be possible to make big changes there (as mentioned, because these basic assumptions can't easily be changed), so another good name could be "Patch-by-Patch Context".

With a microservice architecture, another option to deal with legacy contexts becomes what Eric calls the "Exposed Legacy Asset" (yet another nice term!). This will be a legacy application which starts to adapt to the microservices environment by producing messages that will be useful for actual microservices in that environment. For instance, database triggers could be used to produce events. The external events themselves don't have to be as low-level as the internal events that caused them.

Eric touches on several other interesting aspects of microservice architecture, but I wanted to briefly mention some other relevant ideas here. Eric looked back at 15 years of Domain-Driven Design and proposed that by now we maybe need a definition of DDD itself. He doesn't want DDD to be just a club, but asks for intellectual honesty. If you try to apply DDD and somehow it fails, you should share this story. If you're skeptical about some aspect of DDD, talk about it. I like how it boils down to focusing on the core domain, exploring models together, and speaking a ubiquitous language in an explicitly bounded context. Nice!

Rebecca Wirfs-Brock: Growing Your Design Heuristics Toolkit

This one was a workshop with limited access, so I was lucky I could attend it. Rebecca had spoken in a previous edition of the conference about heuristics, which triggered my interest in the idea. The workshop was about the process behind it. It had some interesting pointers, like a PDF about the concept and a book by Billy Vaughn Koen: Discussion of the Method. Definitely things to check out!

Rebecca defined "heuristic" as a practical method, a rule of thumb. Heuristics are things you collect over time, when you find out what works best in a certain situation. They may be hints about how to solve a design problem, or how to find out what to do next, or how you should relate to something.

An example of a difficult situation in software development is when you encounter "smelly" code. Something's wrong. It needs fixing, but: do you do it now? Never? Only in certain cases? One could say: if you see a problem, fix it. Another would say: if it isn't broken, don't fix it. Or maybe: never fix it. Don't touch anything at all, because it might break.

We all have some intuitive approach in this matter, and I find it very interesting how:

  1. This approach changes over time.
  2. Many people will have many different approaches.

Rebecca suggested keeping a journal, describing and reflecting on how you actually tackle design issues.

We practiced collecting heuristics according to a somewhat fixed recipe. Here's an example:

Question: Should I fix code smells whenever I encounter them? Heuristic: Only fix them when you're working on the code. Example: If you're just reading the code, don't improve it just yet. If you dive in and make changes because of a new feature was requested, or a bug had to be fixed, add a test and fix the smell.

You can collect these based on your own work, and when talking to someone else you want to learn from, you can start collecting many more. When finding out which heuristics other people use, it helps to challenge them. Ask questions, find out the exceptions, the many other subtle things that should be taken into account, etc.

I found the workshop very interesting and was happy to learn more about this useful concept of a "heuristic".I feel that writing about technology has always been about exposing my own heuristics, or rules of thumb, and by sharing, maybe I can help others too. I think there's a fundemental issue with writing though: it may only "click" with the reader when their experience somewhat overlaps with the experience the writer had. You have to recognize at least part of what's in the book/article, before you can relate to it, or can "accept" it as something useful.

PHP DDD conference Comments