Defining multiple similar services with Docker Compose
Matthias Noback
For my new workshop - “Building Autonomous Services” - I needed to define several Docker containers/services with more or less the same setup:
- A PHP-FPM process for running the service’s PHP code.
- An Nginx process for serving static and dynamic requests (using the PHP-FPM process as backend).
To route requests properly, every Nginx service would have its own hostvcname. I didn’t want to do complicated things with ports though - the Nginx services should all listen to port 80. However, on the host machine, only one service can listen on port 80. This is where reverse HTTP proxy Traefik did a good job: it is the only service listening on the host on port 80, and it forwards requests to the right service based on the host name from the request.
This is the configuration I came up with, but this is only for the “purchase” service. Eventually I’d need this configuration about 4 times.
services:
purchase_web:
image: matthiasnoback/building_autonomous_services_purchase_web
restart: on-failure
networks:
- traefik
- default
labels:
- "traefik.enable=true"
- "traefik.docker.network=traefik"
- "traefik.port=80"
volumes:
- ./:/opt:cached
depends_on:
- purchase_php
labels:
- "traefik.backend=purchase_web"
- "traefik.frontend.rule=Host:purchase.localhost"
purchase_php_fpm:
image: matthiasnoback/building_autonomous_services_php_fpm
restart: on-failure
env_file: .env
user: ${HOST_UID}:${HOST_GID}
networks:
- traefik
- default
environment:
XDEBUG_CONFIG: "remote_host=${DOCKER_HOST_NAME_OR_IP}"
volumes:
- ./:/opt:cached
Using Docker Compose’s extend functionality
Even though I usually favor composition over inheritance, also for configuration, in this case I thought I’d be better of with inheriting some configuration instead of copying it. These services don’t accidentally share some setting, in the context of this workshop, these services are meant to be more or less identical, except for some variables, like the host name.
So I decided to define a “template” for each service in docker/templates.yml:
version: '2'
services:
web:
restart: on-failure
networks:
- traefik
- default
labels:
- "traefik.enable=true"
- "traefik.docker.network=traefik"
- "traefik.port=80"
volumes:
- ${PWD}:/opt:cached
php-fpm:
image: matthiasnoback/building_autonomous_services_php_fpm
restart: on-failure
env_file: .env
user: ${HOST_UID}:${HOST_GID}
networks:
- traefik
- default
environment:
XDEBUG_CONFIG: "remote_host=${DOCKER_HOST_NAME_OR_IP}"
volumes:
- ${PWD}:/opt:cached
Then in docker-compose.yml you can fill in the details of these templates by using the extends key (please note that you’d have to use “version 2” for that):
services:
purchase_web:
image: matthiasnoback/building_autonomous_services_purchase_web
extends:
file: docker/templates.yml
service: web
depends_on:
- purchase_php
labels:
- "traefik.backend=purchase_web"
- "traefik.frontend.rule=Host:purchase.localhost"
purchase_php_fpm:
extends:
file: docker/templates.yml
service: php-fpm
We only define the things that can’t be inherited (like depends_on), or that are specific to the actual service (host name).
Dynamically generate Nginx configuration
Finally, I was looking for a way to get rid of specific Nginx images for every one of those “web” services. I started with a Dockerfile for every one of them, and a specific Nginx configuration file for each:
server {
listen 80 default_server;
index index.php;
server_name purchase.localhost;
root /opt/src/Purchase/public;
location / {
# try to serve file directly, fallback to index.php
try_files $uri /index.php$is_args$args;
}
location ~ ^/index\.php(/|$) {
fastcgi_pass purchase_php_fpm:9000;
fastcgi_split_path_info ^(.+\.php)(/.*)$;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
fastcgi_param DOCUMENT_ROOT $realpath_root;
# Prevents URIs that include the front controller. This will 404:
# https://domain.tld/index.php/some-path
# Remove the internal directive to allow URIs like this
internal;
}
}
To reuse the Nginx image for every “web” service, I needed a way to use variables in this configuration file. The solution was documented in the description of the official nginx image: Using environment variables in nginx configuration. The trick is to use environment variables in the configuration file, and replace them with their real values when you start the Nginx container. The template configuration file could look something like this:
server {
listen 80 default_server;
index index.php;
server_name ${SERVER_NAME};
root ${ROOT};
location / {
# try to serve file directly, fallback to index.php
try_files $uri /index.php$is_args$args;
}
location ~ ^/index\.php(/|$) {
fastcgi_pass ${PHP_BACKEND}:9000;
fastcgi_split_path_info ^(.+\.php)(/.*)$;
include fastcgi_params;
fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
fastcgi_param DOCUMENT_ROOT $realpath_root;
# Prevents URIs that include the front controller. This will 404:
# https://domain.tld/index.php/some-path
# Remove the internal directive to allow URIs like this
internal;
}
}
The problem is, the configuration file itself contains many strings that look like environment variables (e.g. $realpath_root). When using the proposed solution, all those variables were replaced by empty strings.
After some fiddling (and looking up configuration options for envsubst), I found the solution: you can explicitly mention which variables should be replaced. The only other thing I needed to do is properly escape the names of these variables, to prevent them from being replaced on the spot:
FROM nginx:1.13-alpine
COPY template.conf /etc/nginx/conf.d/site.template
...
CMD sh -c "envsubst '\$SERVER_NAME \$ROOT \$PHP_BACKEND' < /etc/nginx/conf.d/site.template > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"
At this point, I was able to build only one Docker image that could be reused by all the “web” services. I only had to set the correct environment variables in docker-compose.yml:
services:
purchase_web:
# ...
environment:
- SERVER_NAME=purchase.localhost
- PHP_BACKEND=purchase_php
- ROOT=/opt/src/Purchase/public
sales_web:
# ...
environment:
- SERVER_NAME=sales.localhost
- PHP_BACKEND=sales_php
- ROOT=/opt/src/Sales/public
# ...
You’ll find the complete configuration in the workshop project’s repository: “Building Autonomous Services”.