Writing PHP code with PHP is not very easy. You are constantly switching between the context of the code that generates the code and the code that is to be generated (see, you lost it already!). Some variables are available in the first context, some in the second, and you will have to pass the right values in the right way. One of the areas in Symfony-land where you will have to do these things is when you extend Twig by hooking into the parser and defining your own tags. A tag for example is the "for" in

{% for item in items %}

When Twig converts Twig templates to executable PHP code, it first reads the template and distinguishes its (fine-grained) parts, called tokens: names, strings, blocks, variables, etc. (see The Lexer). Then, token parsers are allowed to process the tokens and create nodes, which contain information gathered from the tokens. For instance, the "for" token parser stores the names of the value ("item") and the collection ("items") that were provided (see The Parser). Then, the compiler turns all the nodes into executable PHP code that can be cached inside, for instance, /app/cache/dev/twig (see The Compiler).

To define your own tags, you will have to create a Twig extension, then a token parser (which parses tokens and returns a node) and finally a node class that can output real PHP code.

My goal is to create a {% collect %} tag which collects strings defined in templates, included templates and in the parent template. This would allow you to pre-process templates, or to collect data for rendering a <title> tag or something. I hereby informally continue the series of Kris Wallsmith (see here and here) and I will describe how to combine the collected data using a node visitor.

Add the Twig extension

The Twig extension could be very simple, it should just return an instance of the yet to be defined token parser:

namespace Acme\DemoBundle\Twig\Extension;

use Acme\DemoBundle\Twig\TokenParser\CollectorTokenParser;

class CollectorExtension extends \Twig_Extension
{
    public function getTokenParsers()
    {
        return array(
            new CollectorTokenParser(),
        );
    }

    public function getName()
    {
        return 'collector';
    }
}

We can register the extension automatically by defining a service for this class and adding the tag twig.extension to it.

Create the token parser

Whenever the token parser comes across something like this

{% collect "hello" %}

we want to store the "hello" value for later use (in another template) and we eventually want it to be combined in to an array of collected values in this template, its included templates and the template it extends. For this, we would first need a token parser. When parsing a template, it will continue reading strings after the "collect" tag and put them in an array. Then it will create a node and pass the retrieved data to it.

namespace Acme\DemoBundle\Twig\TokenParser;

use Acme\DemoBundle\Twig\Node\CollectorNode;

class CollectorTokenParser extends \Twig_TokenParser
{
    public function parse(\Twig_Token $token)
    {
        $stream = $this->parser->getStream();

        $data = array();

        // keep collecting strings until the block ends
        while (!$stream->test(\Twig_Token::BLOCK_END_TYPE)) {
            $data[] = $stream->expect(\Twig_Token::STRING_TYPE)->getValue();
        }

        $stream->expect(\Twig_Token::BLOCK_END_TYPE);

        return new CollectorNode($data, $token->getLine(), $this->getTag());
    }

    public function getTag()
    {
        return 'collect';
    }
}

Define the node type

As said before, the template compiler will use a node to render PHP code. A first implementation of the CollectorNode would be:

namespace Acme\DemoBundle\Twig\Node;

class CollectorNode extends \Twig_Node
{
    public function __construct(array $data, $lineno = 0, $tag = null)
    {
        // store the data collected by the parser as the attribute "data"

        parent::__construct(array(), array('data' => $data), $lineno, $tag);
    }

    public function compile(\Twig_Compiler $compiler)
    {
        $compiler
            ->write('if (!isset($context["collected_data"])) { $context["collected_data"] = array(); }')
            ->raw(";\n");

        // merge the data provided in this template with the existing data
        $compiler
            ->write('$context["collected_data"] = array_merge($context["collected_data"], ')
            ->repr($this->getAttribute('data'))
            ->raw(");\n");
    }
}

So now we are creating PHP code on-the-fly. The compiler takes care of indenting and outdenting the generating code, when we call write(). When we use raw() it will ignore indentation. As you can see we use the value $context - an array which will surely be available at this place. It contains all values that are available inside the template. Once we add a key to the array it also becomes available for output, so inside the template we could now add

{% for data in collected_data %}
    {{ data }}
{% endfor %}

and it would output all the strings gathered from the {% collect %} tag.

Try this with a template which extends some other template. In the parent template, add the lines above to display the values that have been collected.

Included templates

Maybe you didn't try it, but all this works out of the box for templates and their parents. This is because the modified context will be passed to the parent template. In the compiled template (somewhere in app/cache/dev/twig) you'll see:

if (!isset($context["collected_data"])) { $context["collected_data"] = array(); }
$context["collected_data"] = array_merge($context["collected_data"], array(0 => "hello"));

// the modified context will be passed to the parent template
$this->parent->display($context, array_merge($this->blocks, $blocks));

Unfortunately, when included templates modify the context, the changes will not be preserved. This is why Kris in his article proposes to use a node visitor. Node visitors are called after a template is parsed and they are able to modify or replace nodes. But they can also be used to collect some information about the nodes and use it elsewhere. Creating a node visitor is exactly the right thing to do in this case. The strategy would be:

• The collected data from templates should be merged with the data defined in the parent template

• Collected data from templates will consist of collected data of the template itself and of the templates that are included by the template

Adding the node visitor

The node visitor should gather both the collected data and the names of the included templates:

namespace Acme\DemoBundle\Twig\NodeVisitor;

use Acme\DemoBundle\Twig\Node\CollectorNode;

class CollectorNodeVisitor implements \Twig_NodeVisitorInterface
{
    private $data = array();
    private $includes = array();

    public function enterNode(\Twig_NodeInterface $node, \Twig_Environment $env)
    {
        if ($data = $this->getCollectedData($node)) {
            // this is a CollectorNode, so collect the data
            $this->data = array_merge($this->data, $data);
        }
        else if ($includedTemplate = $this->getIncludedTemplateName($node)) {
            // this is an include tag, so collect the included template name for later reference
            $this->includes[] = $includedTemplate;
        }

        return $node;
    }

    private function getCollectedData(\Twig_NodeInterface $node)
    {
        if ($node instanceof CollectorNode) {
            return $node->getAttribute('data');
        }

        return null;
    }

    private function getIncludedTemplateName(\Twig_Node $node)
    {
        if ($node instanceof \Twig_Node_Include) {
            /* @var $node \Twig_Node */
            return $node->getNode('expr')->getAttribute('value');
        }

        return null;
    }

    public function leaveNode(\Twig_NodeInterface $node, \Twig_Environment $env)
    {
        if ($node instanceof \Twig_Node_Module) {
            // reset the gathered data: the current template ends here
            $this->data = array();
            $this->includes = array();
        }

        return $node;
    }

    public function getPriority()
    {
        return 0;
    }
}

When entering a node (when applicable) we collect the data from the CollectorNode or store the included template name. When leaving a node we do nothing, except when it is a module node, i.e. it is the main node of the template, which means we should reset the data that we gathered, to prevent it from ending up in another template.

One thing before we can take the final step: the node visitor should be registered inside the Twig extension:

namespace Acme\DemoBundle\Twig\Extension;

use Acme\DemoBundle\Twig\NodeVisitor\CollectorNodeVisitor;

class CollectorExtension extends \Twig_Extension
{
    // ...

    public function getNodeVisitors()
    {
        return array(
            new CollectorNodeVisitor(),
        );
    }
}

Extending the module node

The node visitor provides us with a way of collecting the data and the names of the included templates. We can use this to combine the collected data from a template, the included templates and the parent template. It can be accomplished by replacing the default module node from Twig by one of our own. This should be done inside the leaveNode() method:

use Acme\DemoBundle\Twig\Node\CollectorModuleNode;

class CollectorNodeVisitor implements \Twig_NodeVisitorInterface
{
    // ...

    public function leaveNode(\Twig_NodeInterface $node, \Twig_Environment $env)
    {
        if ($node instanceof \Twig_Node_Module) {
            $node->setAttribute('collected_data', $this->data);
            $node->setAttribute('collected_includes', $this->includes);

            $newModuleNode = new CollectorModuleNode($node);

            $this->data = array();
            $this->includes = array();

            return $newModuleNode;
        }

        return $node;
    }
}

We add some attributes to the original node, then create a new CollectorModuleNode. This node will add a method to the compiled PHP template class:

namespace Acme\DemoBundle\Twig\Node;

class CollectorModuleNode extends \Twig_Node_Module
{
    public function __construct(\Twig_Node_Module $originalNode)
    {
        \Twig_Node::__construct($originalNode->nodes, $originalNode->attributes, $originalNode->lineno, $originalNode->tag);
    }

    protected function compileClassFooter(\Twig_Compiler $compiler)
    {
        // add the collectData() method just before the class footer

        $this->compileCollectDataMethod($compiler);

        parent::compileClassFooter($compiler);
    }

    private function compileCollectDataMethod(\Twig_Compiler $compiler)
    {
        $collectedIncludes = $this->getAttribute('collected_includes');
        $collectedData = $this->getAttribute('collected_data');

        // add a method collectData()
        $compiler
            ->write("\n", 'public function collectData(array $context)', "\n", "{\n")
            ->indent();

        // initialize $data with the collected data for this template
        $compiler
            ->write('$data = ')
            ->repr($collectedData)
            ->raw(";\n");

        // merge $data with the collect data from the context
        // this data comes from a template which extends the current template
        $compiler
            ->write('if (isset($context["collected_data"])) {', "\n")
            ->indent()
            ->write('$data = array_merge($data, $context["collected_data"]);', "\n")
            ->outdent()
            ->write("}\n");

        // merge $data with the collected data from all included templates
        foreach ($collectedIncludes as $includedTemplate) {
            $compiler
                ->write("\n", '$data = array_merge($data, $this->env->loadTemplate(')
                ->repr($includedTemplate)
                ->raw(')->collectData($context));')
                ->raw("\n");
        }

        $compiler
            ->raw("\n")
            ->write('return $data;', "\n");

        $compiler
            ->outdent()
            ->write("}\n");
    }
}

Quite a lot happens here, but the new collectData() method boils down to this (for example):

/* AcmeDemoBundle:Demo:hello.html.twig */
class __TwigTemplate_f190ce8086eed8bd8619a188a19cbd3e extends Twig_Template
{
    // ...

    public function collectData(array $context)
    {
        $data = array(0 => "hello");
        if (isset($context["collected_data"])) {
            $data = array_merge($data, $context["collected_data"]);
        }

        $data = array_merge($data, $this->env->loadTemplate("AcmeDemoBundle:Demo:included.html.twig")->collectData($context));

        return $data;
    }
}

The final step would be to modify the CollectorNode class to use the new collectData() method to set the collected data as a key of the $context array:

class CollectorNode extends \Twig_Node
{
    public function __construct(array $data, $lineno = 0, $tag = null)
    {
        parent::__construct(array(), array('data' => $data), $lineno, $tag);
    }

    public function compile(\Twig_Compiler $compiler)
    {
        $compiler
            ->write('$context["collected_data"] = $this->collectData($context);', "\n\n");
    }
}

Finally! In the parent template all the data collected from the child template and the included templates is now available as a single array. Try

{% dump(collected_data) %}