How to change PHP code with Abstract Syntax Tree

2017-11-06

We need to make clear what are we talking about right at the beginning. When we say "PHP AST", you can talk about 2 things:

1. php-ast

This is native extension which exports the AST internally used by PHP 7.0+. It allows read-only and is very fast, since it's native C extension. Internal AST was added to PHP 7.0 by skill-full Nikita Popov in this RFC. You can find it on GitHub under nikic/php-ast.

2. PHP AST

This is AST of PHP in Object PHP. It will take your PHP code, turn into PHP object with autocomplete in IDE and allows you to modify code. You can find it on GitHub under nikic/PHP-Parser.

Nikita explains differences between those 2 in more detailed technical way. Personally I love this human reason the most:

"Why would I want to have a PHP parser written in PHP? Well, PHP might not be a language especially suited for fast parsing, but processing the AST is much easier in PHP than it would be in other, faster languages like C. Furthermore the people most probably wanting to do programmatic PHP code analysis are incidentally PHP developers, not C developers."

Which one would you pick? If you're lazy like me and hate reading code and writing code over and over again, the 2nd one.

What work can nikic/PHP-Parser do for us?

Saying that, we skip the read-feature of this package - it's used by PHPStan or BetterReflection - and move right to the writing-feature. Btw, back in 2012, even Fabien wanted to use it in PHP CS Fixer, but it wasn't ready yet.

When we say modify and AST together, what can you brainstorm?

• change method name
• change class name
• rename property
• change property to method call
• move method from one class to another
• split class to multiple ones
• refactor $this->get('name') to constructor injection in Symfony App
• upgrade App from Symfony 3.0 to 4.0
• refactor Laravel App to Symfony App
• ...

It can do many things for you, depends on how much work you put in it. Today we will try to change method name.

4 Steps to Changing a name

1. Parse code to Nodes

composer require nikic/php-parser

Create parser and parse the file:

use PhpParser\Parser;
use PhpParser\ParserFactory;

$parser = (new ParserFactory)->create(ParserFactory::PREFER_PHP7); # or PREFER_PHP5, if your code is older
$nodes = $parser->parse(file_get_contents(__DIR__ . '/SomeClass.php'));

2. Find Method Node

The best way to work with Nodes is to traverse them with PhpParser\NodeTraverser:

$nodeTraverser = new PhpParser\NodeTraverser;
$traversedNodes = $nodeTraverser->traverse($nodes);

Now we traversed all nodes, but nothing actually happened. Do you think we forgot to invite somebody in?

Yes, we need PhpParser\NodeVisitor - an interface with 4 methods. We can either implement all 4 of them, or use PhpParser\NodeVisitorAbstract to save some work:

use PhpParser\NodeVisitorAbstract;

final class ChangeMethodNameNodeVisitor extends NodeVisitorAbstract
{
}

We need to find a ClassMethod node. I know that, because I use this package often, but you can find all nodes here. To do that, we'll use enterNode() method:

use PhpParser\Node;
use PhpParser\NodeVisitorAbstract;
use PhpParser\Node\Stmt\ClassMethod;

final class ChangeMethodNameNodeVisitor extends NodeVisitorAbstract
{
    public function enterNode(Node $node)
    {
        if (! $node instanceof ClassMethod) {
            return false;
        }

        // so we got it, what now?
    }
}

3. Change Method Name

No we find it's name and change it!

use PhpParser\Node;
use PhpParser\Node\Name;
use PhpParser\Node\Stmt\ClassMethod;
use PhpParser\NodeVisitorAbstract;

final class ChangeMethodNameNodeVisitor extends NodeVisitorAbstract
{
    public function enterNode(Node $node)
    {
        if (! $node instanceof ClassMethod) {
            return false;
        }

        $node->name = new Name('newName');

        // return node to tell parser to modify it
        return $node;
    }
}

To work with class names, interface names, method names etc., we need to use PhpParser\Node\Name.

Oh, I almost forgot, we need to actually invite visitor to the NodeTraverser like this:

$nodeTraverser = new PhpParser\NodeTraverser;
$traversedNodes->addVisitor(new ChangeMethodNameNodeVisitor);
$traversedNodes = $nodeTraverser->traverse($nodes);

4. Save to File

Last step is saving the file (see docs). We have 2 options here:

A. Dumb Saving

$prettyPrinter = new PhpParser\PrettyPrinter\Standard;
$newCode = $prettyPrinter->prettyPrintFile($traversedNodes);

file_put_contents(__DIR__ . '/SomeClass.php', $newCode);

But this will actually removes spaces and comments. How to make it right?

B. Format-Preserving Printer

It requires more steps, but you will have output much more under control.

Without our code, it would look like this:

use PhpParser\Lexer\Emulative;
use PhpParser\NodeTraverser;
use PhpParser\NodeVisitor;
use PhpParser\NodeVisitor\CloningVisitor;
use PhpParser\Parser\Php7;
use PhpParser\PrettyPrinter\Standard;

$lexer = new Emulative([
    'usedAttributes' => [
        'comments',
        'startLine', 'endLine',
        'startTokenPos', 'endTokenPos',
    ],
]);

$parser = new Php7($lexer);
$traverser = new NodeTraverser;
$traverser->addVisitor(new CloningVisitor);

$oldStmts = $parser->parse($code);
$oldTokens = $lexer->getTokens();

$newStmts = $traverser->traverse($oldStmts);

// our code start

$nodeTraverser = new NodeTraverser;
$nodeTraverser->addVisitor($nodeVisitor);

$newStmts = $traversedNodes = $nodeTraverser->traverse($newStmts);

// our code end

$newCode = (new Standard)->printFormatPreserving($newStmts, $oldStmts, $oldTokens);

Congrats, now you've successfully renamed method to newName!

Advanced Changes? With Rector!

Do you want to see more advanced operations, like those we brainstormed in the beginning? Look at package I'm working on which should automate application upgrades - RectorPHP.

Let me know in the comments, what would you like to read about AST and its Traversing and Modification. I might inspire by your ideas.

Happy coding!