How to Measure Your Type Coverage

Found a typo? Edit me

When we come to a new code base, we look for a code quality metric that will tell us how healthy the code base is. We can have CI tools like PHPStan and PHPUnit. PHPStan reports missing or invalid types, and PHPUnit reports failing tests.

But how do we know if 10 passing or 100 passing tests is enough? What if there are over 10 000 cases we should test?

That's where test coverage gives us a hint. Which project would you join if you could pick: the one with 20 % test coverage or the one with 80 % test coverage? I'd always go with the latter, as tests give great confidence.


Yet, tests are not the only thing that can help us access code quality quickly. With PHP 7.0, 7.4, and 8.0, type declarations became a sign of project health. But how can we measure those?


Do you measure your type declaration completeness with @phpstan already?

You should 😉

It's such a great and safe feeling to see 99 % type-coverage 😎 pic.twitter.com/cYyDVYKqG8

— Tomas Votruba 🇺🇦 (@VotrubaT) October 26, 2022


What is the "Type Coverage"?

If the test coverage is % of all possible code runs, what is the "type coverage" then?

function run($name)
{
    return $name;
}

Here we have 1 param and 1 function return. That's 2 possible type declarations that we're missing:


How can we increase it? We add a type declaration into the function param:

function run(string $name)
{
    return $name;
}

Here we have 1 param with type declaration, and 1 return without it.


How do we get to 100 %? Exactly, we add the return type declaration:

function run(string $name): string
{
    return $name;
}


We do the same for typed properties as well:

private $name;

private $surname;

private $age;


What about this code?

private string|Name $name;

private ?string $surname = null;

/**
 * @var callable
 */
private $addressCallable;

This code has 100 % type declaration coverage. How is that possible? Nullable, union, and callable docblock type declarations are valid and the most strict types.

Pretty simple, right?


We can run the type coverage check quickly with PHPStan on any project. Even if it's legacy or full of magic - no autoloading is needed.


I love this Metric, Because...


It's amazing to see it grow in time. This is how type coverage evolved in one project I work on:


3 Steps to Measure it

The type coverage is measured by 3 custom PHPStan rules with 3 custom collectors. They work the same way as described above in the code sample.

  1. Install the symplify/phpstan-rules package
composer require symplify/phpstan-rules --dev

The package is available on PHP 7.2+, as downgraded.


  1. Add Rules to phpstan.neon

The easiest type declaration to add is a return one, then the param one. On the other hand, the typed property is available as late as PHP 7.4. That's why we have 3 different rules for them, with one collector per each:

services:
    -
        class: Symplify\PHPStanRules\Rules\Explicit\PropertyTypeDeclarationSeaLevelRule
        tags: [phpstan.rules.rule]
        arguments:
            minimalLevel: 0.99

    -
        class: Symplify\PHPStanRules\Rules\Explicit\ParamTypeDeclarationSeaLevelRule
        tags: [phpstan.rules.rule]
        arguments:
            minimalLevel: 0.99

    -
        class: Symplify\PHPStanRules\Rules\Explicit\ReturnTypeDeclarationSeaLevelRule
        tags: [phpstan.rules.rule]
        arguments:
            minimalLevel: 0.99

The minimalLevel argument defines minimal required type coverage in every rule. Notice the value 0.99, meaning at least 99 % type coverage is required. We'll get back to that later.


  1. Add Collectors to phpstan.neon

At the moment, we've registered the rules, but they do not have any effect. We have to add collector services too:

services:
    -
        class: Symplify\PHPStanRules\Collector\FunctionLike\ParamTypeSeaLevelCollector
        tags: [phpstan.collector]
    -
        class: Symplify\PHPStanRules\Collector\FunctionLike\ReturnTypeSeaLevelCollector
        tags: [phpstan.collector]
    -
        class: Symplify\PHPStanRules\Collector\ClassLike\PropertyTypeSeaLevelCollector
        tags: [phpstan.collector]


Now run to see the results:

vendor/bin/phpstan

The failed error message is more than meets the eye. It shows you where you can complete the type declarations, so you can find them in the code and improve.

How to Find Your Current Type Coverage

Now we get back to the 0.99 resp. 99 % required type coverage. The CI fails on such a high value, but that's our intention. The error message actually tells us the current type coverage value:

Out of 81 possible param types, only 60 % actually have it. Add more param types to get over 99 %

In this case, we take the current value of 60 and put it into the config, so our codebase will remain on this code coverage:

 services:
     -
         class: Symplify\PHPStanRules\Rules\Explicit\ParamTypeDeclarationSeaLevelRule
         tags: [phpstan.rules.rule]
         arguments:
-            minimalLevel: 0.99
+            minimalLevel: 0.60

This value can be different for param, return, and property, so adjust it accordingly to make the CI pass.

Now we re-run PHPStan, and everything is fine. We commit, open pull-request, and merge.

Lean Type Coverage Improvement

Once a week, we run the same command again, trying to bump it 2-3 % (depending on your codebase size) and open a pull request. This way, we can improve the codebase gradually, without any big bang.

We also use these rules to monitor type coverage improvement on the project we work on. That way, both developers and the client knows we're going in the right direction.


Happy coding!


Have you find this post useful? Do you want more?

Follow me on Twitter, RSS or support me on GitHub Sponsors.