How to Measure Your Type Coverage

Found a typo? Edit me
This post was updated at December 2022 with fresh know-how.
What is new?

Update to new package with simple PHPStan configuration.


When we come to a new code base, we look for a code quality metric that will tell us how healthy the code base is. We can have CI tools like PHPStan and PHPUnit. PHPStan reports missing or invalid types, and PHPUnit reports failing tests.

But how do we know if 10 passing or 100 passing tests is enough? What if there are over 10 000 cases we should test?

That's where test coverage gives us a hint. Which project would you join if you could pick: the one with 20 % test coverage or the one with 80 % test coverage? I'd always go with the latter, as tests give great confidence.


Yet, tests are not the only thing that can help us access code quality quickly. With PHP 7.0, 7.4, and 8.0, type declarations became a sign of project health. But how can we measure those?


Do you measure your type declaration completeness with @phpstan already?

You should 😉

It's such a great and safe feeling to see 99 % type-coverage 😎 pic.twitter.com/cYyDVYKqG8

— Tomas Votruba 🇺🇦 (@VotrubaT) October 26, 2022


What is the "Type Coverage"?

If the test coverage is % of all possible code runs, what is the "type coverage" then?

function run($name)
{
    return $name;
}

Here we have 1 param and 1 function return. That's 2 possible type declarations that we're missing:


How can we increase it? We add a type declaration into the function param:

function run(string $name)
{
    return $name;
}

Here we have 1 param with type declaration, and 1 return without it.


How do we get to 100 %? Exactly, we add the return type declaration:

function run(string $name): string
{
    return $name;
}


We do the same for typed properties as well:

private $name;

private $surname;

private $age;


What about this code?

private string|Name $name;

private ?string $surname = null;

/**
 * @var callable
 */
private $addressCallable;

This code has 100 % type declaration coverage. How is that possible? Nullable, union, and callable docblock type declarations are valid and the most strict types.

Pretty simple, right?


We can run the type coverage check quickly with PHPStan on any project. Even if it's legacy or full of magic - no autoloading is needed.


I love this Metric, Because...


It's amazing to see it grow in time. This is how type coverage evolved in one project I work on:


3 Steps to Measure it

The type coverage is measured by 3 custom PHPStan rules. They work the same way as described above in the code sample.


  1. Install the tomasvotruba/type-coverage package
composer require tomasvotruba/type-coverage --dev

The package is available on PHP 7.2+, as downgraded.


  1. With PHPStan extension installer, the rules are already installed.

To enable them, increase the minimal coverage on particular location:

# phpstan.neon
parameters:
    type_coverage:
        return_type: 50
        param_type: 30
        property_type: 70

The number defines minimal required type coverage in particular group. E.g. 30 means at least 30 % type coverage is required.


Now run to see the results:

vendor/bin/phpstan

The failed error message is more than meets the eye. It shows you where you can complete the type declarations, so you can find them in the code and improve.

How to Find Your Current Type Coverage

Now we get back to the 99 resp. 99 % required type coverage. The CI fails on such a high value, but that's our intention. The error message actually tells us the current type coverage value:

Out of 81 possible param types, only 60 % actually have it. Add more param types to get over 99 %


In this case, we take the current value of 60 and put it into the config, so our codebase will remain on this code coverage:

 # phpstan.neon
 parameters:
     type_coverage:
+        return_type: 99
-        return_type: 60

Adjust values accordingly to make the CI pass.

Then re-run PHPStan, and everything is fine. We commit, open pull-request, and merge.

Lean Type Coverage Improvement

Once a week, we run the same command again, trying to bump it 2-3 % (depending on your codebase size) and open a pull request. This way, we can improve the codebase gradually, without any big bang.

We also use these rules to monitor type coverage improvement on the project we work on. That way, both developers and the client knows we're going in the right direction.


Happy coding!


Have you find this post useful? Do you want more?

Follow me on Twitter, RSS or support me on GitHub Sponsors.