Effective Debug Tricks: Narrow Scoping

Writing a code that only you work with is easy. Debugging such code is a bit harder. Writing a code for someone else review is quite hard. The code must be understandable to the other reader to pass the code review.

How hard is reading a code that someone else wrote three months ago?

But what about debugging a code that someone wrote a year ago?

Effective debugging is not about finding the responsible line, nor about fixing the bug.

Effective debugging is about getting to the minimal amount of code that is probably causing the error with the minimal energy invested.

Why it's important?

When we report a bug in a project's issue tracker, we're usually interested in its fix. Why would we report if we don't care? The faster bug gets fixed, the better for us, so we can continue with our original coding goal. We need instant feedback to stay in the flow.


Today we look at one technique that I use for open-source project reports and for teaching private projects to debug their legacy code with peace.

We also have it as a standard in Rector and Symplify projects. It saves us many hours wasted on issues tracker comment ping-pong and focuses on the cooperative solution.

You might notice it's similar to the minimum common denominator process that you probably know from 5th grade—what a powerful concept.


Level 1: "There is a bug" report

What will happen if we provide a report like this?

What can the maintainer do? Can they fix the error and close the issue? Probably not, because there is no information about where to look first. So what will the maintainer do? They will ask us for more information:


This interaction is a problem already because you've lost energy that you're willing to invest to reporting the bug and the maintainer lost energy that is willing to put into narrow the scope of the bug.

It's like going to a mall, stopping right in front of the first door and shouting "open the door". It will take a couple of minutes for security to get to you and figure out what you want. Why not just take the handle yourself? Good luck next time you'll report a robbery to them - they will to help will be a bit lowered, if not depleted.


Bug Fix Effectiveness

Remember, the goal of reporting an issue is to invest as little energy from our side and maintainer's side and get the bug fixed at the same time.

The bug fix effectiveness formula states:

Bug Fix Effectiveness = (Reporter Work + Maintainer Work) / 100


So can we do it better for everyone?

Level 2: "There is a bug with this full message" report

The printing does not work, it ends with a fatal error.

Here is the output I got with --debug/-vvv: ...

I use Rector v0.9.4


Good job! We've just saved both ourselves and the maintainer a significant amount of energy. Now the maintainer know what happens to use and how it looks like.


Have you seen any detective movies lately? There is a murder, the detective arrives at the scene, and the first witness shows the body and location. That's what we just did in our issue report. "Here is somebody dead, I saw him at 22:00". What happens next? The detective smiles, says thank you, and goes home... that would be weird, right? They always ask the same question:

The detective is looking for prerequisites. They need this information about you to get a specific idea of how you fit the whole picture. Maybe you're innocent; maybe you're a murderer. They don't know, but they have to decide - so they ask.


Could you figure out what the report is missing? Yes, the prerequisites.

We know what is wrong with your issue now. But the maintainer doesn't know, what did you do before that you got yourself into this state.


How can we do it better?

Level 3: "I did this, then this happened" report

I tried to run Rector on my project with this rector.php

I used vendor/bin/rector process p src

The printing does not work, it ends with a fatal error. Here is the output I got with --debug/-vvv:

...

I use Rector v0.9.4


Great job!

Now both you and the maintainers now:


Do you remember Bug Fix Effectiveness Formula?

BFE = (Reporter Work + Maintainer Work) / 100

From level 1 to 3, we invested a lot of more energy on first issue reporting:

Level 1:

BFE = (20 + 380) / 100 = 4.0

Level 3:

BFE = (40 + 160) / 100 = 2.0

But because the maintainer doesn't have to ask us more questions and doesn't have to reply to them, we also increased the chance to get a bug fixed by ~100 %.


There is one more step that we found to be the most effective. It usually requires only one comment from the report and one reply from the maintainer. Well, if you count closing the issue with pull-request as a comment.

How can we do it better? Don't worry; it's not about learning project test conventions and sending a failing pull-request.

Level 4: "I did exactly this, then this happened" report

Let's look at the narrow scoping. What can we read from this report?

I tried to run Rector on my project with this rector.php

I used vendor/bin/rector process p src

The question is: what rule and what file is causing this bug?

We need to narrow the scope of INF INF to 1 1. How can we do it?

Half-Half Cutting

This technique can be applied to services, to PHPStan rule, to Rector rules, to a coding standard, to registered event subscribers... to anything. A similar algorithm is used to sort files in an array.

The idea is comment out half of the configuration, run the tool and see if the bug still remains:

 // rector.php
 return static function (ContainerConfigurator $containerConfigurator): void {
     $parameters->set(Option::AUTO_IMPORT_NAMES, true);

     $parameters->set(Option::SETS, [
         SetList::PHP_74,
         SetList::PHP_80,
-        SetList::CODE_QUALITY,
+//      SetList::CODE_QUALITY,
-        SetList::CODING_STYLE,
+//      SetList::CODING_STYLE,
    ]);

Is the bug still there? Remove the commended lines and repeated the half-commented/half-active process with the other half:

 // rector.php
 return static function (ContainerConfigurator $containerConfigurator): void {
     $parameters->set(Option::AUTO_IMPORT_NAMES, true);

     $parameters->set(Option::SETS, [
         SetList::PHP_74,
-        SetList::PHP_80,
+//      SetList::PHP_80,
-        SetList::CODE_QUALITY,
-        SetList::CODING_STYLE,
    ]);

Is the bug gone? It must be one of those commented sets, so comment out half of them and let the ofher half active:

 // rector.php
 return static function (ContainerConfigurator $containerConfigurator): void {
     $parameters->set(Option::AUTO_IMPORT_NAMES, true);

     $parameters->set(Option::SETS, [
-        SetList::PHP_74,
-        SetList::PHP_80,
         SetList::CODE_QUALITY,
-        SetList::CODING_STYLE,
+//      SetList::CODING_STYLE,
    ]);

This way, we discover quickly which exact set and which exact rule is causing the problem.


But, running the whole codebase to find out a single rule might take hours. How can we avoid it? Let's apply narrow scoping on the filesystem first. Instead of running tool globally:

vendor/bin/rector p src

Let's run it only on one directory:

vendor/bin/rector p src/Controllers

Is it not there? Pick another directory:

vendor/bin/rector p src/Repository

Is it there? Bingo!


Then we can report all the information that the maintainer needs:

I tried to run Rector on my project with rector.php


When I run TypedPropertyRector rule like this:

vendor/bin/rector process p src/Repository/AbstractRepository.php

it fails with...

We just reached the most effective value:

BFE = (60 + 60) / 100 = 1.2

Now we get 4-times more bugs fixed then with level 1. Thank you for reporting!


Happy coding!