How to make a Dynamic Matrix in GitHub Actions

Found a typo? Edit me

Do you want split monorepo for each package? Instead of 20 workflows with copy-paste steps, you can define just one with a static matrix for packages.

Yet, nothing in real life is static but rather dynamic. A new package can be added, old can be removed. How could we automate this even more with a dynamic matrix?

In the last post we talked about monorepo split with GitHub Actions.

Today we'll look on a rather general idea for any GitHub Action - dynamic matrix.

Static Matrix

We've already talked about the use case for the split of many packages into many repositories. Instead of repeating each workflow with a different package, we can use a static matrix.

A typical static matrix looks like this:

        runs-on: ubuntu-latest

                    # list your packages here
                    - coding-standard
                    - phpstan-rules

After you define the strategy, add steps that use ${{ matrix.package }}.

        # ...
            -   uses: actions/[email protected]

                uses: symplify/[email protected]
                    GITHUB_TOKEN: ${{ secrets.ACCESS_TOKEN }}
                    package-directory: 'packages/${{ matrix.package }}'
                    split-repository-organization: 'symplify'
                    split-repository-name: '${{ matrix.package }}'

For each item in package: a new workflow run will be triggered. So, in this case - matrix triggers 3 parallel runs.

Matrix is an Array

The above could be written in PHP like:

$matrix = [
    'packages' => [

foreach ($matrix['packages'] as $package) {
    $packageDirectory = 'packages/' . $package;
    $splitRepositoryName = $package;
    // ... do the magic

Is there new package? Add it to the matrix:

                    # list your packages here
                    - coding-standard
                    - phpstan-rules
+                   - easy-coding-standard

That's it!

Any static opens up the door to troubles... How easy do you think is to forget to extend the workflow after adding a new package?

From Static Array to JSON

GitHub Actions are ready to make our life easier. Since April 2020, there is a fromJson() function to help us. What does it do?

It converts a json to an array, like this:

-                 package:
-                   - coding-standard
-                   - phpstan-rules
+                 package: ${{ fromJson(["coding-standard", "phpstan-rules"]) }}

You are probably thinking, "Well, that's just terrible, Tomas". Thank you, and you're right.

The Symplify\MonorepoBuilder is using this trick to get all the packages from /packages directory in handy json format:

vendor/bin/monorepo-builder packages-json


This version is not final, but very roughly the command above would be written like this:

-           matrix: ${{ fromJson(["coding-standard", "phpstan-rules"]) }}
+           matrix: ${{ fromJson(vendor/bin/monorepo-builder packages-json) }}

From Json to Fully Dynamic Matrix

If we run the command above as it is, it would fail. Setting up a matrix is like the setUp() method in a PHPUnit test case - there is zero code executed before it.

We have to do the same here:

How does the Workflow look Like?

    # phase 1
        runs-on: ubuntu-latest

            # git clone + use PHP + composer install
            -   uses: actions/[email protected]
            -   uses: shivammathur/[email protected]
                    php-version: 7.4
            -   run: composer install --no-progress --ansi

            # here we create the json, we need the "id:" so we can use it in "outputs" bellow
                id: set-matrix
                run: echo "::set-output name=matrix::$(vendor/bin/monorepo-builder packages-json --names)"

        # here, we save the result of this 1st phase to the "outputs"
            matrix: ${{ steps.set-matrix.outputs.matrix }}

    # phase 2
        # this means, wait for the "provide_packages_json" phase 1 to finish
        needs: provide_packages_json

        runs-on: ubuntu-latest
            # ↓ the real magic happens here - create dynamic matrix from the json
                package: ${{ fromJson(needs.provide_packages_json.outputs.matrix) }}

            # ...

That's it!

This way, we'll never forget to split nor test a new package. Never.

Check fully functional .github/workflows/split_monorepo.yaml in Symplify monorepo for more details.

Is this something niche? Not really. This week, Kamil has added this approach to Sylius too.

Use for Monorepo Split Anything Dynamic!

This fromJson() trick is not something exclusive to monorepo package splitting. It's just one of the possible use cases.

The primary use case is already in your mind.

Happy coding!

Have you find this post useful? Do you want more?

Follow me on Twitter, RSS or support me on GitHub Sponsors.