How to Split Test Monorepo with Composer 2

2020-10-19

Composer 2 was released this week. It brings massive composer install/update performance improvement of 150-200 %.

That means faster feedback from CI and faster monorepo testing.

Today, we'll look on how to use Composer in Github Actions with monorepo split testing and what to avoid.

Monorepo split testing is made easier by symplify/monorepo-builder package. This is how it "works" for Composer 1. Why "works"? There are 2 problems, that we fix during upgrade to Composer 2.

Let's say we have a symplify/symplify monorepo, where we develop all symplify package. We need to test all packages together with their local version, even in pull-request.

1. Installing Packages that needs Local version Package

We want to test symplify/coding-standard package:

the symplify/coding-standard package needs symplify/autowire-array-parameter package
- the symplify/autowire-array-parameter needs the symplify/package-builder package

In short:

we're testing a
a needs b
- b needs c

This the result of localized package composer install on GitHub Action with Composer 1:

Package a and b and used in local version ✅
Package c is used from packagist ❌

What happens with changes of symplify/package-builder in this pull-request? They're ignored, and last stable version is used instead. ❌

This is the same process, with Composer 2:

Package a and b and used in local version ✅
Package c is used in local version ✅

2. State Local packages as `dev-master`

So we upgrade to Composer 2 with Symplify 9 and that's it? No.

Local and GitHub Action development are different. Locally, Composer can see them as dev-master branch, which works with using branch-alias. But on GitHub Actoin (or your any favorite CI, the pull-request branch is dev-<commit-hash>).

The composer install will lead to a conflict with these 2 version:

What now?

I asked on Composer repository and after less than hour of work got a working solution with proper explanation. Thank you Jordi!

Trick is using COMPOSER_ROOT_VERSION=dev-master env, that will explicitly make version dev-master for all environments.

This is the final working GitHub Action (here you can see it the action):

name: Split Tests

on:
    pull_request: null

jobs:
    split_testing:
        runs-on: ubuntu-latest

        strategy:
            fail-fast: false
            matrix:
                package_name:
                    - coding-standard
                    # and all the other packages...

        name: Split Tests of ${{ matrix.package_name }}

        steps:
            -   uses: actions/[email protected]

            -   uses: shivammathur/[email protected]
                with:
                    php-version: 7.4
                    coverage: none
                    tools: composer:v2

            -   run: composer install --no-progress --ansi

            # tell composer to use local package version
            -   run: vendor/bin/monorepo-builder localize-composer-paths packages/${{ matrix.package_name }}/composer.json --ansi

            -
                working-directory: packages/${{ matrix.package_name }}
                run: composer update --no-progress --ansi
                env:
                    # see https://github.com/composer/composer/issues/9368#issuecomment-718112361
                    COMPOSER_ROOT_VERSION: "dev-master"

            -
                working-directory: packages/${{ matrix.package_name }}
                run: vendor/bin/phpunit

How to Split Test Monorepo with Composer 2

1. Installing Packages that needs Local version Package

2. State Local packages as dev-master

2. State Local packages as `dev-master`