Travis and Monorepo Split is dead now. Use faster GitHub Action instead.
Do you use a monorepo? Then you know How to maintain multiple Git repositories with ease. If you're not there yet, you may wonder How to Merge 15 Repositories to 1 Monorepo and Keep their Git History.
Are you and your monorepo ready? Today we'll focus on fast, secured and outsourced monorepo auto split - all that under 10 minutes.
It's great to be alive in this era. We have solved maintaining multiple repositories, even merge migration of their git history to one repository. Creating huge code bases was more never cost effective and never had a steeper learning curve.
The same way it was never easier to drive an autonomous car. You just sit in the car, press the button of your destination, and Tesla will drive you there when you check all the news on /r/PHP.
Well, the monorepo paradigm is not there yet but it's getting there.
One of the last problems I'm aware of is splitting the monorepo to particular packages. Imagine you have Symfony monorepo and you're trying to split it to all standalone packages like symfony/console, symfony/dependency-injection and so on.
This whole "take a code from this directory and put it into this repository in
master branch and last tag" process is now:
Instead, we want it to be:
Why? So you could amaze your friends at the party that you just set-up a monorepo split and they can enjoy merged PRs in a matter of minutes (even if you know almost nothing about git or PHP).
Do you think we can get there? You'll see.
Feel free to explore these following solutions. I did it for you and here are a few blockers that hold them from being massively adopted.
On the other hand, I'm grateful for each one of them, because they're pushing the evolution further and further. Thanks to them we don't have to reinvent the wheel and we can build on their shoulders.
Before diving into solution and how to do it, I try to stop and go into a wonderland. What would the ideal solution look like? How would I use it? How would I explain it to others? How fast would it be? Try to break away from your know-how limits (because they're limiting your thinking) and be free to come up with absolutely non-sense answers:
If we put it in the code, it might look like:
composer require symplify/monorepo-builder --dev
# monorepo-builder.yml parameters: directories_to_repositories: packages/MonorepoBuilder: '[email protected]:Symplify/MonorepoBuilder.git'
That could do, right? At least from a developer's experience view.
But what would security expert Michal Špaček say to lousy code like that?
"So, anyone can now push to your repository whatever he wants?"
This is a valid question that was probably scratching your mind when you saw Github + Travis + git + open-source combination. Travis is basically a terminal, that runs few command lines. What would prevent someone from using this to "play with" your repository?
Let's look at the repository address in our example:
This basically means we need to make ssh key or username and a password public. Does that sound like a good idea to you?
Don't worry, Github and Travis thought about these cases - with a hashed
GITHUB_TOKEN environment variable.
First, you need to create a custom token, that will authorize access to your Github repositories from any command line where it will be used.
Read the Github docs or use tl;dr;:
GITHUB_TOKENin Repository Settings on Travis
Then we need to store this token to Travis build, so Travis can be authorized to manipulate with your repositories without any password or ssh key.
Read the Travis docs or use tl;dr;:
GITHUB_TOKENwith the value from Github
In the end, it should look like this:
If you got lost in this tl;dr;s, try this nice post with so many screenshots.
Now the best part. If you accidentally commit your access token in
.travis.yml (like I did while testing), GitHub will immediately disable it and sends you an email (to bad I found that out the next day after 4 hours of debugging with that token).
And if you add the token to your repository on Travis as above, it will hide it in all logs for you. No need to hash it.
So instead of insecure
[email protected]:Symplify/MonorepoBuilder.git # or https://[email protected]/Symplify/MonorepoBulder.git
everyone will see:
Sound and safe!
Now we have:
symplify/monorepo-builderpackage as a local dependency
GITHUB_TOKENin Travis settings for your monorepo repository
Something is missing...
Oh right, when will the monorepo be split? Do we have to do it manually? How often? Should Travis CRON do it on daily basis?
In times like these, get back to our ideal product:
It often happens we merge fix or feature to monorepo and we want to try it before rushing to tagging a stable release. We want to do it as soon as possible, without manually triggering Travis to do it. Also, we don't want Travis to waste energy on pull-requests that are not merged to master. That would only slow the whole CI process down and frustrate contributors and maintainer.
Saying that how
.travis.yml should look like?
language: php # required for "git tag" presence for MonorepoBuilder split and ChangelogLinker git tags resolver # see https://github.com/travis-ci/travis-ci/issues/7422 git: depth: false matrix: include: - php: 7.2 env: MONOREPO_SPLIT=true # ... other builds install: - composer install # ... other scripts after_script: # split monorepo to packages - only on merge to master - | if [[ $TRAVIS_EVENT_TYPE == "push" && $MONOREPO_SPLIT == true && $TRAVIS_BRANCH == "master" ]]; then vendor/bin/monorepo-builder split -v fi
That way the split command is run only merge to
master and exactly once after each merge. So you can test your feature in a matter of minutes...
Wait wait, no vague statements like a matter of minutes. How fast it really is? To give you an idea, this is Symplify Travis build with a split of 10 packages:
It takes under 7,5 minutes including all the tests, static analysis and code style validation.
That's all folks. You're ready to go and try it on your monorepo.
So, do you think you're ready to fascinate your friends tonight with all your brand new monorepo split setup?