Distribute Patches to many Git repositories with Patchbot – Notes by Dan Untenzu (pixelbrackets)

tl;dr Patchbot is a tool to automate the distribution of patches to your own Git repositories and helps to reduce technical debt.

Technical debt

Technical debt is a common problem in software development and just like financial debt there may be good or bad reasons for it, but an accumulation of technical debt may lead to unrecoverable risks.

Innovation and rapid development is a catch-22 situation here, since this may increase technical debt causing a loss on innovation and development speed.

There are many ways to acknowledge, address, and pay off technical debt eventually. However, the hardest part is to work against the “No, not right now“ conation.

Real World Examples

I had to face this issue some time ago, when numerous Git repositories I manage were affected by some general decisions.

Required changes:

Streamline the name of the Changelog file to comply with recommended naming patterns
Add a configuration file needed to introduce a new code quality tool
The main domain address of the organisation changed - reflect this in documentation and meta files

Ideas how solve these:

Open an issue for each project repository, wait until solved – Large overhead, what happens if the project is quiet?
Add these tasks to a project health checklist, and one of these tasks needs to be done every time a project is opened
- How to check that a task is solved everywhere and no longer necessary?
Do a walkthrough - get it done
- This… is… monotonous… and requires all attention over a period of time

Those are small issues. But a lot of work to implement them in all projects. All proposed solutions have a great potential for the omnious “No, not right now” excuse.

However, there was a keyword: »monotonous«… Wait… 🤔 This could be automated!

Automate

First step: Write a patch script to apply the changes.

A patch is a set of changes to a computer program or its supporting data designed to update, fix, or improve it.

It is not possible solve the requirements with a Git patch file, since they require conditions or depend on other tools. A custom migration script will do however.

Second step: Write a bot to run the patch script against all repositories.

A (internet) bot is a software application that runs automated tasks (scripts) over the Internet.

The bot needs to checkout every selected repository, create a feature branch, run the patch script, commit the changes, push the branch. Let's do it.

Patchbot

I build a prototype and used it to solve the given tasks.

Patch + Bot = Patchbot

Now, this a not a completely new area. I write migration scripts all the time when I refactor projects. I delete them as soon as they are executed though. Tomas Votruba calls these one-time migration scripts »Mandala Scripts«. I love this synonym.

Since the purpose of my patch scripts was to reuse them across many repositories I designed a lean file structure which encourages this concept. ♻

Usage

Setup

Create a new repository which will contain a collection of all patch scripts.

composer create-project pixelbrackets/patchbot-skeleton my-patches
cd my-patches

Each patch directory always contains at least a patch script and a commit message named commit-message.txt. Patch scripts can be written in different languages: PHP (patch.php), Shell (patch.sh), Python (patch.py), or even as a plain Git diff file (patch.diff). You can create a new patch using the create command:

./vendor/bin/patchbot create

Import Patches

Instead of writing every patch from scratch, you can import ready-to-use patches from Git repositories or Gists.

There is a patchbot-examples repository which covers several real-world use cases like find-and-replace, adding packages, introducing config files or renaming files.

For a quick start, you can import all example patches at once and customize them to your needs:

./vendor/bin/patchbot import https://github.com/pixelbrackets/patchbot-examples

Apply a Patch

Run the following command to apply the patch template to a repository:

./vendor/bin/patchbot patch template git@gitlab.com:user/repo.git

Repeat the same command with different repository URLs to distribute the patch to them as well - or use patch:many to apply a patch to all repositories listed in a repositories.json file at once:

./vendor/bin/patchbot patch:many template

Patchbot creates a random name for the feature branch and uses the default branch as base. When you want to create the feature branch based on a different branch, like development, and declare a fixed name for the feature branch, like feature-1337, you may run this command:

./vendor/bin/patchbot patch template git@gitlab.com:user/repo.git --source-branch=development --branch-name=feature-1337

Hint: Run ./vendor/bin/patchbot discover to quickly create a first repositories.json storage file.

Merge

Patchbot uses a feature branch and does not commit into existing branches right away. When you reviewed the feature branch and all tests are successful then you can use Patchbot again to merge the branch.

Example command to merge branch bugfix-add-missing-lock-file into branch main:

./vendor/bin/patchbot merge bugfix-add-missing-lock-file main git@gitlab.com:user/repo.git

Passing the option --create-mr can start a GitLab merge request automatically.

Contributions welcome

This script is Open Source, so please use, patch, extend or fork it.

Contributions are welcome!

Update: Patchbot now supports import of patches from Git repositories and Gists, and patches written in different languages. The above post was changed accordingly.