Calling GitHub CI from a Wikimedia GitLab repo

I want to spend some more time developing a-wikimedia-cli (previosuly called mwcli), expanding what it can do, and adding integration tests for more of its functionality along the way. Having tests is useful, but having to wait for them every time you change something makes that slightly less appealing. Especially when your development system seems to be intense bursts of work and branches on the odd evening, or odd weekend, and not much development the rest of the time.

Some of the more involved tests create MediaWiki environments, start databases, install things, turn them off, turn them back on again and check that the CLI has done what it was supposed to. These have been running using Docker-in-Docker (or dind for short) on Wikimedia Cloud VPS machines for some time now, alongside the other checks and builds managed by Wikimedia GitLab. However the are only 2 small workers in the pool for the Docker-in-Docker tests, which are limited to 4 simultaneous jobs due to CPU and RAM constraints. not to mention it’s a waste leaving them on most of the time…

Meanwhile, the project also has a repository on GitHub which is mirrored to from GitLab, where GitHub Actions has machines quite capable of running these tests.

I have thought about moving entirely over to GitHub to solve this problem before, and have spun up forks there with CI before to make sure integration style tests pass before promoting them into branches in GitLab. However I never made a complete move. I have also dabbled with doing cross repo CI before for this setup, however it often ended in a disappointing mess.

However, maybe this time it will work *wink wink*…

Read more

mwcli CI in Wikimedia GitLab (docker in docker)

mwcli is a golang CLI tool that I have been working on over the past year to replace the mediawiki-docker-dev development environment that I accidently created a few years back (among other things). I didn’t start the CLI, but I did this mediawiki-docker-dev like functionality.

As some point through the development journey it became clear that one of the ways to set the new and old environments apart would be through some rigorous CI and testing.

This started with CI running on a Qemu node as part of the shared Wikimedia Jenkins CI infrastructure that is hooked up to Gerrit, where the code was being developed. This ended up being quite slow, and involved lots of manual steps.

A next iteration saw the majority of development take place in my own fork on Github, making use of Github Actions. Changes would then be copied over to Gerrit for final review once CI tests had run.

And finally the repository moved to the new Wikimedia GitLab instance (work in progress), where I could make use of GitLab Runners powered by a machine in Wikimedia Cloud VPS.

Screenshot of GitLab pipelines in action for the mwcli project

Read more