Currently the Wikibase Registry(setup post) is deployed using the shoehorning approach described in one of my earlier posts. After continued discussion on the Wikibase User Group Telegram chat about different setups and upgrade woes I have decided to convert the Wikibase Registry to use the prefered approach of a custom Dockerfile building a layer on top of one of the wikibase images.
I recently updated updated the Wikibase registry from Mediawiki version 1.30 to 1.31 and described the process in a recent post, so if you want to see what the current setup and docker-compose file looks like, head there.
As a summary the Wikibase Registry uses:
- The wikibase/wikibase:1.31-bundle image from docker hub
- Mediawiki extensions:
Creating the Dockerfile
Our Dockerfile will likely end up looking vaugly similar to the wikibase base and bundle docker files, with a fetching stage, possible composer stage and final wikibase stage, but we won’t have to do anything that is already done in the base image.
FROM ubuntu:xenial as fetcher # TODO add logic FROM composer as composer # TODO add logic FROM wikibase/wikibase:1.31-bundle # TODO add logic
Modifying the logic that is used in the wikibase Dockerfile the extra Wikibase Registry extensions can be fetched and extracted.
Note that I am using the convenience script for fetching Mediawiki extensions from the wikibase-docker git repo matching the version of Mediawiki I will be deploying.
FROM ubuntu:xenial as fetcher RUN apt-get update && \ apt-get install --yes --no-install-recommends unzip=6.* jq=1.* curl=7.* ca-certificates=201* && \ apt-get clean && rm -rf /var/lib/apt/lists/* ADD https://raw.githubusercontent.com/wmde/wikibase-docker/master/wikibase/1.31/bundle/download-extension.sh /download-extension.sh RUN bash download-extension.sh ConfirmEdit;\ bash download-extension.sh Nuke;\ tar xzf ConfirmEdit.tar.gz;\ tar xzf Nuke.tar.gz
None of these extensions require a composer install, so there will be no composer step in this example. If Nuke for example required a composer install, the stage would look like this.
FROM composer as composer COPY --from=fetcher /Nuke /Nuke WORKDIR /Nuke RUN composer install --no-dev
The Wikibase stage needs to pull in the two fetched extensions and make any other modifications to the resulting image.
In my previous post I overwrote the entrypoint to something much simpler removing logic to do with ElasticSearch that the Registry is not currently using. In my Dockerfile I have simplified this even further inlining the creation of a simple 5 line entrypoint, overwriting what was provided by the wikibase image.
I have left the default LocalSettings.php in the image for now, and I will continue to override this with a docker-compose.yml volume mount over the file. This avoid the need to rebuild the image when all you want to do is tweak a setting.
FROM wikibase/wikibase:1.31-bundle COPY --from=fetcher /ConfirmEdit /var/www/html/extensions/ConfirmEdit COPY --from=fetcher /Nuke /var/www/html/extensions/Nuke RUN echo $'#!/bin/bash\n\ set -eu\n\ /wait-for-it.sh $DB_SERVER -t 120\n\ sleep 1\n\ /wait-for-it.sh $DB_SERVER -t 120\n\ docker-php-entrypoint apache2-foreground\n\ ' > /entrypoint.sh
If the composer stage was used to run a composer command on something that was fetched then you would likely need to COPY that extension –from the composer layer rather than the fetcher layer.
Building the image
I’m going to build the image on the same server that the Wikibase Registry is running on, as this is the simplest option. More complicated options could involve building in some Continuous Integration pipeline and publishing to an image registry such as Docker Hub.
I chose the descriptive name “Dockerfile.wikibase.1.31-bundle” and saved the file alongside my docker-compose.yml file.
There are multiple approaches that could now be used to build and deploy the image.
- I could add a build configuration to my docker-compose file specifying the location of the Dockerfile as described here then building the service image using docker-compose as described here.
- I could build the image separate to docker-compose, giving it an appropriate name and then simply use that image name (which will exist on the host) in the docker-compose.yml file
I’m going with option 2.
docker build --tag wikibase-registry:1.31-bundle-1 --pull --file ./Dockerfile.wikibase.1.31-bundle .
docker build documentation can be found here. The command tells docker to build an image from the “Dockerfile.wikibase.1.31-bundle” file, pulling new versions of any images being used and giving the image the name “wikibase-registry” with tag “1.31-bundle-1”
The image should now be visible in the docker images list for the machine.
root@wbregistry-01:~/wikibase-registry# docker images | grep wikibase-registry wikibase-registry 1.31-bundle-1 e5dad76c3975 8 minutes ago 844MB
Deploying the new image
In my previous post I migrated from one image to another having two Wikibase containers running at the same time with different images.
For this image change however I’ll be going for more of a “big bang” approach and I’m pretty confident.
The current wikibase service definition can be seen below. This includes volumes for the entry point, extensions, LocalSettings and images, some of which I can now get rid of. Also I have removed the requirement for most of these environment variables by using my own entrypoint file and overriding LocalSettings entirely.
wikibase-131: image: wikibase/wikibase:1.31-bundle restart: always links: - mysql ports: - "8181:80" volumes: - mediawiki-images-data:/var/www/html/images - ./LocalSettings.php:/var/www/html/LocalSettings.php:ro - ./mw131/Nuke:/var/www/html/extensions/Nuke - ./mw131/ConfirmEdit:/var/www/html/extensions/ConfirmEdit - ./entrypoint.sh:/entrypoint.sh depends_on: - mysql environment: MW_ADMIN_NAME: "XXXX" MW_ADMIN_PASS: "XXXX" MW_SITE_NAME: "Wikibase Registry" DB_SERVER: "XXXX" DB_PASS: "XXXX" DB_USER: "XXXX" DB_NAME: "XXXX" MW_WG_SECRET_KEY: "XXXX" networks: default: aliases: - wikibase.svc - wikibase-registry.wmflabs.org
The new service definition has an updated image name, removed redundant volumes and reduced environment variables (DB_SERVER is still used as it is needed in the entrypoint I added).
wikibase-131: image: wikibase-registry:1.31-bundle-1 restart: always links: - mysql ports: - "8181:80" volumes: - mediawiki-images-data:/var/www/html/images - ./LocalSettings.php:/var/www/html/LocalSettings.php:ro depends_on: - mysql environment: DB_SERVER: "mysql.svc:3306" networks: default: aliases: - wikibase.svc - wikibase-registry.wmflabs.org
For the big bang switchover I can simply reload the service.
root@wbregistry-01:~/wikibase-registry# docker-compose up -d wikibase-131 wikibase-registry_mysql_1 is up-to-date Recreating wikibase-registry_wikibase-131_1 ... done
Using the docker-compose images command I can confirm that it is now running from my new image.
root@wbregistry-01:~/wikibase-registry# docker-compose images | grep wikibase-131 wikibase-registry_wikibase-131_1 wikibase-registry 1.31-bundle-1 e5dad76c3975 805 MB
- This should probably be documented in the wikibase-docker git repo which everyone seems to find, and also in the README for the wikibase image.
- It would be nice if there were a single place to pull the download-extension.sh script from, perhaps with a parameter for version?