wikibase-docker, Mediawiki & Wikibase update
Today on the Wikibase Community User Group Telegram chat I noticed some people discussing issues with upgrading Mediawiki and Wikibase using the docker images provided for Wikibase.
As the wikibase-registry is currently only running Mediawiki 1.30 I should probably update it to 1.31, which is the next long term stable release.
This blog post was written as I performed the update and is yet to be proofread, so expect some typos. I hope it can help those that were chatting on Telegram today.
Starting state
Documentation
There is a small amount of documentation in the wikibase docker image README file that talks about upgrading, but this simply tells you to run update.php.
Update.php has its own documentation on mediawiki.org.
None of this helps you piece everything together for the docker world.
Installation
The installation creation process is documented in this blog post, and some customization regarding LocalSettings and extensions was covered here.
The current state of the docker-compose file can be seen below with private details redacted.
This docker-compose files is found in /root/wikibase-registry on the server hosting the installation. (Yes I know that’s a dumb place, but that’s not the point of this post)
version: '3'
services:
wikibase:
image: wikibase/wikibase:1.30-bundle
restart: always
links:
- mysql
ports:
- "8181:80"
volumes:
- mediawiki-images-data:/var/www/html/images
- ./LocalSettings.php:/var/www/html/LocalSettings.php:ro
- ./Nuke:/var/www/html/extensions/Nuke
- ./ConfirmEdit:/var/www/html/extensions/ConfirmEdit
depends_on:
- mysql
environment:
MW_ADMIN_NAME: "private"
MW_ADMIN_PASS: "private"
MW_SITE_NAME: "Wikibase Registry"
DB_SERVER: "mysql.svc:3306"
DB_PASS: "private"
DB_USER: "private"
DB_NAME: "private"
MW_WG_SECRET_KEY: "private"
networks:
default:
aliases:
- wikibase.svc
- wikibase-registry.wmflabs.org
mysql:
image: mariadb:latest
restart: always
volumes:
- mediawiki-mysql-data:/var/lib/mysql
environment:
MYSQL_DATABASE: 'private'
MYSQL_USER: 'private'
MYSQL_PASSWORD: 'private'
MYSQL_RANDOM_ROOT_PASSWORD: 'yes'
networks:
default:
aliases:
- mysql.svc
wdqs-frontend:
image: wikibase/wdqs-frontend:latest
restart: always
ports:
- "8282:80"
depends_on:
- wdqs-proxy
environment:
BRAND_TITLE: 'Wikibase Registry Query Service'
WIKIBASE_HOST: wikibase.svc
WDQS_HOST: wdqs-proxy.svc
networks:
default:
aliases:
- wdqs-frontend.svc
wdqs:
image: wikibase/wdqs:0.3.0
restart: always
volumes:
- query-service-data:/wdqs/data
command: /runBlazegraph.sh
environment:
WIKIBASE_HOST: wikibase-registry.wmflabs.org
networks:
default:
aliases:
- wdqs.svc
wdqs-proxy:
image: wikibase/wdqs-proxy
restart: always
environment:
- PROXY_PASS_HOST=wdqs.svc:9999
ports:
- "8989:80"
depends_on:
- wdqs
networks:
default:
aliases:
- wdqs-proxy.svc
wdqs-updater:
image: wikibase/wdqs:0.3.0
restart: always
command: /runUpdate.sh
depends_on:
- wdqs
- wikibase
environment:
WIKIBASE_HOST: wikibase-registry.wmflabs.org
networks:
default:
aliases:
- wdqs-updater.svc
volumes:
mediawiki-mysql-data:
mediawiki-images-data:
query-service-data:
Code language: PHP (php)
Backups
docker-compose.yml
So that you can always return to your previous configuration take a snapshot of your docker-compose file.
If you have any other mounted files it also might be worth taking a quick snapshot of those.
Volumes
The wikibase docker-compose example README has a short section about backing up docker volumes using the loomchild/volume-backup docker image.
So let’s give that a go.
I’ll run the backup command for all 3 volumes used in the docker compose file which cover the 3 locations that I care about that persist data.
docker run -v wikibase-registry_mediawiki-mysql-data:/volume -v /root/volumeBackups:/backup --rm loomchild/volume-backup backup mediawiki-mysql-data_20190129
docker run -v wikibase-registry_mediawiki-images-data:/volume -v /root/volumeBackups:/backup --rm loomchild/volume-backup backup mediawiki-images-data_20190129
docker run -v wikibase-registry_query-service-data:/volume -v /root/volumeBackups:/backup --rm loomchild/volume-backup backup query-service-data_20190129
Code language: JavaScript (javascript)
Looking in the /root/volumeBackups directory I can see that the backup files have been created.
ls -lahr /root/volumeBackups/ | grep 2019
-rw-r--r-- 1 root root 215K Jan 29 16:40 query-service-data_20190129.tar.bz2
-rw-r--r-- 1 root root 57M Jan 29 16:40 mediawiki-mysql-data_20190129.tar.bz2
-rw-r--r-- 1 root root 467 Jan 29 16:40 mediawiki-images-data_20190129.tar.bz2
I’m not going to bother checking that the backups are actually complete here, but you might want to do that!
Prepare the next version
Grab new versions of extensions
The wikibase-registry has a couple of extension shoehorned into it mounted through mounts in the docker-compose file (see above).
We need new versions of these extensions for Mediawiki 1.31 while leaving the old versions in place for the still running 1.30 version.
I’ll do this by creating a new folder, copying the existing extension code into it, and then changing and fetching the branch.
# Make copies of the current 1.30 versions of extensions
root@wbregistry-01:~/wikibase-registry# mkdir mw131
root@wbregistry-01:~/wikibase-registry# cp -r ./Nuke ./mw131/Nuke
root@wbregistry-01:~/wikibase-registry# cp -r ./ConfirmEdit ./mw131/ConfirmEdit
# Update them to the 1.31 branch of code
root@wbregistry-01:~/wikibase-registry# cd ./mw131/Nuke/
root@wbregistry-01:~/wikibase-registry/mw131/Nuke# git fetch origin REL1_31
From https://github.com/wikimedia/mediawiki-extensions-Nuke
* branch REL1_31 -> FETCH_HEAD
root@wbregistry-01:~/wikibase-registry/mw131/Nuke# git checkout REL1_31
Branch REL1_31 set up to track remote branch REL1_31 from origin.
Switched to a new branch 'REL1_31'
root@wbregistry-01:~/wikibase-registry/mw131/Nuke# cd ./../ConfirmEdit/
root@wbregistry-01:~/wikibase-registry/mw131/ConfirmEdit# git fetch origin REL1_31
From https://github.com/wikimedia/mediawiki-extensions-ConfirmEdit
* branch REL1_31 -> FETCH_HEAD
root@wbregistry-01:~/wikibase-registry/mw131/ConfirmEdit# git checkout REL1_31
Branch REL1_31 set up to track remote branch REL1_31 from origin.
Switched to a new branch 'REL1_31'
Code language: PHP (php)
Define an updated Wikibase container / service
We can run a container with the new Mediawiki and Wikibase code in alongside the old container without causing any problems, it just needs a name.
So below I define this new service, called wikibase-131 using the same general details as my previous wikibase service, but pointing to the new versions of my extensions, and add it to my docker-compose file.
Note that no port is exposed, as I don’t want public traffic here yet, and also no network aliases are yet defined. We will switch those from the old service to the new service at a later stage.
wikibase-131:
image: wikibase/wikibase:1.31-bundle
restart: always
links:
- mysql
volumes:
- mediawiki-images-data:/var/www/html/images
- ./LocalSettings.php:/var/www/html/LocalSettings.php:ro
- ./mw131/Nuke:/var/www/html/extensions/Nuke
- ./mw131/ConfirmEdit:/var/www/html/extensions/ConfirmEdit
depends_on:
- mysql
environment:
MW_ADMIN_NAME: "private"
MW_ADMIN_PASS: "private"
MW_SITE_NAME: "Wikibase Registry"
DB_SERVER: "mysql.svc:3306"
DB_PASS: "private"
DB_USER: "private"
DB_NAME: "private"
MW_WG_SECRET_KEY: "private"
Code language: JavaScript (javascript)
I tried running this service as is but ran into an issue with the change from 1.30 to 1.31. (Your output will be much more verbose if you need to pull the image)
root@wbregistry-01:~/wikibase-registry# docker-compose up wikibase-131
wikibase-registry_mysql_1 is up-to-date
Creating wikibase-registry_wikibase-131_1 ... done
Attaching to wikibase-registry_wikibase-131_1
wikibase-131_1 | wait-for-it.sh: waiting 120 seconds for mysql.svc:3306
wikibase-131_1 | wait-for-it.sh: mysql.svc:3306 is available after 0 seconds
wikibase-131_1 | wait-for-it.sh: waiting 120 seconds for mysql.svc:3306
wikibase-131_1 | wait-for-it.sh: mysql.svc:3306 is available after 1 seconds
wikibase-131_1 | /extra-entrypoint-run-first.sh: line 3: MW_ELASTIC_HOST: unbound variable
wikibase-registry_wikibase-131_1 exited with code 1
Code language: PHP (php)
The wikibase:1.31-bundle docker image includes the Elastica and CirrusSearch extensions which were not a part of the 1.30 bundle, and due to the entrypoint infrastructure added along with it I will need to change some things to continue without using Elastic for now.
Fix MW_ELASTIC_HOST requirement with a custom entrypoint.sh
The above error message shows that the error occurred while running extra-entrypoint-run-first.sh which is provided as part of the bundle.
It is automatically loaded by the base image entry point.
The bundle now also runs some extra steps as part of the install for wikibase that we don’t want if we are not using Elastic.
If you give the entrypoint file a read through you can see that it does a few things:
- Makes sure the required environment variables are passed in
- Waits for the DB server to be online
- Runs extra scripts added by the bundle image
- Does the Mediawiki / Wikibase install on the first run (if LocalSettings does not exist)
- Run apache
This is a bit excessive for what the wikibase-registry requires right now, so lets strip this down, saving next to our docker-compose file, so /root/wikibase-registry/entrypoint.sh for the wikibase-registry
#!/bin/bash
REQUIRED_VARIABLES=(MW_ADMIN_NAME MW_ADMIN_PASS MW_WG_SECRET_KEY DB_SERVER DB_USER DB_PASS DB_NAME)
for i in ${REQUIRED_VARIABLES[@]}; do
eval THISSHOULDBESET=\$$i
if [ -z "$THISSHOULDBESET" ]; then
echo "$i is required but isn't set. You should pass it to docker. See: https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file";
exit 1;
fi
done
set -eu
/wait-for-it.sh $DB_SERVER -t 120
sleep 1
/wait-for-it.sh $DB_SERVER -t 120
docker-php-entrypoint apache2-foreground
Code language: PHP (php)
And mount it in the wikibase-131 service that we have created by adding a new volume.
volumes:
- ./entrypoint.sh:/entrypoint.sh
Run the new service alongside the old one
Running the service now works as expected.
root@wbregistry-01:~/wikibase-registry# docker-compose up -d wikibase-131
wikibase-registry_mysql_1 is up-to-date
Recreating wikibase-registry_wikibase-131_1 ... done
Code language: PHP (php)
And the service appears in the list of running containers.
root@wbregistry-01:~/wikibase-registry# docker-compose ps
Name Command State Ports
-------------------------------------------------------------------------------------------------
wikibase-registry_mysql_1 docker-entrypoint.sh mysqld Up 3306/tcp
wikibase-registry_wdqs-frontend_1 /entrypoint.sh nginx -g da ... Up 0.0.0.0:8282->80/tcp
wikibase-registry_wdqs-proxy_1 /bin/sh -c "/entrypoint.sh" Up 0.0.0.0:8989->80/tcp
wikibase-registry_wdqs-updater_1 /entrypoint.sh /runUpdate.sh Up 9999/tcp
wikibase-registry_wdqs_1 /entrypoint.sh /runBlazegr ... Up 9999/tcp
wikibase-registry_wikibase-131_1 /bin/bash /entrypoint.sh Up 80/tcp
wikibase-registry_wikibase_1 /bin/bash /entrypoint.sh Up 0.0.0.0:8181->80/tcp
Code language: PHP (php)
Update.php
From here you should now be able to get into your new container with the new code.
root@wbregistry-01:~/wikibase-registry# docker-compose exec wikibase-131 bash
root@40de55dc62fc:/var/www/html#
Code language: PHP (php)
And then run update.php
In theory updates to the database, and anything else, will always be backward compatible for at least 1 major version, which is why we can run this update while the site is still being served from Mediawiki 1.30
root@40de55dc62fc:/var/www/html# php ./maintenance/update.php --quick
MediaWiki 1.31.1 Updater
Your composer.lock file is up to date with current dependencies!
Going to run database updates for wikibase_registry
Depending on the size of your database this may take a while!
{snip boring output}
Purging caches...done.
Done in 0.9 s.
Code language: PHP (php)
Switching versions
The new service is already running alongside the old one, and the database has already been updated, now all we have to do is switch the services over.
If you want a less big bangy approach you could probably setup a second port exposing the updated version and direct a different domain or sub domain to that location, but I don’t go into that at all here.
Move the “ports” definition and “networks” definition from the “wikibase” service to the “wikibase-131” service. Then recreate the container for each service using the update configuration. (If you have any other references to the “wikibase” service in the docker-compose.yml file such as in depends-on then you will also need to change this.
The reason that we run the “up” command specifically on the 2 “wikibase” services is to avoid touching any of the other services at all.
root@wbregistry-01:~/wikibase-registry# docker-compose up -d wikibase
wikibase-registry_mysql_1 is up-to-date
Recreating wikibase-registry_wikibase_1 ... done
Code language: PHP (php)
root@wbregistry-01:~/wikibase-registry# docker-compose up -d wikibase-131
wikibase-registry_mysql_1 is up-to-date
Recreating wikibase-registry_wikibase-131_1 ... done
Code language: PHP (php)
If everything has worked you should see Special:Version reporting the newer version, which we now see on the wikibase-registry.

Cleanup
Now that everything is updated we can stop and remove the previous “wikibase” service container.
root@wbregistry-01:~/wikibase-registry# docker-compose stop wikibase
Stopping wikibase-registry_wikibase_1 ... done
root@wbregistry-01:~/wikibase-registry# docker-compose rm wikibase
Going to remove wikibase-registry_wikibase_1
Are you sure? [yN] y
Removing wikibase-registry_wikibase_1 ... done
Code language: PHP (php)
You can then do some cleanup:
- Remove the “wikibase” service definition from the docker-compose.yml file, leaving “wikibase-131” in place.
- Remove any files or extensions (older versions) that are only loaded by the old service that you have now removed.
Further notes
There are lots of other things I noticed while writing this blog post:
- It would be great to move the env vars out of the docker-compose and into env var files.
- The default entrypoint in the docker images is quite annoying after the initial install and if you don’t use all of the features in the bundle.
- We need a documentation hub? ;)
Hi,
thank you for this tutorial! At some point you write “I’m not going to bother checking that the backups are actually complete here, but you might want to do that!”. Could you explain how to do this? Is is just checking that the bz2 file is fine, like with bunzip2 -t dump?
Thank you
D063520
Checking that the bz2 file is okay is definitely a good first check. But the ultimate check would be restoring the backup volume to a different container and ensuring that everything looks good from there.