WBStack 2020 Update 1

April 19, 2020 1 By addshore
This entry is part 4 of 12 in the series WBStack

WBStack has now been up and running for 6 months. During that time it has helped 70 people create 178 MediaWiki installs running Wikibase, a SPARQL query service and quickstatements, all at the click of a button, with a total of around 200,000 edits across all sites.

The most active site is currently virus-taxonomy.wiki.opencura.com which was developed during the Virtual Biohackathon on COVID-19 as a staging environment for “improving the taxonomy of viruses on Wikidata”. It currently stands at 20,000 edits, around 7000 Items.

Screenshot of the virus-taxonomy Wikibase Main Page, 19 April 2020

Thanks again to Rhizome, who run their very own Wikibase, for their support paying the Google Cloud bill in the early stages of this project.

Updates

2020 has so far seen 135 commit to the currently private git repo (38 a month). Today the git repo hit 1038 commits, the first being on 29 December 2017.

For previous update posts see the 2019 October Introduction, November Review and January 2020 Infrastructure Overview.

MediaWiki & Wikibase

MediaWiki and extensions have all be updated to include the latest security fixes, this is MediaWiki 1.33.3. You can find the release notes here.

MediaWiki has had a large number of commonly used extensions enabled. These include: JsonConfig, Kartographer, Math, Score, PageImages, Scribunto, Cite, TemplateSandbox, CodeEditor, WikiEditor, SecureLinkFixer, Echo, Graph, Poem, TemplateData, AdvancedSearch, ParserFunctions, MobileFrontend, DeleteBatch, MultimediaViewer and EmbedVideo. The MinervanNeue skin was also added.

Wikibase has seen the addition of a few new datatypes that have already been on Wikidata for quite some time, these include Musical Notation and Mathematical Expression.

WikibaseLexeme is also deployed to the platform and enabled by request on some sites. If you want to try out this extension please get in touch as a feature toggle has not yet made its way into the UI.

If you want to read up move on the skins, extensions of data types then I advise that you click one of the many links above.

You’ll also notice a shameless bit of self promotion that has been added to the bottom of all sites. Hopefully we can add this to Wikibase soon (Wikibase task).

For site managers

Site managers in this context are the users that created the site on wbstack.com. Currently that is limited to 1 manager per site, but that will be changing in the future.

Many wbstack.com UI bugs have been fixed, these include confusing form input errors when creating accounts and sites. More content is now included on the main dashboard and, if you forget your password, there is now a password reset flow available from the login page.

Site managers can now set a Logo for the site that will automatically be sized and applied to MediaWiki. And to get rid of all of those pesky test sites, there is finally a delete button!

When creating a site you now also have the option to use a custom domain name.

Screenshot of the wbstack site management page, 19 April 2020

Queryservice

The WBStack query service updater has been a terrible hack since day 1. It was a PHP script wrapping around the main Java updater, and the Java updater would be shelled out to based on events from MediaWiki. This is of course slow, and the JVM for the updater could regularly take 30 seconds to fully initiate, all to send a single update to the backend.

Finally this has been totally rewritten in Java, so you should see less delays to your query service. Though currently specific to WBStack, this multi site updater should make its way into the main Wikidata query git repo for use by others if needed. You can find the current Gerrit patch here.

The query service itself has also seen an expansion to the whitelisted SPARQL endpoints. You can see the full list here.

Under the hood

If you want to know more about how all of the moving parts tie together take a look at my 2019 infrastructure post.

Backups always existed of all sites, however these were taken by hand, now automatic snapshots of all sites are taken every single night!

The main platform is powered by Laravel, and was recently updated from version 5.8 to 6.18. The main wbstack.com UI is written in VueJS using Vuetify which has been updated from version 1.5 to 2.2, which has lead to some UI improvements and layout changes. Other backend services such as Redis, MariaDB, Nginx, and Cert-Manager have also been kept up to date with security fixes and new releases.

The whole platform is powered using Docker images on Kubernetes running in the Google Cloud. One part of deploying to the site involves building docker images. The build pipeline now makes use of Kaniko build caching which dramatically speeds up the time to live for needed changes. The “wikibase-docker” images are not used on WBStack, and the images that are used are not really fit for use outside of the infrastructure. However learnings will continue to filter into “wikibase-docker”.

The future

The future is bright, and although WBStack is still very much in an alpha state the platform has proved itself to be scalable, managale, maintainable and of use to people.

In the last month I presented the idea of WBStack at a remote version of EMWcon (Enterprise MediaWiki con). One of the quotes from the slide deck is:

WMDE work around “Wikibase as a Service” is planned in this area during the second half of 2020.

EMWCon 2020 slides

If you are interested in this effort then please contact the Product manager for Wikibase, Samantha Alipio.

Planned changes

One of the biggest changes which should happen in the next 3 months will be the upgrade from MediaWiki 1.33 to 1.35. This will bring a variety of features including some new special pages, a new REST API and a PHP based Parsoid service. The PHP based parsoid service should allow VisualEditor to more easily be deployed on WBStack (something I have been waiting for).

As well as VisualEditor I am keen to try and deploy some sort of collaborative editor for wiki pages. This could be the VisualEditor “CollabPad”, or some other, possibly new extension.

Login is not currently where I want it to be. MediaWiki “user identity” extensions don’t currently offer the level of flexibility that I would like. This being custom usernames, but easy registration and login making use of common authentication methods such as Google, Twitter or others.

I want to push more settings into the site manager view such site language, site name and default skin selection, and also open up the ability to have a site managed by multiple people. Sites should also be more discoverable on the wbstack website itself.

A documentation hub of some description is also long overdue. This could be added to the main wbstack site, or created as a wiki itself, possibly sitting at wiki.wbstack.com. Dog food is good for you after all!

Series Navigation<< WBStack Infrastructure (2020)WBStack 2020 Update 2 (May) >>