Addshore

It's a blog

Tag: wikibase

Customizing Wikibase config in the docker-compose example

Just over a month ago I setup the Wikibase registry project on Wikimedia Cloud VPS using the docker-compose example provided by Wikibase docker images. The Wikibase registry is the first Wikibase install that I control that uses the Wikibase docker images, so I’ll be using it as an example showing how the docker images can be manipulated to configure MediaWiki, Wikibase, and load custom extensions and skins.

The example docker-compose file at the time of writing this post can be found at https://github.com/wmde/wikibase-docker/blob/5919016eac16c5f0aefc448240fdf6a09bb56bec/docker-compose.yml

Since the last blog post new wikibase image tags have been created (the ‘bundle’ tags) that include some extensions you might want to enable, as well a quickstatements image for the quickstatements service used on Wikidata written by Magnus Manske.

Continue reading

Wikibase of Wikibases

The Wikibase registry was one of the outcomes of the first in a series of Federated wikibase workshops organised in partnership with the European research council.

The aim of the registry is to act as a central point for details of public Wikibase installs hosted around the web. Data held about the installs currently includes the URL for the home page, Query frontend URL and SPARQL API endpoint URL (if a query service exists).

During the workshop an initial data set was added, and this can be easily seen using the timeline view of the query service and a query that is explained within this post.

Continue reading

WMDE: Wikibase Docker intro

This is an internal WMDE presentation made to introduce people to Docker and the Wikibase docker images already introduced in this post.

Wikibase docker images

This is a belated post about the Wikibase docker images that I recently created for the Wikidata 5th birthday. You can find the various images on docker hub and matching Dockerfiles on github. These images combined allow you to quickly create docker containers for Wikibase backed by MySQL and with a SPARQL query service running alongside updating live from the Wikibase install.

A setup was demoed at the first Wikidatacon event in Berlin on the 29th of October 2017 and can be seen at roughly 41:10 in the demo of presents video which can be seen below.

Continue reading

Building applications around Wikidata (a beer example)

Wikidata provides free and open access to entities representing real world concepts. Of course Wikidata is not meant to contain every kind of data, for example beer reviews or product reviews would probably never make it into Wikidata items. However creating an app that is powered by Wikidata & Wikibase to contain beer reviews should be rather easy.

Continue reading

Wikimedia Grafana graphs of Wikidata profiling information

I recently discovered the Wikimedia Grafana instance. After poking it for a little while here are some slightly interesting graphs that I managed to extract.

Continue reading

Removing use of Mediawiki’s API ‘rawmode’ in Wikibase

Rawmode was a boolean value used to determine if an API result formatter in Mediawiki needed extra metadata in order to correctly format the result output. The main use of said metadata was in the XML output of the Mediawiki API. How hard can removing it be? This is the story of the struggle to remove the use of this single boolean value from the Wikibase codebase.

Overview

The first commit for this task was made on the 6th July 2015 and the final commit was about to be merged on the 27th August. So the whole removal took just under 2 months.

During this two months roughly 60 commits were made and merged working towards removal.

Overall 9290 lines were removed and 5080 lines were added.

I’m glad that is all done. (This analysis can be found on Google sheets). Sorry there are not more pictures in this post…..

Reason for removal

Well, rawmode is being remove from Mediawiki to remove API complexity. Instead of having to check what the API formatters need they will instead just accept all metadata and simply use what they need and discard the rest.

The change to “Finish killing ‘raw mode'” can be seen on Gerrit and has been around since April of this year. The relevant task can be found on Phabricator.

Process overview

The first step on the path was to remove the old serialization code from Wikibase (otherwise known as the lib serialization code) and replace all usages with the new WikibaseDataModelSerialization component. This component was already used in multiple other places in the code but not in the API due to its reliance on the way the lib serialization code handled the rawmode requirement of the API at the time.

Removal of the lib serialization code was the the first of the two major parts of the process and after around 50 commits I managed to remove it all! Hooray for removing 6000 lines with no additions in a commit…

The next and final step was to make the ResultBuilder class in Wikibase always provide metadata for the API and to remove any dirty hacks that I had to introduce in order to kill the lib code. Again this was done over the course of multiple commits, mainly adding tests for the XML output which at the time was barely tested. Finally a breaking change had to be made to remove lots of the hacks that I had added and the final uses of raw mode.

The final two commits can be seen at http://gerrit.wikimedia.org/r/#/c/227686/ and http://gerrit.wikimedia.org/r/#/c/234258/

Final notes

Look! You can even see this on the GitHub code frequency graph (just….)

You can also find my draft API break announcement post here.

— edit —

It looks like I did break 1 thing incorrectly: https://phabricator.wikimedia.org/T110668 , thought a fix is on it’s way our to the beta site! :)

© 2018 Addshore

Theme by Anders NorenUp ↑