wikibase

How much is “Wikibase Suite” (and deploy) used

by
This entry is part 3 of 3 in the series Wikibase ecosystem

At the start of this year, I spent some time visualizing the Wikibase ecosystem by making use of the data that has been collected on wikibase.world. As part of that, I tried to focus in on Wikibase Suite, trying to determine how many possible installations there were making use of the container images, and or the newer Wikibase Suite Deploy thing.

I came to the number 33, based on the fact that there were this many sites online on wikibase.world that have an exact match to the MediaWiki versions that have been released as part of the Wikibase Suite container images. And in all cases, this would be an overestimate, given that these versions would also be installed by some not using the images, so the likely number would be closer to ~16…

And of the 33 sites that might possibly be using “suite” as they are on the same version at least, probably 50% are installed via other means, so the “suite” installations probably account for ~16 of the wikibases in wikibase.world at a guesstimate, with ~50 using other methods and 711 using wikibase.cloud.

9th December 2025 Edit

Apparently this post attracted some attention, and I want to make it clear that “Wikibase suite” here is specifically talking about the packaging up of the container images / docker images into some single installable reusable magic component, and support system around it.

I personally believe the container images themselves are a great asset, and the idea of a suite of recommended extensions and applications that should be delivered alongside a Wikibase is also an asset that the ecosystem does need.

Also looking at the “Wikibase Suite Team” board for “Sprint 9 (Nov 25 – Dec 9)” federation is a key topic that is currently being worked on, and tasks like T404547 [Self-Hosting Ops] Define metric for ease of self-hosting show that the team is / has moved away from only thinking about the magic packaged layer.

Updated count

10 months on from this first look, while visiting the Wikimedia Germany offices, I found the need once again to try to come up with concrete numbers in terms of the users of Wikibase Suite, where my general motive would be to convince WMDE that resources are better spent in other places, such as supporting the underlying software, not just this fancy wrapper on top.

And with this new analysis, my revised number is roughly 18, of which 9 are possibly active, and 2 are likely lost to bot spam. You can find the list via this rather lengthy wikibase.world query.

Read more

What is Wikibase “Federated Properties” in 2025

by

I recently wrote a post looking at the history of the Wikibase “Federated Properties” feature. While at Wikimania 2025 the topic of federation came up a few times, particularly given the current discussions ongoing on the Wikidata project chat page including discussions about wikicite, and the recent Wikidata graph split.

All the code for the “Federated Properties” feature still exists in Wikibase code, despite a ticket being open on phabricator to potentially delete it. And it turns out that the configuration for it still exists on wikibase.cloud too, where the feature was initially presented to the communities to try out.

So with a little bit of sneaky “hacking”, I can try to summarize the current / final state of the “Federated Properties” feature, after development during the MVP stopped some years ago.

This also means you can still try out the feature on your own wiki using the setting.

$wgWBRepoSettings['federatedPropertiesEnabled'] = true;

Creating a local property

Firstly, we need a property, and the creation workflow is exactly the same as on a normal Wikibase.

Read more

What was Wikibase “Federated Properties”

by

The “Federated Properties” feature allows / allowed a local Wikibase instance to access and utilise properties directly from a remote Wikibase, primarily Wikidata. Its primary purpose is to enable partial federation between a local Wikibase and Wikidata, broadening the base of available data without needing to create a property set from scratch.

I’m split between using the present and past tense here, as all of this code still exists within the Wikibase extension, however no one has used it since 2022, and it certainly doesn’t seem to be on the short or medium term (or maybe even long term) roadmaps.

This overview comes from the Wikibase – Federated Properties Phabricator project, which I’ll quote the whole of here for prosperity.

Federated Properties v2 (2021)
An initiative to give users the ability to access remote properties from their local Wikibase and use them in combination with custom local properties. The primary use case is enabling partial federation between a Wikibase and Wikidata. This version of the feature will allow you to:

  • Opt-in to use Wikidata’s properties in addition to your own custom local properties
  • Create and view statements about local entities that contain both local and federated properties
  • Query your Wikibase using both local and federated properties

Federated Properties v1 (2020-2021)
An initiative to give users the ability to access remote properties from their local Wikibase (no local properties were possible in this MVP). This version was launched in the Wikibase Spring Release in May 2021.

As far as I remember, the project died with v2, and I don’t even recall if v2 really saw the light of day outside WMDE internal testing and or hidden testing on wikibase.cloud.

Read more

Wikimedia Commons Depicts statements over time

by

Wikimedia Commons now uses Structured Data on Commons (SDC) to make media information multilingual and machine-readable. A core part of SDC is the ‘depicts’ statement (P180), which identifies items clearly visible in a file. Depicts statements are crucial for MediaSearch, enabling it to find relevant results in any language by using Wikidata labels, as well as having pre precise definition and structure than the existing category structures.

SDC functionalities began to roll out in 2019. Multilingual file captions were introduced early that year, enabling broader accessibility, followed by the ability to add depicts statements directly on file pages and through the UploadWizard.

Although there are numbers floating around showing a general increase in usage of structured data on Commons, there didn’t seem to be any concrete numbers around the growth in use of depicts statements.

I was particularly interested in this, as must tool WikiCrowd is steadily becoming a more and more efficient way of adding these statements en masse. So I decided to see what data I could come up with.

Read more

Wikibase ecosystem in Q1 2025, according to wikibase.world

by

I wrote a post in February 2025 looking at what the Wikibase ecosystem (might) look like, according to the data that had at that point been collected on wikibase.world. Now that data has had some time to evolve and expand, we can take a little look at how it has changed throughout the last 2 months.

In the future, I’ll try to remember to write something up every quarter or so (for now), until someone else feels like taking this over ;)

The latest notebooks for generating this are in git, and the latest XML file dumped from wikibase.world is on archive.org.

Site count and status

We have gone from tracking 777 sites, up to 873, so an increase of nearly 100 in 2 months.

However, we need to look at the tracked status to determine how big the current ecosystem actually might be. So I added a little extra counting to the notebook previously used to count the wikis based on P13 (availability status).

Back in Feb, there were 774 online, and only 3 marked as offline. This was primarily as Addbot was not often marking sites as offline, however I added automatic detection of deleted sites for wikibase.cloud and went through and checked a bunch of sites that the scripts were failing to lookup.

Looking at the April data, we have ~847 online, and ~26 offline, so an increase of around 3%.

Graph

Most of the growth in sites seems to come from wikibase.cloud, however many sites on wikibase.cloud are test sites and may not have much content.

So when displaying the graph this time, I’ll filter out everything that doesn’t have a highest Item ID of at least 25, this roughly cuts the size of the graph in half.

Read more

Profiling Wikibase APIs and import speed

by

There has been some recent chat once again on the Wikibase telegram groups around importing, and the best approach to import a large amount of data into a Wikibase instance. 2 years ago I started a little GitHub project aimed at profiling the speed of loading using the action API, and various settings, DB versions etc, as well as trying out a bulk load API. And I have just taken the opportunity to take another look at it and try to visualize some of the comparisons given changes through the last 2 years.

In case you don’t want to read and follow everything below, the key takeaways are:

  • EPS (edits per second) of around 150 are achievable on a single laptop
  • When testing imports, you really need to test at least 50k items to get some good figures
  • The 2 ID generation related settings are VERY IMPORTANT if you want to maximise import times
  • Make async requests, but not too many, likely tuned to the number of CPUs you have serving web requests. You wan near 100% utilization
  • A batch API, such as FrozenMink/batchingestionextension would dramaticly increase import times

Some napkin math benchmarks for smallish items, I would hope:

  • 1 million items, 2 hours (validated)
  • 10 million items, 1 day
  • Wikidata (116 million) items, 14 day+

Read more

Visualizing Wikibase ecosystem, using wikibase.world

by
This entry is part 2 of 3 in the series Wikibase ecosystem

In October last year, I wrote a post starting to visualize the connections between Wikibases in the ecosystem that had been found and collected on wikibase.world thanks to my bot that I occasionally run. That post made use of the query service visualizations, and in this post I’ll take the visualizations a step further, making use of IPython notebooks and plotly.

Previously I reported the total number of Wikibases tracked in wikibase.world being around 784, with around 755 being active (however I didn’t write down exactly how I determined this). So I’m going to take another stab at that with some code backing up the determinations, rather than just my late night data ramblings.

All of the data shown in this post is generated from the IPython notebook available on Github, on 16 Feb 2025, based on the data on wikibase.world which is maintained as a best effort system.

General numbers

MetricValue
Wikibases with properties777
Wikibases with properties, and more than 10 pages600
Wikibases with properties, and more than 10 pages, and 1 or more active users264
Wikibases with properties, and more than 10 pages, and 2 or more active users129
Wikibases that link to other wikibases194
Wikibases that only link to non Wikimedia Foundation wikibases5
Wikibases that link to other wikibases, excluding Wikimedia Foundation35

A few things of note:

  • “with properties” is used, as a clear indicator that Wikibase is not only installed, but also used in at least a very basic way. (ie, it has a created Wikibase property). I would use the number of items ideally as a measure here, however as far as I can tell, this is hard to figure out?)
  • “with more than 10 pages” is my baseline measure of the site having some content, however this applies across all namespaces, so can also be wikitext pages…
  • “active users” are taken from MediaWiki statistics, and apply across all namespaces. These numbers also rely on MediaWiki being correctly maintained and these numbers actually being updated. (Users who have performed an action in the last 30 days)
  • “link to other wikibases” are links extracted from sites by Addbot either via external links or specific properties that state they are links to other wikibases. (The code is not pretty, but gives us an initial view)

And summarized in words:

  • 264 Wikibases with some content that have been edited in the past 30 days
  • 194 Wikibases link in some way to other Wikibases
    • Excluding links to Wikidata and Commons, this number comes down to 35 (So Wikidata is very much the centre)

And of course, take all of this with a pinch of salt, these numbers are an initial stab at trying to have an overview of the ecosystem.

An updated web

My October post included some basic visualizations from the query service of wikibase.world.

However, it’s time to get a little more fancy and interactive. (As well as showing all wikibases, not just the linked ones)

Read more

Visualizing Wikibase connections, using wikibase.world queries

by
This entry is part 1 of 3 in the series Wikibase ecosystem

Over the past week I have spent some time writing some code to start running a little bot on the wikibase.world project, aimed at expanding the number of Wikibases that are collected there, and automating collection of some of the data that can easily be automated.

Over the past week, the bot has imported 650 Wikibase installs that increases the total to 784, and active to 755.

I mainly wanted to do this to try and visualize “federation” or rather, links between Wikibases that are currently occurring, hence creating P55 (links to Wikibase) and P56 (linked from Wikibase).

251 Wikibases seem to link to each other, and Wikidata is very clearly at the centre of that web.

Read more

2 years of wikibase.cloud by WMDE

by

It’s been somewhere between 2 and 3 years since WMDE took over WBStack, turned it into wikibase.cloud. During this time, my techy focus has slowly shifted away from the world of Wikibase, though I still enjoy following along and working on other Wikimedia areas.

Here I will ramble on about what I saw in terms of potential for wikibase.cloud within the Wikibase ecosystem, as well as what developments have happened within the past years.

The initial problems, goals and dreams

From An introduction to WBStack, I said:

The idea behind the project is to provide Wikibase and surrounding services, such as a blazegraph query service, query service ui, quick statements, and others on a shared platform where installs, upgrades and maintenance are handled centrally.

Now, this is fairly obvious, and clearly something that wikibase.cloud still offers today, however I didn’t write why!? And this is potentially something that has gotten lost through the years of multiple PMs, multiple engineers, multiple project names etc.

Read more

Cloudflare workers for wikibase.cloud uptime & status

by

Recently I wanted to create a live status page for wikibase.cloud, that also tracking the status of the various services and response times, so that people in the Telegram group might be able to try and correlate their experiences (possibly slow behaviour) with what was seen by others in other locations on other sites, without needing to message in the Telegram group.

In a way, this could be seen as an iteration on the current status page for the service, which is maintained as a static site on Github, making use of cState, a static status page.

Screenshot of the current status page

I initially chose to experiment with Cloudflare Workers to do the minutely checks, after looking around at the current offerings for free online code running (thinking Heroku style etc).

Why Workers?

Read more