Wikidata, instance of and subclass of through time (P31 & P279)

by

Last month I looked at all Wikimedia Commons revisions and managed to generate some data and graphs for the usage of depicts statements since they were introduced on the project.

This month, I have applied the same analysis on Wikidata but looking at instance of and subclasses of items. A slightly bigger data set, however essentially the same process.

This will enable easy updating, of various pie charts that have been published over the years, such as

In future, this could be easily adapted to show per Wikipedia project graphs, such as those that are currently at Wikidata:Statistics/Wikipedia

Method

The details of the method can be seen in code in my previous post about depicts statements, and this mostly stays the same.

In words:

  • Look at every revision of Wikidata ever
  • Parse the JSON to determine what values there are for P31 and P279 for each revision
  • Find the latest revision of each item in each given month, and thus find the state of all items in that month
  • Plot the data by number of items that are P31 or P279 of each value item

There are some minor defects to this logic currently that could be cleaned up with future iterations:

  • Deleted items will continue being counted, as I don’t consider the point items are deleted
  • Things will be double counted in this data, as 1 item may have multiple P31 and P279 values, and I don’t try to join these into higher level concept at all

We make an OTHER and UNALLOCATED count as part of the final data summarization. OTHER accounts for things that have not made it into the top 20 items by count, and UNALLOCATED means that we didn’t have a P31 or P279 value in the latest revision.

2025

For August 2025 (or at least part way through it), this is the current state of Wikidata per the above method.

You can now find a PNG of this pie chart on Wikimedia Commons https://commons.wikimedia.org/wiki/File:Wikidata_P31_%26_P279_analysis_August_2025.png

Read more

Online RDS column type modification, using pt-online-schema-change from EC2

by

I’m using percona-tools to do an online schema modification today, and thought I would document the process, especially as even the installation guides seem to be badly linked, out of date, and did not work out of the box…

EC2 instance

This is all running on a t3.micro EC2 instance with Ubuntu. I deliberately didn’t go with Amazon Linux, as I wanted to be able to use apt. For simplicities’ sake, I’ll be using the EC2 Instance Connect feature, which allows connection to a session in a web browser! (although the copy and paste via this is annoying)

This instance of course also needs access to your MySQL server, in this case an RDS instance. So I’ll go ahead and add it to the security group.

Percona toolkit

Percona Toolkit is a powerful open-source collection of advanced command-line tools designed to help MySQL and MariaDB DBAs perform tasks like online schema changes, replication troubleshooting, and data auditing safely and efficiently.

It’s used a Wikimedia for online database migrations (the reason I know about it), however I have never actually used it myself!

Read more

What is Wikibase “Federated Properties” in 2025

by

I recently wrote a post looking at the history of the Wikibase “Federated Properties” feature. While at Wikimania 2025 the topic of federation came up a few times, particularly given the current discussions ongoing on the Wikidata project chat page including discussions about wikicite, and the recent Wikidata graph split.

All the code for the “Federated Properties” feature still exists in Wikibase code, despite a ticket being open on phabricator to potentially delete it. And it turns out that the configuration for it still exists on wikibase.cloud too, where the feature was initially presented to the communities to try out.

So with a little bit of sneaky “hacking”, I can try to summarize the current / final state of the “Federated Properties” feature, after development during the MVP stopped some years ago.

This also means you can still try out the feature on your own wiki using the setting.

$wgWBRepoSettings['federatedPropertiesEnabled'] = true;

Creating a local property

Firstly, we need a property, and the creation workflow is exactly the same as on a normal Wikibase.

Read more

What was Wikibase “Federated Properties”

by

The “Federated Properties” feature allows / allowed a local Wikibase instance to access and utilise properties directly from a remote Wikibase, primarily Wikidata. Its primary purpose is to enable partial federation between a local Wikibase and Wikidata, broadening the base of available data without needing to create a property set from scratch.

I’m split between using the present and past tense here, as all of this code still exists within the Wikibase extension, however no one has used it since 2022, and it certainly doesn’t seem to be on the short or medium term (or maybe even long term) roadmaps.

This overview comes from the Wikibase – Federated Properties Phabricator project, which I’ll quote the whole of here for prosperity.

Federated Properties v2 (2021)
An initiative to give users the ability to access remote properties from their local Wikibase and use them in combination with custom local properties. The primary use case is enabling partial federation between a Wikibase and Wikidata. This version of the feature will allow you to:

  • Opt-in to use Wikidata’s properties in addition to your own custom local properties
  • Create and view statements about local entities that contain both local and federated properties
  • Query your Wikibase using both local and federated properties

Federated Properties v1 (2020-2021)
An initiative to give users the ability to access remote properties from their local Wikibase (no local properties were possible in this MVP). This version was launched in the Wikibase Spring Release in May 2021.

As far as I remember, the project died with v2, and I don’t even recall if v2 really saw the light of day outside WMDE internal testing and or hidden testing on wikibase.cloud.

Read more

Wikimedia Commons Depicts statements over time

by

Wikimedia Commons now uses Structured Data on Commons (SDC) to make media information multilingual and machine-readable. A core part of SDC is the ‘depicts’ statement (P180), which identifies items clearly visible in a file. Depicts statements are crucial for MediaSearch, enabling it to find relevant results in any language by using Wikidata labels, as well as having pre precise definition and structure than the existing category structures.

SDC functionalities began to roll out in 2019. Multilingual file captions were introduced early that year, enabling broader accessibility, followed by the ability to add depicts statements directly on file pages and through the UploadWizard.

Although there are numbers floating around showing a general increase in usage of structured data on Commons, there didn’t seem to be any concrete numbers around the growth in use of depicts statements.

I was particularly interested in this, as must tool WikiCrowd is steadily becoming a more and more efficient way of adding these statements en masse. So I decided to see what data I could come up with.

Read more

Easy WSL Windows path switching alias

by

I have been primarily developing on WSL for some years now, and still love the combination in terms of all around flexibility. When primarily working on Linux based or focused applications, everything is lovely! However, I’m spending more time straying into the land of hardware, USB devices, and custom IDEs and debug interfaces that are … Read more

AI Code assistant experience comparison (golang-kata-1)

by

If you’re reading this, and thinking about trying an IDE integrated coding agent, or thinking about switching, maybe stick around, have a read and watch some of the videos. There is at least 6 hours worth of experience wrapped up in this 20 minuite read!

I’m watching a thread on the GitHub community forums, where people are discussing how GitHub Copilot has potentially gone slightly downhill. And in some ways I agree, so I through I’d spend a little bit more time looking at the alternatives, and how they behave.

This post tries to compare 9 different setups, and will primarily look at the differences in presentation within the VS Code IDE that each of these different coding assistants have. How the default user interactions work, and how the tasks are broken down and presented to the user, and generally what the user experience is like between these different assistants.

I’ll try to flag up some other useful information along the way, such as time comparisons, amount of human interaction needed, and overall satisfaction with what the thing is doing, and if this all presents itself nicely in this post, I might find myself writing more in the future…

However, I will not be looking at cost, setup, resource usage or what’s happening with my data along the way…

Assistant, LLM combinations

AssistantModelMain tasks @Tests @Second app @
Github CopilotGPT 4o~ 5:00~ 24:45~ 32
Github CopilotGPT 4.1~ 15:00~ 17:40~ 35
Github CopilotClaude Sonnet 4~ 17:00 (inc tests)~ 17:00~ 28
Gemini Code AssistantGemini Something ?~ 11:20~ 14:30~ 25
AmazonQClaude Sonnet 4~ 7:20~ 15:50~ 28
RoocodeGPT 4.1 (via Github Copilot)~ 5:30~ 10:00~ 18
RoocodeClaude Sonnet 4 (via Anthropic)~ 15:30~ 20:00~ 37
Claude CodeClaude Sonnet 4~ 9:30~ 17:40~ 24
Claude CodeClaude Opus 4~ 10:00N/AN/A

I have setup this post, and the code problem in such a way that I should be able to easily add more combinations and comparisons in the future, and directly compare the performance back to this post. Ideally, at some stage I’d try some other models via Ollama, and also some other pay per requests LLM APIs…

Read more

Toit: jag monitor proxy

by

While developing on ESP32 boards at Lightbug on some of our newer products, I have repeatedly wanted to run Toit and Jaguar without WiFi enabled during a development setting. Either to have WiFi off to keep power consumption lower, turn off the default network so that I can make use of a secondary network or set up my own hotspot, and most recently, so that I can quickly iterate while using ESP-NOW.

Many months ago @floitsch told me about the --proxy option for the jag monitor command in Discord, and I finally have gotten around to trying it out this week. Here is how it went…

Using –proxy

The --proxy argument exists on the jag monitor command, which is essentially a slightly fancy serial / COM port monitor.

Adding this argument will attempt to proxy the device to the local network via your computer, rather than via the ESPs own WiFi. As a result, you can turn the WiFi off!

~ jag monitor --help
Monitor the serial output of an ESP32

Usage:
  jag monitor [flags]

Flags:
  -a, --attach            attach to the serial output without rebooting it
      --baud uint         the baud rate for serial monitoring (default 115200)
      --envelope string   name or path of the firmware envelope
  -l, --force-plain       force output to use plain ASCII text
  -r, --force-pretty      force output to use terminal graphics
  -h, --help              help for monitor
  -p, --port string       port to monitor (default "/dev/ttyUSB0")
      --proxy             proxy the connected device to the local network
Code language: Shell Session (shell)

Read more

WikiCrowd for 2025

by

I wrote the first version of WikiCrowd back in 2022 and haven’t really iterated on it much since, beyond adding the odd new set of image categories, and removing features that I decided were not optimum.

At the 2025 Wikimedia Hackathon however, WikiCrowd came up as both an entertaining little game to show people during beers, and also a project similar (ish) to something Daanvr was working on (I think it was Suggestion-Engine-Commons-prototype ?)

Upgrades

During the hackathon, and in the weeks following, WikiCrowd went through quite a number of changes

  • The YAML config files for the pre-calculated depicts statements are now on Commons for all to edit
  • Generation of the questions has been spruced up to stop it breaking as it gets deeper into category trees
  • Generation can now be triggered in the UI, as can deleting pending questions
  • The old one by one image mode was removed, and instead replaced by a grid mode
  • More categories and depict options were added
  • A custom grid view was added, allowing users to specify their own category and or Wikidata item
  • Ability to zoom in on an image being displayed
  • Addition of “levels” of questions
  • Display of Wikidata labels and descriptions in the UI (Making use of the new REST API)

Read more

Wikibase ecosystem in Q1 2025, according to wikibase.world

by

I wrote a post in February 2025 looking at what the Wikibase ecosystem (might) look like, according to the data that had at that point been collected on wikibase.world. Now that data has had some time to evolve and expand, we can take a little look at how it has changed throughout the last 2 months.

In the future, I’ll try to remember to write something up every quarter or so (for now), until someone else feels like taking this over ;)

The latest notebooks for generating this are in git, and the latest XML file dumped from wikibase.world is on archive.org.

Site count and status

We have gone from tracking 777 sites, up to 873, so an increase of nearly 100 in 2 months.

However, we need to look at the tracked status to determine how big the current ecosystem actually might be. So I added a little extra counting to the notebook previously used to count the wikis based on P13 (availability status).

Back in Feb, there were 774 online, and only 3 marked as offline. This was primarily as Addbot was not often marking sites as offline, however I added automatic detection of deleted sites for wikibase.cloud and went through and checked a bunch of sites that the scripts were failing to lookup.

Looking at the April data, we have ~847 online, and ~26 offline, so an increase of around 3%.

Graph

Most of the growth in sites seems to come from wikibase.cloud, however many sites on wikibase.cloud are test sites and may not have much content.

So when displaying the graph this time, I’ll filter out everything that doesn’t have a highest Item ID of at least 25, this roughly cuts the size of the graph in half.

Read more