Welcome to the third installment of my tech lead digest digest. In order to allow myself some extra space to write, and also to provide these public updates and thoughts on a more regular basis, this is now becoming a monthly digest.
I’m going to try to incorporate some of the ongoings from other Wikidata / Wikibase projects, as well as my regular digest and reading.
🧑🤝🧑Wikidata & Wikibase
Work continues on the next iteration of Wikibase Federated Properties (phabricator board). This work will allow use of properties from multiple sources at once, such as Wikidata and also the local Wikibase.
Work also continues on the Wikidata Mismatch Finder (phabricator board) which is a tool to enable finding mismatches between Wikidata’s data and data in other databases.
The Campsite continues to work on a variety of smaller tasks, in the last month including a new release of our Design System, dealing with the Query Builder security review and preparing for deployment, performing some maintenance on WBStack including preparing for 1.36 and adding Elasticsearch. We also continue to support a university team in deploying a new Property Suggester algorithm (announcement coming soon), work towards tagging all edits made from the UI (T236893), as well as many other smaller tasks.
A recent Wikibase email list post on the topic of Wikibase and bulk imports caused me to write up a mostly human readable version of what happens, in what order, and when, for Wikibase action API edits, for the specific case of item creation.
There are a fair few areas that could be improved and optimized for a bulk import use case in the existing APIs and code. Some of which are actively being worked on today (T285987). Some of which are on the roadmap, such as the new REST APIs for Wikibase. And others which are out there, waiting to be considered.
This post is is written looking at Wikibase and MediaWiki 1.36 with links to Github for code references. Same areas may be glossed over or even slightly inaccurate, so take everything here with a pinch of salt.
Reach out to me on Twitter if you have questions or fancy another deep dive.
If you’re working with legacy code, chances are you’ve inherited some technical debt. Infact, if you’re working with code, chances you’re already surrounded by technical debt of varying sizes, at least by some measures.
Some believe that technical debt is something to be avoided, and that technical debt that exists is a dirty secret that should be hidden. The reality is that technical debt is a fact of life when code iteratively changes to deliver product solutions.
Striving for programming perfection is great in principle, but ultimately code is meant to deliver features, and there is always a good, better and best approach, with many other variations in-between.
Over the last year at Wikimedia Deutschland we have worked on refining how we record, triage, prioritize and tackle technical debt within the Wikidata and Wikibase product family.
There are many thoughts out there about how to track, tackle, and prioritize technical debt. This post is meant to represent the current status of the Wikidata / Wikibase team. Hopefully you find this useful.
This is the second installment of my tech lead digest digest with my tech lead hat on for the Wikidata Wikibase team.
This is a digest of my internal digest emails. These contain lots of links to reading, podcasts and general goings on that could be useful to a wider audience.
🧑🤝🧑Wikidata & Wikibase
Federated Properties, Mismatch finder & WBaaS maintenance teams have spun up from the Campsite
The “experimental” patch for Github Dependabot -> Gerrit was merged
Browser tests came up as a hot topic. A deep dive and some central analysis occurred seemingly correlation failures with “memory compaction” on VMs. This is out of our control, so we increased timeouts in some key areas.
MediaWiki-Docker-Dev (or MWDD) is a development environment for MediaWiki, based on Docker and docker-compose. It was created back in 2017 at the Wikimedia Hackathon in Vienna where it had a slightly difference feature set and focus. (Original Slides).
Since inception the git repo now has 180 commits from 20 authors over the course of 4 years, of which 7 have been WMF employees and 11 have been WMDE employees, though the project has had no “official” support from either organization. Counting forks we have 12 WMF employees and 16 WMDE employees.
Due to the nature of the project (being setup from a git clone), it is quite hard to figure out how many users it has. We can infer that in the last year, thanks to a custom image that has been required, it has been set up roughly 1200 times, by checking the pull stats of silvanwmde/nginx-proxy.
The number of Github repositories that I end up maintaining in one way or another ends up growing week by week. And keeping all of the descriptions and settings up to date in sync can be painful todo by hand.
A little while ago I migrated my addwiki project to use a monorepo, and thus needed to bulk update all of the github repository descriptions. While doing so I made use of the github cli and created a single bash script to let me configure all of the repositories at once.
Assuming you already have the github cli install and configured getting started with this is easy.
The command
The below command is one of many in my bash script for repo configuration. This sets a description, homepage and various other flags that I want to be consistent across repositories.
gh api --method PATCH repos/addwiki/addwiki \
--field description='Monorepo containing all addwiki libraries, packages and applications'\
--field homepage='https://addwiki.github.io/'\
--field has_issues='true'\
--field has_projects='false'\
--field has_wiki='false'Code language:JavaScript(javascript)
Recently I found myself altering some virtual images for loading onto a qemu machine. I wanted to increase the disk space on the root partition, but couldn’t find any straightforward guides. So here is a little guide for future me, and anyone else. Install libguestfs-tools libguestfs is a set of tools for accessing and modifying … Read more