addshore

Altering a Gerrit change (git workflow)

by

I don’t use git-review for Gerrit interactions. This is primarily because back in 2012/2013 I couldn’t get git-review installed, and someone presented me with an alternative that worked. Years later I realized that this was actually the documented way of pushing changes to Gerrit.

As a little introduction to what this workflow looks, and a comparison with git-review I have created 2 overview posts altering a gerrit change on the Wikimedia gerrit install. I’m not trying to convince you, either way, is better, merely show the similarities/difference and what is happening behind the scenes.

Be sure to take a look at the other post “Altering a Gerrit change (git-review workflow)“

I’ll be taking a change from the middle of last year, rebasing it, making a change, and pushing it back for review. Fundamentally the 2 approaches do the same thing, just one (git-review) requires an external tool.

Read more

Small commits

by

There are many blog posts and articles out there about making small git commits. I’m sure most people (including me) bring up the same few topics around why small commits are good and why we should all probably be making smaller commits.

In this post, I’ll look at some of the key topics from my perspective, and try to tie these topics to concrete examples from repositories that I have worked on. The topics are in no particular order, so be sure to give them all a read.

One thing to note is that “small” doesn’t necessarily mean small in terms of lines of code. Small here is also relative. Also, small commits can benefit you in many different places, but to stand the test of time they must end up in your main branch.

Git features during development

Git only takes full responsibility for your data when you commit

Commit Often, Perfect Later, Publish Once: Git Best Practices

Read more

Wikibase a history

by

I have had the pleasure of being part of the Wikibase journey one way or another since 2013 when I first joined Wikimedia Germany to work on Wikidata. That long-running relation to the project should put me in a fairly good position to give a high-level overview of the history, from both a technical and higher-level perspective. So here it goes.

For those that don’t know Wikibase is code that powers wikidata.org, and a growing number of other sites. If you want to know more read about it on Wikipedia, or the Wikibase website.

For this reason, a lot of the early timeline is quite heavy on the Wikidata side. There are certainly some key points missing, if you think they are worthy of mentioning then leave a comment or reach out!

Read more

Profiling a Wikibase item creation on test.wikidata.org

by

Today I was in a Wikibase Stakeholder group call, and one of the discussions was around Wikibase importing speed, data loading, and the APIs. My previous blog post covering what happens when you make a new Wikibase item was raised, and we also got onto the topic of profiling.

So here comes another post looking at some of the internals of Wikibase, through the lens of profiling on test.wikidata.org.

The tools used to write this blog post for Wikimedia infrastructure are both open source, and also public. You can do similar profiling on both your own Wikibase, or for your requests that you suspect are slow on Wikimedia sites such as Wikidata.

Wikimedia Profiling

Profiling of Wikimedia sites is managed and maintained by the Wikimedia performance team. They have a blog, and one of the most recent posts was actually covering profiling PHP at scale in production, so if you want to know the details of how this is achieved give it a read.

Throughout this post I will be looking at data collected from a production Wikimedia request, by setting the X-Wikimedia-Debug header in my request. This header has a few options, and you can find the docs on wikitech.wikimedia.org. There are also browser extensions available to easily set this header on your requests.

I will be using the Wikimedia hosted XHGui to visualize the profile data. Wikimedia specific documentation for this interface also exists on wikitech.wikimedia.org. This interface contains a random set of profiled requests, as well as any requests that were specifically requested to be profiled.

Profiling PHP & MediaWiki

If you want to profile your own MediaWiki or Wikibase install, or PHP in general, then you should take a look at the mediawiki.org documentation page for this. You’ll likely want to use either Tideways or XDebug, but probably want to avoid having to setup any extra UI to visualize the data.

This profiling only covered the main PHP application (MediaWiki & Wikibase extension). Other services such as the query service would require separate profiling.

Read more

WBStack close and migration

by
This entry is part 11 of 12 in the series WBStack

The time is approaching for the end of life of the WBStack alpha platform (don’t worry, it’s still some months away, and there is a migration path to a new platform etc :)).

In this post you’ll find an update on the current state of WBStack, another introduction to Wikibase.Cloud, some rough dates and connections to other communications. If you don’t know what WBStack is then you can start with this introduction.

Wikibase.Cloud

Following the pre launch announcement of Wikibase.Cloud at WikidataCon 2021, the WMDE team has been working on getting the new platform set up and ready to replace wbstack.com. This includes updates to components such as MediaWiki, Wikibase, reworking components, modifying code bases to be more easily maintained by a team, and generally getting to grips with the platform.

This new platform uses the same codebases, and architecture as wbstack.com does currently, but it is maintained by a team at Wikimedia Deutschland, rather than me, an individual.

You can read more on the launch from the WMDE perspective in the mailing list post that will be sent at the same time as this blog post.

Initially, Wikibase.cloud will launch as a closed beta for WBStack users who registered before February 2nd, 2022 with a waiting list for later expansion. To join the waiting list, please fill out this form https://lime.wikimedia.de/index.php/717538. Please note that the waitlist will be considered after WBStack migration has completed.

For current users of WBStack, you do not need to join this waitlist. You will receive an email with details on how you can opt-in to a migration to wikibase.cloud in March.

wikibase-cloud mailing list February 2022

As migration time approaches, I will be reaching out to the current users of WBStack about the options and approach to migration. And if you are a current user, it’s worth reading the rest of this post.

Read more

cobra: unable to redefine ‘h’ shorthand

by

“Cobra is both a library for creating powerful modern CLI applications as well as a program to generate applications and command files”

I’m currently using Cobra in the MediaWiki CLI project and recently came across an error while trying to auto-generate docs.

panic: unable to redefine 'h' shorthand in "login" flagset: it's already used for "hostname" flag

goroutine 1 [running]:
github.com/spf13/pflag.(*FlagSet).AddFlag(0xc0000b5d00, 0xc00053d680)
        /go/pkg/mod/github.com/spf13/pflag@v1.0.5/flag.go:874 +0x6f0Code language: PHP (php)

I’m not actually trying to redefine this flag, but it looks like by default cobra always uses the -h flag for a help command, if you don’t define a help flag that doesn’t have the shorthand.

Read more

Wikidata ontological tree of Trains

by

While looking working on my recent WikiCrowd project I ended up looking at the ontological tree of both Wikidata entities and Wikimedia Commons categories.

In this post, I’ll look at some of the ontology mappings that happen between projects, some of the SPARQL that can help you use this ontology in tools, and also some tools to help you explore this complex tree.

I’m using trains as I think they are fairly easy for most folks to relate to, and also don’t have a massively complex tree.

Commons & Wikidata mapping

Depicts questions in WikiCrowd are entirely generated from these Wikimedia Commons categories, such as Category:Trains & Category:Steam locomotives. These are then mapped to items on Wikidata such as Q870 (train) & Q171043 (steam locomotive).

Wikimedia Commons categories quite often contain infoboxes on the right-hand side that link to a variety of resources for the thing the category is covering. And quite often there is a Wikidata item ID present, this is the case for the categories above.

Likewise on Wikidata statements for P373 (Commons category) will often exist for entities that are depicted on Commons.

Read more

Pre-launch Announcement of Wikibase.Cloud [WikidataCon Writeup]

by
This entry is part 10 of 12 in the series WBStack

WikidataCon 2021 was in October 2021, and one of the sessions that I spoke in was a “Pre-launch Announcement and Preview of Wikibase.Cloud”.

The recording is now up on YouTube, and below you’ll find a write-up summary of what was said.

You can also find:

So what is wikibase.cloud?

It’s a new platform that has yet to be launched, that is based on WBStack code, but that will be managed and maintained by Wikimedia Deutschland (or Wikimedia Germany).

This is a Wikibase as a service platform, that exists to offer open knowledge projects a new way to create their own Wikibase very quickly and very easily.

Read more

A first look at WikiCrowd

by

I have quite enjoyed the odd contribution to an app by Google called Crowdsource. You can find it either on the web, or also as an app.

Crowdsource allows people to throw data at Google in controlled ways to add to the massive pile of data that Google uses to improve its services and at the end of the day beat its competition.

It does this by providing a collection of micro contribution tasks in a marginally gamified way, similar to how Google Maps contributions get you Local Guide points etc. In Crowdsource you get a contribution count, a level, and a metric for agreements.

While I enjoy making the odd contribution when bored out of my mind and enjoy looking at the new challenges (currently at 2625 contributions), I always think that data like this should just be going out into the world under a free licence to benefit everyone.

So finally, introducing WikiCrowd, an interface, and soon to be app, that I developed over the new year period.

WikiCrowd Overview

WikiCrowd is hosted on toolforge and can be found at https://wikicrowd.toolforge.org/ (Source code on Github)

In order to contribute, you need some knowledge of the world, a Wikimedia account and that’s it!

Screenshot showing the wikicrowd application, listing various groups of questions users can contribute to

Read more

Pushing patches to sr.ht git repos

by

I recently wrote a blog post using the taavi/laravel-socialite-mediawiki PHP library which is hosted on sr.ht.

sr.ht, or sourcehut is “is a collection of tools useful for software development”, including git repository hosting.

I had to make a change to the library, and this was my first time contributing to a sr.ht repository. The workflow was quite different from what I am used to and required quite some Googling before I came up with something that I was happy with. So here goes my workflow.

Workflow

Firstly, if you haven’t used sr.ht before you’ll need to create an account and set up some way of pushing code (SSH keys etc). There are some fairly good docs for this already for Setting up your account and first git repository, but you can stop at Creating a git repository.

Once you have an account set up you need to find the sr.ht repository that you want to work with. In my case that is at the URL https://git.sr.ht/~taavi/laravel-socialite-mediawiki

You can find a clone URL over on the right-hand side of the repo page, and you want to go ahead and clone the repo.

git clone https://git.sr.ht/~taavi/laravel-socialite-mediawikiCode language: PHP (php)

You then also want to fork the repo into your own sr.ht account. The button is also on the right.

(You can also use your local clone to send a git email, but I didn’t want to install this tooling)

Screenshot of a sr.ht repository with the clone information highlighted

Read more