Grafana, Graphite and maxDataPoints confusion for totals
The title is a little wordy, but I hope you get the gist. I just spent 10 minutes staring at some data on a Grafana dashboard, comparing it with some other data, and finding the numbers didn’t add up. Here is the story in case it catches you out. The dashboard The dashboard in question…
Using Hue & Hive to quickly determine Wikidata API maxlag usage
Hue, or Hadoop User Experience is described by its documentation pages as “a Web application that enables you to easily interact with an Hadoop cluster”. The Wikimedia Foundation has a Hue frontend for their Hadoop cluster, which contains various datasets including web requests, API usage and the MediaWiki edit history for all hosted sites. The install…
Python3, Using some shared state in 2 async methods
The asyncio module was added to Python in version 3.4 to “provides infrastructure for writing single-threaded concurrent code using coroutines, multiplexing I/O access over sockets and other resources, running network clients and servers, and other related primitives”. If you want an introduction to asyncio take a look at this blog post. This post will look…
The Wikimedia Server Admin Logs
The Wikimedia Server Admin Log or SAL for short is a timestamped log of actions performed on the Wikimedia cluster by users such as roots and deployers. The log is stored on the WikiTech Wikimedia project and can be found at the following URL: https://wikitech.wikimedia.org/wiki/Server_Admin_Log An example entry in the log could be: 09:04 addshore: addshore@terbium:~$…
Customizing Wikibase config in the docker-compose example
2019 Edit: If you want to use a Dockerfile and custom image take a look a here. Just over a month ago I setup the Wikibase registry project on Wikimedia Cloud VPS using the docker-compose example provided by Wikibase docker images. The Wikibase registry is the first Wikibase install that I control that uses the…
Wikibase of Wikibases
The Wikibase registry was one of the outcomes of the first in a series of Federated wikibase workshops organised in partnership with the European research council. The aim of the registry is to act as a central point for details of public Wikibase installs hosted around the web. Data held about the installs currently includes…
WMDE: CI, Deploys & Config changes intro
This is an internal WMDE presentation made to introduce people to the land of Wikimedia CI, MediaWiki deployment and config changes. This briefly covers: Jenkins & Zuul CI Config Beta Cluster SWAT The MediaWiki train Monitoring mediawiki-config The Docker & Kubernetes future?
Wikidata Map March 2018
It’s time for the first 2018 installation of the Wikidata Map. It has been roughly 4 months since the last post, which compared July 2017 to November 2017. Here we will compare November 2017 to March 2018. For anyone new to this series of posts you can check back at the progression of these maps…
From 0 to Kubernetes cluster with Ingress on custom VMs
While working on a new Mediawiki project, and trying to setup a Kubernetes cluster on Wikimedia Cloud VPS to run it on, I hit a couple of snags. These were mainly to do with ingress into the cluster through a single static IP address and some sort of load balancer, which is usually provided by…
gitgraph.js and codepen.io for git visualization
I was looking for a new tool for easily visualizing git branches and workflows to try and visually show how Gerrit works (in terms of git basics) to clear up some confusions. I spent a short while reading stackoverflow, although most of the suggestions weren’t really any good as I didn’t want to visualize a…