It's a blog

Category: Tech (Page 1 of 13)

WBStack setting changes, Federated properties, Wikidata entity mapping & more

During the first 3 months of 2021, some Wikimedia Deutschland engineers, from the Wikidata / Wikibase team, spent some time working on WBStack as part of an effort to explore the WBaaS (Wikibase as a service) topic during the year, as outlined by the development plan.

We want to make it easier for non-Wikimedia projects to set up Wikibase for the first time and to evaluate the viability of Wikibase as a Service.

Wikibase 2021 Development plan

This has lead to a few new Wikibase features being exposed through the WBStack dashboard for sites that run on the platform. These features are primarily features developed by the Wikibase team in 2020 and 2021. The work also brought some other quality of life improvements for the settings pages.

Here is a quick rundown of what’s new and improved.

Continue reading

WBStack Infrastructure

WBStack is a platform allowing shared scalable hosting of Wikibase and surrounding services.

A year ago I made an initial post covering the state of WBStack infrastructure. Since then some things have changed, and I have also had more time to create a clear diagram. So it is time for the 2021 edition.

WBStack currently runs on a single Google Cloud Kubernetes Engine cluster, now made up of 3 virtual machines, one e2-standard-2 and two e2-medium. This results in roughly 4 vCPUs and 12GB of allocatable memory.

Continue reading

Twitter bot powered by Github Actions (WikidataMeter)

Recently 2 new Twitter bots appeared in my feed, fullyjabbed & fullyjabbedUK, created by iamdanw and powered entirely by Github Actions (code).

I have been thinking about writing a Twitter bot for some time and decided to copy this pattern running a cron based Twitter bot on Github Actions, with an added bit of free persistence using jsonstorage.net.

This post if my quick walkthrough of my new bot, WikidataMeter, what it does and how it works. You can find the code version when writing this blog post here, and the current version here.

Continue reading

Programmatically convert Github master branch to main

Back in 2020 Github said that the default settings for new repositories would change.

On Oct. 1, 2020, any new repositories you create will use main as the default branch, instead of master

Some blog post on Medium

Github provided some advice for renaming branches focused around how this can be done in the UI. But with the Github CLI tool that was also release at the end of 2020 you can programatically make this change too.

The Github CLI tool is called gh. The README for the tool has installation instructions for a variety of platforms. One of the commands that it enables is called api, which can be used to “Make an authenticated GitHub API request”.

The Github API obviously allows you to perform most actions that can be performed by the UI, this we can use the gh cli tool to rename the branches of repositories.

For example, for the addwiki/addwiki repository, we can rename the master to main branch using the following command:

gh api --method POST repos/addwiki/addwiki/branches/master/rename --field new_name='main'
Code language: JavaScript (javascript)

And you can use such a command in whatever loops etc you want to, to quickly rename a whole bunch of branches in a whole bunch of repositories.

You can read more on the API docs here and the api cli command here.

Testing WDQS Blazegraph data load performance

Toward the end of 2020 I spent some time blackbox testing data load times for WDQS and Blazegraph to try and find out which possible setting tweaks might make things faster.

I didn’t come to any major conclusions as part of this effort but will write up the approach and data nonetheless incase it is useful for others.

I expect the next step toward trying to make this go faster would be via some whitebox testing, consulting with some of the original developers or with people that have taken a deep dive into the code (which I started but didn’t complete).

Continue reading

PHP 7 composer docker image digest

At some point in the last weeks the composer docker image that can be found on docker hub switched from PHP7 to PHP8. Some projects, such as wikibase-docker, still require PHP7 for installation.

It looks like the composer image is not going to be providing an additional tag using a pre PHP 8 version, thus an old digest will be needed.

tldr; here is the digest that you want for composer with PHP7 (tag-details)

docker pull composer@sha256:d374b2e1f715621e9d9929575d6b35b11cf4a6dc237d4a08f2e6d1611f534675
Continue reading

Go Docker SDK, Raw Terminal Ctrl+C handling

I spent my weekend in working on a new project called dockerit. It’s a simple wrapper around Docker written in Go and making use of the Docker SDK.

One of the biggest sticking points for me, being fairly new with the Golang world, was trying to pass stdin stdout and stderr between the container and host terminal correctly, while also having good performance and doing the expected things (like Ctrl+C to cancel).

The full code for setting up and, interacting with and removing my container can be found here. The main steps are broken down below.

Continue reading

Simple Go defer code example

In Go, a defer statement will execute a function call just before the function it is called from returns.

I found that most of the examples of a Go defer call online seemed to do complicated things with numbers. So here is a nice simple example with just text output.

Example

You can run this code yourself in The Go Playground! You can also find a similar example in the Go Tour.

package main import ( "fmt" ) func main() { // This will output first (in order) fmt.Println("One") // This will output last (before this function finishes) defer fmt.Println("Two") // This will be called before the above defer call "Two" secondary() } func secondary() { // This will output after Four (before this function finishes) defer fmt.Println("Three") // This will output (in order) fmt.Println("Four") }
Code language: JavaScript (javascript)
Continue reading

My Git Aliases

Overtime key presses really add up, especially when you use certain tools all throughout the day.

Here are the bash and git aliases that I use for git to avoid doing some of these keypresses.

I’m not sure how many times a day I use these shortcuts, or how many key presses I skip, but I’m fairly sure my keyboard is thanking me.

Continue reading
« Older posts

© 2021 Addshore

Theme by Anders NorénUp ↑