Addshore

It's a blog

Page 2 of 14

Github repo settings sync, using the Github cli

The number of Github repositories that I end up maintaining in one way or another ends up growing week by week. And keeping all of the descriptions and settings up to date in sync can be painful todo by hand.

A little while ago I migrated my addwiki project to use a monorepo, and thus needed to bulk update all of the github repository descriptions. While doing so I made use of the github cli and created a single bash script to let me configure all of the repositories at once.

Assuming you already have the github cli install and configured getting started with this is easy.

The command

The below command is one of many in my bash script for repo configuration. This sets a description, homepage and various other flags that I want to be consistent across repositories.

gh api --method PATCH repos/addwiki/addwiki \ --field description='Monorepo containing all addwiki libraries, packages and applications'\ --field homepage='https://addwiki.github.io/'\ --field has_issues='true'\ --field has_projects='false'\ --field has_wiki='false'
Code language: JavaScript (javascript)
Continue reading

Resizing a qemu image root disk partition

Recently I found myself altering some virtual images for loading onto a qemu machine. I wanted to increase the disk space on the root partition, but couldn’t find any straightforward guides. So here is a little guide for future me, and anyone else.

Install libguestfs-tools

libguestfs is a set of tools for accessing and modifying virtual machine (VM) disk images. You can find details in the docs.

apt-get install libguestfs-tools
Code language: JavaScript (javascript)

Create a new resized image

First you need to create a new empty image of the size that you want.

For me that is 20GB

truncate -s 20G ./out.img

Then use virt-resize to expand the existing disk to fill all of the space in the new image that we created. (virt-resize docs)

virt-resize --expand /dev/sda1 ./vm.img ./out.img

Verify it worked

libguestfs-tools also provides a way to view file system information by only using the image file. (virt-filesystems docs)

virt-filesystems --long --parts --blkdevs -h -a ./out.img

You’ll see something like this:

Name Type MBR Size Parent /dev/sda1 partition 83 20G /dev/sda /dev/sda device - 20G -
Code language: PHP (php)

WBStack setting changes, Federated properties, Wikidata entity mapping & more

During the first 3 months of 2021, some Wikimedia Deutschland engineers, from the Wikidata / Wikibase team, spent some time working on WBStack as part of an effort to explore the WBaaS (Wikibase as a service) topic during the year, as outlined by the development plan.

We want to make it easier for non-Wikimedia projects to set up Wikibase for the first time and to evaluate the viability of Wikibase as a Service.

Wikibase 2021 Development plan

This has lead to a few new Wikibase features being exposed through the WBStack dashboard for sites that run on the platform. These features are primarily features developed by the Wikibase team in 2020 and 2021. The work also brought some other quality of life improvements for the settings pages.

Here is a quick rundown of what’s new and improved.

Continue reading

WBStack Infrastructure

WBStack is a platform allowing shared scalable hosting of Wikibase and surrounding services.

A year ago I made an initial post covering the state of WBStack infrastructure. Since then some things have changed, and I have also had more time to create a clear diagram. So it is time for the 2021 edition.

WBStack currently runs on a single Google Cloud Kubernetes Engine cluster, now made up of 3 virtual machines, one e2-standard-2 and two e2-medium. This results in roughly 4 vCPUs and 12GB of allocatable memory.

Continue reading

Twitter bot powered by Github Actions (WikidataMeter)

Recently 2 new Twitter bots appeared in my feed, fullyjabbed & fullyjabbedUK, created by iamdanw and powered entirely by Github Actions (code).

I have been thinking about writing a Twitter bot for some time and decided to copy this pattern running a cron based Twitter bot on Github Actions, with an added bit of free persistence using jsonstorage.net.

This post if my quick walkthrough of my new bot, WikidataMeter, what it does and how it works. You can find the code version when writing this blog post here, and the current version here.

Continue reading

If you bake 2 lasagnes, and stack them, do you have 1 or 2 lasagnes?

This is the question that I asked a group of around 50 people this evening as part of a food for thought exercise. 37 replied before I decided to write this blog post, and here is a quick summary…

The question was “If you bake 2 lasagnes, and then put one on top of the other, do you have 1 or 2 lasagnes?”

The answers ranged dramatically, and the group was fairly divided, but slightly leaning toward an answer of 1, with ~24 saying that you have 1 lasagne, and ~10 saying 2 lasagnes. Some of the answers were a bit vague and hard to stick into one of these categories.

Word cloud of the responses
Continue reading

VSCode & PHP XDebug from Localhost, WSL2 and Docker containers

VSCode seems to be one of the up and coming IDEs over the last year. I personally switched from Jetbrains IDEs to VSCode fo most of my development work at some point in 2020.

Apparently up until now I have avoided running the PHP debugger Xdebug. I had to do a little search around to figure out the correct settings for my setup decided to write them down in this handy blog post.

Continue reading

dockerit v0.0.5 (Easier than docker run)

dockerit is a small CLI tool that I have been working on during the start of 2021. It’s intended to make running one off commands and CLI tooling easier in docker. Rather than having to write a long string of parameters for docker run, instead you can just use dockerit. This applies to both CLI usage, but also via bash aliases.

You can download a build binary of dockerit from the Github releases page.

Continue reading

Programmatically convert Github master branch to main

Back in 2020 Github said that the default settings for new repositories would change.

On Oct. 1, 2020, any new repositories you create will use main as the default branch, instead of master

Some blog post on Medium

Github provided some advice for renaming branches focused around how this can be done in the UI. But with the Github CLI tool that was also release at the end of 2020 you can programatically make this change too.

The Github CLI tool is called gh. The README for the tool has installation instructions for a variety of platforms. One of the commands that it enables is called api, which can be used to “Make an authenticated GitHub API request”.

The Github API obviously allows you to perform most actions that can be performed by the UI, this we can use the gh cli tool to rename the branches of repositories.

For example, for the addwiki/addwiki repository, we can rename the master to main branch using the following command:

gh api --method POST repos/addwiki/addwiki/branches/master/rename --field new_name='main'
Code language: JavaScript (javascript)

And you can use such a command in whatever loops etc you want to, to quickly rename a whole bunch of branches in a whole bunch of repositories.

You can read more on the API docs here and the api cli command here.

Testing WDQS Blazegraph data load performance

Toward the end of 2020 I spent some time blackbox testing data load times for WDQS and Blazegraph to try and find out which possible setting tweaks might make things faster.

I didn’t come to any major conclusions as part of this effort but will write up the approach and data nonetheless incase it is useful for others.

I expect the next step toward trying to make this go faster would be via some whitebox testing, consulting with some of the original developers or with people that have taken a deep dive into the code (which I started but didn’t complete).

Continue reading
« Older posts Newer posts »

© 2021 Addshore

Theme by Anders NorénUp ↑