It's a blog

Tag: wbstack

Delete all Github releases, using the Github cli

If you are like me, at some point while developing projects on Github you will have started and stopped using releases, opting to switch back to only creating tags.

Github defaults leave you in a bit of an awkward position here where the “Releases” section in the side bar will continue to show your old releases, even if you have made hundreds of tags since.

If you’re managing lots of repositories and have made lots of legacy releases, removing them can be a pain (particularly in the UI). So, once again, GitHub CLI to the rescue!

In the repository of your choice run the following, and all of your releases will be removed!

gh release list | awk '{print $1;}' | xargs -L1 gh release delete
Code language: JavaScript (javascript)

If you want to see them ahead of removing them, list them first.

gh release list
Code language: PHP (php)

Enjoy your time saving, and your nice clear releases list!

WBStack setting changes, Federated properties, Wikidata entity mapping & more

During the first 3 months of 2021, some Wikimedia Deutschland engineers, from the Wikidata / Wikibase team, spent some time working on WBStack as part of an effort to explore the WBaaS (Wikibase as a service) topic during the year, as outlined by the development plan.

We want to make it easier for non-Wikimedia projects to set up Wikibase for the first time and to evaluate the viability of Wikibase as a Service.

Wikibase 2021 Development plan

This has lead to a few new Wikibase features being exposed through the WBStack dashboard for sites that run on the platform. These features are primarily features developed by the Wikibase team in 2020 and 2021. The work also brought some other quality of life improvements for the settings pages.

Here is a quick rundown of what’s new and improved.

Continue reading

WBStack Infrastructure

WBStack is a platform allowing shared scalable hosting of Wikibase and surrounding services.

A year ago I made an initial post covering the state of WBStack infrastructure. Since then some things have changed, and I have also had more time to create a clear diagram. So it is time for the 2021 edition.

WBStack currently runs on a single Google Cloud Kubernetes Engine cluster, now made up of 3 virtual machines, one e2-standard-2 and two e2-medium. This results in roughly 4 vCPUs and 12GB of allocatable memory.

Continue reading

Open Sourcing WBStack

wbstack organization on Github

Open Sourcing the code and config for WBStack has always been part of the plan, although functionality came first throughout the first year or so. Finally there is a github organization for wbstack containing 16 public repositories that make up the entire deployment for wbstack.com.

This effort took a few weeks trying to split sensible components out of the original mono repo that was started back in 2017 that now has over 1600 commits, making sure that no secrets were swept up along the way, and also trying to preserve git history where possible.

Although everything is now on Github that doesn’t mean that it is clearly understandable just yet, or in the most sensible layout, that will come with time.

Continue reading

Reducing Java JVM memory usage in Containers and on Kubernetes

For a while I have been running a Wikibase query service update script for WBStack, which is a Java application on a Kubernetes cluster. Part of that journey has included the updater using all available memory, hitting into the kubernetes memory limit and being OOM killed. The title of the post is a little verbose, but I wanted to include all of the keywords that might help people find the answers to the memory issues that I was running into.

Before getting into my weeds, if you want the answers head straight to https://developers.redhat.com/blog/2017/04/04/openjdk-and-containers/

UPDATE: This blog post was written with Java 8. Java 10+ now automatically recognizes memory limits and enforces them. https://www.docker.com/blog/improved-docker-container-integration-with-java-10/

Continue reading

WBStack 2020 Update 2 (May)

WBStack is now in its 7th month with 76 user accounts who have created 226 MediaWiki sites running Wikibase, of which 145 are currently online (81 deleted sites). 295,000 edits have now been made in total, which is an increase of 95,000 in the last month, which roughly equates to 2 edits a minute for the month.

The most active site is currently UniTest which is “a Wikibase sandbox with information about the research ecosystem”. Second and third come School of Design and Hercules Demo.

Screenshot of the WESO UniTest Main Page, 17 May 2020
Continue reading

WBStack 2020 Update 1

WBStack has now been up and running for 6 months. During that time it has helped 70 people create 178 MediaWiki installs running Wikibase, a SPARQL query service and quickstatements, all at the click of a button, with a total of around 200,000 edits across all sites.

The most active site is currently virus-taxonomy.wiki.opencura.com which was developed during the Virtual Biohackathon on COVID-19 as a staging environment for “improving the taxonomy of viruses on Wikidata”. It currently stands at 20,000 edits, around 7000 Items.

Screenshot of the virus-taxonomy Wikibase Main Page, 19 April 2020

Thanks again to Rhizome, who run their very own Wikibase, for their support paying the Google Cloud bill in the early stages of this project.

Continue reading

WBStack Infrastructure (2020)

UPDATE: You can find an up to date 2021 version of this post here.

WBStack currently runs on a Google Cloud Kubernetes cluster made up of 2 virtual machines, one e2-medium and one e2-standard-2. This adds up to a current total of 4 vCPUs and 12GB of memory. No Google specific services make up any part of the core platform at this stage meaning WBStack can run wherever there is a Kubernetes cluster with little to no modification.

A simplified overview of the internals can be seen in the diagram below where blue represents the Google provided services, with green representing everything running within the kubernetes cluster.

Continue reading

WBStack – November review

It’s been roughly 1 month since WBStack appeared online, and it’s time for a quick review of what has been happening in the first month. If you don’t already know what WBStack is, then head to my introduction post.

The number of users and wikis has slowly been increasing. In my last post I stated ” 20 users on the project with 30 Wikibase installs”. 3 weeks after that post WBStack now sits at roughly 38 users with roughly 65 wikibases. Many of these wikibases are primarily users test wikis, but that’s great, the barrier to trying out Wikibase is definitely lowered.

If you would like an invite code to try WBStack, or have any related thoughts of ideas, then please get in touch.

What’s changed

As WBStack is a shared platform, all changes mentioned in this blog post are immediately visible on all hosted Wikibases. In the future there will be various options to turn things on and off, but at this early stage things are being kept simple.

Continue reading

An introduction to WBStack

WBStack is a project that I have been working on for a couple of years that finally saw the light of day at Wikidatacon 2019. It has gone through a couple of different names along the way, MWaas, WBaas, WikWiki, OpenCura and finally WBStack.

The idea behind the project is to provide Wikibase and surrounding services, such as a blazegraph query service, query service ui, quick statements, and others on a shared platform where installs, upgrades and maintenance are handeled centrally.

Continue reading

© 2021 Addshore

Theme by Anders NorénUp ↑