Open Sourcing WBStack

Open Sourcing the code and config for WBStack has always been part of the plan, although functionality came first throughout the first year or so. Finally there is a github organization for wbstack containing 16 public repositories that make up the entire deployment for wbstack.com.

This effort took a few weeks trying to split sensible components out of the original mono repo that was started back in 2017 that now has over 1600 commits, making sure that no secrets were swept up along the way, and also trying to preserve git history where possible.

Although everything is now on Github that doesn’t mean that it is clearly understandable just yet, or in the most sensible layout, that will come with time.

The Repositories

• docs – A new VuePress docs site, currently on Github pages.
• deploy – All deployment code and config (minus secrets) for Google Cloud, and Kubernetes for wbstack.com
• ui – The main VueJS based frontend for wbstack.com
• api – The main platform API in PHP using Laravel. Currently acting as a bit of a kitchen sink for prototyping
• mediawiki – A full version of MediaWiki, skins, extensions and config logic for all wbstack.com wikis
• queryservice – Tiny modifications to the regular query service
• queryservice-updater – Java wrapper around the regular query service updater to make it better for running on multiple sites
• queryservice-gateway – Custom node application to route different SPARQL queries to different backend hosts and namespaces
• queryservice-ui – Slightly modified queryservice frontend, mainly styles and config
• mediawiki-extensions-WikibaseInWikitext – A custom MediaWiki extension for a <sparql> tag in wikitext
• quickstatements – Fork of quickstatments with style and config modifications
• widar – Fork of widar with style and config modifications
• cradle – Fork of cradle with style and config modifications
• magnustools – Fork of magnustools library with config modifications
• docker-build-util – Docker utility image used in some build steps
• bitnami-charts – Fork with some small modifications that I should upstream

Builds, Deployment & Reuse

All of the repositories that get deployed now have Github Actions built Docker images, as well as tagged images and a CHANGELOG. The images that are available on Github are the images that are used for wbstack.com. You can currently find a full list of the Docker images here.

I believe all of these images should be usable off the shelf if anyone wanted to try and deploy all of the components, the exception being the ui, which has the API url built into the image. Though any reuse of these components would likely want a custom UI anyway. Anything that seems to get in the way of reuse should be easily fixable.

Upstreaming

Lots of the customizations in these repositories work around the fact that most components have some sort of hardcoded or simplistic way of determining configuration. This is normally a file that can be provided for a single site.

The modifications mostly turn this into a dynamic decision. For both frontend and backend application this means switching or configuring based on the request domain.

Almost all of the modifications that have been made for the wbstack.com use case can likely be upstreamed in one way or another, and I look forward to working toward that in 2021.

Series Navigation

<< WBStack 2020 Update 2 (May)WBStack Infrastructure >>