WBStack Infrastructure (2020)
UPDATE: You can find an up to date 2021 version of this post here.
WBStack currently runs on a Google Cloud Kubernetes cluster made up of 2 virtual machines, one e2-medium and one e2-standard-2. This adds up to a current total of 4 vCPUs and 12GB of memory. No Google specific services make up any part of the core platform at this stage meaning WBStack can run wherever there is a Kubernetes cluster with little to no modification.
A simplified overview of the internals can be seen in the diagram below where blue represents the Google provided services, with green representing everything running within the kubernetes cluster.
Other utility services exist around the core platform both running on kubernetes and as Google services. These includes:
- Snapshots, Backups and storage
- cert-manager for certificates
- Docker image build system
- Log and metric system
And some of the platform services are made up of multiple parts, such as:
- Different MediaWiki services for UI, API and job / backend requests
- Different Platform APIs for user or services sourced requests
- Other Platform elements such as a job queue and scheduler.
- Replication of the storage layer (redis, mysql, blazegraph)
Right now many of the Google priced variables are still within, or mostly provided by the free tier meaning the cost per month is essentially the cost of the CPU, RAM and Load Balancer.
Moving forward it would be nice to move this to a location more supportive of the WBStack project to enable continued cost effective growth and development at this early stage.
Hi Addshore-
I am interested in your implementation as I am doing something similar and have wikibase installed on AWS EKS. I had a question about whether or not you had made your instance to use TLS. I am planning to install a cert on the load balancer that fronts the whole platform but was wondering what issues you may have encountered and what configs might need to be set specifically.
A cert on the load balancer is similar to what wbstack currently has.
More specifically cert-manager is being used with an nginx ingress with certificates coming from Let’s Encrypt.
No other special config really, other than loading different config from the rest of the platform for each requested wiki, in generic terms, something like https://www.mediawiki.org/wiki/Manual:Wiki_family.
You say wbstack already has TLS. This confuses me. As I look at the apache2 configs for wikibase and the default-ssl.conf is not enabled. If it was it doesn’t look like it would work as the pem and key referenced to in the conf file do not exist.
I guess you mean the defsult-ssl.conf in the base wikibase docker image?
If so, wbstack uses a custom docker image (not currently published anywhere).
Also, the TLS termination is happening in the nginx-ingress which sits between the load balancer and any of the internal services such as wikibase.
Yes, thats what I was referring to in the docker image. What is the difference in WBstack and https://github.com/wmde/wikibase-docker ?
For the implementation I was considering I would indeed terminate TLS at the K8s ingress.
WBStack uses its own custom docker image which includes some custom code for loading the relevant settings per wiki and also a custom set of extensions.
The WBStack image is currently made up of 3 layers 1) base MediaWiki 2) MediaWiki + Extensions & Skin code 3) WBStack configs and extra code.
Thanks. I went back and reviewed your older posts and the demo video to get a better idea of what WBstack is.
[…] For previous update posts see the 2019 October Introduction, November Review and January 2020 Infrastructure Overview. […]
[…] WBStack Infrastructure, by […]
[…] year ago I made an initial post covering the state of WBStack infrastructure. Since then some things have changed, and I have also […]