Addshore

It's a blog

Tag: docker (page 1 of 2)

Changing the concept URI of an existing Wikibase with data

Many users of Wikibase find themselves in a position where they need to change the concept URI of an existing Wikibase for one or more reasons, such as a domain name update or desire to have https concept URIs instead of HTTP.

Below I walk through a minimal example of how this can be done using a small amount of data and the Wikibase Docker images. If you are not using the Docker images the steps should still work, but you do not need to worry about copying files into and out of containers or running commands inside containers.

Continue reading

Your own Wikidata Query Service, with no limits (part 1)

The Wikidata Query Service allows anyone to use SPARQL to query the continuously evolving data contained within the Wikidata project, currently standing at nearly 65 millions data items (concepts) and over 7000 properties, which translates to roughly 8.4 billion triples.

Screenshot of the Wikidata Query Service home page including the example query which returns all Cats on Wikidata.

You can find a great write up introducing SPARQL, Wikidata, the query service and what it can do here. But this post will assume that you already know all of that.

Guide

Here we will focus on creating a copy of the query service using data from one of the regular TTL data dumps and the query service docker image provided by the wikibase-docker git repo supported by WMDE.

Continue reading

Creating a Dockerfile for the Wikibase Registry

Currently the Wikibase Registry(setup post) is deployed using the shoehorning approach described in one of my earlier posts. After continued discussion on the Wikibase User Group Telegram chat about different setups and upgrade woes I have decided to convert the Wikibase Registry to use the prefered approach of a custom Dockerfile building a layer on top of one of the wikibase images.

I recently updated updated the Wikibase registry from Mediawiki version 1.30 to 1.31 and described the process in a recent post, so if you want to see what the current setup and docker-compose file looks like, head there.

As a summary the Wikibase Registry uses:

  • The wikibase/wikibase:1.31-bundle image from docker hub
  • Mediawiki extensions:
    • ConfirmEdit
    • Nuke
Continue reading

wikibase-docker, Mediawiki & Wikibase update

Today on the Wikibase Community User Group Telegram chat I noticed some people discussing issues with upgrading Mediawiki and Wikibase using the docker images provided for Wikibase.

As the wikibase-registry is currently only running Mediawiki 1.30 I should probably update it to 1.31, which is the next long term stable release.

This blog post was written as I performed the update and is yet to be proofread, so expect some typos. I hope it can help those that were chatting on Telegram today.

Starting state

Documentation

There is a small amount of documentation in the wikibase docker image README file that talks about upgrading, but this simply tells you to run update.php.

Update.php has its own documentation on mediawiki.org.
None of this helps you piece everything together for the docker world.

Installation

The installation creation process is documented in this blog post, and some customization regarding LocalSettings and extensions was covered here.
The current state of the docker-compose file can be seen below with private details redacted.

This docker-compose files is found in /root/wikibase-registry on the server hosting the installation. (Yes I know that’s a dumb place, but that’s not the point of this post)

Continue reading

A firewall is blocking sharing between Windows and the containers – Docker

I recently encountered this error while trying to run one of my docker setups.

I have encountered errors like this before and it has always ended up being related to docker and sharing my drives to the linux VM that actually runs my containers.

Checking the shared drives menu of the docker UI everything seemed to be fine.

However when removing the drive share and re sharing the drive I got an error message saying that there was a “Firewall detected” and that “A firewall is blocking file Sharing between Windows and the containers. See documentation for more info”.

The docs seem to suggest that my firewall rule had vanished somehow. I spent a few minutes digging around and tinkering with firewall rules before determining the rule itself was not the issue. The next suggestion was to “reinstall the File and Print sharing service on the Hyper-V virtual network card

Opening PowerShell as administrator and running the commands suggested on stackoverflow (switching the DockerNAT network) seemed to do the trick.

I still have no idea why this happened. Did I accidently set the DockerNAT network to be public? Could the Docker UI really not check the firewall rule itself and propose the fix?

Customizing Wikibase config in the docker-compose example

2019 Edit: If you want to use a Dockerfile and custom image take a look a here.

Just over a month ago I setup the Wikibase registry project on Wikimedia Cloud VPS using the docker-compose example provided by Wikibase docker images. The Wikibase registry is the first Wikibase install that I control that uses the Wikibase docker images, so I’ll be using it as an example showing how the docker images can be manipulated to configure MediaWiki, Wikibase, and load custom extensions and skins.

The example docker-compose file at the time of writing this post can be found at https://github.com/wmde/wikibase-docker/blob/5919016eac16c5f0aefc448240fdf6a09bb56bec/docker-compose.yml

Since the last blog post new wikibase image tags have been created (the ‘bundle’ tags) that include some extensions you might want to enable, as well a quickstatements image for the quickstatements service used on Wikidata written by Magnus Manske.

Continue reading

Wikibase of Wikibases

The Wikibase registry was one of the outcomes of the first in a series of Federated wikibase workshops organised in partnership with the European research council.

The aim of the registry is to act as a central point for details of public Wikibase installs hosted around the web. Data held about the installs currently includes the URL for the home page, Query frontend URL and SPARQL API endpoint URL (if a query service exists).

During the workshop an initial data set was added, and this can be easily seen using the timeline view of the query service and a query that is explained within this post.

Continue reading

WMDE: CI, Deploys & Config changes intro

This is an internal WMDE presentation made to introduce people to the land of Wikimedia CI, MediaWiki deployment and config changes.

This briefly covers:

  • Jenkins & Zuul
  • CI Config
  • Beta Cluster
  • SWAT
  • The MediaWiki train
  • Monitoring
  • mediawiki-config
  • The Docker & Kubernetes future?

From 0 to Kubernetes cluster with Ingress on custom VMs

While working on a new Mediawiki project, and trying to setup a Kubernetes cluster on Wikimedia Cloud VPS to run it on, I hit a couple of snags. These were mainly to do with ingress into the cluster through a single static IP address and some sort of load balancer, which is usually provided by your cloud provider. I faffed around with various NodePort things, custom load balancer setups and ingress configurations before finally getting to a solution that worked for me using ingress and a traefik load balancer.

Below you’ll find my walk through, which works on Wikimedia Cloud VPS. Cloud VPS is an openstack powered public cloud solution. The walkthrough should also work for any other VPS host or a bare metal setup with few or no alterations.

Continue reading

Cambridge Analytica, #DeleteFacebook, and adding EXIF data back to your photos

2019 Update: This script now exists with an easy to use GUI

Back in 2016 I wrote a short hacky script for taking HTML from facebook data downloads and adding any data possible back to the image files that also came with the download. I created this as I wanted to grab all of my photos from Facebook and be able to upload them to Google Photos and have Google automatically slot them into the correct place in the timeline. Recent news articles about Cambridge Analytica and harvesting of Facebook data have lead to many people deciding the leave the platform, so I decided to check back with my previous script and see if it still worked, and make it a little easier to use.

Continue reading

« Older posts

© 2019 Addshore

Theme by Anders NorénUp ↑