Tag: wikidata query service

A Wikidata query service JNL file for public use

Back in 2019 I wrote a blog post called Your own Wikidata Query Service, with no limits which documented loading a Wikidata TTL dump into your own Blazegraph instance running within Google cloud, a near 2 week process. I ended that post speculating that part 2 might be using a “pre-generated Blazegraph journal file to…

By addshore October 2, 2022 4

Wikidata query service updater evolution

Posts Tech

The Wikidata Query Service (WDQS) sits in front of Wikidata and provides access to query its data via a SPARQL API. The query service itself is built on top of Blazegraph, but in many regards is very similar to any other triple store that provides a SPARQL API. In the early days of the query…

By addshore April 12, 2022 2

Wikidata maxlag, via the ApiMaxLagInfo hook

Posts Tech

Wikidata tinkers with the concept of maxlag that has existed in MediaWiki for some years in order to slow automated editing at times of lag in various systems. Here you will find a little introduction to MediaWiki maxlag, and the ways that Wikidata hooks into the value, altering it for its needs. As you can…

By addshore March 4, 2022 0

Wikidata ontological tree of Trains

Posts Tech

While looking working on my recent WikiCrowd project I ended up looking at the ontological tree of both Wikidata entities and Wikimedia Commons categories. In this post, I’ll look at some of the ontology mappings that happen between projects, some of the SPARQL that can help you use this ontology in tools, and also some…

By addshore January 7, 2022 0

Testing WDQS Blazegraph data load performance

Posts Tech

Toward the end of 2020 I spent some time blackbox testing data load times for WDQS and Blazegraph to try and find out which possible setting tweaks might make things faster. I didn’t come to any major conclusions as part of this effort but will write up the approach and data nonetheless incase it is…

By addshore February 10, 2021 1

Faster munging for the Wikidata Query Service using Hadoop

Posts Tech

The Wikidata query service is a public SPARQL endpoint for querying all of the data contained within Wikidata. In a previous blog post I walked through how to set up a complete copy of this query service. One of the steps in this process is the munge step. This performs some pre-processing on the RDF…

By addshore October 14, 2020 3

Using OpenRefine with Wikidata for the first time

Posts Tech

I have long known about OpenRefine (previously Google Refine) which is a tool for working with data, manipulating and cleaning it. As of version 3.0 (May 2018), OpenRefine included a Wikidata extension, allowing for extra reconciliation and also editing of Wikidata directly (as far as I understand it). You can find some documentation on this…

By addshore July 7, 2020 2

Changing the concept URI of an existing Wikibase with data

Posts Tech

Many users of Wikibase find themselves in a position where they need to change the concept URI of an existing Wikibase for one or more reasons, such as a domain name update or desire to have https concept URIs instead of HTTP. Below I walk through a minimal example of how this can be done…

By addshore November 5, 2019 6

Your own Wikidata Query Service, with no limits

Posts Tech

The Wikidata Query Service allows anyone to use SPARQL to query the continuously evolving data contained within the Wikidata project, currently standing at nearly 65 millions data items (concepts) and over 7000 properties, which translates to roughly 8.4 billion triples. You can find a great write up introducing SPARQL, Wikidata, the query service and what…

By addshore October 23, 2019 15