Linked Data and the World Cup

In January of 2010, I attended the News Linked Data Summit. This was a collection of several organizations involved in news production and distribution, looking to see if there was a way to collaborate on moving forward the Semantic Web (and particularly Linked Data) for news. It was a very interesting discussion, lead mainly by the BBC and The Guardian. At the end of the day, the group decided to move forward together by producing Linked Data for the UK Election. (In January, the election was clearly going to happen in the near future, but not yet announced - it wound up happening in May 2010).

I had a counter proposal for a Linked Data experiment - the World Cup. This made more sense for my employer and seemed to be of interest to at least some others. It was not to be, however. (I later spoke to some folks about whatever happened with the UK Election Linked Data experiment. As far as I can tell, the Guardian did produce some election data. But it isn't clear to me that this turned into anything bigger).

By Shine 2010 http://www.flickr.com/photos/shine2010/3292491879/

However, it turns out that the BBC did use the Semantic Web to power their World Cup “microsite” (actually their World Cup site has more pages than the non-SemWeb Sports section). In "The World Cup and a call to action Around Linked Data", BBC Architect John O'Donovan gives an overview of how they used fine-grained metadata to be able to produce their site with far less need for editorial curation of individual pages. In an accompanying piece, Jem Rayfield describes the technical details of the "dynamic semantic publishing".

Although their descriptions are couched in the terms of the Semantic Web, much of what they describe is more to do with the application of fine-grained metadata than with the particular data formats they use. They do describe how they make use of certain Semantic Web technologies - such as an RDF Triplesore / SPARQL system - to apply derived properties. And it is almost certainly the case that the use of the RDF model has given them more flexibility than can be the case when you use RDBMS systems - or even XML-based schema. However, fundamentally, what they are describing is the potential for what could be done via the application of metadata to content. It is interesting to see it in action.

The England Team didn't do so well at the World Cup - thanks Doug88888! http://www.flickr.com/photos/doug88888/4550561194/

It is also interesting how much of a revelation this is to people. (The first article was widely distributed via Twitter. And the comments are quite breathless in their admiration).

Hopefully, the work that the IPTC is doing on Linked Data and the Semantic Web will help other news organizations (including the AP!) unlock some of the great metadata work that is going on behind the scenes...

Towards NITF 4.0 - Experimental Support for "Foreign" Namespaces

One of the critiques that has been leveled against NITF for many years is that it cannot be customized by including "other" XML namespaces [1]. In NITF 3.5, we completed the step taken towards opening up the NITF 3.4 schema by fixing a bug in namespace support in enriched text [2].

However, we decided that full support for foreign namespaces was such a big change, that this would constitute one of the major pieces of work for NITF 4.0 [3]. I've created an *experimental* NITF XSD with foreign namespace support. It can be downloaded from

http://groups.yahoo.com/group/nitf/files/schema/

I've performed a number of tests with this schema and it seems to me to be going in the right direction. I've also discovered a problem with the NITF 3.5 namespace support, which I've fixed in this experimental XSD [4]. I promise to write a future note about the choices I made in adding foreign namespace support. However, I wanted to get the current version out there, to give people a chance to download and try it out [5].

--
[1] See for example, the excellent discussion of schemas and extensibility by Bob DuCharme and how NITF is too closed http://snee.com/xml/xml2005/industryschemas.html#d50e406

[2] Get the NITF schema at http://www.iptc.org/std/NITF/3.5/specification/

[3] Discussion of the plans for NITF 4.0 (amongst other things) can be reviewed in http://www.slideshare.net/smyles/nitf-2010-spring-working-group

[4] A special prize for the first person to figure out what the bug was

[5] Note that the NITF 4.0 experimental schema contains the documentation I copied over from the NITF 3.5 DTD. I'm still keen to get feedback on this too!

SKOS and Protoge HOWTO

In the IPTC, we are doing some work to figure out how to represent the IPTC Controlled Vocabularies as Linked Data.  We've decided to use SKOS as the RDF Vocabulary.  One of the things we wanted to do was to use a tool that "understands" SKOS.  We decided to look at Protoge for this.  Here are the steps we figured out to make it work (for some values of work):

Download version 4.0.2 from http://protege.stanford.edu/download/registered.html
-          Install it in your PC
-          Add the SKOSed plugin (use Check for plugins... item under File)
-          Add the Pellet Reasoner plugin
-          (you have to restart Protege before the new plugins are active)
-          Add Views to the Individuals tab: SKOSed view -> Inferred Concept Hierarchy + SKOS Usage
-          Then you may load a SKOS vocabulary – but only with narrower and broader relationships, does not work with the ...Transitive variants.
-          Then you should run the Pellet Reasoner against this vocabulary
-          Only then you should see the hierarchy in the Inferred Concept Hierarchy frame.
(JPE later adds "I have declared the narrower and broader properties as "transitive" using protégé and it  works.")

Posting them here, to make it easier for me to find (and maybe to help others).

Recently, I was asked for pointers to introductory material on the Semantic Web, specifically for such topics as N3 and Turtle.  I found "The Semantic Web 1-2-3" the most helpful in starting to get to grips with the mysteries of the SemWeb.  Note, however, that it is somewhat outdated now (it refers to DAML+OIL for example).  But it is still a good foundation and it has more links to great semwebby material than you can shake a stick at, if that's your idea of fun.  A couple of extra links that might be of help are

• timbl's primer on N3
• the Rough Guide to N3
• a six page short (!) introduction to RDF, plus a longer thirty page introduction
• W3C's best practices for publishing RDF vocabularies
• the SKOS primer 

I can't really find anything good that explains Turtle, other than the formal spec.  But my twitter-length explanation is that Turtle is N3 minus the reasoning extensions but plus internationalization (i.e. it is a more exact rendition of RDF than N3 is).

Other great links out there?

Experimenting with Bull Fighting and the Semantic Web

Paul Kelly and I took a look at what it would take to represent NewsML-G2 style Knowledge Items as SKOS.


 Here's a typical G2 style concept from the IPTC subject code vocabulary:

<concept>
<conceptId qcode="subj:01003000" created="2008-01-01T00:00:00+00:00" />
<type qcode="cpnat:abstract">
<name xml:lang="en-GB">Abstract concept</name>
</type>
<name xml:lang="en-GB">bullfighting</name>
<name xml:lang="de">Stierkampf</name>
<name xml:lang="fr">Tauromachie</name>
<name xml:lang="es">toros</name>
<name xml:lang="es">??</name>
<definition xml:lang="en-GB">Classical contest pitting man against the bull</definition>
<definition xml:lang="de">Klassischer Wettkampf Mann gegen Stier.</definition>
<definition xml:lang="fr">Tauromachie, combat entre un homme et un taureau</definition>
<definition xml:lang="es">Clasico enfrentamiento entre Toro y Hombre.</definition>
<definition xml:lang="es">????????????????</definition>
<broader qcode="subj:01000000" type="cpnat:abstract" />
</concept>

Here's what I think that would render as in SKOS - first in N3:

@prefix dc: <http://purl.org/dc/elements/1.1/>.
@prefix skos: <http://www.w3.org/2004/02/skos/core#>.
@prefix subj: <http://iptcsubj.example.com/>.
@prefix rdf: <http://www.w3.org/1999/02/22-rdf-syntax-ns#>.

subj:01003000 rdf:type skos:Concept;
dc:created "2008-01-01T00:00:00+00:00";
skos:prefLabel "bullfighting"@en-GB;
skos:prefLabel "Stierkampf"@de;
skos:prefLabel "Tauromachie"@fr;
skos:prefLabel "toros"@es;
skos:prefLabel "闘牛"@es;
skos:definition "Classical contest pitting man against the bull"@en-GB;
skos:definition "Klassischer Wettkampf Mann gegen Stier."@de;
skos:definition "Tauromachie, combat entre un homme et un taureau"@fr;
skos:definition "Clasico enfrentamiento entre Toro y Hombre."@es;
skos:definition "人間を雄牛と戦わせる伝統的な競技"@es;
skos:broader subj:01000000.

Some conclusions. G2 concepts are easy to map to SKOS! But there are some choices that need to be made (e.g. do we use skos:broader or owl:broader?) It feels like a G2 conceptset is almost identical to a SKOS Concept Scheme.

And there's a bug in the IPTC mapping of the subject codes (the Japanese definition and name is labeled as Spanish).

¡Olé! by karwik

http://www.flickr.com/photos/karwik/4878855534/

Connecting oXygen and eXist

What could be more natural than connecting oXygen and eXist? After all, they both share that second upper case X.

In fact, I'm trying to teach myself how to perform XQueries using the eXist-db, the "open source native XML database". This is somewhat tricky to start. Although it is relatively easy to download eXist, it does require ensuring that you have a JDK installed on your Windows machine. But then I ran into the real problem: there is no "hello world" example that tells you completely step-by-step how to write your first XQUERY, upload it to eXist and then execute it via the web interface. Or, at least, I couldn't find anything.

I have, however, figured out a partial workaround: I have figured out how to use oXygen as a sort of front end to eXist, to allow me to upload, edit and run my XQueries. (I still haven't figured out how to execute the XQueries directly from the webserver. Perhaps one day...).

I followed this screencast which shows how to configure your oXygen editor to connect to eXist. Although I had to use the jars listed in this howto, rather than the ones shown in the screencast! Just in case either that howto or that screencast disappears, here's what I did:

-1. Download a JDK (a JRE isn't sufficient)
0. Download eXist
1. In oXygen, switch to the Database Perspective
2. Configure an eXist datasource:
- 2a. Click on "New" under the top panel labeled "Data Sources"
- 2b. Select "eXist" from the "Type" dropdown
- 2c. Give it a name (such as "Exist Datasource")
- 2d. Click "Add"
- 2e. Select the following JARs from within the exist library:

• exist/exist.jar
• exist/lib/core/xmldb.jar
• exist/lib/core/xmlrpc-client-3.1.3.jar
• exist/lib/core/xmlrpc-common-3.1.3.jar
• exist/lib/core/ws-commons-util-1.0.2.jar

- 2f. Click "Open" then "OK"
3. Create an Exist Connection
- 3a. Click "New" under the bottom panel, labeled "Connections"
- 3b. In the Data Source dropdown, select the one you created in step two (which I suggested you call "Exist Datasource")
- 3c. Give your connection a name (such as "Exist Connection")
- 3d. In the XML DB URI field, edit the placeholder to be the host address of your Exist installation (and modify anything else that differs, such as the port number)
- 3e. Replace the username and password fields with the correct values
- 3f. Hit "OK"
4. You should now have an exist connection which you can open by double clicking!

Help Haiti or Apalachia?

A friend of mine on Facebook opined

wonders hwo throwing money at Haiti will fix anything? How about we send that money and that Red Cross aid to the people in Appalachia? They're our own citizens starving and living in hovels. But no, we go to Haiti.....*looks around....steps off soapbox*

I responded:

although it seems as though a lot of people have died in haiti, the real risk is what happens next. if there is no significant help sent right away, then there will likely be more deaths due to shortages of food and water and from epidemics (cholera, typhoid, dysentry, hepatitis) due to lack of sanitation facilities. in 2004, in the aftermath of the indian ocean earthquake, "only" about 200,000 to 300,000 people died because the international response was relatively good. (the usa sent about $2.8bn - which works out to around $9.81 per person).

and it isn't a one way street. in the aftermath of katrina, countries all around the world offered to send aid to the usa in their hour of need.

finally, it isn't a case of either or, is it? most of us could probably afford to send $9.81 to haiti and $9.81 to help people in the slow motion crisis of apalachia.