Open and Linked Data

Dane WrightIT Strategy Manager

Brent Council

October 2010

Conservative Party conference 2010Francis Maude

Thousands of commercial and social entrepreneurs have been frustrated by their inability to obtain and reuse datasets. I'm sorry to say that some councils spend time and money deliberately making data unusable to anyone else.

Open Data

Open data is information that is freely available on the web to everyone without restrictions on its re-use

Data on the web becomes more useful when it can be connected (or linked) with other data

Linked data is a central component of the Semantic Web where information is understandable not just by human beings but also by computers – machine readable

A history of the Internet/Web1940s Vannevar Bush's “memex” vision for automated

information management

1960s McLuhan's “global village, concept of online hypertext

1970s TCP/IP protocols agreed, Arpanet > Internet

1980s Tim Berners-Lee develops WWW concept at CERN, first web servers on the Internet

1990s CERN makes WWW freely available, websites and web browsers developed, first UK council website in 1996

2000s Global public use of the Internet, search engines, Google,dot.com boom and bust, Web 2.0, social media, 1 trillion web pages in 2008

2010s Mobile devices, cloud computing, Semantic Web

The Semantic Web

“I have a dream for the Web [in which computers] become capable of analysing all the data on the Web – the content, links, and transactions between people and computers. A ‘Semantic Web’, which should make this possible, has yet to emerge, but when it does, the day-to-day mechanisms of trade, bureaucracy and our daily lives will be handled by machines talking to machines.” Tim Berners-Lee 1999 • Additional languages etc - RDF, OWL, SKOS, SPARQL• Ontologies – taxonomies with formal relationships• Linked open data – structured data in standard formats

Linked DataLinking web pages together to allow machine readability•Everything identified by URIs – Universal Resource Identifer

•URIs give information in standard formats eg XML/RDF•Include links to other related URIs

Tim Berners-Lee at TED 2009 – “raw data now”BBC World Cup 2010 website used linked data engine to build/manage the website and its navigation esd standards controlled lists are published as RDFTalis Connected Commons offer hosting for RDF/SPARQL LeGSB working on linked data ontology for £500 spend data

History of open data in government 1

1766 Freedom of the Press Act in Sweden

1966 Freedom of Information Act in US (Lyndon Johnson)

2000 Freedom of Information Act in UK (Tony Blair)

2004 Civic enthusiasts repurpose Hansard & US Congress data

2003 EU Reuse of Public Sector Information directive

2005 Google Maps and Google Earth open up mapping to the public

2006 Guardian Free our Data campaign begins

2006 EU Inspire directive for open spatial data

2007 Cabinet Office Power of Information Review

2009 Tim Berners-Lee “Raw data now” speech at TED

2009 Tim Berners-Lee appointed as expert advisor by Gordon Brown

2009 US federal and state open data stores – data.gov, datasf.org etc

History of open data in government 2 2010 Jan - data.gov.uk and data.london.gov.uk launched

2010 Apr - Ordnance Survey makes some mapping data open

2010 May - David Cameron letter re central & local data transparency

2010 May - Tim Berners Lee, Nigel Shadbolt & Tom Steinberg appointed to Cabinet Office Transparency Board

2010 Jun - Eric Pickles announces local government publication of £500+ spend/contracts by Jan 2011

2010 Oct - UK Open Government Licence released

2010 Oct - Ordnance Survey makes more mapping data open

2010 Oct - David Cameron & Francis Maude reiterate support for open data

2011 Jan - All local authorities in England publish spend and contract data

Drivers for open data in government

3 factors driving government open data -

Pressure from civil society enthusiasts to re-purpose government datasets and make them available for re-useSupport from government officersTop level mandate to publish open data

Plus the capacity – FoI legislation, availability of data etc

Open Data Study – research by Becky Hogge, May 2010

Licensing

• Crucial for re-use in machine readable form• Variety of licences in the past – Click-use, Creative

Commons etc• Now there is the new Open Government Licence (OGL)

which covers access, re-use and database rights • Still some issues with Ordnance Survey licensing around

derived data – details of new PSMA to be revealed soon• LAs can still charge for data where necessary• LG Group is working on a practitioners guide

£500 spend & contracts

May 2010 – Prime Minister letter on opening up dataJune – letter from Eric PicklesSept - spend guidance from Local Public Data PanelOct – Letter/video from Eric Pickles Oct – spend & salary guidance from LG GroupOct - Open Government Licence publishedOct – 35 councils with spend data published? – contracts guidance Jan 2011 – deadline for spend to be published monthly

Spend/contract/salary issues

• England only, not statutory• Council commitment to open data or just the minimum?• Personal data redaction – manual or automatic?• Format – CSV not PDF, maybe RDF as well? • Consistency across boroughs?• Classification schemes – BVACOP, ProClass• Publishing large volume of data files• FOI requests – increase or decrease? • Potential for fraud?

Examples

Openly LocalBrentLichfield – Jonathan Harrison (Pezholio)WarwickshireTraffordWalsallLG Group Transparency Guidance

What to do next

Feedback to LG Group Transparency ProgrammePublish more than just the minimumUse the recommended CSV formatsInclude classifications, links wherever possibleUse the Open Government LicenceEncourage OS to allow us to make GIS data openEncourage conversion to RDF and linked dataMake more data sets open – GIS, Govt returns …

HindsightFreedom of Information. Three harmless words. I look at those words as I write them, and feel like shaking my head till it drops off my shoulders. You idiot. You naive, foolish, irresponsible nincompoop. There is really no description of stupidity, no matter how vivid, that is adequate. I quake at the imbecility of it.Once I appreciated the full enormity of the blunder, I used to say - more than a little unfairly - to any civil servant who would listen: Where was Sir Humphrey when I needed him? We had legislated in the first throes of power. How could you, knowing what you know have allowed us to do such a thing so utterly undermining of sensible government?

Tony Blair - A Journey - September 2010

Open and Linked Data

Dane WrightBrent Council

dane.wright@brent.gov.uk@pluto9

Useful links are on brentwebdev.wordpress.com