openstreetmap - the quality issue

OpenStreetMapThe Quality Issue

Oliver O’Brien

CENTRE FOR ADVANCED SPATIAL ANALYSIS

Contents

• Quality & Completeness– Sources of Data

• Scans of Imagery for Tracing• Imports

– Studies, Tools and Methods– Comparison– Other Considerations

• OSM in the Real World


Quality & Completeness

• OpenStreetMap – a crowdsourced spatial database of the world.– How good is the data compared with real

life?– Is everything important on the ground in

the database?


GPS Traces

• In the beginning (~2004)– Tracks from GPS receivers

• Consumer-grade handheld devices• Prone to random and systematic error

– Particularly in urban areas (multipath)– Particularly if walking– Walk both ways down street, take the “average”?– In theory quality improves with multiple traces

• Metadata doesn’t include measured quality

– eCourier van GPS traces for London


GPS Traces


Out-of-Copyright Map Tracing

• Crown Copyright expires after 50 years• Not so useful for urban areas

– Great for rural areas – less likely to change – Old, low resolution map better than nothing

• Started with Richard Fairhurst’s collection in early 2006– Quality dependent on scan resolution and

quality of rectification– Paper warps, especially over 50 years


Out-of-Copyright Map Tracing


Yahoo Aerial Imagery Tracing

• Agreement in late 2006 that allowed tracing– In the UK, some large

urban areas covered– Accuracy better than GPS

but not as good as newer imagery sets (e.g. Google)

• Phantom roads• Hidden barriers



• Coverage of many countries– Iraq: “The whole road and rail network of

the Baghdad area has been mapped using the Yahoo! imagery. As have some parks, forested areas, etc.”

– Australia (Sydney): “Completion of the greater metropolitan railway network… several complex road interchanges not practical to map on the bike”


NaPTAN

• Public transport nodes– Once-off copy of DtT database in 2009– Most useful for “less glamorous” bus stops

as railway station coverage was already complete in OpenStreetMap

– Rich metadata– Envisaged as two-way, OSM would fix poor

locations and might be fed back– Imported on a county-by-county basis


NaPTAN

• NOVAM-Viewer– Bus stops– For reconciling

NaPTAN/OSM– Graphical tool– Manage manual

verification of the locations and metadata


Walking Papers

• Designed for large scale POI addition• The source is annotations to existing

OpenStreetMap data so ultimately dependent on the other sources

• Specially rendered to print out and add Points of Interest to

• Automatic georeferencing for tracing– If you want it


Walking Papers


Imports

• A common theme so far – tracing– Inherently poor quality due to user error

• Tiger (U.S.)• AND (Netherlands)• OS Open Data


Tiger (U.S.)

• From US Government• Imported in late 2007 (data from 2005)

– Data quality low but good coverage– More difficult to create a community to fix

rather than map• Specialist tools: CloudMade has realised

potential of a cleaned-up set and has created web tools to help

– [animation]


AND (Netherlands)

• July 2007– Relatively straightforward as there was

little existing OSM mapping for the Netherlands

• From a digital mapping data company with navigation and LBS products

• First “complete” country in OSM• “15-20m” offsets in places between

Yahoo aerial imagery and the AND data


OS OpenData (U.K.)

• No bulk imports (c.f. Andy Robinson)• Community still debating

– UK coverage already too comprehensive– Possible licensing concerns– Building outlines possibly the biggest win

• Some test imports – automated feature detection from Street View rasters

• Isle of Sheppey


OS OpenData (U.K.)


Summary of Data Sources (UK)

SourceStartDate

QualityU.K.

CoverageCompleteness Currency

Consumer GPS 2004 Very poor Most of U.K. Very poor Variable

eCourier GPS 2005 Quite poorCentral London

OK (roads only) Quite old

Out-of-Copyright Maps

2006 Quite poorAll

eventuallyReasonable Very old

Yahoo Imagery 2006 ReasonableCertain

urban areasVery good Quite old

NaPTAN 2009 GoodAll

eventuallyVery good

(bus/stops only)Fairly new

Walking Papers 2009 Variable Variable Excellent Variable

OS OpenData Meridian

2010 Quite good All (Britain)Good (certain features only)

New

OS OpenData Street View

2010 Very good All (Britain)Very good

(certain features only - no paths)

New


Studies on Quality

• Muki Haklay (UCL CEGE)– “Haklay, M., 2010, How good is

volunteered geographical information? A comparative study of OpenStreetMap and Ordnance Survey datasets” Environment and Planning B: Planning and Design 37(4) 682 – 703• “on average within about 6 m of the position recorded

by the OS, and with approximately 80% overlap of motorway objects between the two datasets”

• Analysis in 2008 (Meridian 2)


Studies on Quality

• Muki Haklay and Aamer Ather– Beyond good enough? Spatial Data Quality

and OpenStreetMap data• OSM vs MasterMap I.T.N., looking at road lengths• “The analysis shows that when A-roads, B-roads and

a motorway from ITN are compared to OSM data, the overlap can reach values that are over 95%. … OSM is of better quality than Meridian 2.”

• “Positional accuracy is satisfactory for many applications. Attribute accuracy is also satisfactory.”

• Spring 2009


Studies on Quality

• “Just how good is it?”– As good as many consumer-grade online

mapping websites– Depends on the care and keenness of the

“local champion”


Studies on Completeness

• Muki Haklay– Completeness in volunteered geographical

information – the evolution of OpenStreetMap coverage (2008-2009)• “OpenStreetMap already covers 65% of the area of

England, although when details such as street names are taken into consideration, the coverage is closer to 25%. Significantly, this 25% of England’s area covers 45% of its population.”

• Affluence bias• Covers to October 2009



• Muki Haklay: OSM vs Meridian 2 (red)

Left: Features only

Right: With attributes



• Peter Reed– DfT highway length

figures per county• “Is it complete yet?”

– Depends on what you need

– Depends on where you need it

– Improving!


Tools for Completeness

• ITO World OS Locator


Tools for Completeness

• “Lonely Buses” tool – 20m buffer


Getting Completeness

• Mapping Parties– May 2006: “Mapchester”

• Chris Perkins, Andy Robinson

– January 2007: Central London• Steve Chilton• Richard Fairhurst

– May 2009: Milton Keynes • Me, Andy, Steve

– October 2009: Atlanta



• Milton Keynes– 2 days– 25 people– [animation]



• London Mapping Marathon series– One hour of mapping followed by pub

• 2008: Streets• 2009: POIs• 2010: Building outlines

– Build and keep the community• Crises

– January 2010: Haiti earthquake• High quality aerial imagery


Comparison

• Quality vs Google Maps– POIs– (Over)simplification of roads

• Completeness vs Google Maps – POIs again

• Databases vs crowdsourcing

• Coverage…– A few surprises – Pyongyang in OSM


Forbidden City in Beijing


Other Considerations


It Started Life as a Street Map

• It’s not Ordnance Survey MasterMap– Roads are line

features, not areal• Not yet, anyway

– OpenStreetMap is just a database• Anything can go in

it…


Individual Trees in Bern!


Loose Taxonomy

• Need to consider the metadata too – is it on the map, is it in the right place, is it tagged appropriately?– Community proposes and documents on

wiki but doesn’t enforce– Worldwide considerations– Metadata sometimes present for source,

always present for input/update date & user


Loose Taxonomy

• Partial order ensured by Steve Chilton’s OSM Mapnik stylesheet– If you want to see your data appear on the

map that “most” people see, follow the guidelines


Curves

• ...or lack of...– Ability to tag ways as curved– Still straight lines in DB


OSM in the Real World


CycleStreets

“We'll be launching a page for Local Authorities v. soon about embedded versions of the CycleStreets planner on their sites”

- CycleStreets


Skobbler

• Commercial Sat-Nav application based on OSM data with custom additions– Motorway lanes for junction approaches

• Completeness crucial– Display Google Maps on website for

address finding (Google Local) but use OSM for routing and app 3D view

• Accuracy not so important


Skobbler


Skobbler

• Started in Germany– The map is more complete

• Just launched in U.K.– The map is probably good enough, though

missing a lot of turn restrictions• Users report bugs

– Skobbler fix remotely or highlight to OSM– Both communities benefit


Flickr

• Beijing Olympics in 2008– Flickr is owned

by Yahoo


Waze in Chile

• Recent accidental unattributed inclusion of OSM data from “unreliable source”– Fine with

attribution


Background for Visualisation

• http://oobrien.com/vis/bikes/ (London)• http://osm.mapki.com/bikes/ (Minneapolis)

http://oobrien.com/vis/bikes/

http://osm.mapki.com/bikes/


Attribution

• Various – Imagery of OpenStreetMap data which are CC-By-SA OpenStreetMap and contributors.

• Various – Imagery of Ordnance Survey OpenData Street View rasters which are Crown Copyright and Database Right 2010.

• p4 – Photograph of GPS unit by Steve Coast.• p9 – Imagery of out-of-copyright map by Richard Fairhurst• p16 – Photograph of Walking Papers in pub by Harry Wood.• p27 – Maps produced by Muki Haklay.• p28 – Map produced by Peter Reed.• p42 – Screenshot of motorway junctions in OSM by Barry Crabtree.• p43 – Photograph of ambulance by DBH NHS media services.• p50 – Photograph of Boris Bike by Pocket Lint website.

Other imagery may be copyright of the producers referred to on the same page.


More

• http://www.oliverobrien.co.uk/– My research blog

• @oobr on Twitter• [email protected]

http://www.oliverobrien.co.uk/

openstreetmap - the quality issue

Technology

advanced spatial analysisout

crowdsourced spatial

london centre

spatial interaction

metadata centre

years centre

bike centre

copyright map tracing