8/14/2019 SoC Cloudsourcing

1/50

Neodata:From Crowdsourcing

To Cloudsourcing AndWhere It Will Take Us

Christopher Osborne

I was feeling a little mischevious and threw in as many buzzwords into this presentation title as possible. I even made up a few, but I will be talking about how data is changing and the new ways we are trying to visualise it.

8/14/2019 SoC Cloudsourcing

2/50

About Me

In order to understand my perspective you need to know a little about where I come from

8/14/2019 SoC Cloudsourcing

3/50

#geomob - gmdlondon.ning.com

Taken by @sigizmund

I run a bimonthly geoweb/neogeography meetup called #geomob, attended by lots of internet startups where we discuss the latest developments in web mapping and location technology

8/14/2019 SoC Cloudsourcing

4/50

I actually started ofas a geographer, with proper paper maps and everything

8/14/2019 SoC Cloudsourcing

5/50

A turning point was when I became a web GIS developer at Brent Council, and realised there was a better way to present maps online, and started doing web development in my free time

8/14/2019 SoC Cloudsourcing

6/50

I then moved to Transport for London, where I redesigned their data infrastructure. It was there that I became fascinated with theidea of sensor networks. They manage a huge infrastructure network of roads, railways, assets such as buses and trains, andthere are sensors all over the network. Induction loops under the road that measure the amount of trac passing above, Oystercard transactions, air quality sensors, GPS on buses, cameras that scan licence plate numbers... A huge network of sensorsacross London

8/14/2019 SoC Cloudsourcing

7/50

http://www.readwriteweb.comUnfortunately all of this data was in silos and going to waste. I designed a data architecture to make all the data available in realtime and spatially referenced so that real-time information can be used to make decisions influencing the road network

http://mike.teczno.com/notes/de2008.htmlhttp://mike.teczno.com/notes/de2008.html

8/14/2019 SoC Cloudsourcing

8/50

Neogeography

MASHUP:

{data source A + datasource B} * remix = newservice

Cloud is my database

Open data is innovationdriver

Lots of talk of neogeography and mashups. A mashup is just a remix of several different data sources into one new service, and we have seen a huge number of map mashups being produced as itgets . This is wonderful, we have people able to quickly and easily create web mapping, but as you may have seen a lot of cartographically poor mashups. Its important to distinguish between thequick and dirty mashups and the professional polished mashups.

A map of my holiday snaps doesnt need much design or cartography but to convey a large amount of information you need to pay attention to cartography and visualisation.

When we talk about neogeography I define it as a state of mind, the internet is my database, a huge nebulous entity of millions of atoms that forms what we call the cloud

The key thing here is that the availability of open data or data that is open enough that enables innovation to take place at zero or low cost

8/14/2019 SoC Cloudsourcing

9/50

http://www.where-can-i-live.comSo talking about doing proper cartography in mashups, here is one that I made: Where Can I Live?

A tool to show you locations in London within a convenient commute time from your place of work, within your budget. You specify your work location, how long you wish to travel and howmuch you can afford for the type of property you want to buy.

We tried really hard to show just the right level of detail on the map, and keep the cartography in keeping with the style of the site, we layer on tube lines and visualise the locations with a littleclock to show you how long it takes to travel there. We also graph house price data and display individual property listings

This is what a professional mashup should look like, paying attention to cartography and style

http://www.where-can-i-live.com/http://www.where-can-i-live.com/

8/14/2019 SoC Cloudsourcing

10/50

This is a simple example of a typical mashup architecture, its simple, lightweight, open source, but the main distinction is that I dont own any of this data. The cloud is my database and I pull indata from wherever I want and display it in a web browser.

8/14/2019 SoC Cloudsourcing

11/50

So doing all this work in Web2.0 and working for a large government body I learnt a lot about architecture and data.

You start with a huge amount of different systems and data sources, each one designed for a specific task

8/14/2019 SoC Cloudsourcing

12/50

Government / Enterprise

8/14/2019 SoC Cloudsourcing

13/50

http://www.flickr.com/photos/su-lin/

Unfortunately this is what government and enterprise IT systems looks like, a huge greasy mess of spaghetti. Its virtuallyimpossible to track down what systems do what let alone provide real-time data and information.

http://www.flickr.com/photos/su-lin/http://www.flickr.com/photos/su-lin/

8/14/2019 SoC Cloudsourcing

14/50

Web 2.0

8/14/2019 SoC Cloudsourcing

15/50

http://www.flickr.com/photos/thatguyfromcchs08/Whereas web2.0 is all about speed, lightweight architectures that allow data to flow at near real-time speedsI was lucky that Keir Clarke introduced mashups and some real time mapping yesterday, Im going to talk about the challenges ofvisualising huge amounts of real-time data

http://www.flickr.com/photos/thatguyfromcchs08/http://www.flickr.com/photos/thatguyfromcchs08/

8/14/2019 SoC Cloudsourcing

16/50

Current Web Mapping

8/14/2019 SoC Cloudsourcing

17/50

http://mike.teczno.com/notes/de2008.htmlTiles, slippy maps. Amazing five years ago and you can throw some data on top but its pre-rendered and you cant change anyof the mapping

http://mike.teczno.com/notes/de2008.htmlhttp://mike.teczno.com/notes/de2008.html

8/14/2019 SoC Cloudsourcing

18/50

http://www.opencyclemap.orgOpenStreetMap gives you access to the underlying data and you can create your own beautiful maps. OpenCycleMap is the classicexample: features relevant to cyclists are highlighted, contours to show relief, POIs such as pubs

http://www.opencyclemap.org/http://www.opencyclemap.org/

8/14/2019 SoC Cloudsourcing

19/50

http://www.cartagen.orgCartagen is vector rendering of OpenStreetMap data in the browser. No plugin and no pre-rendered tiles, its early days and is abit rough and ready but the map can change according to user interaction

http://www.opencyclemap.org/http://www.opencyclemap.org/

8/14/2019 SoC Cloudsourcing

20/50

2.2.4 no output or display from the ConsumerWebsite Service contains Licensed Data inVector Format or allows Licensed Data inVector Format to be reconstructed

OS Licence

None of this can happen with Ordnance Survey data, the licence is too restrictive

8/14/2019 SoC Cloudsourcing

21/50

Crowdsourced GeodataOpenStreetMap

At Ito World we use and contribute to OpenStreetMap

8/14/2019 SoC Cloudsourcing

22/50

You can see just how much wider and deeper the data coverage is becoming

8/14/2019 SoC Cloudsourcing

23/50

The UK and Europe are now very well mapped, particularly the Netherlands etc thanks to the contribution of commercial ANDdata

8/14/2019 SoC Cloudsourcing

24/50

And its happening all over the world

8/14/2019 SoC Cloudsourcing

25/50

http://www.vimeo.com/2598878

We made this wonderful visualisation of all the OpenStreetMap edits for 2008, gives an idea of the power of OSM

http://www.vimeo.com/2598878http://www.vimeo.com/2598878

8/14/2019 SoC Cloudsourcing

26/50

A Map = Data x Visualisation

the basics, this hasnt changed but the data we use is

8/14/2019 SoC Cloudsourcing

27/50

Im showing this because the man who created those streams in OpenStreetMap, Steve Chilton, is sitting right there.

8/14/2019 SoC Cloudsourcing

28/50

We are working on wonderful ways of renderingmaps

8/14/2019 SoC Cloudsourcing

29/50

Data is Changing

Visualisation is Changing

Maps are Changing

So the data we use is changing, we need to visualise it diferently and we are creating new maps

8/14/2019 SoC Cloudsourcing

30/50

http://www.race.u-tokyo.ac.jp/~uchida/blogdata/Because of this, the internet. This is a visualisation of the blogosphere, you can see these large bubbles are blogs with a lot ofreaders and links back to them. You see how this is all connected

http://www.race.u-tokyo.ac.jp/~uchida/blogdata/http://www.race.u-tokyo.ac.jp/~uchida/blogdata/

8/14/2019 SoC Cloudsourcing

31/50

8/14/2019 SoC Cloudsourcing

32/50

8/14/2019 SoC Cloudsourcing

33/50

http://www.flickr.com/photos/bryce/58299511/Great example of the way a simple web service to share photographs has evolved with user input and is now a huge source ofmetadata

http://www.flickr.com/photos/bryce/58299511/http://www.flickr.com/photos/bryce/58299511/

8/14/2019 SoC Cloudsourcing

34/50

http://code.flickr.com/blog/Flickr Alphashapes - using geotagged photos to create polygons for country and area boundaries. Psychogeographical space

http://code.flickr.com/blog/http://code.flickr.com/blog/

8/14/2019 SoC Cloudsourcing

35/50

http://code.flickr.com/blog/Imagine what happens when we have billions of geotagged photos

http://code.flickr.com/blog/http://code.flickr.com/blog/

8/14/2019 SoC Cloudsourcing

36/50

Cloudsourcing -The Sensor Web

this is just a flippant term I coined to describe the idea of huge amounts of data that you can pull from the cloud. We are nowseeing the start of the sensor web

8/14/2019 SoC Cloudsourcing

37/50

we are becoming connected to the web everywhere. You are sending data back whenever you interact with your location enabledsmartphone

8/14/2019 SoC Cloudsourcing

38/50

8/14/2019 SoC Cloudsourcing

39/50

We hold the equivalent of the tricorder in our hands right now

8/14/2019 SoC Cloudsourcing

40/50

I want to invent a tricorderfor planet earth - Mike

Liebhold

8/14/2019 SoC Cloudsourcing

41/50

8/14/2019 SoC Cloudsourcing

42/50

http://tonchidot.com/

8/14/2019 SoC Cloudsourcing

43/50

We Are All Sensors Now

8/14/2019 SoC Cloudsourcing

44/50

http://www.citysense.com/Citysense creates heatmaps and shows you the hot areas of the city where people are, showing you where all the popular barsand nightlife areas are. An old visualisation applied to real-time data

http://www.citysense.com/http://www.citysense.com/

8/14/2019 SoC Cloudsourcing

45/50

Not the best visualisation but if you use Google Maps then you are sending back data about the speed of your movement, usedto generate trac speed data

8/14/2019 SoC Cloudsourcing

46/50

Self Organising Maps

So how do we visualise huge amounts of real-time data - one way is to use a self organising map

8/14/2019 SoC Cloudsourcing

47/50

http://senseable.mit.edu/nyte/visuals.html

8/14/2019 SoC Cloudsourcing

48/50

http://neworleans.citymurmur.org/

8/14/2019 SoC Cloudsourcing

49/50

I blogwww.cloudsourced.com

I tweet@osbornec

http://www.cloudsourced.com/http://www.cloudsourced.com/

8/14/2019 SoC Cloudsourcing

50/50

www.itoworld.com

http://www.itoworld.com/http://www.itoworld.com/