![Page 1: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/1.jpg)
Using open data for smart cities developmentNOVEMBER 8-10, 2017
ANDREIWID SHEFFER CORRÊA
CC BY 4.0
![Page 2: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/2.jpg)
Open Data and Smart Cities
A smart city is an open city!(https://www.youtube.com/watch?v=BHrsllHJHeo)
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
2
(images from https://www.madhattertech.ca/chatter/open-data-and-journey-innovative-governments)
![Page 3: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/3.jpg)
Data-driven cities
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
3
(images from http://www.trentonian.com/article/TT/20140616/NEWS/140619760)(images from http://seevider.com/solution.htm)
Parking places assisted by sensors Triangulating soundwave from gunshot
![Page 4: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/4.jpg)
Data-driven cities
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
4
(images from http://www.libelium.com/smart_cities/)(images from https://www.smartcityinitiative.com/tag/traffic-light/)
Intelligent traffic lights with video and sensors Sensors measure noise, pollution, dust quantities and garbage levels
![Page 5: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/5.jpg)
What’s Open DataThe Open Data Definition
o “Open data and content can be freely used, modified, and shared by anyone for any purpose”
Open Data ≠ Big Data
o Big Data is defined by its size, complexity, and changeabilityo e.g. Data from sensors, mobile phone GPS signals
o Open Data is defined by its useo Must be available to the public
o May come from Big Data
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
5
(images from http://opendefinition.org/)
![Page 6: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/6.jpg)
What’s Open Data
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
6
(images from http://5stardata.info/en/)
5-star deployment scheme (first appeared 2006)
o OPEN LICENSE (OL)▪ Make your stuff available on the Web (whatever format)
under an open license
o STRUCTURED DATA (RE)▪ Make it available as structured data (e.g., Excel instead of
image scan of a table)
o NON-PROPRIETARY FORMAT (OF)▪ Make it available in a non-proprietary open format (e.g.,
CSV instead of Excel)
o USE OF URIs (URI) – identifiers
o LINKED DATA (LD)
![Page 7: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/7.jpg)
What’s Open DataThe 8 Principles of Open Government Data
o In 2007 a group of 30 advocates wrote a set o principles
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
7
Principle Description Recommendations
1-Complete All public data is made available Made available electronically
2-Primary As collected at the source (not in aggregate, highest granularity) Consider multiple views of data
3-Timely Available as quickly as necessary to preserve the value of the data Update data! Update data!
4-Accessible Available to the widest range of users for the widest range of purposes Provide APIs – automated access
5-Machine processable Data is reasonably structured to allow automated processing Avoid PDFs and images. Think tabularly.
6-Non-discrimatory Available to anyone, with no requirement of registration Allow anonymous access
7-Non-proprietary Available in a format over which no entity has exclusive control Use multiple formats
8-License-free Not subject to any copyright, patent, trademark or trade secret regulation
Be clear about licensing (mix of public records, personal info, copyrighted work..)
![Page 8: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/8.jpg)
How Open Data benefits cities?o Promotes new ideas and businesses (products and services)
o Allows third party apps instead of government apps
o Opportunity for small businesses
o Improves city efficiency
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
8
(images from https://smartcitiesworld.net/news/news/smart-cities-services-worth-225bn-by-2026-1618)
![Page 9: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/9.jpg)
Benchmarkingo Means evaluating and ranking countries, organizations and projects
o Improves accountability
o Emphasizes best practices
o Understand use open data for solving problems
o Mainly evaluates country-level initiatives
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
9
![Page 10: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/10.jpg)
Benchmarking: Global Open Data Index
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
10
(images from https://index.okfn.org/place/)
![Page 11: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/11.jpg)
Benchmarking: Open Data Barometer
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
11
(images from /?_year=2016&indicator=ODB)
![Page 12: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/12.jpg)
How Open Data looks like?Open Data lifecycle – Acquisition, Presentation and Dissemination
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
12
Data Acquisition
• Sensor
• Mobile App
• Collaboration
• Government
• Private sector
Data Presentation
• Search & discovery
• Metadata
• Multiple data formats
Data Dissemination
• API
• Advanced visualization
Open Data Catalog
![Page 13: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/13.jpg)
How Open Data looks like?Open Data catalogs
o Software to make data available▪ It’s possible to implement Open Data Principles
o There are many solution in the market
o The most famous is CKAN - Comprehensive Knowledge Archive Network
o Free & open source software
o Developed by Open Knowledge (http://okfn.org)
o Available + languages (including French, Portuguese and English)
o Sandbox at http://demo.ckan.org
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
13
![Page 14: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/14.jpg)
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
14
Natal-RN Data Portalhttp://dados.natal.rn.gov.br
Seems to be a test instance… With only four datasets!(as of Nov, 2017)
![Page 15: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/15.jpg)
How Open Data looks like?Some other CKAN Instances around the world
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
15
And +140 instanceshttps://ckan.org/about/instances/
![Page 16: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/16.jpg)
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
16
State-of-the-art report and evaluation of Open Data Catalogs (2015)
Summary of catalog features
Extracted from http://routetopa.eu/deliverable-2-1-state-of-the-art-report-and-evaluation-of-existing-open-data-platforms-now-available/
![Page 17: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/17.jpg)
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
17
State-of-the-art report and evaluation of Open Data Catalogs (2015)
Availability of extensibility
Extracted from http://routetopa.eu/deliverable-2-1-state-of-the-art-report-and-evaluation-of-existing-open-data-platforms-now-available/
![Page 18: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/18.jpg)
Know by examples: TransportationBlind Square – Smart Cities For All
o Use data from Foursquare and OpenStreetMap
o App provides audio directions for blind residents
(images from http://www.perkinselearning.org)
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
18
![Page 19: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/19.jpg)
Know by examples: TransportationTurnaround: Fixing New York City’s Buses
o An initiative of a group of New Yorkers riders
o Using open data to raise awareness about bus system conditions
o Report Cards with speed, accuracy, and ridership volume
(images from http://busturnaround.nyc)
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
19
![Page 20: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/20.jpg)
Know by examples: City PlanningBA Obras Buenos Aires
o Website for monitoring public works
o Based on official data
o Rich interface with maps and graphics
o Suggestions for data visualization
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
20
![Page 21: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/21.jpg)
Know by examples: City PlanningOpenStreetMap: “the Wikipedia ofmaps”
Current OSM map vs. Google Map of Sochi, Russia where the 2014 Olympic Games begin on Feb. 7
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
21
(images from https://stevecoast.com/2014/01/30/its-time-to-make-openstreetmap-your-only-street-map/)
![Page 22: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/22.jpg)
Know by examples: Resilient CityBaltimore Open Air Project
o Project to distribute stations to monitor air quality
o They call those stations WeatherCubes
o Monitor temperature, humidity, ozone, and nitrogen dioxide
o Everyone can apply for hosting a WeatherCube and be part of the network
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
22
![Page 23: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/23.jpg)
Know by examples: Resilient CityInfoAmazonia Colombia
o A project to reduce deforestation
o It uses data from satellites
o Also data crowdsourced from data journalists
o Provide alerts to citizens when identify new construction projects
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
23
![Page 24: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/24.jpg)
Know by examples: EngagementSmart Dublin
o A project to engage with smart technology providers
o Providers have benefits from local authorities
o Data is shared to solve challenges and improve city life
o There is an engagement process
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
24
![Page 25: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/25.jpg)
Know by examples: Public and PrivateThe City Data Exchange (Copenhagen)
o Integrates data from public and private data owners
o Tries to avoid silos of data
o Based on a proprietary platform from Hitachi
o Owners can charge to access data
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
25
(images from https://www.hitachivantara.com/en-us/pdf/solution-profile/hitachi-solution-profile-city-data-exchange.pdf)
![Page 26: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/26.jpg)
Know by examples: Spreading the word!School of data (https://schoolofdata.org)
o Global network committed to advancing data literacy
o Aims at empowering civil society, journalists and citizens
o Promote training, workshops and community events
o 13 organizations and 101 active individuals (as of Nov 2017)
o Presence in developing countries
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
26
![Page 27: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/27.jpg)
Some points to think about (carefully!)o Keep the mantra: “OPEN BY DEFAULT”
o Opening data is not only for governments
o Legislation is not enough!
o Disclosing data is not enough: we need social benefits (open data impact)
o Clear understanding of licensing issues
o The spread of documents instead of data (specially at municipal level)
o Go beyond hackathons to a continuous development of smart cities
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
27
![Page 28: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/28.jpg)
Data openness in Brazilian local governmentsResearch paper highlights
o Understand how current data portals comply with open government data (OGD)
o Focus on local governments (city halls)
o Assessed 561 municipalities
o Findings showed the vast majority of assessed data portals did not comply with OGD
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
28
![Page 29: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/29.jpg)
Data openness in Brazilian local governments
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
29
Andreiwid Sh. Corrêa, Evandro Couto de Paula, Pedro Luiz Pizzigatti Corrêa, Flávio Soares Corrêa da Silva, (2017) "Transparency and open government data: A wide national assessment of data openness in Brazilian local governments", Transforming Government: People, Process and Policy, Vol. 11 Issue: 1, pp.58-78, doi: 10.1108/TG-12-2015-0052
![Page 30: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/30.jpg)
Data openness in Brazilian local governments
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
30
Andreiwid Sh. Corrêa, Evandro Couto de Paula, Pedro Luiz Pizzigatti Corrêa, Flávio Soares Corrêa da Silva, (2017) "Transparency and open government data: A wide national assessment of data openness in Brazilian local governments", Transforming Government: People, Process and Policy, Vol. 11 Issue: 1, pp.58-78, doi: 10.1108/TG-12-2015-0052
![Page 31: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/31.jpg)
Current researcho Aims at assessing government data portals according to OGD
o Look into the Deep Governmental Web
o Dynamic-generated data (HTML)▪ Data behind HTML forms and parameters
o Preliminary tests in the 27 Brazilian capitals▪ Crawled ~20 millions of links
▪ Currently processing potential deep web pages (with certain kinds of HTML forms)
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
31
(images from https://www.deepweb-sites.com/)
![Page 32: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/32.jpg)
Interesting sources of informationo Open Data Charter (2015)▪ https://opendatacharter.net
o Open Data Institute (2012)▪ https://theodi.org
o Open Data for Development (2010)▪ http://od4d.net
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
32
![Page 33: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/33.jpg)
Thank you!
http://andreiwid.info
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
33
![Page 34: Using open data for smart cities developmentandreiwid.info/support/2017_ifrn/using_open_data.pdf · Acquisition, Presentation and Dissemination. 2nd French-Brazilian School on Big](https://reader034.vdocuments.mx/reader034/viewer/2022043010/5fa205c1aaf5501970156d79/html5/thumbnails/34.jpg)
Bibliographyhttps://carto.com/blog/forty-brilliant-open-data-projects-preparing-smart-cities-2018/
http://busturnaround.nyc/
http://www.blindsquare.com/
https://smartnet.niua.org/sites/default/files/resources/Open-Data-Guide-8-24-2015.pdf
https://opengovdata.org/
https://www.pwc.ru/en/government-and-public-sector/assets/ddc_eng.pdf
http://cdn.cs50.net/2016/fall/seminars/smart_cities/smart_cities.pdf
https://www.hitachivantara.com/en-us/pdf/solution-profile/hitachi-solution-profile-city-data-exchange.pdf
http://routetopa.eu/deliverable-2-1-state-of-the-art-report-and-evaluation-of-existing-open-data-platforms-now-available/
2nd French-Brazilian School on Big Data and Smart Cities November 8-10, 2017
34