visions of the web of open data - files.meetup.com chris davis.pdf · visions of the web of open...
TRANSCRIPT
![Page 2: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/2.jpg)
Who am I?
● Postdoc Energy & Industry, TBM, TU Delft
● Focus on Industrial Ecology, Open Data, Collaborative Software, Modeling, Visualization, Analytics, etc.
![Page 3: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/3.jpg)
Motivations
● Energy and sustainability are some of the most important topics of the 21st century
● Need both aggregated and fine-grained data
● Research can be data intensive● There's a lot out there, but
connecting it is tedious● Researchers often duplicate effort● It would be great to revolutionize
how we deal with this data● The energy sector is only slowly
embracing the ICT & Open Data revolutions
![Page 4: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/4.jpg)
“Information wants to be free”
![Page 5: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/5.jpg)
Information wants to be free because it has become so cheap
to distribute, copy, and recombine - too cheap to meter.
Stewart Brand
There's a Tension...
![Page 6: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/6.jpg)
It wants to be expensive because it can be
immeasurably valuable to the recipient.
Stewart Brand
There's a Tension...
![Page 7: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/7.jpg)
That tension will not go away. It leads to endless wrenching debate
about price, copyright, “intellectual property,” and the moral rightness of casual distribution,
because each round of new devices makes the tension worse, not better.
Stewart Brand
There's a Tension...
![Page 8: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/8.jpg)
If you cling blindly to the expensive part
of the paradox, you miss all the action
going on in the free part.
The pressure of the paradox forces information
to explore incessantly.Stewart Brand
There's a Tension...
![Page 9: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/9.jpg)
![Page 10: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/10.jpg)
Enipedia.tudelft.nl
![Page 11: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/11.jpg)
11
![Page 12: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/12.jpg)
12
![Page 13: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/13.jpg)
13
![Page 14: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/14.jpg)
14
![Page 15: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/15.jpg)
15
A tale of one (or four?) power stations and seven data sets
![Page 16: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/16.jpg)
![Page 17: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/17.jpg)
17
How the European Commission manages data
Large Combustion Plants Directivehttp://ec.europa.eu/environment/air/pollutants/stationary/lcp/legislation.htm
![Page 18: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/18.jpg)
Coupling of Power Production to Water Consumption
"Water becoming a serious constraint for power generation" Aditi Nigam, The Hindu, July 18, 2012
![Page 19: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/19.jpg)
![Page 20: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/20.jpg)
![Page 21: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/21.jpg)
Transparency?
Copyright
Unless specifically prohibited by a notice published on any page, you may make a print copy of such parts of the Web-site as you may reasonably require for your use provided that any copy has attached to it any relevant proprietary no-tices and/or disclaimers. You agree not for yourself or through or by way of assistance to any third party to distribute, decompile, reverse engineer, disassemble or otherwise deal in or with the Website or materials therein or otherwise commercially exploit such material or content otherwise than as permitted by law.
[...]
Costs of Access
You shall be responsible for obtaining access to the internet in order to make use of the Website and shall pay any ser-vice fees, telephone charges or other costs associated with such access.
Data for further analysing purposes are to be downloaded using the means available at the website. The use of crawlers, robots or similar tools will be seen as offensive and will lead to a temporarily or permanent disclosure of a user/company from the website.
[...]
The downloads shall be based on fair use, unproportional downloads (7.5 times more than the average user of the same category) may lead to a withdrawal of the user rights without prior notice.
http://www.gas-roads.eu/gte_tp/html/termsandconditions
http://www.entsoe.net/res/disclaimer.pdf
![Page 22: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/22.jpg)
26
Officially Curated vs. Crowdsourced data
● Crowdsourcing generally OK for easily verifiable data● Officially curated data needed for comprehensive, hard
to verify data, small specialized communities● Crowdsourced data is only possible because of revision
control.● Crowdsourced data needs an incentive
● General interests, hobbies, gamification
![Page 23: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/23.jpg)
27
![Page 24: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/24.jpg)
28
![Page 25: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/25.jpg)
29
Data Quality as a Product Data Quality as a Process
![Page 26: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/26.jpg)
How to Measure Data Quality?
DataQuality
ResearcherSkill/Experience
# Viewers/Editors
Ease of IndependentVerification
= X X
Low Editor Diversity
High Editor Diversity
![Page 27: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/27.jpg)
31
How to Measure Data Quality?
● Eric Raymond – “With many eyes all bugs are shallow”● But... not all eyes are evenly distributed
![Page 28: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/28.jpg)
Distributed Air Quality Sensors
http://airqualityegg.wikispaces.com/
![Page 29: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/29.jpg)
Distributed Air Quality Sensors
http://airqualityegg.wikispaces.com/
![Page 30: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/30.jpg)
Two Long Tails
Contributors
Data
![Page 31: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/31.jpg)
Two Long Tails
Contributors
Data Diminishing Marginal Returns
Diversity of User Knowledge
![Page 32: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/32.jpg)
![Page 33: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/33.jpg)
![Page 34: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/34.jpg)
Club of 27?
![Page 35: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/35.jpg)
Loosely Coupled
Open Data?
LinkedOpen Data?
![Page 36: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/36.jpg)
![Page 37: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/37.jpg)
41
![Page 38: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/38.jpg)
42
![Page 39: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/39.jpg)
enipedia.tudelft.nl/mapsenipedia.tudelft.nl/maps
![Page 40: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/40.jpg)
![Page 41: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/41.jpg)
http://skytruth.org/viirs/
![Page 42: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/42.jpg)
![Page 43: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/43.jpg)
![Page 44: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/44.jpg)
48
Big Data?
![Page 45: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/45.jpg)
49
Big Data
http://uncyclopedia.wikia.com/wiki/Rocket_Propelled_Chainsaw
![Page 46: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/46.jpg)
50
Back to Basics
●API●REST●GET●POST●CSV●XML●JSON●CC BY-SA
![Page 47: Visions of the Web of Open Data - files.meetup.com Chris Davis.pdf · Visions of the Web of Open Data Chris Davis - @cbdvs ... Unless specifically prohibited by a notice published](https://reader033.vdocuments.mx/reader033/viewer/2022052020/6034331bf3aa41231815b0e7/html5/thumbnails/47.jpg)
Conclusions
● Data is embedded in a socio-technical system● Co-evolution of Data, Platforms, and Communities● Official data needs crowdsourced data & vice versa● Data Quality as a product vs. a process