Mashups

Ryan McKeel, ryan.mckeel@nrel.gov

Digital Assets Applications Engineer, OpenEI.org

Twitter: @openenergyinfo @rmckeel

What is a mashup?

a mashup uses and

combines data,

presentation or

functionality from two

or more sources to

create new services

combination, visualization and aggregation

implies easy, fast

integration, frequently using

open APIs

enriched results that were

often not the original intent

behind the raw source data

A brief history of mashups

Mashup from 1854

1869

multiple data sources, multiple axes

Mashup from 1945:

Heads up displays

1945 British combat aircraft:

Radar display projected

onto the aircraft’s windscreen

2004 US Navy F-18 HUD

National Geographic 2009

compiled from global server records

Popular mashup

gapminder.org

438 data sources! now, free software

Effort to create a mashup

1854 1945 2009 2020

paper electronic web mobile

Future of mashups

Visual Programming Languages:

DIY Mashups

Max/MSP – musical random note generator

This visual source code

allows a search of NYC

apartments that are within n

miles of a search item

Displayed are available NYC apartments

that are within 1 mile of a hospital

2005 heads up display

Engineer test of DARPA

COORDINATORs work

2010 LAYAR on mobile

phones

a “mobile mashup”

17 lines of custom code

What’s speeding adoption?

Standard data formats

Open licensing

Reusable software

Simply powerful GIS tools

Community growth and collaboration

Linked Data 2007

Open Distribution

Linked Data 2008

Open Distribution

Linked Data 2009

Open Distribution

Linked Data 2010

What’s holding us back?

Intellectual property rights

Security

Data quality

Data is hard to find

Data is challenging to integrate

Which attitudes hold us back?

Managers “If we don’t share, we can differentiate ourselves”

“We’re too busy to take two days off”

“We don’t market our products. If it’s good, it will become known”

Developers “This has to be perfect before I release it!”

“The IT department won’t let me do much, I better stick with Excel macros”

IT Department “God forbid we have custom code on one of our websites!”

More concern over losing your job than not making progress

Jumping on the bandwagon

What can we do?

Change "we made this all ourselves" to "we

helped build something great”

Managers, remove legal roadblocks

Open licensing

Provide a data dictionary

Make tasty machine food (XML, RDF, RDFa…)

Show developers how to get started with your

data

How to host a mashathon

Involve people who are enthusiastic and capable

Small groups

Planning wiki

Cut red tape beforehand, then stand back as planner

Have potential stories and data sources ready

Quick build then fine-tune later

Provide free food, snacks and caffeine

Social and (if possible) financial incentives

Making a good mashup

Follows the FIST philosophy:

Fast

Inexpensive

Simple

Tiny

For more info on FIST philosophy,

search the Internet for “FIST Dan

Ward”

Make the mashup quickly

Quotes from Lt Col Dan Ward & RPL team:

– Creative constraints foster creativity. Adding time

and/or money generally does not improve

outcomes.

– Iteration drives learning, discovery and efficiency.

– Complexity is a cost.

– Simplicity scales. Complexity doesn’t.

Demo

http://en.openei.org/apps/mashathon2010/

Peered Production http://en.openei.org/wiki

Centralized Production http://en.openei.org/datasets

Open Distribution http://en.openei.org/sparql

• Web-based submission process

• Contributor-only write access

• Open read access

• Open commenting and rating

• Scalable distributed storage

• Semantic Wiki

• Open write access

• Open read access

• Forms-based authoring

• SPARQL Endpoint

• Linked Open Data (RDF/HTTP)

• Named Graphs for Segmentation

Crowdsourced Data Entry

“EIA does not collect or publish data on electricity rates, or tariffs, for the sale or purchase of electricity, or on demand charges for electricity service, nor does

EIA publish retail electricity rates or prices for peak or off-peak periods

(sometimes referred to as time-of-use-rates). … EIA is not aware of a publicly

available source for this information other than individual utilities.”

EIA Electricity FAQ http://tonto.eia.doe.gov/ask/electricity_faqs.asp#electric_rates