canarie: providing essential digital infrastructure for canada · 2015-05-26 · middleware for...
TRANSCRIPT
www.canarie.ca
Mark Wolff; CTO
CANARIE: Providing Essential Digital Infrastructure for Canada
April 16, 2014
www.canarie.ca
DATA volume and complexity
A Transformation of the Science Paradigm
thousands of years ago
last few hundred years
last few decades
today tomorrow
empirical
theoretical
computational
data exploration (e-science)
2
www.canarie.cawww.canarie.ca 3
www.canarie.ca
The Digital Transformation of Research
4
not to scale.
Yottabyte1,000,000,000,000,000,000,000,000
Zettabyte1,000,000,000,000,000,000,000
Exabyte1,000,000,000,000,000,000
Petabyte1,000,000,000,000,000
Terabyte1,000,000,000,000
Gig
a 1
,00
0,0
00
,000
,00
0
1 PB of MP3songs would
take 2000 years to play
25 PB / year: Data generated
by the Large Hadron Collider
350 EB / year:Data produced by the
Square Kilometre Array
www.canarie.ca
The e-Science Data Deluge
Software to Support Use of the Network:
5
VolumeTerabytes +
VarietyStructured,
unstructured
VelocityReal-time, requires
immediate response
ValueComplex,
predictive analysis
www.canarie.ca
The Challenge for Digital Infrastructure
6
Digital infrastructure is more than advanced networks.
Unified Digital Infrastructure
Advanced Computing
Applications & Software
Tools
Distributed Data Storage
Expertise
Advanced High-Speed Networks
www.canarie.ca
Building Tomorrow’s Digital Infrastructure Today
7
Cloud Services for
Entrepreneurs
Software for Research
Network & Network Services
www.canarie.ca
Why?
8
To accelerate Canadian innovation through research in both the public and private sectors
To drive the adoption of transformative technologies among our stakeholder communities
To ensure Canada continues to develop a vibrant knowledge economy that benefits all Canadians
www.canarie.ca
Network & Network Services
9
www.canarie.ca
Network & Network Services
10
www.canarie.cawww.canarie.ca
CANARIE Connects to the World
Acknowledgements - The Global Lambda Integrated Facility (GLIF) Map 2011 visualization was created by Robert Patterson of the Advanced Visualization Laboratory (AVL) at the National Center for Supercomputing Applications (NCSA) at the University of Illinois at Urbana-Champaign (UIUC), using an Earth image provided by NASA with texture retouching by Jeff Carpenter, NCSA. Data was compiled by Maxine D. Brown of the Electronic Visualization Laboratory (EVL) at the University of Illinois at Chicago (UIC). Support was provided by GLIF, NCSA/UIUC, the State of Illinois, and US N
11
www.canarie.ca
Network & Network Services
12
www.canarie.cawww.canarie.ca
Network & Network ServicesCanadian Access Federation
13
Ensuring a trusted, unified, digital identity for Canada’s research and
education communities.
WiFi access worldwide using your home institution credentials
Single Sign On : Use one credential to access multiple services
Services access worldwide using your home institution credentials
www.canarie.ca
But Networks Alone Do Not Drive the Adoption of Digital Infrastructure
• Commercial toolsets require cost-prohibitive development
• Minimal shared infrastructure among disciplines
• Barrier to adoption & leverage of powerful infrastructure
Software to Support Use of the Network:
14
www.canarie.ca
The Missing Piece:Software for Researchers
15
• Funds the development of tools and software that help researchers focus on research rather than technologies and equipment
• Facilitates use of CANARIE network
• Focused on “big data” tools and platforms
www.canarie.ca
Middleware for Canadian Research: Network-Enabled Platforms (NEPs) and Research Platform Interfaces (RPIs)
NEP = a software platform optimized to
leverage digital infrastructure
for big data analysis in support of
research, discovery and innovation
RPI = a toolkit of reusable software
services that accelerate the
development of domain-specific
research platforms
A more efficient way to turn research data
into knowledge
Specifically designed by researchers for
researchers
16
www.canarie.ca
The Power of Research Software Platforms
• Existing CBRAIN NEP (soon to be an RPI contributor) best illustrates a network-enabled platform
https://brainbrowser.cbrain.mcgill.ca/surface-viewer#
17
www.canarie.ca
Why RPI?
• Excellent capabilities (services) are being built within research platforms
• But services are locked inside (most) platforms
– Platforms were/are not designed to use or provide external services
18
John Gress/Reuters
www.canarie.cawww.canarie.ca
NEP Program - Platform Architecture
Service 1
Service 2
Service 3
Service 4
Platform 1
Service 1
Service 2
Platform 2
Service 1
Service 2
Service 3
Platform 3
Service 1
Service 2
Service 3
Service 4
Platform 4
Service 5
19
www.canarie.cawww.canarie.ca
Platform Architecture after RPI
Service 1
Service 2
Platform 1
Service 1
Service 2
Platform 2
Service 1
Service 2
Service 3
Platform 3
Service 1
Service 2
Service 3
Service 4
Platform 4
Service A
Service B
Service C
20
www.canarie.cawww.canarie.ca
Platform Architecture after NEP-RPI
Service 1
Service 2
Platform 1
Service 1
Platform 2
Service 1
Service 2
Platform 3
Service 1
Service 2
Platform 4
Service A
Service B
Service C
Service D
Service E
Service F
Service G
Service I
Service H
21
www.canarie.ca
Use Case: Ocean Networks Canada Observatory Platform
Chilean Earthquakeand Tsunami of27 Feb 2010
ONC makes their dataOGC compliant and
releases the data as a Service (an RPI).
Example: water pressure
www.canarie.ca
Disaster Recovery Platform - Sendai Japan Earthquake & Tsunami, March 11, 2011
The Disaster Recovery Platform adds OCG data compliance, and connects to the ONC Observatory RPI for real-time data.
www.canarie.ca
The result is a highly advanced tsunami detector
www.canarie.ca
Efficient Reuse of Software Elements from Platforms
25
Software Platforms (NEP)
Previous Mandate:21 Platforms Funded
21
12
15
47
Software Elements (RPI)New RPIs created
…leveraged by new Platforms
…spinoff additional RPIs
www.canarie.ca
A New Paradigm for Research Software Development
• Unlocks functional elements built within research Software Platforms
• Supports the culture of collaboration and reuse within scientific research community
• Maximizes efficiency by eliminating duplication of effort and investment
• Minimizes time to discovery: 1. Browse / search software element repository
2. Reuse existing software elements to build customized platform for research requirements
3. Share new, discreet software elements with the global community
4. Contribute to innovative software ecosystem
26
CANARIE’s toolkit of software elements is available to the research community at no cost:
https://science.canarie.ca
Interested in providing a software element?
www.canarie.ca
The Cloud Transforms the Provisioning and Delivery of Software and Services
Web-based access to centralized computing resources & software applications
On-demand, pay-per-use access
Pool of shared, configurable resources (servers, storage, network)
Rapid provisioning with minimal management effort or service provider interaction
27
www.canarie.cawww.canarie.ca
Digital Technology Megatrends
28
18x: size of mobile data traffic in 2013 vs global internet traffic in 2000
81%: global mobile data traffic growth in 2013
500M: mobile devices added in 2013 alone
Mobile Devices
Big Data2.5 Exabytes: volume of data created daily, doubling each month
44x: forecasted increase in data volume 2009-2020
1 week: time to process decoding of human genome today vs 10 years initially
Social Media100: hours of video uploaded to YouTube each minute
2: # of new Facebook users each second
93%: adoption of social media by marketers
30+ Petabytes: volume of user generated data stored, accessed & analyzed by Facebook
Internet of Things2008: year in which # of things connected to Internet surpassed world population
50B: forecasted # of things connected to the Internet by 2020Source: Cisco Visual Networking Index, IBM, IDC, The Economist, Huffington Post, Wikibon, YouTube, Pew Research, Business Insider, Global Web Index
www.canarie.cawww.canarie.ca
All Enabled By Cloud Computing!!!
29
www.canarie.cawww.canarie.ca
What is Cloud Computing?
A network of remote servers on the Internet to store, manage, and process data…
…instead of on local servers or personal computers.
30
www.canarie.cawww.canarie.ca
Cloud Computing is Already Ubiquitous
31
www.canarie.ca
Digital Accelerator for Innovation & Research (DAIR)
32
• Application development accelerator provisioned on the CANARIE network
• Powerful test-bed for SMEs to develop, test, & demonstrate new cloud applications
• Access to cloud computing, network & storage infrastructure (IaaS --Infrastructure as a Service)
A kick-start to new cloud service offerings
Mobilizes new cloud ICT businesses in Canada
www.canarie.ca
Digital Accelerator for Innovation & Research (DAIR)
33
• 1st users on DAIR June 2011, among first IaaS clouds in Canada
• Edmonton and Sherbrooke regions operated by Compute Canada with cloud software support from Cybera, using OpenStack
• Close to 200 companies have used DAIR to speed their time to market.
www.canarie.ca
Summary: CANARIE Mandate
34
• Network Operations:• Continue to operate the CANARIE Network as essential research
infrastructure;
• Technology Innovation:• Develop, demonstrate, and implement next-generation technologies to
advance the CANARIE Network as a leading-edge research network; and
• Private Sector Innovation:• Leverage the CANARIE Network to assist firms operating in Canada,
and Canadian universities, to advance innovation and commercialization of products and services to bolster Canada’s
technology innovation capabilities.
www.canarie.cawww.canarie.ca 35
www.canarie.ca