Download - Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds
![Page 1: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/1.jpg)
November 2013 slide 1
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
Big Data & Collaboration
The Theory, Practice & Opportunity, a view from the University of Leeds
Rhys Davies, IT Director University of Leeds & Chairman YHMAN Ltd
Friday 8th November 2013
Location: Aldwark Manor Golf & Spa Hotel, Alne Nr York
Meeting: 22nd Annual NYHDIF Conference
![Page 2: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/2.jpg)
November 2013 slide 2
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
Agenda
• Introduction & background
• The Theory
– Approach, Vision & Plan
• The Practice
– Preparation, Timing, Execution & Outcomes
• The Opportunity
– What this might mean for Health Sciences
– Where we are now
• What next ?
• Questions ?
![Page 3: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/3.jpg)
November 2013 slide 3
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Theory …
My Role
Approach
Vision
![Page 4: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/4.jpg)
November 2013 slide 4
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Theory …
My Role
Approach
Vision
“To create a Collaborative Centre of Excellence
which can be used for Research Computing
for the benefit of Academia, Health, Commerce and the greater good;
to be built on a mutually beneficial ‘model’
with the belief that by sharing assets (equipment, intellect, funding)
we can deliver more, better, cheaper for all concerned”
![Page 5: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/5.jpg)
November 2013 slide 5
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Preparation
What were the required ingredients ?
People:
Shared Vision; Skills; Energy; Appetite for Risk; Trust
Process:
Collaboration; Consistency; Secure; Sustainability
Technology:
Network; Compute; Storage; Data; Service
![Page 6: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/6.jpg)
November 2013 slide 6
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Preparation, Timing & Execution
What were the required ingredients ?
People:
Shared Vision; Skills; Energy; Appetite for Risk; Trust
Process:
Collaboration; Consistency; Secure; Sustainability
Technology:
Network; Compute; Storage; Data; Service
![Page 7: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/7.jpg)
November 2013 slide 7
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Networks #1
Beyond the UK JANET Services offer onward connect to:
UK Internet Peering
Europe (GEANT Network)
US Internet, Abilene & ESnet2
Japan (NI) & China (CERNET)
Beyond the UK JANET Services offer onward connect to:
UK Internet Peering
Europe (GEANT Network)
US Internet, Abilene & ESnet2
Japan (NI) & China (CERNET)
The National and International picture
Secure,Free for research
![Page 8: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/8.jpg)
November 2013 slide 8
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Networks #2
The Local picture
•Secure,•Resilient,•‘Limitless’ bandwidth, •Free at point of consumption
![Page 9: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/9.jpg)
November 2013 slide 9
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Compute #1
• The UK’s first triangulated datacentre
• One of the worlds’ largest spanning datacentres
• Has the unique capability to span to more than 3 hubs
![Page 10: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/10.jpg)
November 2013 slide 10
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Compute #2
• Innovative
• Award winning
• Secure
• Resilient
• Virtual
• Highly available
• Linked to HPC
![Page 11: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/11.jpg)
November 2013 slide 11
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Compute #2
• Innovative
• Award winning
• Secure
• Resilient
• Virtual
• Highly available
• Linked to HPC
• This also covers Security; Storage; Services
![Page 12: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/12.jpg)
November 2013 slide 12
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, next step…
Fundamental Building Blocks are in place…
The physical network
The shared virtual data centre = ‘safe haven’
•The Supercomputer
•The skills necessary to exploit
![Page 13: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/13.jpg)
November 2013 slide 13
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Super Compute #1
![Page 14: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/14.jpg)
November 2013 slide 14
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Super Compute #2
![Page 15: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/15.jpg)
November 2013 slide 15
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Compute #3
N8 HPC ‘Approach & Way of Working’
Research Impact & Industrial Growth
Research-led (Vertical) Themes Network
Research-led (Vertical) Themes Network
Cross-cutting (horizontal) themes in methods and techniques
Cross-cutting (horizontal) themes in methods and techniques
• Institutional, specialist research computing support
• Specialist Facility Support• N8 Industry Innovation Forum• Business Engagement Teams• Research Computing Training• Doctoral Training (CDT)
• Institutional, specialist research computing support
• Specialist Facility Support• N8 Industry Innovation Forum• Business Engagement Teams• Research Computing Training• Doctoral Training (CDT)
Centr
e o
f Exce
llence
Infr
ast
ruct
ur
eR
ese
arc
h
![Page 16: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/16.jpg)
November 2013 slide 16
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Super Compute #4
N8 HPC • EPSRC funded in March 2012
– Capital– First year set-up and running costs
• Aims– Establish a Tier 2 HPC facility– Develop a computational science research network– Share support and training expertise– Develop collaborative links with Tier 1 partners– One stop shop for business – key themes for
engagement
• Future running costs underwritten by partners
![Page 17: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/17.jpg)
November 2013 slide 17
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Execution, Super Compute #5
5312 2.6Ghz Intel Sandy Bridge cores2:1 blocking QDR infiniband4GB/core (256 cores @16GB/core)174TB Lustre parallel filesystemCentOS/Redhat 6.3 based.SGE scheduler, Intel/GNU Compilers, OpenMPI/IntelMPI/MVAPICH2Locally- and centrally-provided software.
Co-located with 4500-core Leeds HPC
Purchased through Esteem framework agreement: SGI hardware, Alces integration
#291 in June 2012 Top500
![Page 18: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/18.jpg)
November 2013 slide 18
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Outcomes #1, Proctor & Gamble
![Page 19: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/19.jpg)
November 2013 slide 19
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Outcomes #2, Proctor & Gamble
![Page 20: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/20.jpg)
November 2013 slide 20
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Outcomes #3, BBC
Opportunity & Challenge•Relocation of BBC to Salford
– Exploring opportunities to deepen relationship with University of Manchester
•“Making Musical Moods Metadata”
– 128,000 audio files
– 53 transformations (classifying mood as f(time))
– On current tech: Over a year of processing time
Outcome
•Over a year of processing down to 12 hours.
•“The entire dataset was processed in only 12 hours, creating the world's largest time-varying musical feature database. Their combination of cutting-edge facilities and outstanding support was of huge benefit in getting the project completed and we look forward to working with them again.” – Chris Baume
![Page 21: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/21.jpg)
November 2013 slide 21
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Outcomes #4, A VRE
![Page 22: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/22.jpg)
November 2013 slide 22
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Practice … Outcomes #5, Secure Storage ITC
HE Community Storage •Secure•UK Located•Cost effective•Functionally richer than anything else in the current market•Institutional scale•Single point of access to all data sources•Authenticated to your systems
“UNIVAULT”
![Page 23: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/23.jpg)
November 2013 slide 23
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Opportunity
![Page 24: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/24.jpg)
November 2013 slide 24
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
University of Leeds
Leeds Teaching Hospitals Trust
SharedInfrastructure
InvestmentClinical
InfrastructureInvestment
Research Infrastructure
Investment
PPMN8 / HPC
Consent systemData extraction
The Opportunity, What might this mean for Health #1
![Page 25: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/25.jpg)
November 2013 slide 25
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
University of Leeds
Leeds Teaching Hospitals Trust
SharedInfrastructure
InvestmentClinical
InfrastructureInvestment
Research Infrastructure
Investment
PPMN8 / HPC
Consent systemData extraction
PhenobankingBiobanking & Analysis
The Opportunity, What might this mean for Health #1
![Page 26: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/26.jpg)
November 2013 slide 26
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Opportunity, What might this mean for Health #2
![Page 27: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/27.jpg)
November 2013 slide 27
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
The Opportunity, What might this mean for Health #3
Figure 3
![Page 28: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/28.jpg)
November 2013 slide 28
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
What Next ?
Shared Virtual Data Centre– we have our first customer and are investigating options
with other HE
N8 High Performance Datacentre- we are looking to develop the next iteration of this
Secure Storage in the cloud- we are starting to market this and looking for BETA testers
Big Data Collaboration with LTHT- MRC decision expected this month
![Page 29: Big Data & Collaboration The Theory, Practice & Opportunity, a view from the University of Leeds](https://reader036.vdocuments.mx/reader036/viewer/2022070415/56814f89550346895dbd4401/html5/thumbnails/29.jpg)
November 2013 slide 29
Document Reference : NYHDIF Conference Presentation November 2013, Rhys Davies
Questions…
Thank you
???