the outlook is cloudy

25
The Outlook is Cloudy Benefits, pitfalls and issues in building large-scale clouds Terry Harmer 12 May 2011 EduServ 2011 Symposium 1 http://www.besc.ac.uk

Upload: eduserv

Post on 27-Nov-2014

851 views

Category:

Education


1 download

DESCRIPTION

A talk by Terry Harmer at the Eduserv Symposium 2011 - Virtualisation and the Cloud.

TRANSCRIPT

Page 1: The Outlook is Cloudy

1

The Outlook is CloudyBenefits, pitfalls and issues in building large-scale clouds

Terry Harmer

12 May 2011 EduServ 2011 Symposium

http://www.besc.ac.uk

Page 2: The Outlook is Cloudy

2

What do I do?Technical Director of Belfast e-Science

• Develop project ideas for digital economy applications• Form consortia to bid for funding

– usually write the project funding proposals– funding from EPSRC, TSB, LDA and private companies

• Lead Technical architect for projects• Project Manager

… also do software development

I propose, design, manage and (help) build large-scale service centric applications.These projects are (and are increasingly) cloud based using utility infrastructure consisting of owned and multiple utility vendors.12 May 2011 EduServ 2011 Symposium

Page 3: The Outlook is Cloudy

3

Talk OutlineOrganisation

1. BeSC? 2. How we started with

clouds 3. Project Overviews

1. Financial services2. Digital media

4. Issues

Objective• To present some large-scale

projects that are in or were in field deployment with established user groups• Dynamic and utility focused• Why this approach and what

advantages has this approach given us.

• Some technology that we use.• Issues, advantages, problems,

pitfalls…

12 May 2011 EduServ 2011 Symposium

Cloud – Hype or Reality? For us this has been part of how we been doing things for a number of years.

Page 4: The Outlook is Cloudy

4

Belfast e-Science Centre? • Belfast e-Science was established in 2002 with funding from

EPSRC and the DTI under the UK e-Science programme.– EPSRC Platform Award funded and by TSB, EPSRC, INI, MoD,

QinetiQ– BeSC is entirely self funding (and has been since 2002)

•Don’t really use shared resources within a University infrastructure• Have close connections with companies, commercial users and organisations

•We have the attitude and tend to operate like a small R&D company•Have a tight budget and (perhaps too) big ambitions.

• The accidental decision to focus on commercial / industrial applications was driven by the challenges they offered.

– The challenges made us unique– We met a bunch of people who got the idea!– But they wanted to do large-scale and practical examples.

12 May 2011 EduServ 2011 Symposium

Page 5: The Outlook is Cloudy

BeSC Infrastructure – Shared Services

• Email has been hosted by a provider (Jan 2007)• Project calendars, email lists and chat rooms are all hosted by

utility providers (besc.ac.uk)• All of our project shared services have migrated to utility

resources (Autumn 2007 onward).• …projects have progressively moved to utility cloud services

– Use a range of commercial providers– Turning off internal kit that we own and shared with our

commercial partners

Page 6: The Outlook is Cloudy

Infrastructure circa

12 May 2011 EduServ 2011 Symposium 6

2002 2003 2004

BT DC

2005

THN

BBC

Page 7: The Outlook is Cloudy

Infrastructure 2007

12 May 2011 EduServ 2011 Symposium 7

Page 8: The Outlook is Cloudy

Financial Services

12 May 2011 EduServ 2011 Symposium 8

Futures Commodities Branches

Users

Server Farm

JobManagement

DRMirror

Capability Cloud

Capability Cloud

Page 9: The Outlook is Cloudy

Physical Resource View (2006)

12 May 2011 EduServ 2011 Symposium 9

Page 10: The Outlook is Cloudy

Digital media

12 May 2011 EduServ 2011 Symposium 10

• Working in the evolving on-demand media environment– Started pre- iPlayer and YouTube!

• Concern early was on better resource utilisation in and expensive and highly dynamic environment.– Early model of pooled resources

• Most recently in on-demand media infrastructures– Project PRISM with BBC/QinetiQ/BT (completed 2009)

• Supported game console to Phone to set-top box access.

– Currently supporting film and on-demand services.

• Stats– BBC archive ~52PB– iPlayer 7PB of data transfer per month

Page 11: The Outlook is Cloudy

A Circuit-based Infrastructure (2002)

12 May 2011 EduServ 2011 Symposium 11

Large-scaleContent Store

NetworkController

Scheduling Automation

BroadcastTransmitters

Uplinkto Satellite

I nternet

Content Store

Presentation Suite

Scheduling Automation

Page 12: The Outlook is Cloudy

12 May 2011 EduServ 2011 Symposium 12

BBCNorthern I reland

BBCScotland

BBCWales

BBCNetwork

BBCNorthern I reland

BBCScotland

BBCWales

BBCNetwork

High SpeedNetwork

BBC Broadcast Resources (slide from 2004)

Page 13: The Outlook is Cloudy

Mobile Non-geographic services (slide from 2005)

Services

BBCNorthern I reland

Services

BBCScotland

Services

BBCWales

Services

BBCNetwork

High SpeedNetwork

BBCYorkshireServices

BBCWest

Services

BBCSouth

Services

BBCE Midlands

Services

BBCW Midlands

Services

BBCNE

Services

BBCEast

Services

BBCSouth West

Services

BBCEY & L

Services

BBCLondonServices

BBCNW

Services

BBCSatellite

12 May 2011 EduServ 2011 Symposium 13

Page 14: The Outlook is Cloudy

MediaEconomy

12 May 2011 EduServ 2011 Symposium 14

Page 15: The Outlook is Cloudy

12 May 2011 EduServ 2011 Symposium 15

Content Cloud

Transcoding ContentRelease

StreamingContent

Exchange

ContentManagement

MetadataRelease

ContentContent

ContentContent

Infrastructure needs to scale rapidly to reflect demand

Page 16: The Outlook is Cloudy

12 May 2011 EduServ 2011 Symposium 16

A Dynamic Utility Resource Cloud (2007)

Page 17: The Outlook is Cloudy

Infrastructure Summary

• Dynamic collections of services– Managing real user groups

• Services scale and contract to established SLAs– We attempt to keep our deployed infrastructure low

• Cheaper … and… a small attack surface

• Our infrastructure is a mix of owned and utility infrastructure– increasingly the utility part is the majority– Buying capacity and storage on demand is our norm.

• Play the provider market…

12 May 2011 EduServ 2011 Symposium 17

Page 18: The Outlook is Cloudy

Issues - General

• Utility resource market is immature– The offerings can be difficult to compare

• no standard unit of compute/storage– simply specifying 1 GiB is pretty meaningless, need to talk about uptime, chance of data loss, etc

• Prices will be dependant on the user usage pattern• What you get and what you can buy varies widely

– Some attempts at customer lock-in to providers– Multi-provider clouds can be (relatively) expensive

• Not being multi-provider can be expensive in reliability

• Provider APIs and features constantly changing.– No standard API – New services and providers appearing. – APIs not very well documented

• Weak SLAs from providers– best SLAs guarantee 100% uptime and refund when there's downtime. – There's always downtime. – The cost of your downtime will be much higher than what you are paying for the resources.

12 May 2011 EduServ 2011 Symposium 18

Page 19: The Outlook is Cloudy

Issues - Technical

• Machine performance unpredictable. – CPU features especially unpredictable and can make a big difference to

compute-heavy tasks • e.g. we are heavy video transcoding users.

– individual instances can be unreliable (hosts DO crash)

• Bandwidth unpredictable and can be costly– relative to JANET…more reasonable relative to real commercial rates

• Required to manage OS images– proliferation of images; – using anything but vendor images requires trust in creator.

• nobody has a trust framework-have to trust that user

– creating own images (or using other peoples) means more machines to keep up to date!

12 May 2011 EduServ 2011 Symposium 19

Page 20: The Outlook is Cloudy

Issues - Technical/development

• You need significant metadata on your applications– OS versions, Software stack, security configuration, operational

behaviour, versions and compatibility– What you are prepared to pay and how to model

• We develop this metadata as part of our software development process– As software is developed and integrated– Develop, manage, test costing models and scenarios for deployment

• Your applications need to be developed to cope with and recover from failure– MTTR rather than MTTF

Page 21: The Outlook is Cloudy

Issues - Security

• low latency to other consumers' boxes decreases attacker cost and time to perform timing attacks

• nefarious, rich attackers can get on your box and slow you down or potentially compromise key generation

• See http://people.csail.mit.edu/tromer/papers/cloudsec.pdf

• DDoS on cloud providers can be very damaging to everyone in it; – Larger providers just increase the cost of the attack but reward is also high

(see http://www.theregister.co.uk/2009/10/05/amazon_bitbucket_outage/ )

• No (meaningful) security QoS• post-attack analysis challenging - in many clouds you cannot inspect a disk to see logs

without starting up machine

• Potential data security issues: who has access to physical boxes? • e.g. Amazon recommends all data on disks be encrypted

12 May 2011 EduServ 2011 Symposium 21

Page 22: The Outlook is Cloudy

Issues - Staff

• Utility cloud approaches require – Staff with more rounded understanding of end-to-end

deployment• Understand software development• Understand and can model operational behaviour• Understand security• Deployment• In service management

– Fewer staff but ones that are better qualified• …and harder to find…and harder to keep…and more expensive

12 May 2011 EduServ 2011 Symposium 22

Page 23: The Outlook is Cloudy

Don’t go cloud

• Half-heartedly– Picking up your server room and placing it in a provider– Here is my cloud kit…I have a cloud

• without knowing why you are doing it and what you want to achieve• without assessing the process and organisational changes you will need

to make it work • without knowing your applications and their behaviour• because you think you will save money quickly

– You probably won’t initially…

• and think in terms of resources– Clouds of services provide a better model of use– Integrate owned and commercial services to meet user needs

• and adopt a single vendor or provider approach– Single point of failure– your technical roadmap will be controlled by someone else– Academic applications have different characteristics to commercial applications

12 May 2011 EduServ 2011 Symposium 23

Page 24: The Outlook is Cloudy

Headline Advantages

• You own what you need to own for as long as you need to own it and it can be configured for your needs.

• Focus on application rather on the supporting resources

• Requires applications to address resilience and surviving failure

• Use and be part of a marketplace of application services

• Can enable the application to remain immune from underlying technology changes

• Develop an infrastructure that suits the application we are deploying.– The cost of ownership is pretty low.– R&D organisation we can punch above our small size and relatively small budget.– Reach out to real user groups – Experiment with great flexibility– ….. Unconstrained by (often entirely justified) corporate/academic infrastructure procedures.

• Security and procedure appropriate for the application

12 May 2011 EduServ 2011 Symposium 24

Page 25: The Outlook is Cloudy

25

Thank you

12 May 2011 EduServ 2011 Symposium