dlf fall forum 2012, tales from the cloud
DESCRIPTION
Title: Tales from the Cloud: Experiences and Challenges Event: DLF Fall Forum 2012, Nov. 4-6, Denver Colorado Desription: This panel presentation offered first-hand institutional experiences of cloud adoption when developing and working with cloud-based solutions for the purpose of digital object preservation.TRANSCRIPT
Our Bits Sit on AWSTexas Digital Library and Amazon
Ryan SteansSenior Program Coordinator, Texas Digital Library
Agenda
• Who we are• Fall 2010 – where we were• Why we looked at Amazon• Spring 2011• Today and Looking Forward
TDL DLF Presentation
What is Texas Digital Library?
TDL DLF Presentation
• 14 Member Consortium of Libraries at academic/ research institutions across Texas
• TDL provides centralized software support and development• Cost share• Provide scholarly communication
options to schools all at the same level
Texas Digital Library Services
TDL DLF Presentation
• DSpace internet repositories (DuraSpace)• Vireo ETD Management System (TDL)• Open Journal System (PKP)• Open Conference System (PKP)• WordPress for research group websites and
blogs• MediaWiki for research group data sharing and
management• Training, consultation, conference, community
TDL Activities – in the works
TDL DLF Presentation
• Digital Preservation Network• Fedora/ Hydra• Data management planning• Shared research repositories/ communities• Vireo users community (thesis and dissertation
community)
Fall 2010
TDL DLF Presentation
• TDL located on UT Austin Campus• TDL hardware at UT Data Center• Data Center move scheduled for December
2010 for all of UT campus• TDL hardware was very close to expiration
of service contracts, etc…• TDL began looking at AWS for a backup
option during the move
TDL DLF Presentation
Texas Digital Library Inventory - AustinMake Model QuantityUsage
Servers Sun SunFire 490 2Database / Application Server / DNS / LDAP / Handle / Shib
Sun Sunfire T5220 2Dspace / WWW / Blogs / Wikis / Faculty Pages / Journals / Mail / Services / Support / SVN
Sun Sunfire 245 2 Dspace Labs / DevelopmentSun Sunfire T5120 1 Preservation (iRods/SAM-QFS)
Sun Sunfire X4150 2Old content management servers (positioned to help w/ web applications )
NetworkCisco Cataylst 3750 2 Network connectivityCitrix Netscaler 2 Load BalancersBrocadeSilkworm 300E 2 SAN ConnectivityAvocentCycladed ACS 2 Serial Console Connectivity
Storage NetApp Filer 3020 2Dspace Assets / Web Application Data / Home Directories / Backup
Sun StorageTek 614 1Preservation (iRods / SAM-QFS) / Virtual Machines / Databases
Sun SL500 1 Preservation (iRods / SAM-QFS)
Texas Digital Library Inventory - College StationServers Sun T 5220 2 Dspace
Sun T 5120 1 Preservation
StorageNetApp Filer 3020 1 Dspace Assets / Backup
2010 ‐ Our existing hardware
TDL DLF Presentation
• Very locked down with space• Had to over‐buy in order to guess need• Limited staff was worrying about physical
hardware• All eggs in one basket• Tied to university security protocols that were
difficult with multiple members• IT infrastructure had led to bad user experience –
shared hardware, machines, etc…
Looking at Amazon Web Services
TDL DLF Presentation
• ELASTIC• Could increase size of a machine• Could increase number of machines• Removed guess work and bad estimates
from the equation• Elastic cost
• Service Contracts were highly reasonable• Road tested. AWS launched in 2006.
Spring 2011
TDL DLF Presentation
• We moved to AWS for all services
Why?
TDL DLF Presentation
• Allowed us to separate service instances, bring each up on a VM
• Created far greater user customization/ access/ control
• Rapid deployment of services from master image
• Accounting
How?
TDL DLF Presentation
• Set up a duplicate system in AWS• Used load balancer as the fulcrum• Caused a minimum of downtime
Today
TDL DLF Presentation
• Running 100 separate AWS instances as of 11/19/2012
• Deploying services easily• Literally nothing is on UT’s hardware nor at
any TDL member school• Depend on standard security and AWS• Cost
TDL DLF Presentation
It’s actually working just fine
TDL DLF Presentation
• A bit of “beg forgiveness instead of asking permission”
• Costs are down• Members don’t notice• Our staff is handling it well (2 longtime
university IT people as sys admins)
An overly simple idea of service provisioning
TDL DLF Presentation
But…
TDL DLF Presentation
• Network cost could become burdensome• Cost savings per unit no longer as obvious• UT and our own team more comfortable
with VMs and could move to purchased hardware
• AWS billing not easy to manage month‐to‐month
And…
TDL DLF Presentation
• As we look at the Digital Preservation Network ‐ we will rebuild on campus• Security concerns• Goal of group is to do this in academia• It may also affect how we think of
implementing Fedora/ Hydra
Week of October 22nd…
TDL DLF Presentation
• TDL learned of outage at 1:15 PM Central• Services were visible, but not accessible• Contacted members• Watched logs, news reports, etc…• 7:30 PM we were back up and running
with no data loss• Received 7 communications through
helpdesk
It’s bad, but…
TDL DLF Presentation
• AWS was in communication with users• We feel confident in them, and they got it
back up• Checking our systems was fairly easy• Took some pressure off our staff• Outages are not limited to AWS
Looking forward
TDL DLF Presentation
• We will always rethink our strategy on AWS
• AWS is providing us with a sales person and a consultant before end of year
• Cost has to remain reasonable for uploading, downloading
• Will be moving preservation efforts to campus
Ryan Steans, Program CoordinatorKristi Park, Program Coordinator
Andrew Woodshttp://duracloud.org
04 Nov 2012
Tales from the Cloud: Experiences and Challenges
2
Why might I even be interested in “the cloud”
?
3
What are some risks
?
4
How is it different from local infrastructure
?
5
What about this “DuraCloud”
?