aws webcast - library systems on the aws cloud

35
Library Workloads on Amazon Web Services May 27 th , 2015 Sri Elaprolu Manager, Solutions Architecture Worldwide Public Sector

Upload: amazon-web-services

Post on 23-Jul-2015

1.197 views

Category:

Technology


0 download

TRANSCRIPT

Page 1: AWS Webcast - Library Systems on the AWS Cloud

Library Workloads on Amazon

Web Services

May 27th, 2015

Sri Elaprolu – Manager, Solutions Architecture

Worldwide Public Sector

Page 2: AWS Webcast - Library Systems on the AWS Cloud

AWS Overview

Library Workloads and Use Cases

Getting Data into AWS

Storage Services

Transcoding Service

Content Delivery Service

Q & A

Agenda

Page 3: AWS Webcast - Library Systems on the AWS Cloud

What is AWS?

Page 4: AWS Webcast - Library Systems on the AWS Cloud

Application Services

Compute Storage Databases

Networking

AWS Global Infrastructure

Deployment & Administration

Amazon Web Services

Page 5: AWS Webcast - Library Systems on the AWS Cloud

11 Regions

29 Availability Zones

53 Edge locations

AWS Global Infrastructure

Customer Decides Where Applications and Data Reside

Page 6: AWS Webcast - Library Systems on the AWS Cloud

AWS Availability Zone (AZ) View

- Multiple Isolated locations within a Region

- Availability Zone = 1 or more “data center”

- Independent Failure Zone

- Physically separated

- On separate Low Risk Flood Plains

- Discrete UPS

- Onsite backup generation facilities

- Fed from different segments of utility provider

- Redundantly connected to multiple tier-1 ISP’s

- No “Disaster Recovery Datacenter”

- Built for Continuous Availability

- Customer decides Availability Zone for Compute

Availability

Zone AAvailability

Zone B

Availability

Zone C

Sample US Region

~ Data Center

Page 7: AWS Webcast - Library Systems on the AWS Cloud

Architected for Government Security Requirements

http://aws.amazon.com/security

Certifications and accreditations for workloads that matter

AWS CloudTrail and AWS Config - Call logging and configuration management for governance & compliance

• Log, review, alarm on all user actions

• Browse and query database of current and previous state of cloud resources

Page 8: AWS Webcast - Library Systems on the AWS Cloud

Experience

Since 2006 supporting

large numbers of

customers across 190

countries

Innovation

Rapid delivery of new

services and features based

on customer feedback

Robust Platform

Number of services and

features, virtually to

support every use case

imaginable

Simple Pricing

Philosophy

48 Price reductions

Expect more reductions

in the future

Global Footprint

11 Regions

29 Availability Zones

53 Edge Locations

Eco system

Thousands of partners

(ISV, SI, consulting)

23 categories; 2100 apps

in Marketplace

AWS Differentiators

Page 9: AWS Webcast - Library Systems on the AWS Cloud

Library Workloads

Page 10: AWS Webcast - Library Systems on the AWS Cloud

Library Use Cases

Online Public

Access Catalogs

Library Catalogs

Online databases

Institutional

Repositories

Online Archive

Intellectual Output

Digital Asset

Storage

Protect from Loss and

Degradation

Offsite Storage

Redundancy and

Durability

Backups

Offsite

Redundant

Development Space

Disposable Environments

Start and Stop Frequently

Page 11: AWS Webcast - Library Systems on the AWS Cloud

Dspace

Open Journal Systems

Open Conference Systems

Thesis and Dissertation Systems

Web Properties – WordPress

DuraCloud Preservation System

• Consortium of higher education institutions in

Texas that has provided shared digital library

services since 2005

• The mission of the Texas Digital Library (TDL) is

to enable each of its member libraries to advance

a program of digital initiatives in support of

research, scholarship, and learning.

Page 12: AWS Webcast - Library Systems on the AWS Cloud

Getting Data into AWS

Page 13: AWS Webcast - Library Systems on the AWS Cloud

Data Ingestion Options

AWS Direct ConnectDedicated bandwidth between

your site and AWS

InternetTransfer data in a secure SSL tunnel over the

public Internet

AWS Import/ExportPhysical transfer of media into and

out of AWS

Page 14: AWS Webcast - Library Systems on the AWS Cloud

AWS Ingestion Options - Internet

1. Multipart upload

2. Request rate optimization

3. TCP window scaling

4. TCP selective

acknowledgement

AWS has customers that ingest roughly 1 PB per day

Page 15: AWS Webcast - Library Systems on the AWS Cloud

AWS Ingestion Options - AWS Direct Connect

• Private connectivity to AWS– Physical connection – 1 Gbps or 10 Gbps port

• Consistent network performance

• Consider burst models on ingest

• Reduces costs for bandwidth-heavy outbound workloads

• US Locations

• CoreSite 32 Avenue of the Americas, NY

• CoreSite One Wilshire & 900 North Alameda, LA

• Equinix DC1 – DC6 & DC10 - DC11, Ashburn, VA

• Equinix SV1 & SV5, San Jose, CA

• Equinix SE2 & SE3, Seattle, WA

Page 16: AWS Webcast - Library Systems on the AWS Cloud

AWS Ingestion Options - AWS Import/Export

• Rapidly move data into

and out of AWS

• Portable storage device

shipment to AWS

• Supports– Amazon EBS

– Amazon S3

– Amazon Glacier

• Use cases– Initial data migration

– Content distribution via portable

devices

– Disaster recovery

Page 17: AWS Webcast - Library Systems on the AWS Cloud

Amazon Storage Services

Page 18: AWS Webcast - Library Systems on the AWS Cloud

Amazon Simple Storage Service (S3)Highly scalable object storage

1 byte to 5 TB in size

99.999999999% durability

Amazon Elastic Block Store (EBS)High-performance block storage device

1 GB to 16 TB in size

Mount as drives to instances with

snapshot/cloning functionalities

Magnetic and General Purpose SSD

Amazon GlacierLong-term object archive

Extremely low cost per gigabyte

99.999999999% durability

AWS Storage and Archive Options

Page 19: AWS Webcast - Library Systems on the AWS Cloud

Amazon Elastic Block Store (EBS)

• High I/O block storage for Amazon

EC2

• Point-in-time snapshots to Amazon S3• 99.999999999% Durability

• Snapshot software is FREE

• Point-in-time snapshots across

regions

Page 20: AWS Webcast - Library Systems on the AWS Cloud

Amazon Simple Storage Service (S3)

• Durable and low cost

• Unlimited number of objects and volume

• Back up to Amazon S3 buckets via

HTTP/HTTPS

– Create scripts using PowerShell,

Perl, Python…

– Numerous solutions for data backup

• Authentication mechanisms ensure data

is kept secure

• Reduced redundancy storage (RRS)

option

Page 21: AWS Webcast - Library Systems on the AWS Cloud

Amazon Glacier

• $0.01 per GB/mo, $120 per TB/yr

• 3-5 hour data retrieval latency

• Archives: single file or zipped files

• Vaults: collection of archives

• Infinite archival storage

• 99.999999999% durability

• Immutable, encrypted by default

Page 22: AWS Webcast - Library Systems on the AWS Cloud

Object Life Cycle Management Amazon S3 → Amazon Glacier

• Seamlessly move data from Amazon S3 → Amazon Glacier

• 3-5 hour asynchronous retrieval

• Data lifecycle policies

• $0.01 per GB for Amazon Glacier costs

Page 23: AWS Webcast - Library Systems on the AWS Cloud

Why AWS for Storage and Archiving?

• Protect digital content from fragility

• Protect digital assets from loss and degradation

• Promote learning

• Share research

Page 24: AWS Webcast - Library Systems on the AWS Cloud

TCO: On-Premise Cost Considerations

1. Primary storage hardware (primary / remote site)

2. Storage growth (cost of upgrades)

3. Storage management software and 3rd party tools

4. Professional services

5. Hardware maintenance

6. Software maintenance

7. Backup software

8. Backup hardware (primary / remote site)

9. Offsite tape storage / vault

10. Archive software

11. Archive hardware

12. Power

13. Cooling

14. Space

15. Labor

16. Cost of capital

17. Training

18. Asset depreciation

19. Migration

20. Decommission / remove

21. Recycle

22. …

Page 25: AWS Webcast - Library Systems on the AWS Cloud

Storage on AWS

10 TB S3 = $ 3,631.20 per YEAR

5 TB S3 | 5 TB Glacier = $ 2,433.12 per YEAR

10 TB Glacier = $ 1,228.80 per YEAR

Price based on US-EAST-1 region; correct as of May 22nd, 2015

Page 26: AWS Webcast - Library Systems on the AWS Cloud

Amazon Elastic

Transcoder

Page 27: AWS Webcast - Library Systems on the AWS Cloud

• Managed Transcoding Service built on EC2

– No software to buy or manage

– No need to manage capacity

– Seamless integration with AWS S3 and Amazon CloudFront CDN

• Transcode Content for Any Device

– Select from over 30 transcode presets

– Define up to 50 presets per AWS Account!

• Process jobs in parallel and on demand

Amazon Elastic Transcoder: Overview

Page 28: AWS Webcast - Library Systems on the AWS Cloud

• Self-Service Control

• Un-Matched On Demand Capacity

• Industry Leading Reliability

• Lowest Cost Transcoding Service

• Highly Secure

• Global Availability

• Rapidly Releasing New Features

Amazon Elastic Transcoder: Why Customers prefer the service

Page 29: AWS Webcast - Library Systems on the AWS Cloud

Pipeline 1 Pipeline 2

Input Bucket 1 Input Bucket 2

Output BucketPipelines, jobs, & outputs

ALL run in parallel

:

:

:

:

:

:

:

Job N (Progressing)

Job N+1 (Complete)

Job N+2 (Progressing)

:

:

:

:

:

:

:

:

:

:

Job M (Progressing)

Job M+1 (Progressing)

Job N+3 (Progressing)

Job N+4 (Submitted)

SNS Topic

Amazon Elastic Transcoder: How it works?

Page 30: AWS Webcast - Library Systems on the AWS Cloud

Amazon Content Delivery

Service

Page 31: AWS Webcast - Library Systems on the AWS Cloud

• Full Feature Caching Network

• Global Infrastructure

• Tuned for Optimal Performance

• Massively Scalable

• Highly Secure

• Robust Analytics

• Self Service

• Priced to Minimize Cost

Amazon CloudFront: Content Delivery Network

Page 32: AWS Webcast - Library Systems on the AWS Cloud

• Media and Entertainment

• Gaming

• Digital Catalogs

• Digital Advertising

• Software Downloads

• Dynamic Websites and Applications

Amazon CloudFront: For Any Market Segment

Page 33: AWS Webcast - Library Systems on the AWS Cloud

Video StreamingOn-demand & Live Streaming

RTMP (Flash) and HTTP(S)

Adaptive Bitrate Live Streaming

Microsoft Smooth Streaming

Whole Site DeliveryStatic & Dynamic Content

Mobile Detect, CORS Support

Multiple Cache Behaviors

Multiple Origin Servers

SecurityPrivate Content (Signed URLs)

Custom SSL (Dedicated IP & SNI)

Geo Restriction

HTTP to HTTPS Redirect

High Availability99.9% SLA

Automatic Origin Failover

Custom Error Pages

Serve Stale Content when Origin unavailable

High PerformanceLatency Based Routing

TCP Optimization

Persistent Connections

EDNS Client Subnet

Low TCOPay for use

Commit-Based lower pricing

Price Classes

Preferential Pricing for AWS origins

Amazon CloudFront: Popular Features

Page 34: AWS Webcast - Library Systems on the AWS Cloud

Dynamic

StaticVideo

User

Input

SSL

Amazon CloudFront: Deliver all your site

Page 35: AWS Webcast - Library Systems on the AWS Cloud

Thank You!