aws apac webinar week - understanding aws storage options

aws.amazon.com/webinars/apac/webinar-week | #AWSWebinarWeek

Understanding AWS Storage OptionsOlivier KleinAWS Solutions Architect

v

Agenda

• AWS Infrastructure• Storage Options• Data Transfer• Live Demo

vOregonBeijing

Tokyo

Singapore

IrelandGovCloud

NorthernCalifornia

Sydney

São Paulo

Frankfurt

N. Virginia

AWS Global Infrastructure

• 11 Regions• 30 Availability Zones• 53 Edge Locations

Continuous Expansion

v

Core Services Compute Storage Database Networking

Infrastructure RegionsAvailability Zones Edge Locations

Platform Services

Analytics App Deployment Mobile

Virtual Desktops

Collaboration & Sharing App Delivery E-Mail

Access Control Auditing Monitoring EncryptionSecurity

ApplicationsAPI

&

SDKs

Web Services

v

Core Services Compute Storage Database Networking

Infrastructure RegionsAvailability Zones Edge Locations

Platform Services

Analytics App Deployment Mobile

Virtual Desktops

Collaboration & Sharing App Delivery E-Mail

Access Control Auditing Monitoring EncryptionSecurity

ApplicationsAPI

&

SDKs

Web Services

v

AWS Storage Options

• Object Storage• Archival Storage• Block Storage• File Storage Service• Gateway Solution

v

Object Storage

v

Simple Storage Service

• Highly scalable, fully managed object storage• Virtually unlimited storage capacity - no

provisioning needed• 1 byte to 5 TB in size per object• Designed for 99.999999999% durability• Replicated across 3 facilities• Different pricing tiers for hot (Standard), warm

(Standard - IA) or cold storage (Glacier)• Trillions of unique customer objects• Millions of transactions per second

v

Simple Storage Service

1 PB raw storage

800 TB usable storage

600 TB allocated storage

400 TB application data

Traditional Storage Amazon S3

Pay only for what you use!

v

S3 – Buckets and Objects

• Stored in buckets• Versioning• Access control lists and bucket policies• AES-256 bit encryption at rest• Private by default• Addressable over the internet if public• Allows event notifications to trigger

further action• Enable cross-region replication with a

click of a button• Ideal for images, videos, application data,

backups and more

v

S3 – Static Content Website

• S3 becomes your static webserver• Offload static content to S3 and run

dynamic content on other services such as EC2 or Elastic Beanstalk

v

S3 – When to use?

Use Amazon S3 when you need:• Web-scale storage capacity and performance for

web applications• High data durability• Storage for log files• Storage for backup and active archives• Origin store for Content Delivery Networks such

as Amazon CloudFront• Ingestion point or data lake for Big Data

applications

Amazon S3

v

• Uses Amazon S3 to store daily logs and static assets such as images

• Generates more than 1.2TBs of logs per day

• Big Data - Runs ~250 Hadoop jobs per day, processing more than 30TBs of data

v

Archival Storage

v

Amazon Glacier

• Long term low-cost archiving service• Optimal for infrequently accessed data• 99.999999999% durability• 3-5 hours retrieval time• $0.007 per GB / month• $120 per TB / year

v• Provides secure backup and recovery solutions for SaaS applications including Salesforce.com and Google Apps

• Over 200,000 customers• Stores multiple petabytes of data in

Amazon S3 and Amazon Glacier• Choose AWS to build a scalable backup

storage platform without requiring significant initial investment

Use Case: Backupify

v

Amazon S3 and Amazon Glacier IntegrationPolicy-based archiving service

v

Amazon S3 Lifecycle Policies

• S3 Lifecycle policies allow to transition, archive or permanently delete objects based on age

• Set rules per S3 bucket• Apply rule to whole bucket or on a

specific prefix (e.g. logfiles/)

• Example:• Transition to S3-IA after 7 days• Move object to Glacier after 30 days• Delete object after 365 days

v

v

Block Storage

v

EC2 Instance Storage

• Every EC2 comes with instance storage• Local, direct attached resource• Size is based on EC2 instance type• Storage optimized instances for up to

365,000 r/s and 315,000 w/s• Choice of SSD or magnetic

It’s volatile!• No persistence• All data is automatically deleted

when an EC2 instance stops, fails or is terminated

v

Elastic Block Store

• High performance block storage• Persistent block storage• 1GB to 16TB in size per volume• Mount as drives to EC2 instances• SSD or Magnetic• Burstable or provisioned throughput

* Upcoming: Larger and faster EBS volumes for up to 16TB

v

Elastic Block Store

• You can attach multiple EBS volumes• RAID to increase performance or storage

capacity

v

Elastic Block Store

• You can reattach your EBS volume to a new instance

• Data is persisted

v

Elastic Block Store

• You can snapshot your EBS volume into our highly durable storage service

• Create new EBS volumes from snapshots or clone drives

v

EBS Performance

• EBS Magnetic• 40-200 IOPS

• EBS General Purpose• SSD backed• 3 IOPS / GB• Burstable to 10,000 IOPS

• EBS Provisioned IOPS• SSD backed• Up to 20,000 IOPS consistently

IOPS = I/O per second for up to 256KB blocks

v

File Storage Service

v

Amazon Elastic File System

• Fully managed file system for EC2 instances

• Sharable across multiple EC2 instances

• Automatically grows or shrinks on petabyte scale – no provisioning required

• Pay only for what you use

• Highly available and durable

• NFS v4-based• Works with standard OS file systems

• Throughput and IOPS automatically scale

• Consistent low latency regardless of file system size

vAmazon EFS is designed for a broad range of use cases, such as:

• Content repositories• Development environments• Home directories• Data lakes (Big Data)

Amazon Elastic File System

v

Gateway Solution

v

Gateway Solution

v

AWS Storage Gateway

• Connect an on-premises software appliance to provide integration with Amazon S3

• Supports three configuration• Gateway-Cached Volumes• Gateway-Stored Volumes• Gateway-Virtual Tape Library (VTL)

v

AWS Storage Gateway

• Cached – Data is stored in Amazon S3 with frequently accessed files kept locally

• Stored – Asynchronous point-in-time backup snapshots to Amazon S3

Servers

On-premises Host

AWS Storage

Gateway VM

Direct Attached or Storage Area Network Disks

iSCSI

SSL

Amazon Simple Storage Service

(S3)

AWS Storage Gateway Service

v

AWS Storage Gateway

• VTL– Expose an industry standard virtual tape library

• Write to tape S3• Put in shelf Glacier

Backup Application On-premises

Host

Tape DriveiSCSI

SSL Amazon S3

AWS Storage Gateway ServiceMedia

ChangeriSCSI

Amazon Glacier

v

AWS Storage Gateway

• Backup data into Amazon S3• Disaster Recovery of applications to EC2• Archive into Amazon Glacier

v

Use Case: Disaster Recovery

v

Use Case: Disaster Recovery

v

Use Case: Disaster Recovery

v

AWS Direct ConnectDedicated bandwidth between your

site and AWS

AWS Import/ExportPhysical transfer of media into and out of

AWS

How to get my data into AWS?

InternetTransfer data through secure encrypted

tunnel over the public internet

v

What is Snowball? Petabyte scale data transport

E-ink shipping label

Ruggedizedcase

“8.5G Impact”

All data encrypted end-to-end

50 TB10G network

Rain & dust resistant

Tamper-resistant case & electronics

v

How does Snowball work?

v

Reduce costs Reduce on-premise

Change processes

Reduce CAPEX while dramatically increasing scalability

Eliminate the need for secondary sites

Eliminate on premise equipment to manage archives

Consolidate on-premise and augment with cloud

Eliminate capacity planning

Eliminate provisioning for peak demand

Remove aging technologies

Remove tape archives

Cycle out aging disk arrays

Why AWS for storage?

v

Demo Time!

Online Labs & Training

Gain confidence and hands-on experience with AWS.

Watch free Instructional Videos and explore Self-Paced Labs

Instructor Led Classes

Learn how to design, deploy and operate highly available, cost-

effective and secure applications on AWS in courses led by qualified

AWS instructors

Validate your technical expertise with AWS and use practice exams to help you

prepare for AWS Certification

AWS Certification

More info at http://aws.amazon.com/training