aws.amazon.com/webinars/apac/webinar-week | #AWSWebinarWeek
Understanding AWS Storage OptionsOlivier KleinAWS Solutions Architect
v
Agenda
• AWS Infrastructure• Storage Options• Data Transfer• Live Demo
vOregonBeijing
Tokyo
Singapore
IrelandGovCloud
NorthernCalifornia
Sydney
São Paulo
Frankfurt
N. Virginia
AWS Global Infrastructure
• 11 Regions• 30 Availability Zones• 53 Edge Locations
Continuous Expansion
v
Core Services Compute Storage Database Networking
Infrastructure RegionsAvailability Zones Edge Locations
Platform Services
Analytics App Deployment Mobile
Virtual Desktops
Collaboration & Sharing App Delivery E-Mail
Access Control Auditing Monitoring EncryptionSecurity
ApplicationsAPI
&
SDKs
Web Services
v
Core Services Compute Storage Database Networking
Infrastructure RegionsAvailability Zones Edge Locations
Platform Services
Analytics App Deployment Mobile
Virtual Desktops
Collaboration & Sharing App Delivery E-Mail
Access Control Auditing Monitoring EncryptionSecurity
ApplicationsAPI
&
SDKs
Web Services
v
AWS Storage Options
• Object Storage• Archival Storage• Block Storage• File Storage Service• Gateway Solution
v
Object Storage
v
Simple Storage Service
• Highly scalable, fully managed object storage• Virtually unlimited storage capacity - no
provisioning needed• 1 byte to 5 TB in size per object• Designed for 99.999999999% durability• Replicated across 3 facilities• Different pricing tiers for hot (Standard), warm
(Standard - IA) or cold storage (Glacier)• Trillions of unique customer objects• Millions of transactions per second
v
Simple Storage Service
1 PB raw storage
800 TB usable storage
600 TB allocated storage
400 TB application data
Traditional Storage Amazon S3
Pay only for what you use!
v
S3 – Buckets and Objects
• Stored in buckets• Versioning• Access control lists and bucket policies• AES-256 bit encryption at rest• Private by default• Addressable over the internet if public• Allows event notifications to trigger
further action• Enable cross-region replication with a
click of a button• Ideal for images, videos, application data,
backups and more
v
S3 – Static Content Website
• S3 becomes your static webserver• Offload static content to S3 and run
dynamic content on other services such as EC2 or Elastic Beanstalk
v
S3 – When to use?
Use Amazon S3 when you need:• Web-scale storage capacity and performance for
web applications• High data durability• Storage for log files• Storage for backup and active archives• Origin store for Content Delivery Networks such
as Amazon CloudFront• Ingestion point or data lake for Big Data
applications
Amazon S3
v
• Uses Amazon S3 to store daily logs and static assets such as images
• Generates more than 1.2TBs of logs per day
• Big Data - Runs ~250 Hadoop jobs per day, processing more than 30TBs of data
v
Archival Storage
v
Amazon Glacier
• Long term low-cost archiving service• Optimal for infrequently accessed data• 99.999999999% durability• 3-5 hours retrieval time• $0.007 per GB / month• $120 per TB / year
v• Provides secure backup and recovery solutions for SaaS applications including Salesforce.com and Google Apps
• Over 200,000 customers• Stores multiple petabytes of data in
Amazon S3 and Amazon Glacier• Choose AWS to build a scalable backup
storage platform without requiring significant initial investment
Use Case: Backupify
v
Amazon S3 and Amazon Glacier IntegrationPolicy-based archiving service
v
Amazon S3 Lifecycle Policies
• S3 Lifecycle policies allow to transition, archive or permanently delete objects based on age
• Set rules per S3 bucket• Apply rule to whole bucket or on a
specific prefix (e.g. logfiles/)
• Example:• Transition to S3-IA after 7 days• Move object to Glacier after 30 days• Delete object after 365 days
v
v
Block Storage
v
EC2 Instance Storage
• Every EC2 comes with instance storage• Local, direct attached resource• Size is based on EC2 instance type• Storage optimized instances for up to
365,000 r/s and 315,000 w/s• Choice of SSD or magnetic
It’s volatile!• No persistence• All data is automatically deleted
when an EC2 instance stops, fails or is terminated
v
Elastic Block Store
• High performance block storage• Persistent block storage• 1GB to 16TB in size per volume• Mount as drives to EC2 instances• SSD or Magnetic• Burstable or provisioned throughput
* Upcoming: Larger and faster EBS volumes for up to 16TB
v
Elastic Block Store
• You can attach multiple EBS volumes• RAID to increase performance or storage
capacity
v
Elastic Block Store
• You can reattach your EBS volume to a new instance
• Data is persisted
v
Elastic Block Store
• You can snapshot your EBS volume into our highly durable storage service
• Create new EBS volumes from snapshots or clone drives
v
EBS Performance
• EBS Magnetic• 40-200 IOPS
• EBS General Purpose• SSD backed• 3 IOPS / GB• Burstable to 10,000 IOPS
• EBS Provisioned IOPS• SSD backed• Up to 20,000 IOPS consistently
IOPS = I/O per second for up to 256KB blocks
v
File Storage Service
v
Amazon Elastic File System
• Fully managed file system for EC2 instances
• Sharable across multiple EC2 instances
• Automatically grows or shrinks on petabyte scale – no provisioning required
• Pay only for what you use
• Highly available and durable
• NFS v4-based• Works with standard OS file systems
• Throughput and IOPS automatically scale
• Consistent low latency regardless of file system size
vAmazon EFS is designed for a broad range of use cases, such as:
• Content repositories• Development environments• Home directories• Data lakes (Big Data)
Amazon Elastic File System
v
Gateway Solution
v
Gateway Solution
v
AWS Storage Gateway
• Connect an on-premises software appliance to provide integration with Amazon S3
• Supports three configuration• Gateway-Cached Volumes• Gateway-Stored Volumes• Gateway-Virtual Tape Library (VTL)
v
AWS Storage Gateway
• Cached – Data is stored in Amazon S3 with frequently accessed files kept locally
• Stored – Asynchronous point-in-time backup snapshots to Amazon S3
Servers
On-premises Host
AWS Storage
Gateway VM
Direct Attached or Storage Area Network Disks
iSCSI
SSL
Amazon Simple Storage Service
(S3)
AWS Storage Gateway Service
v
AWS Storage Gateway
• VTL– Expose an industry standard virtual tape library
• Write to tape S3• Put in shelf Glacier
Backup Application On-premises
Host
Tape DriveiSCSI
SSL Amazon S3
AWS Storage Gateway ServiceMedia
ChangeriSCSI
Amazon Glacier
v
AWS Storage Gateway
• Backup data into Amazon S3• Disaster Recovery of applications to EC2• Archive into Amazon Glacier
v
Use Case: Disaster Recovery
v
Use Case: Disaster Recovery
v
Use Case: Disaster Recovery
v
AWS Direct ConnectDedicated bandwidth between your
site and AWS
AWS Import/ExportPhysical transfer of media into and out of
AWS
How to get my data into AWS?
InternetTransfer data through secure encrypted
tunnel over the public internet
v
What is Snowball? Petabyte scale data transport
E-ink shipping label
Ruggedizedcase
“8.5G Impact”
All data encrypted end-to-end
50 TB10G network
Rain & dust resistant
Tamper-resistant case & electronics
v
How does Snowball work?
v
Reduce costs Reduce on-premise
Change processes
Reduce CAPEX while dramatically increasing scalability
Eliminate the need for secondary sites
Eliminate on premise equipment to manage archives
Consolidate on-premise and augment with cloud
Eliminate capacity planning
Eliminate provisioning for peak demand
Remove aging technologies
Remove tape archives
Cycle out aging disk arrays
Why AWS for storage?
v
Demo Time!
Online Labs & Training
Gain confidence and hands-on experience with AWS.
Watch free Instructional Videos and explore Self-Paced Labs
Instructor Led Classes
Learn how to design, deploy and operate highly available, cost-
effective and secure applications on AWS in courses led by qualified
AWS instructors
Validate your technical expertise with AWS and use practice exams to help you
prepare for AWS Certification
AWS Certification
More info at http://aws.amazon.com/training