Intro to AWS: Storage Services

Download Intro to AWS: Storage Services

Post on 15-Jul-2015




1 download


2015, Amazon Web Services, Inc. or its affiliates. All rights reservedIntro to AWS: Storage ServicesGuy Farber, AWS Storage Business DevelopmentAWS storage options Scalable object storage Inexpensive archive storage Persistent block storage Shared file system Turnkey gateway solutionWe are constantly producing more dataFrom all types of industriesObject StorageAmazon S3Simple Storage Service2012 2013 2014 2015102% year-over-year increase in data transfer to and from S3(Q4 2014 vs Q4 2013, not including Amazon use)S3 usage99.999999999%durabilityNice try kidReduced redundancy option99.99% saves ~20%S3 resources: buckets and objectsS3 website: static content1 PB raw storage800 TB usable storage600 TB allocated storage400 TB application dataS3 capacity pricingpay only for what you use!Amazon S3S3 continuous cost reduction Available through 11 regions globally Priced at per GB-month rates 8 price reductions since launch 51% average S3 capacity fee reduction on 4/1/2014 TCO: comparing on-premises to S3 Can be challenging for some customers We can help!S3 Amazon EMR Amazon S3New FeaturesS3 event notificationsDelivers notifications to Amazon SNS, Amazon SQS, or AWS Lambda when events occur in S3S3EventsSNS topicSQS queueLambda functionNotificationsFoo() {} Preserve, retrieve, and restore every version of every object stored in your bucket S3 automatically adds new versions and preserves deleted objects with delete markers Easily control the number of versions kept by using lifecycle expiration policies Easy to turn on in the AWS Management ConsoleS3 versioningS3 cross-region replicationAutomated, fast, and reliable asynchronous replication of data across AWS regionsSource(Virginia)Destination(Oregon) Only replicates new PUTs. Once S3 is configured, all new uploads into a source bucket will be replicated Entire bucket or prefix based 1:1 replication between any 2 regions Versioning requiredUse cases: Compliancestore data hundreds of miles apart Lower latencydistribute data to regional customers) Securitycreate remote replicas managed by separate AWS accountsCross Region Replication DemoS3 use cases Web-scale storage capacity and performance for web applications Single-origin store with delivery through Amazon CloudFront Staging and persistent store for Big Data applications Storage target for backup and active archiveAmazon GlacierLow-cost archiving service$0.01per GB-month$120per TB/year99.999999999%durability35 hoursdata retrievalStorage costsvs.retrieval costsGlacier benefits Reduce cost for long-term archiving Leverage unlimited storage capacity Replace tape museums Improve durabilityAmazon S3 Glacier integrationPolicy-based archiving serviceS3 lifecycle policies Key prefix logs/Transition objects to Glacier 30 days after creationDelete 365 days after creation datearchive-in-30-dayslogs/Enabled30GLACIER365SoundCloudleveraging Glacier for audio transcoding Worlds leading social sound platform Audio files must be transcoded and stored in multiple formatsS3GlacierUse cases for S3 lifecycle policies Data tiering in the cloud Manage object versioning for logical data protection Deletion of data from Glacier by policyBlock StorageInstance storageAmazon EBSElastic Block Storage1GB16 TBEBSGeneral Purpose (SSD)Up to 16 TB10,000 IOPSUp to 160 MbpsProvisioned IOPS (SSD)Up to 16 TB20,000 IOPSUp to 320 MbpsPrice PerformanceEBSMagnetic General Purpose Provisioned IOPSUse cases Infrequent data accessBoot volumesSmall to med DBsDev and TestI/O intensiveRelational DBsNoSQL DBsStorage media Magnetic disk-backed SSD-backed SSD-backedMax IOPS 40200 IOPS 10,000 IOPS 20,000 IOPSLatency (random read)2040 ms 12 ms 12 msAvailability Designed for 99.999% Designed for 99.999% Designed for 99.999%Price$.05/GB-month$.05/million I/O$.10/GB-month$.125/GB-month$.065/provisioned IOPSAmazon EBSsnapshotsEBS use cases Persistent block storage for Amazon EC2 Transactional workloads File system for an instanceNTFS, ExtFS, and so onIntroducing Amazon Elastic File System Cloud-based file systemWhat is EFS? Fully managed file system for EC2 instances Provides standard file system semantics Elastically grows to petabyte scale Delivers performance for a wide variety of workloads Highly available and durablesimple elastic scalable1 2 3EFS is designed for a broad range of use cases, such as Content repositories Development environments Home directories Big DataEFS is simple Fully managed No hardware, network, or file layer Create a scalable file system in seconds! Seamless integration with existing tools and apps NFS v4open and non-proprietary Standard file system semantics Works with standard OS file system APIs Simple pricing = simple forecasting EFS price: $0.30/GB-month1EFS is elastic File systems grow and shrink automatically as you add and remove files No need to provision storage capacity or performance You pay only for the storage space you use, with no minimum fee2 File systems can grow to petabyte scale Throughput and IOPS scale automatically as file systems grow Consistent low latencies regardless of file system size Support for thousands of concurrent NFS connectionsEFS is scalable3How does it all fit together?AVAILABILITY ZONE 1REGIONAVAILABILITY ZONE 2AVAILABILITY ZONE 3VPCEC2EC2EC2EC2Customers file systemWhy does this matter? app owners and developers?to your business? Easy to move existing code/applications/tools used today with existing NFS servers to the AWS cloud Simple shared file storage solution for new cloud-native applications Predictable pricing with no up-front investment Increased agility Spend less time managing file storage and more time focusing on your businessto IT administrators? Eliminates need to manage and maintain file system storage at scaleAmazon Storage GatewayStorage GatewayYour on-ramp to AWS cloud storage: Back up into S3 DR on EC2 Archive into Amazon Glacier iSCSI or VTL interfaceSummary: the Amazon storage portfolioS3 Object storage: data presented as buckets of objects Data access by using APIs over the InternetEFS File storage (analogous to NAS): data presented as a file system Shared low-latency access from multiple EC2 instancesGlacier Archival storage: data presented as vaults/archives of objects Lowest-cost storage, infrequent access by using APIs over the InternetStorage Gateway Back up and archive data into S3 and Amazon GlacierEBS Block storage (analogous to SAN): data presented as disk volumes Lowest-latency access from single EC2 instancesSAN FRANCISCO


View more >