Red Hat Storage Server for AWS Craig Carl
Solution Architect, Amazon Web Services Tushar Katarki
Principal Product Manager, Red Hat
Veda Shankar Principal Technical Marketing Manager, Red Hat
GlusterFS Community Project l Community driven Open Source @gluster.org l Distributed file system on top of local file system l No metadata server with elastic hashing algorithm l Spreads files across local file system directories called bricks l Stores metadata in extended attributes of local file system l Modular stackable architecture based on user-space FUSE
What is Red Hat Storage l Open, Software Defined Storage Solution for On-Premise,
Virtualized and Cloud environments l Enterprise implementation of GlusterFS technology l RHEL + XFS + GlusterFS l Support Subscription l 24x7 Premium Support
RED HAT STORAGE CONCEPTS
VOLUME
A namespace presented as a
POSIX mount point and is comprised of bricks.
BRICK
The basic unit of storage, represented by an export
directory on a server
SERVER/NODES
Contain the bricks
Elastic Hash Algorithm
• No central metadata • No Performance Bottleneck • Eliminates risk scenarios • Location hashed intelligently on filename • Unique identifiers, similar to md5sum • The “Elastic” Part • Files assigned to virtual volumes • Virtual volumes assigned to multiple bricks
Why run RHS on AWS?
You need shared access to a file system GlusterFS, NFS and CIFS You need a really large file system I’ve created multiple 2PB clusters
Things to think about -
Run at least replica 2 in two availability zone Single AZ = no durability commitments Dual AZ = 99.95% available Durability
Backing store Performance
Network interface EBS interface EBS performance
Backing stores
EBS between 0.1% – 0.5% AFR per volume
Ephemeral
hs1.8xlarge 48 terabytes of storage across 24 hard disk drives
i2.8xlarge ~5.7 terabytes of storage across 8 SSDs
Running RHS on EC2
Use Reserved Instances! RHS is a perfect use case for reserved instances. Discounts of up to 65% Purely a billing construct, no technical implications. m3.xlarge is perfect for RHS @ $.25/hr (1yr heavy RI)
VOLUME CONFIGURATION
l Global namespace l Bricks are XFS filesystems l Volumes are made of bricks l Distributed l Replicated l Distributed and replicated l Geo-replication
UNIFIED FILE AND OBJECT Object storage • Objects • Containers • Accounts Standards based • OpenStack Swi5 interface • GlusterFS underneath • RESTful • APIs NAS protocol support • NFS / CIFS / GlusterFS • No translaCon required
Clients talk directly to necessary nodes
DATA FLOW: Gluster native client
Red Hat Storage User Perspective (distributed volumes)
server1:/exp1 server2:/exp1
DISTRIBUTED VOLUME
FILE 1 FILE 2 FILE 3
BRICK BRICK
MOUNT POINT server1:/exp1 server2:/exp1
DISTRIBUTED VOLUME
BRICK BRICK
Red Hat Storage user perspective (distributed replicated volumes)
MOUNT POINT
Replicated Volume 0
DISTRIBUTED VOLUME
FILE 1 FILE 2
BRICK (exp1)
Replicated Volume 1
BRICK (exp2)
server1 server2
BRICK (exp3)
BRICK (exp4)
server3 server4
Red Hat Storage Server for Public cloud
RED HAT STORAGE FOR PUBLIC CLOUD
EBS
Scale out performance, capacity, and availability
Scal
e up
ca
paci
ty
• GlusterFS Amazon Machine Images (AMIs) • The only way to achieve high availability of Elastic Block Storage (EBS) • Multiple EBS devices pooled • POSIX compatible (no application to rewrite required to run on Amazon EC2) • Scale out capacity and performance as needed
SINGLE GLOBAL NAMESPACE
...
...
EC2
... ... ... ... ... ...
Red Hat Storage Use Cases in AWS EC2
Data Availability on Amazon Web Services
DNS Gateway
EBS
Red Hat Storage
Web and App Tier Web and App Tier
EBS
Red Hat Storage
Load-balancers Load-balancers
Amazon East Zone A
Amazon East Zone B
EBS
Red Hat Storage
Amazon Region West
Synchronous Replication
Master Slave
Replication
Web and App Tier
Load-balancers
Web Servers
Content
Processing
Load Balancers
Geo Rep
MetaData Store
Distributed, Replicated RHS Vols
Content Delivery Network
Content Delivery Network
Content is available In RHS Volumes
EC2 West Region EC2 East Region
Content Uploads
Content Downloads
Global Access
Scale with Growth
High Availability
Disaster Recovery
Open Source
Multiple Access
Methods
Object Access
Posix
CONTENT SHARING WITH RHS
App Servers
Map Reduce Processing With RHS Hadoop Plugin
Load Balancers
Ad Profile DB
Distributed, Replicated RHS Vols
Impression Log Info
Click Thru Requests
RHS Hadoop Plugin
Scale with Growth
High Availability
Open Source
Map Reduce With RHS
AD SERVING WITH RHS
Click-Thru Logs
Compute Storage
Co-residency
Ad Serving
Demo
Demo Architecture
Check Out Other Red Hat Storage Activities at The Summit
• Enter the raffle to win tickets for a $500 gift card or trip to LegoLand! • Entry cards available in all storage sessions - the more you attend, the more chances you
have to win!
• Talk to Storage Experts: • Red Hat Booth (# 211)
• Infrastructure • Infrastructure-as-a-Service
• Storage Partner Solutions Booth (# 605)
• Upstream Gluster projects • Developer Lounge
Follow us on Twitter, Facebook: @RedHatStorage