riak cs build your own cloud storage
DESCRIPTION
Riak CS Build Your Own Cloud Storage by Pavan VenkateshTRANSCRIPT
Build Your Own Cloud Storage
About Basho Our Mission is to Be The Leader in Distributed Systems
• Founded January 2008
• 115+ employees
• Headquartered in Cambridge, with regional offices in San Francisco, Washington DC, London and Tokyo
• Makers of Riak & Riak CS
• Thousands of Users Worldwide including over 20% of the Fortune 50
• 30,000+ downloads per month now up from 19,500 in Dec 2011
• Strategic Partners include Citrix, IDC Frontier, Yahoo! Japan, and Microsoft
Product Suite Basho’s distributed systems technologies
EnterpriseDS
Open Source Distributed Database
Commercial Distributed Database
Distributed Cloud Storage Pla7orm
• Always-‐available, scalable, ops-‐friendly NoSQL database
• Over 35,000 downloads per month
• Thousands of users worldwide
• Version 1.0 unveiled September 2011
• Adds mulK-‐data center replicaKon & Basho 24x7 support
• Enables Secondary Search and MapReduce Clusters
• Requires commercial contract and secure download
• Version 1.4 in May 2013
• Expands with mulK-‐tenancy, large object support, metering and Amazon S3 API
• Enables public and private cloud storage
• SoXware released to open source on March 20th
Overview
Riak CS is...
enterprise cloud storage
Riak S3-compatibility
multi-tenancy
per user reporting
built on top of
large object storage
Enabling you to host your own
PUBLIC PRIVATE
& CLOUDS
or…. Reliable Storage Behind Apps
Riak CS Use Cases
Public Cloud Storage
S3 Without AWS Cloud Drive (General Content Storage)
Backup-as-a-Service
Archival and Preservation
Storage for Internal
Clients / Depts.
Riak CS
Riak Stanchion
Riak CS node for every
node of Riak 1
How It Works
RiakNode
RiakNode
RiakNode
RiakNode
RiakNode
Large Object
Riak CS
S3 API
Reporting API
Riak CS
S3 API
Reporting API
Riak CS
S3 API
Reporting API
Riak CS
S3 API
Reporting API
Riak CS
S3 API
Reporting API
1. User uploads an object
1 MB
2. Riak CSbreaks object
into 1 MB chunks
1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB 1 MB
3. Riak CSstreams chunks to Riak nodes
4. Riak replicates and stores chunks
Under The Hood
Riak Riak is a distributed NoSQL key-value store
Store anything: text, images
More Flexible
Riak Remains flexible in the long run, because of design
Riak High availability
Low-latency
Horizontal scalability
Fault-tolerance
Ops friendliness
Riak Masterless • No master/slave or different roles • All nodes are equal • Write availability and scalability • All nodes can accept/route requests
Riak No Sharding • Consistent hashing • Prevents “hot spots” • Lowers operational burden of scale • Data rebalanced automatically
Riak Availability and Fault-Tolerance • Automatically replicates data • Read and write data during
hardware failure and network partition
• Hinted handoff
USERS multi-tenancy:
Riak CS will track individual usage/stats
access_key secret_key users identified by users authenticated by
BUCKETS users create buckets.
buckets are like folders. store objects in buckets.
names are globally unique.
OBJECTS stored in buckets.
store any file type.
Basic Operations S3 API Commands���
s3cmd mb s3://pavan-test
s3cmd ls s3://pavan-test
s3cmd put README.txt
s3://pavan-test s3cmd get s3://pavan-test/README.txt
Features
Riak CS Large Object Support • Started with 5GB / object • Now have multipart upload • Content agnostic
Riak CS S3-Compatible API • Use existing S3 libraries and tools • RESTful operations • Multipart upload • S3-style ACLs for object/bucket
permissions • S3 authentication scheme
Riak CS Administration and Users • Interface for user creation,
deletion, and credentials • Configure so only admins can
create users
Riak CS New Stuff in Riak 1.3 • Multipart upload: parts between
5MB and 5GB • Restrict access to buckets based on
source IP
Multi-Datacenter Replication • For active backups, availability
zones, disaster recovery, global traffic
• Real-time or full-sync • 24/7 support • Per-node or storage-based pricing
“[deploying Riak CS] reduces the risk of using AWS and allows customers to store their data in their own data centers, on their
own terms.” - Alex Williams, TechCrunch
Riak CS
Operations
OPERATIONAL STATS exposed via HTTP resource: /riak-cs/stats
block bucket object GET, PUT, DELETE LIST KEYS, CREATE,
DELETE, GET/PUT ACL GET, PUT, DELETE
HEAD, GET/PUT ACL
HISTOGRAMS & COUNTERS
TRACK INDIVIDUAL USER’S
THE “USAGE” BUCKET
ACCESS STORAGE
QUERY USAGE STATS
Storage and access statistics tracked on per-user basis, as rollups for slices of time • Operations, Count, BytesIn,
BytesOut, + system and user error • Objects, Bytes
Partnerships
Citrix CloudStack
• Combined platform for compute and storage
• Summer ‘13: integrated authentication support
• Videos on our blog
Roadmap
• Open stack integration • API • Keystone authentication
• Expand S3 API support
• Object versioning support • Cloudstack integration work
Q&A http://docs.basho.com/riakcs/latest/
http://docs.basho.com/riakcs/latest/riakcs-
tutorials/fast-track/
riak-users [email protected]