![Page 1: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/1.jpg)
Q2 2014INTRODUCTION TO CEPH
![Page 2: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/2.jpg)
A STORAGE REVOLUTION
PROPRIETARY HARDWARE
PROPRIETARY SOFTWARE
SUPPORT & MAINTENANCE
COMPUTER
DISKCOMPUTE
RDISK
COMPUTER
DISK
STANDARDHARDWARE
OPEN SOURCE SOFTWARE
ENTERPRISEPRODUCTS &
SERVICES
COMPUTER
DISKCOMPUTE
RDISK
COMPUTER
DISK
![Page 3: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/3.jpg)
Copyright © 2014 by Inktank | Private and Confidential
ARCHITECTURAL COMPONENTS
3
RGWA web services
gateway for object storage, compatible
with S3 and Swift
LIBRADOSA library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby,
PHP)
RADOSA software-based, reliable, autonomous, distributed object store comprised ofself-healing, self-managing, intelligent storage nodes and lightweight monitors
RBDA reliable, fully-distributed block device with cloud
platform integration
CEPHFSA distributed file
system with POSIX semantics and
scale-out metadata management
APP HOST/VM CLIENT
![Page 4: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/4.jpg)
Copyright © 2014 by Inktank | Private and Confidential
ARCHITECTURAL COMPONENTS
4
RGWA web services
gateway for object storage, compatible
with S3 and Swift
LIBRADOSA library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby,
PHP)
RADOSA software-based, reliable, autonomous, distributed object store comprised ofself-healing, self-managing, intelligent storage nodes and lightweight monitors
RBDA reliable, fully-distributed block device with cloud
platform integration
CEPHFSA distributed file
system with POSIX semantics and
scale-out metadata management
APP HOST/VM CLIENT
![Page 5: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/5.jpg)
OBJECT STORAGE DAEMONS
5
FS
DISK
OSD
DISK
OSD
FS
DISK
OSD
FS
DISK
OSD
FS
btrfsxfsext4
M
M
M
![Page 6: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/6.jpg)
RADOS CLUSTER
6
APPLICATION
M M
M M
M
RADOS CLUSTER
![Page 7: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/7.jpg)
RADOS COMPONENTS
7
OSDs: 10s to 10000s in a cluster One per disk (or one per SSD, RAID
group…) Serve stored objects to clients Intelligently peer for replication & recovery
Monitors: Maintain cluster membership and state Provide consensus for distributed decision-
making Small, odd number These do not serve stored objects to
clients
M
![Page 8: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/8.jpg)
WHERE DO OBJECTS LIVE?
8
??APPLICATION
M
M
M
OBJECT
![Page 9: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/9.jpg)
A METADATA SERVER?
9
1
APPLICATION
M
M
M
2
![Page 10: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/10.jpg)
CALCULATED PLACEMENT
10
FAPPLICATION
M
M
MA-G
H-N
O-T
U-Z
![Page 11: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/11.jpg)
EVEN BETTER: CRUSH!
11
RADOS CLUSTER
OBJECT
10
01
01
10
10
01
11
01
10
01
01
10
10
01 11
01
1001
0110 10 01
11
01
![Page 12: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/12.jpg)
CRUSH IS A QUICK CALCULATION
12
RADOS CLUSTER
OBJECT
10
01
01
10
10
01 11
01
1001
0110 10 01
11
01
![Page 13: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/13.jpg)
CRUSH: DYNAMIC DATA PLACEMENT
13
CRUSH: Pseudo-random placement algorithm
Fast calculation, no lookup Repeatable, deterministic
Statistically uniform distribution Stable mapping
Limited data migration on change Rule-based configuration
Infrastructure topology aware Adjustable replication Weighting
![Page 14: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/14.jpg)
Copyright © 2014 by Inktank | Private and Confidential
ARCHITECTURAL COMPONENTS
14
RGWA web services
gateway for object storage, compatible
with S3 and Swift
LIBRADOSA library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby,
PHP)
RADOSA software-based, reliable, autonomous, distributed object store comprised ofself-healing, self-managing, intelligent storage nodes and lightweight monitors
RBDA reliable, fully-distributed block device with cloud
platform integration
CEPHFSA distributed file
system with POSIX semantics and
scale-out metadata management
APP HOST/VM CLIENT
![Page 15: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/15.jpg)
ACCESSING A RADOS CLUSTER
15
APPLICATION
M M
M
RADOS CLUSTER
LIBRADOS
OBJECT
socket
![Page 16: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/16.jpg)
L
LIBRADOS: RADOS ACCESS FOR APPS
16
LIBRADOS: Direct access to RADOS for applications C, C++, Python, PHP, Java, Erlang Direct access to storage nodes No HTTP overhead
![Page 17: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/17.jpg)
Copyright © 2014 by Inktank | Private and Confidential
ARCHITECTURAL COMPONENTS
17
RGWA web services
gateway for object storage, compatible
with S3 and Swift
LIBRADOSA library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby,
PHP)
RADOSA software-based, reliable, autonomous, distributed object store comprised ofself-healing, self-managing, intelligent storage nodes and lightweight monitors
RBDA reliable, fully-distributed block device with cloud
platform integration
CEPHFSA distributed file
system with POSIX semantics and
scale-out metadata management
APP HOST/VM CLIENT
![Page 18: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/18.jpg)
THE RADOS GATEWAY
18
M M
M
RADOS CLUSTER
RADOSGWLIBRADOS
socket
RADOSGWLIBRADOS
APPLICATION APPLICATION
REST
![Page 19: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/19.jpg)
RADOSGW MAKES RADOS WEBBY
19
RADOSGW: REST-based object storage proxy Uses RADOS to store objects API supports buckets, accounts Usage accounting for billing Compatible with S3 and Swift applications
![Page 20: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/20.jpg)
Copyright © 2014 by Inktank | Private and Confidential
ARCHITECTURAL COMPONENTS
20
RGWA web services
gateway for object storage, compatible
with S3 and Swift
LIBRADOSA library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby,
PHP)
RADOSA software-based, reliable, autonomous, distributed object store comprised ofself-healing, self-managing, intelligent storage nodes and lightweight monitors
RBDA reliable, fully-distributed block device with cloud
platform integration
CEPHFSA distributed file
system with POSIX semantics and
scale-out metadata management
APP HOST/VM CLIENT
![Page 21: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/21.jpg)
STORING VIRTUAL DISKS
21
M M
RADOS CLUSTER
HYPERVISORLIBRBD
VM
![Page 22: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/22.jpg)
SEPARATE COMPUTE FROM STORAGE
22
M M
RADOS CLUSTER
HYPERVISORLIBRB
D
VM HYPERVISORLIBRB
D
![Page 23: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/23.jpg)
KERNEL MODULE FOR MAX FLEXIBLE!
23
M M
RADOS CLUSTER
LINUX HOSTKRBD
![Page 24: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/24.jpg)
RBD STORES VIRTUAL DISKS
24
RADOS BLOCK DEVICE: Storage of disk images in RADOS Decouples VMs from host Images are striped across the cluster
(pool) Snapshots Copy-on-write clones Support in:
Mainline Linux Kernel (2.6.39+) Qemu/KVM, native Xen coming soon OpenStack, CloudStack, Nebula,
Proxmox
![Page 25: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/25.jpg)
Copyright © 2014 by Inktank | Private and Confidential
ARCHITECTURAL COMPONENTS
25
RGWA web services
gateway for object storage, compatible
with S3 and Swift
LIBRADOSA library allowing apps to directly access RADOS (C, C++, Java, Python, Ruby,
PHP)
RADOSA software-based, reliable, autonomous, distributed object store comprised ofself-healing, self-managing, intelligent storage nodes and lightweight monitors
RBDA reliable, fully-distributed block device with cloud
platform integration
CEPHFSA distributed file
system with POSIX semantics and
scale-out metadata management
APP HOST/VM CLIENT
![Page 26: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/26.jpg)
SEPARATE METADATA SERVER
26
LINUX HOST
M M
M
RADOS CLUSTER
KERNEL MODULE
datametadata 0110
![Page 27: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/27.jpg)
SCALABLE METADATA SERVERS
27
METADATA SERVER Manages metadata for a POSIX-compliant
shared filesystem Directory hierarchy File metadata (owner, timestamps,
mode, etc.) Stores metadata in RADOS Does not serve file data to clients Only required for shared filesystem
![Page 28: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/28.jpg)
CEPH AND OPENSTACK
28
RADOSGWLIBRADOS
M M
RADOS CLUSTER
OPENSTACK
KEYSTONE CINDER GLANCE
NOVASWIFTLIBRB
DLIBRB
D
HYPER-
VISORLIBRBD
![Page 29: Ceph Day London 2014 - Ceph Ecosystem Overview](https://reader034.vdocuments.mx/reader034/viewer/2022052217/5562edb4d8b42a213b8b4dd6/html5/thumbnails/29.jpg)
Read about the latest version of Ceph. The latest stuff is always at http://ceph.com/get
Deploy a test cluster using ceph-deploy. Read the quick-start guide at http://ceph.com/qsg
Read the rest of the docs! Find docs for the latest release at http://ceph.com/docs
Ask for help when you get stuck! Community volunteers are waiting for you at
http://ceph.com/help
Copyright © 2014 by Inktank | Private and Confidential
GETTING STARTED WITH CEPH
29