1

Deep Dive into

Red Hat Enterprise Linux

Openstack Storage

Sean CohenSr. Product ManagerRed HatDec 9, 2013

2

The Red Hat Way Red Hat’s business model is 100% open source.

We have no alternative commercial solutions, and we never will.

3

From Community to Enterprise

● Enterprise-hardened OpenStack software

● Delivered with an enterprise life cycle

● Six-month release cadence offset from community releases to allow testing

● Aimed at long-term production deployments

● Certified hardware and software through the Red Hat OpenStack Cloud Infrastructure Partner Network

● Supported by Red Hat

● Latest OpenStack software, packaged in a managed open source community

● Facilitated by Red Hat

● Aimed at architects and developers who want to create, test, collaborate

● Freely available, not for sale

● Six-month release cadence mirroring community

● No certification, no support

● Installs on Red Hat and derivatives

● Open source, community-developed (upstream) software

● Founded by Rackspace Hosting and NASA

● Managed by the OpenStack Foundation

● Vibrant group of developers collaborating on open source cloud infrastructure

● Software distributed under the Apache 2.0 license

● No certifications, no support

4

Red Hat Continues to be Top Contributor also in OpenStack Havana

Projects led by Red Hat

5

What's new in HavanaStorage

6

Cinder

7

Block Storage - Cinder

Encrypted Volumes● Cinder volumes are now encrypted

● Data is decrypted and encrypted as needed at read/write time

● Process is transparent to guest instances.

● Encryption is done by Nova using dm-crypt, Cinder is made aware of encryption keys

QEMU Assisted Snapshotting● Provides snapshotting of volumes on backends by storing data as

QCOW2 files on these volumes. With Nova support, this can also enable quiescing via the QEMU guest agent

8

Block Storage - Cinder

Centralized Mount Options● When connecting to NFS or GlusterFS backed volumes, uses mount

options from Cinder

● Was previously set on all Compute nodes

Extend Volume

● Add support for extending the size of an existing volume.

● To resize your volume, you must first detach it from the server.

● Resize the volume by passing the volume ID and the new size as parameters (using the new cinder extend command)

9

Block Storage - Cinder

QoS support for volumes

● Across Block Storage drivers to guarantee applications performance (IOPS / Bandwidth), with settings such as:

● maximum MB/second (maxBWS)

● maximum IO/second (maxIOPS)

Volume host attaching

● Allow client require to attach a volume to a host by api but an instance only.

● This change allow attach_volume API support 'host_name' as a argument but not 'instance_uuid' only.

10

Block Storage - Cinder

Transfer ownership of volumes● Added the support for transferring Cinder Volumes from

one tenant or project to another.

● As both projects can’t use the volume at the same time, you can create a transfer from one tenant, and then accept it from the other

# cinder transfer­create <volume_id> # Tenant A

# cinder transfer­accept <transfer_id> <auth_key> # Tenant B

12

Block Storage - Cinder

Volume Migration

● Administrators are able to migrate a volumes to another host or to an entirely different backend, like so

● # cinder migrate <volume-id> <target>● Check if storage can migrate the volume, if not, create a new

volume● If original volume is detached, Cinder server attaches both and

runs 'dd'● If original volume is attached, Nova performs the copy (KVM-

only in Havana)

● Hot Swap Attached Volumes ● Transparently swap volumes attached to an instance● No reads or writes are lost/discarded

13

Block Storage - Cinder

Extended Quotas● Quotas are operational limits. For example, the number of

gigabytes allowed for each tenant can be controlled so that cloud resources are optimized. Quotas can be enforced at both the tenant (or project) and the tenant-user level.

● Edit default quota settings such as update a particular quota value to prevent system capacities from being exhausted without notification.

● Using the class quotas named `default` as the default editable quotas.

● cinderclient command to update default quota example:

 # cinder quota­class­update default <key> <value>

14

Block Storage - Cinder

● Cinder Backup● Starting the Havana release users may be able to use

an alternative object store than Swift● Backup service improvements to Object Storage so any

driver can take advantage● Enable the generalized backup layer to allow backups

from any iSCSI device that doesn't have internal optimizations

● Added Ceph driver to backup service (allowing Ceph as a backup target with differential backups from Ceph to Ceph)

15

Block Storage - Cinder

● Scheduler hints

Filter Scheduler: Example Flow● Drivers continuously report capabilities and state● Scheduler starts with list of all back-ends● Filters out unsuitable back-ends

● Insufficient free space● Insufficient capabilities

● Sorts according to weights (e.g., available space)● Returns best candidate

16

Block Storage - Cinder

● Scheduler hints● cinderclient's code and to cinder API was introduced

with a flexible hint mechanism which enhances user's ability to design filters and interact with them.

● Chooses back-end to place a new volume on● Configurable plugins for scheduler

● Simple● Chance● Filter

● Most common is the filter scheduler● Has plug-able filters & weights

17

Block Storage - Cinder

iSER Transport Protocol Support● iSCSI over RDMA Increases performance compared to iSCSI over

TCP (up to 5x faster bandwidth and lower CPU overhead), drove by Mellanox in Havana

Support for raw disks without LVM

● In addition to or instead of the base LVM implementation

● libvirt uses the local storage as storage for the instance. The instance will get a new disk, usually a /dev/vdX disk.

Rate Limited Disk Access● QoS parameters extracted from Cinder

● Allows rate limiting per volume

● Can be enforced by Nova (KVM-only in Havana) or by storage

18

Block Storage● Added native GlusterFS support.

● If qemu_allowed_storage_drivers is set to gluster in nova.conf then QEMU is configured to access the volume directly using libgfapi instead of via fuse.

● Added support for the following Gluster volume features:

● Volume Snapshots (QEMU assisted) ● Create● Delete● List● Create volume from snapshot

● Volume Clones● Extend GlusterFS volume● Volume Migration (Host assisted)

19

Block Storage New Vendor Drivers

● Dell EqualLogic volume driver

● VMware VMDK cinder driver

● IBM General Parallel File System (GPFS)

● Microsoft Windows Storage Server driver

Major Additions To Existing Drivers

● Add a NFS Volume Driver to support Nexenta storage in Cinder

● Add Fibre Channel drivers for Huawei storage systems

Backup Drivers

● Allow Ceph as an option for volume backup

● IBM Tivoli Storage Manager (TSM)

20

Block Storage

New Vendor Certifications in Havana

● The following vendors with OpenStack storage drivers are part of our Partner Network, that we are currently working with to test and certify their products on RHEL OSP 4.0:

● Coraid, Dell ,EMC, Hitachi, IBM, Inktank, Mellanox, NetApp, SolidFire, Zadara and many more...

● Vendors can submit their certification results for review once the GA bits are available.

21

22

GlanceDeep Dive

23

Image Service - Glance

● Glance Multi-locations

● Enable image domain object fetch data from multiple locations, allow API client consume image from multiple backend store.

● Glance now supports adding/removing multiple location information to the metadata of an image, an image maybe have more then one location within the backend store.

● Glance Registry service deprecation

● Implement Registry Database Driver for the registry service in order to support legacy deployments based on 2 separate services

24

Image Service - Glance

● Total disk quota for glance users

● Added the ability to limit the usage of some basic image-related resources, such as:

● The number of images stored● The amount of storage in occupied by a set of images

● Direct URL Metadata

● As each storage system have a means to return direct URL specific meta-data to the client when direct_url is enabled

● The direct URL can now provide additional information to the client. For example, with a file:// URL the client may need to know the NFS host that is exporting it, the mount point, and FS type used.

●

25

SwiftDeep Dive

26

Object Storage - Swift

Global clusters● Globally Distributed OpenStack Swift Cluster

● Replication across the world

● A globally replicated cluster is created by deploying storage nodes in each Region. The proxy nodes will have an affinity to a Region and be able to optimistically write to storage nodes based on the storage nodes’ Region.

● Local reads/writes for performance

● Tiered zones

● Added a region tier above zones. This allows for the existing "unique-as-possible" placement strategy to continue to work across a distributed cluster and ensures that data is as protected from failures as possible.

27

Object Storage - Swift

● Proxy affinity (writes) ● In a multi-region scenario, writes are sent to <replica

count> servers in the same region as the proxy. This keeps latency on writes down, and allows WAN traffic to be more strictly controlled, eg through a separate replication network.

● Dedicated replication network support ● Added support for using dedicated network for

replication traffic. Separating client-bound traffic between proxy-servers and storage-servers, and improves replication performance.

28

Object Storage - Swift

● Cluster-side crossdomain.xml file● Useful for flash, cross-domain JavaScript

● <?xml version="1.0"?>● <cross­domain­policy>● <allow­access­from domain="*.mirantis.net" />● <allow­access­from domain="*.mirantis.com" />● </cross­domain­policy>

● Configuration Directory

● Allow a single configuration object to be sourced from multiple files (either via swift.utils.readconf or paste.deploy.appconfig).

29

Object Storage - Swift

● Thread Pools

● Use external real threads to allow for actual concurrent reads on multiple disks, ensuring that a single slow disk won't end up with all the threads stuck waiting for it.

● Performance Improvements:

● Optimized storage disk operations ● Memcache pool of connections (to prevent the connection

count from growing without bound) ● Faster Handoff node selection (replicate handoff first)● Cluster-wide crossdomain.xml file to better enable Flash

apps reading content directly from a Swift cluster.● Configuration Directory (ConfD) support to better manage

configurations

30

31

Icehouse Storage Roadmap

Highlights

32

Features in the Works for Icehouse

Cinder Volume Replication

Multi-attach

Volume Retype

ACLs for volumes

Volume export/import

Bare metal volumes

Public Volumes

Attachment notifications

Filtering weighing (as part of placement decision making)

33

Features in the Works for Icehouse

Glance image-recover

New download workflow ("Export")

New Upload Workflow ("Import")

Add multifilesystem store to support NFS servers as backend

Adding image location selection strategy (in multi-location)

34

Features in the Works for Icehouse

Swift Storage Policies

Shard large containers

Pluggable Back-end API (Gluster, Ceph)

Multi-ring servers

Improved Object Replicator- aka Local storage volume (volume in local storage and incremental snapshots are stored in swift)

Object Replicator - 'ssync' (an rsync alternative)

Searchable Metadata (driven by HP and IBM Softlayer)

Cluster Federation

35

We’ve built the world’s largest ecosystem for commercially supported OpenStack deployments

It’s open. It’s innovative.

And it’s all yours.

Join the RDO Community

http://openstack.redhat.comhttp://redhatstack.com