everything you wanted to know about trove but didn't know whom to ask!

Everything you wanted to know about Trove but didn't know

whom to ask!Nikhil Manchanda [HP]

PTL, TroveEmail: [email protected]

IRC: SlickNikTwitter: @SlickNik@hphelioncloud

Amrith Kumar [Tesora]Core Reviewer, TroveEmail: [email protected]

IRC: amrithTwitter: @amrithkumar

@tesoracorp

Tweet about this presentation #openstack #trove

What is Trove• Trove Mission

“To provide scalable and reliable Cloud Database as a Service provisioningfunctionality for both relational and non-relational database engines, and to continue to improve its fully-featured and extensible open source framework.”

https://wiki.openstack.org/wiki/Trove

• Provisioning and de-provisioning.• Administration.• Configuration.• Backup and Restore.• Clustering & Replication.

• Create and manipulate tables, collections, namespaces.• Select, Insert, Update, Delete Data.

11/6/2014All you ever wanted to know about Trove!

OpenStack Summit (Paris), 20142

https://wiki.openstack.org/wiki/Trove

An Overview of Trove



Trove Architecture [1]




Keysto

ne

Cinder VolumeCinder VolumeCinder Volume

No

va-Netw

orkin

g/N

eutro

nTrove API

Message Bus

Trove Conductor

Trove Task Manager

Compute Instance

Guest Agent SQL/NoSQL

Backup

Data

BackupBackups

DB

BackupBackup

Guest Images

DataData

Nova

Cinder

Swift

Glance

Trove




Trove Guest Instance

Operating System

Database Management System

Trove Guest Agent

Trove Controller

Trove API

Message Bus



Trove Taskmanager

Trove Conductor

Where Can I Get Trove?• Use the source, Luke!

• OpenStack Trove Project• https://git.openstack.org/cgit/openstack/trove• Mirrored at - http://github.com/openstack/trove.git

• Trove python client binding and command line client• https://git.openstack.org/cgit/openstack/python-troveclient• Mirrored at – http://github.com/openstack/python-troveclient.git

• Trove design specifications for blueprints• https://git.openstack.org/cgit/openstack/trove-specs• Mirrored at – http://github.com/openstack/trove-specs

• Trove scripts for installation and testing, and elements for building guest images• https://git.openstack.org/cgit/openstack/trove-integration• Mirrored at – http://github.com/openstack/trove-integration.git



http://github.com/openstack/trove.git

http://github.com/openstack/trove.git

https://git.openstack.org/cgit/openstack/python-troveclient

http://github.com/openstack/python-troveclient.git

https://git.openstack.org/cgit/openstack/trove-specs

http://github.com/openstack/trove-specs

https://git.openstack.org/cgit/openstack/trove-integration

http://github.com/openstack/trove-integration.git

trove/guestagent/strategies

backup/

[…]

replication/

restore/

storage/

trove/guestagent/

[…]

backup/

common/

datastore/

[…]

strategies/

[…]

trove/guestagent/strategies/backup

base.py

base.pyc

couchbase_impl.py

couchbase_impl.pyc

__init__.py

__init__.pyc

mysql_impl.py

mysql_impl.pyc

postgresql_impl.py

grep class guestagent/strategies/backup/mysql_impl.py

class MySQLDump(base.BackupRunner):

class InnoBackupEx(base.BackupRunner):

class InnoBackupExIncremental(InnoBackupEx):



Trove Modularity

Getting Started with Trove

• As a Trove user• OpenStack distribution that includes Trove (such as HP Helion Dev Platform)

• http://www8.hp.com/us/en/cloud/hphelion-openstack.html• Tesora DBaaS platform, a Trove packaging tailored for the enterprise

• http://tesora.com/download-tesora-dbaas-platform-enterprise-edition

• As a Trove developer$ git clone http://github.com/openstack/trove-integration$ cd trove-integration/scripts$ ./redstack install$ ./redstack kick-start mysql

• On top of DevStack• Add to localrc:

• ENABLED_SERVICES+=,trove,tr-api,tr-tmgr,tr-cond• Swift should also be enabled for Backup and Restore.

• ENABLED_SERVICES+=,s-proxy,s-object,s-container,s-account• SWIFT_HASH=<swift-hash-here>



http://tesora.com/download-tesora-dbaas-platform-enterprise-edition

http://github.com/openstack/trove-integration

Provisioning a Trove Instance



$ trove create

<instance-name> <flavor-id>

--size <volume-size>

• Support for flavors

• Support for volumes using Cinder

• Optional parameters to create• Image per ‘datastore type’ and ‘version’

• Support AZs using --availability_zone

• Support for Neutron using --nic

{"instance": {

"status": "BUILD","updated": "2014-06-05T19:33:46","name": "test","links": [

{"href": "https://region-a.geo-

1.database.hpcloudsvc.com/v1.0/39745227274204/instances/ce629494-a64d-41ed-b73c-04cb28bb33bb",

"rel": "self"},{

"href": "https://region-a.geo-1.database.hpcloudsvc.com/instances/ce629494-a64d-41ed-b73c-04cb28bb33bb",

"rel": "bookmark"}

],"created": "2014-06-05T19:33:46","id": "ce629494-a64d-41ed-b73c-04cb28bb33bb","flavor": {

"id": "1002","links": [


1.database.hpcloudsvc.com/v1.0/39745227274204/flavors/1002","rel": "self"

},{

"href": "https://region-a.geo-1.database.hpcloudsvc.com/flavors/1002",

"rel": "bookmark"}

]},"datastore": {

"version": "5.5","type": "mysql"

}}

}

Managing the Trove Database

• Resize flavor

• Resize volume

• Datastore specific extensions:• Create Database / Schema

• Create Users

• Grant Users Permissions on Databases

• Enable a Root User



Trove is Tuned

• Trove sets up sane defaults• InnoDB only• Disable load data infile and select into outfile

• Ability to automatically tune settings by flavor – e.g. for my.cnf• Buffer Pool Size, • Log file size• max_connections, etc.

• API to programmatically target configuration groups for settings



Trove is Secure

• Removes anonymous user

• Removes non-localhost users

• Removes local file access

• Mangles root user password

• Sets up Security Groups to manage access (via API)

• User SSH access to datastore not required



Backup and Restore



$ trove backup-create <backup-name> <instance-id>

• Optional params:• Description through --description

• Incremental backups using –parent

• To Restore backup use create:

$ trove create



--backup <backup-id>

{

"backup": {

"instance_id": "9a31e420-aee6-4329-92f7-4b6b3790bcd4",

"status": "NEW",

"updated": "2014-06-05T19:43:28",

"locationRef": null,

"name": "DEMO_instance_test_backup",

"created": "2014-06-05T19:43:28",

"size": null,

"id": "f811a9cf-f397-4e07-a21f-7134094976f8",

"description": null

}

}

Backups in Trove

• Fully Managed

• Triggered and Tracked via API

• Streamed to Swift (OpenStack Object Storage)

• Incremental as well as Full backups

• Multiple formats per datastore supported via strategies:

• XtraBackup (Percona)

• mysqldump



Replica Instances



$ trove create



--replica_of <instance-id>

• Support for async mysql replication (mysql slave instances)

• Manual detach using

$ trove update <instance-id>

--detach-replica-source

{"instance": {

"status": "BUILD","updated": "2014-06-05T19:33:46","name": "test","links": [


1.database.hpcloudsvc.com/v1.0/39745227274204/instances/ce629494-a64d-41ed-b73c-04cb28bb33bb",

"rel": "self"},{

"href": "https://region-a.geo-1.database.hpcloudsvc.com/instances/ce629494-a64d-41ed-b73c-04cb28bb33bb",

"rel": "bookmark"}

],"created": "2014-06-05T19:33:46","id": "ce629494-a64d-41ed-b73c-04cb28bb33bb","flavor": {

"id": "1002","links": [


1.database.hpcloudsvc.com/v1.0/39745227274204/flavors/1002","rel": "self"

},{

"href": "https://region-a.geo-1.database.hpcloudsvc.com/flavors/1002",

"rel": "bookmark"}

]},"datastore": {

"version": "5.5","type": "mysql"

}}

}

Clusters



$ trove cluster-create <datastore> <ds-version>

• Optional parameters to cluster-create• --instance <flavor_id=flavor_id,volume=volume>

• Specify multiple times to create multiple instances for your cluster

• Initial support in Juno added for MongoDB Clusters• Sets up mongo config server, and mongo query routers

• Transparent and driven by configuration options

• Support for adding shards to existing cluster for horizontal scale out.

Juno and Kilo



Completed in Juno

• Async MySQL replication (master-slave)

• Clusters for MongoDB

• Neutron Support

• Support for PostgreSQL

• Config-groups enhancements• Configuration groups per datastore / version

• Config-groups for MongoDB

• Backups for Cassandra and Couchbase

• Additional Tempest Tests

Planned for Kilo

• Building out clusters -- including Semi Synchronous (Galera) mysql clusters

• Async Replication v2 – failover support

• Associate flavors with datastores

• Access datastore logs via API

• Removing deprecated oslo-incubator code

• Upgrade testing through grenade

• YOUR IDEA HERE!

Your Idea Here



• Growing community of contributors• 128 contributors from 28 companies• 1723 commits, and 231829 lines of code

• Open to new ideas, and code

• Lots of room for improvement

• Find us at #openstack-trove on FreeNode.

Thank You!

Got Questions?



Nikhil Manchanda [HP]PTL, Trove

Email: [email protected]: SlickNik

Twitter: @SlickNik@hphelioncloud

Amrith Kumar [Tesora]Core Reviewer, TroveEmail: [email protected]

IRC: amrithTwitter: @amrithkumar

@tesoracorp

Tweet about this presentation #openstack #trove

everything you wanted to know about trove but didn't know whom to ask!

Software