intro to the skysql cloud data suite

V1212.01

Ivan Zoratti

SkySQL Data SuiteA New Open Source Approach to MySQL Distributed Systems

Wednesday, 5 December 12

Who is Ivan

?Wednesday, 5 December 12

SkySQL•#1 provider of MySQL-based

solutions, for software and services

•Backed by Monty Program AB and 3rd parties active in the MySQL ecosystem

•Committed to provide the best solutions around MySQL for the Community and for the Enterprise

Agenda

• SkySQL Cloud Data Suite Architecture

• SkySQL Cloud Data Suite on Amazon EC2

• Components for automated provisioning in AWS

• Components for high availability in EC2

• The Administration Console

• Automated Backup/Restore with S3

4Wednesday, 5 December 12

Agenda

• SkySQL Cloud Data Suite Architecture

• SkySQL Cloud Data Suite on Amazon EC2

• Components for automated provisioning in AWS

• Components for high availability in EC2

• The Administration Console

• Automated Backup/Restore with S3

WARNING!We have lots of slides

Intro to SkySQL Data SuiteWednesday, 5 December 12

SkySQL Data Suite

High Availability

Full-Text

Replication& MHA

Galera

Analytics

Carrier Grade

Online Apps.

SharedStorage

SkySQL Cloud Data Suite

SkySQL Data Suite

High Availability

Full-Text

Replication

Galera

Analytics

Carrier Grade

Online Apps.

SharedStorage

Cloud Data Suitevs

Enterprise Data Suite

Cloud Data Suite

•Deployed in Amazon AWS•Automatic configuration and provisioning•MariaDB 5.5•MySQL Replication with Pacemaker/Corosync

•Administration Console with Backup/Restore•SkySQL Enterprise Monitoring aka MONyog - Trial•PhpMyAdmin•Community Support•Development Release•Fully OSS (with MONyog Trial)

Enteprise Data Suite

•Deployed in Clouds, virtualised env. and bare OS•Standard download packages•MariaDB, MySQL, Percona Servers•MySQL Replication with MHA, DRBD, Galera or shared storage and Pacemaker/Corosync

•ZManda Recovery Manager•SkySQL Enterprise Monitory aka MONyog•SQLyog•Enterprise Technical Support•Production Release•Mix of OSS and Commercial

SkySQL Cloud Data Suite Sep 2012

1. Configure your DB system on config.skysql.com or cloud.skysql.com

2. Deploy the DB system:• Online• Configuration pull from AWS• Manually

3. Connect the DB system toyour application

SkySQL Data Suite Dec 2012

• Rel. 1.1 with bug fixing

• New Administration Console with: • rpm package and installation on Linux OS

• Administration and monitoring of replicated and standalone servers

• Customisable graphs

• Customisable commands

SkySQL Data Suite ArchitectureWednesday, 5 December 12

SkySQL Data Suite Configurator

User Account

ConfigurationEngine

RulesRepository

User Account

ConfigurationEngine

RulesRepository

ProvisioningQueue

User Account

ConfigurationEngine

RulesRepository

ProvisioningEngine

SoftwareRepository

ProvisioningQueue

User Account

ConfigurationEngine

RulesRepository

ProvisioningEngine

SoftwareRepository

ProvisioningQueue

DeploymentQueue

User Account

ConfigurationEngine

RulesRepository

ProvisioningEngine

SoftwareRepository

DeploymentEngine

ProvisioningQueue

DeploymentQueue

User Account

ConfigurationEngine

RulesRepository

ProvisioningEngine

SoftwareRepository

DeploymentEngine

ProvisioningQueue

DeploymentQueue

DatabaseDatabase

Database Database Database

userdata.sh

User Account

Deployment triggered by the first boot of each instance

ConfigurationEngine

RulesRepository

ProvisioningEngine

SoftwareRepository

DeploymentEngine

ProvisioningQueue

DeploymentQueue

DatabaseDatabase

userdata.sh

User Account

Deployment triggered by the first boot of each instance

SkySQL Cloud Data Suite

DatabaseDatabase

$ ssh -i SDS193500.pem ec2-user@174.129.202.133[ec2-user@node1 ~]$ ps -efUID PID PPID C STIME TTY TIME CMDroot 1157 1 0 Nov29 ? 00:00:02 /usr/sbin/glusterd --pid-file=/var/run/glusterd.pidroot 1840 1 2 Nov29 ? 00:11:34 /usr/sbin/glusterfsd -s localhost --volfile-id glust...root 1889 1 0 Nov29 ? 00:00:50 /usr/sbin/glusterfsd -s localhost --volfile-id glust...root 1895 1 0 Nov29 ? 00:00:01 /usr/sbin/glusterfs -s localhost --volfile-id gluste...root 1901 1 0 Nov29 ? 00:00:01 /usr/sbin/glusterfs -s localhost --volfile-id gluste...root 4484 1 8 Nov29 ? 00:36:13 /usr/sbin/glusterfs --volfile-id=/gluster-monyog --......root 1311 1 0 Nov29 ? 00:00:34 corosync220 1317 1311 0 Nov29 ? 00:00:12 /usr/libexec/pacemaker/cibroot 1318 1311 0 Nov29 ? 00:00:05 /usr/libexec/pacemaker/stonithdroot 1319 1311 0 Nov29 ? 00:00:12 /usr/lib64/heartbeat/lrmd220 1320 1311 0 Nov29 ? 00:00:03 /usr/libexec/pacemaker/attrd220 1321 1311 0 Nov29 ? 00:00:01 /usr/libexec/pacemaker/pengine220 1322 1311 0 Nov29 ? 00:00:05 /usr/libexec/pacemaker/crmd...root 1534 1 0 Nov29 ? 00:00:00 sendmail: accepting connectionssmmsp 1542 1 0 Nov29 ? 00:00:00 sendmail: Queue runner@01:00:00 for /var/spool/clientmqueue...root 2293 1 0 Nov29 ? 00:00:00 /bin/sh /usr/bin/mysqld_safe --defaults-file=/etc/my.cnf --p...mysql 2791 2293 0 Nov29 ? 00:00:28 /usr/sbin/mysqld --defaults-file=/etc/my.cnf --basedir=/usr ......root 3467 1 0 Nov29 ? 00:00:01 /usr/sbin/httpd -DSTATUS -f /etc/httpd/conf/httpd.confapache 25416 3467 0 Nov29 ? 00:00:00 /usr/sbin/httpd -DSTATUS -f /etc/httpd/conf/httpd.conf...root 4578 1 1 Nov29 ? 00:07:56 /usr/local/MONyog/bin/MONyog-bin -s

Terminal

Admin Console

PhpMyAdmin

MONyog

SkySQL Data Suite Building Block

Replicator

DatabaseDatabase

Data StoreAdminConsole

SkySQL Cloud Data Suite Sep 2012Technologies and Software

Replicator

DatabaseDatabase

FRONT ENDApacheTomcat

BACK ENDSQLite

MariaDB

Linux 3.2 (Amazon Distro)

MONyogTrial

PHPMyAdmin

MySQL ReplicationGlusterFSPacemaker

Corosync Xtrabackup

SkySQL Cloud Data Suite Sep 2012Data Store

DatabaseDatabase

Data Store

PublicSkySQL AMI

MariaDB 5.5MyISAM, InnoDB, XtraDB, Enhanced

Optimizer, Virtual Columns

From standalone to1 Master + 3 Slaves

System and Datafile systems

Pacemaker controls the automatic failover

SkySQL Cloud Data Suite Sep 2012Administration Console

DatabaseDatabase

AdminConsole

GUI to monitor the status of the

Cluster

Extended Monitoring from MONyog as a trial

PhPMyAdmin as graphical user tool

Backup/Restore to/from S3 using

XtrabackupStart/Stop, Isolate,

Promote nodesmysql command

line interface

Automatic failover for the admin and

monitoring servers

SkySQL Cloud Data Suite Sep 2012Replicator

Replicator

DatabaseDatabase

Replicator

DatabaseDatabase

Standard MySQL Replication in asynchronous and semi-

synchronous mode

Replicator

DatabaseDatabase

synchronous modeGlusterFS on ext4 to replicate the

data collection and monitoring

Replicator

DatabaseDatabase

synchronous modeGlusterFS on ext4 to replicate the

data collection and monitoring

Automatic failover of database servers and console components

SkySQL Data Suite Provisioning

DatabaseDatabase

userdata.sh

User Account

DatabaseDatabase

userdata.sh

User AccountVariable definition - they will

be used later when other scripts are called

DatabaseDatabase

userdata.sh

On-the-fly configuration files, such as my.cnf

DatabaseDatabase

userdata.sh

Execute predefinedscripts already in the AMI

DatabaseDatabase

userdata.sh

Adaptive, distributed setup

DatabaseDatabase

userdata.sh

Elastic IP onthe Master Server

DatabaseDatabase

userdata.sh

Elastic IP onthe Master Server

Currently, the Administration Console is on the Master server

SkySQL Data Suite Deployment

Database

Automated Provisioning in AWS

Starting Here!

You can try it for FREE! Even without an

AWS account

Simplified Interface

Immediate Help

Real Time Check

Advanced Configuration

Cost Estimation

Delivery Mechanism

Online Deployment

The keys will be discarded immediatelyafter the provisioning

Extra AWS & Contact Info

Welcome Email

mysql command line

Administration Console

ssh connection

How to remove SCDS

An inventory of the AWS components

IMPORTANT!Your pem key

Job Done!

SkySQL Cloud Data Suite in Amazon EC2

SecurityIdentity Access Manager Roles

• EIP Address• Associate/Disassociate, Describe

• Instances• Start, Stop, Describe• Attributes, Status, Tags, Net IF

• Create, Describe

• S3 Buckets• List, Create, Delete• Objects

• Get, Put, List, Delete

SecuritySecurity Groups

• ssh

• http

• MySQL

• MONyog

Storage

Components for High Availability in EC2

Automatic Failover with PacemakerCRM Configuration

Replicator

DatabaseDatabase

Database Database

Replicator

DatabaseDatabase

Database Database

Replicator

DatabaseDatabase

Database Database

Replicator

DatabaseDatabase

Database Database

Replicator

DatabaseDatabase

Database Database

Replicator

DatabaseDatabase

Database Database

[root@node2 init.d]# crm status============Last updated: Sat Dec 1 12:42:03 2012Last change: Fri Nov 30 12:43:27 2012 via crm_attribute on node2Stack: openaisCurrent DC: node2 - partition with quorumVersion: 1.1.7-6.el6-148fccfd5985c5590cc601123c6c16e966b85d144 Nodes configured, 4 expected votes14 Resources configured.============

Online: [ node2 node3 node1 node4 ]

Master/Slave Set: ms_resMySQL [resMySQL] Masters: [ node2 ] Slaves: [ node1 node3 node4 ] AmazonEIP (lsb:amazon_master_eip): Started node2 Clone Set: apache-php-myadmin [ApachePhpMyAdmin] Started: [ node1 node2 node3 node4 ] Resource Group: monyog monyog-fs (ocf::custom:Filesystem): Started node2 monyog-service (lsb:MONyogd): Started node2 Resource Group: admin-console console-fs (ocf::custom:Filesystem): Started node2 SkySQL-monitor (lsb:skysql_monitor): Started node2 Tomcat7 (ocf::heartbeat:tomcat): Started node2

Node 2

Elastic IP

Public IPsPublic IPsPublic IPs

admin-console and MONyog group

The Administration ConsoleWednesday, 5 December 12

Admin Console Dec 2012

Access to MONyog

MasterServer

ContextSensitiveContext

Sensitive

Customisable Graphs

Access to PhpMyAdmin

Customisable Commands

Backup/Restore

Administration Console Dec 2012Customizable Graphs

Name SQL Icon Delta Avg Chart

Monitors

Administration Console Dec 2012Customizable Commands

Desc Icon Order

Commands

Desc Icon Script

CommandStep

Automated Backup/Restore with S3Wednesday, 5 December 12

Automated BackupUsing Xtrabackup from the Admin Console

Why is the SkySQL Cloud Data Suite important for you?

• You can test MariaDB/MySQL on AWS for FREE

• You can use and fully control a ready made highly available solution AWS

• You can compare MySQL on EC2 and RDS

• You can study/review security and availability aspects in AWS

• You can use some of the SCDS components in your environment

For more information...

• Try the Configurator and How-To Video: http://cloud.skysql.com

• The Admin Console: http://www.skysql.com/downloads/community/skysql-console

• Forums: https://www.skysql.com/forums/skysql-cloud-data-suite

• Web: https://www.skysql.com/cloud

• Bugs: http://bugs.skysql.com

• Try it! http://config.skysql.com• Free SkySQL account available for testing• Test on your account

Thank You!

ivan@skysql.comizoratti.blogspot.com

www.slideshare.net/izorattiwww.skysql.com

intro to the skysql cloud data suite

skysql data suitewednesday

automated provisioning

mysql ecosystemcommitted

provider of mysql

amazon ec2 components

new administration console

aws components

linux os administration

Technology

crmsuite softwareproduct entellium · intro selectinga...

mariadb: viable mysql replacement - scale 16x | 16x ·...

web media api test suite 2017 - world wide web ......agenda...

collaboration suite mobility suite suite licensing ... suite...

[uengine.org] (eng) uengine bpm product intro(uengine3.5...

intro to photography, course intro

intro sdl suite

intro to intro-2

measuring outcomes in eap and workplace programs: intro of a...

savi course calendar - spring 2019 - pratt institute ·...

www.tpfsoftware.com suite. suite 2 suite 3 integrated ide

demo projekt: erste schritte -...

soa expert series: intro to oracle mft for soa suite...

skysql mariadb 云数据组件

skysql reference architecture (kaj arno)

aqwa-intro 14.5 l03 aqwa suite

basics intro middle mastery senior mastery master...

mechanical intro 14.5 l01 intro

intro. au sc intro. au dr. intro. à sc intro. relations

intro digital publish suite