nosql + sql = mysql - doag.org · mysql is the most popular database . mysql evolution •mysql 5.1...

NoSQL + SQL = MySQL Developing Next Gen Apps with MySQL 8 Carsten Thalheimer

Sales Consultant MySQL GBU EMEA (Carsten.Thalheimer@Oracle.com)

Safe Harbor Statement

The following is intended to outline our general product direction. It is intended for information purposes only, and may not be incorporated into any contract. It is not a commitment to deliver any material, code, or functionality, and should not be relied upon in making purchasing decisions. The development, release, and timing of any features or functionality described for Oracle’s products remains at the sole discretion of Oracle.

2

Agenda

• MySQL “Status Quo Nov 2018”

• Key SQL Updates in MySQL 8

• The new MySQL Shell

• NoSQL – No SQL in MySQL anymore?

• MySQL in 6 months … what you can expect

Explosive Data Growth

5

4.33 Million Videos Views

473,400 Tweets

854,166 Logins

97,222 Hours of Video

Streamed

1,389 Rides

120+ New Professionals

Every 60 Seconds

0

20

40

60

80

100

120

140

160

180

2010 2012 2014 2016 2018 2020 2022 2024

Zett

abyt

es

Source: IDC, 2017 Source: Domo, 2018

Stack Overflow Developer Survey 2018 Developer Survey Results

MySQL is the most popular database

MySQL Evolution

• MySQL 5.1

• MySQL 5.5

• MySQL 5.6

• MySQL 5.7

• MySQL 8.0

7

MySQL Evolution

• MySQL 5.1 Partitioning

• MySQL 5.5

• MySQL 5.6

• MySQL 5.7

• MySQL 8.0

8

MySQL Evolution

• MySQL 5.1 Partitioning

• MySQL 5.5 InnoDB

• MySQL 5.6

• MySQL 5.7

• MySQL 8.0

MySQL Evolution

• MySQL 5.1 Partitioning…

• MySQL 5.5 InnoDB

• MySQL 5.6 Replication

• MySQL 5.7

• MySQL 8.0

10

MySQL Evolution

• MySQL 5.1 Partitioning…

• MySQL 5.5 InnoDB

• MySQL 5.6 Replication

• MySQL 5.7 InnoDB Cluster

• MySQL 8.0 …

11

MySQL Innovation: 5.7 -> 8.0

- 3x Better Performance - Replication Enhancements - Optimizer Cost Model - JSON Support - Improved Security - Sys & Performance Schema - GIS

MySQL 5.7

MySQL InnoDB Cluster - MySQL Group Replication - MySQL Router - MySQL Shell

MySQL 8.0

- 2x Better Performance - Native Data Dictionary - Unicode - Updated Security - New MySQL Clients - Advanced SQL - InnoDB Redo Log - Instant Add Column - Replication improvements - NoSQL Document Store

3+ Years in Development (since 10/2014) 400+ WL (https://dev.mysql.com/worklog/) 5000+ Bugs Fixed 500 New Tests

GA

New MySQL 8.0 means also

• Oracle License (Std and EE) or GPLv2 (Community)…

• New Clients and Connector

• New MySQL Workbench 8

• New Docker Images and MySQL Operator

• New MySQL Enterprise Backup

• New MySQL Enterprise Monitor

• New Security features

• Overworked Plug-Ins

13

Agenda

• MySQL “Status Quo Nov 2018”

• Key SQL updates in MySQL 8

• MySQL Shell

• NoSQL – No SQL in MySQL anymore?

• MySQL in 6 months … what you can expect

Intro: One Giant Leap for SQL: MySQL 8.0 Released

“This is a landmark release as MySQL

eventually evolved beyond SQL-92

and the purely relational dogma.

Among a few other standard SQL

features, MySQL now supports window

functions (over) and common table

expressions (with). Without a doubt,

these are the two most important post-

SQL-92 features.”

https://modern-sql.com/blog/2018-04/mysql-8.0

SQL Updates in MySQL 8.0

• Window Functions

16

SQL Updates in MySQL 8.0

• Window Functions

• Common Table Expression (Cursive/Recursive)

17

SQL Updates in MySQL 8.0

• Window Functions

• Common Table Expression (Cursive/Recursive)

• Invisible Indexes

18

SQL Updates in MySQL 8.0

• Window Functions

• Common Table Expression (Cursive/Recursive)

• Invisible Indexes

• Functional Indexes

19

Index over an expression CREATE TABLE t1 (col1 INT, col2 INT, INDEX func_index ((ABS(col1))));

CREATE INDEX idx1 ON t1 ((col1 + col2));

CREATE INDEX idx2 ON t1 ((col1 + col2), (col1 - col2), col1);

Agenda

• MySQL “Status Quo Nov 2018”

• Key SQL updates in MySQL 8

• The new MySQL Shell

• NoSQL – No SQL in MySQL anymore?

• MySQL in 6 months … what you can expect

MySQL Shell

Vision

“Integrated development and administration shell where all MySQL products will be available through a common scripting interface”

Motivation

• Ease new product integration

• One For All: Same interface for all products, JavaScript/Python

• All In One: Expose common use cases for different products

Introduction

21

MySQL Shell 1.x

The first iteration

Confidential - Oracle Internal

22

JavaScript

SQL

Python

Output Formats (Table, JSON, Tabbed)

Document Store X DevAPI

(Developer Preview)

SQL CLI

InnoDB Cluster

Batch Execution

MySQL Server 5.7

MySQL Shell 8.x

The second iteration

23

JavaScript

SQL

Python

Output Formats (Table, JSON,

Tabbed, Vertical)

Batch Execution

MySQL Server 5.7

MySQL 8.0 Upgrade Checker

Prompt Themes Auto Completion

& Command History

MySQL Server 8.0

Document Store X DevAPI

SQL CLI

InnoDB Cluster

MySQL Shell 8.0

Display useful information

– Session data

– Status, global or session variables.

– Environment variables.

– User defined variables.

Additional features –Auto Completion

–Persistency of the command history across sessions

–The ability to customize the information shown on the Shell prompt

Prompt Themes

25

MySQL Group Replication

InnoDB Cluster

26

App Servers with MySQL Router

InnoDB Cluster

MySQL Group Replication

27

App Servers with MySQL Router

InnoDB Cluster

MySQL Group Replication

MySQL Shell Setup, Manage,

Orchestrate

App Servers with MySQL Router

InnoDB Cluster

MySQL Group Replication

MySQL Shell Setup, Manage,

Orchestrate Clients

1.) mysqlsh - 2.) dba.deploySandboxInstance(3501) 3.) dba.deploySandboxInstance(3502) 4.) dba.deploySandboxInstance(3503) 5.) \connect root@localhost:3501 - 6.) mc = dba.createCluster("mycluster") 7.) mc.addInstance("root@localhost:3502") 8.) mc.addInstance("root@localhost:3503") - mysqlrouter --bootstrap localhost:3501 --user=root

MySQL 8 Upgrade Check

• “mysqlcheck --check-upgrade” - does not detect all problems that might result in failed upgrade

• Goal of Upgrade checker is to point out backwards incompatible changes from the new server version that will affect the specific server it runs on

– Automate checks that need to be performed

– Offer advice targeted advice to encountered problems e.g. • Usage of db objects with names conflicting with reserved keywords in 8.0

• Usage of utf8mb3 charset

• Usage of use ZEROFILL/display length type attributes

• check table x for upgrade

• Table names in the mysql schema conflicting with new tables in 8.0

• Usage of old temporal type

• Foreign key constraint names longer than 64 characters

Help customers *before* the Upgrade

MySQL Server 5.7

MySQL Server 8.0

NoSQL Schemaless JSON Collections

MySQL Relational Tables

X Dev API SQL

CRUD

MySQL

Document Store

MySQL Shell 8.0

"An easy, straight forward way to work with JSON documents in MySQL"

• A Document-oriented database built on top of MySQL

• Relational database aspects are abstracted

– No tables, no columns, no schema

– Just JSON documents in collections

Document Store - Overview

• Collection:

– New Schema Object

– Stored as InnoDB Table

– ACID compliance

– Transaction Support

– Replication Support

– Row Locking

Agenda

• MySQL “Status Quo Nov 2018”

• Key SQL updates in MySQL 8

• MySQL Shell

• NoSQL – No SQL in MySQL anymore?

• MySQL in 6 months … what you can expect

What is the MySQL Document Store?

"An easy, straight forward way to work with JSON documents in MySQL"

JSON: Why is it popular?

• Simple data format that allows programmers to store and communicate sets of values. Its ease of use, speed of processing and easy integration made it extremely popular for Web-based applications

• JSON's lack of a predefined schema makes it easy for developers to store and update documents

• As JSON adoption has grown, JSON-centric document databases have become more popular

MySQL 5.7+ integrated JSON features, resulting in a best-of-both worlds benefit to developers and database administrators

Developer Friendly

How DBAs see data How Developers see data

{ "GNP" : 249704, "Name" : "Belgium", "government" : {

"GovernmentForm" : "Const i tut ional Monarchy, Federation",

"HeadOfState" : "Phi l ippe I " } , "_ id " : "BEL", "IndepYear" : 1830, "demographics" : {

"Population" : 10239000, "LifeExpectancy" : 77.8000030517578

} , "geography" : {

"Region" : "Western Europe", "SurfaceArea" : 30518, "Continent" : "Europe"

} }

Copyright @ 2018 Oracle and/or its affiliates. All rights reserved.

36

MySQL 8.0: Document Store

• Document oriented data storage for MySQL

– Full JSON document support through SQL and new X DevAPI NoSQL interface

• Schema-less and schema based data in the same technology stack

– Use COLLECTIONs of documents & relational TABLEs together

• Rapid Prototyping & Simple CRUD APIs

– Modern APIs using “method chaining” and asynchronous execution (e.g. promises, callbacks, etc.)

• Connectors for many different languages and frameworks

– Node.JS, Java, NET, C++/C, PHP, Python

NoSQL + SQL = MySQL

Basic Concepts

MySQL – SQL (:3306) MySQL – DocumentStore (:33060)

database/schema database

table collection

record document

column field

MySQL Document Store: Architecture

Copyright @ 2018 Oracle and/or its affiliates. All rights reserved.

ACID compliance

Transaction Support

Replication Support InnoDB Engine

Basic Concepts – Databases/Schemas, Table/Collections

Basic Concepts Tables/Collections (Schemaless)

Basic Concepts Tables/Collections

Basic Concepts select (\sql)

Basic Concepts select (\js)

Collection Search – find(), bind(), fields()

• Supports many operators to specify searches

– ||, &&, XOR, IS, NOT, BETWEEN, IN, LIKE, !=, <>, >, >=, <, <=, &, |, <<, >>, +, -, *, /, ~, %.

• Searching

– db.CountryInfo.find("GNP > 500000 and demographics.Population < 100000000")

– db.CountryInfo.find("GNP*1000000/demographics.Population > 30000")

• Binding - bind()

– db.CountryInfo.find("Name = :country").bind("country", "Italy")

• Project Results – fields() – returns specific fields

– db.CountryInfo.find("Name = :country").bind("country", "Italy")

44

Basic Concepts changing data(\js)

js> session.createSchema('name') js> db.getCollections() js> db.createCollection(‘myCollection') js> db.myCollection.add({“param1":“value1", “param2":“value2”}) js> db.myCollection.find()

js> db.myCollection.find().limit(1)

js> db.myCollection.find("_id = '00005af018430000000000000002'")

js> db.myCollection.modify("_id = '1234'").set(“param",“value")

js> db.myCollection.remove("_id = '1234'")

js> session.startTransaction()

js> …

js> session.rollback()

Copyright @ 2018 Oracle and/or its affiliates. All rights reserved.

MySQL Document Store demo “cheat sheet”

Create

Read

Update

Delete

Transaction

CRUD Operations in php NoSQL/Document

/* open a new connection */ $session = mysql_xdevapi\getSession("localhost","root", "secretpwd"); /* CREATE a new collection and document*/ $schema = $session->getSchema("products"); $collection = $schema->createCollection("best_products"); $product = array('name' => 'superdent', ' quantity' => 42); $collection->add($product)->execute(); /* READ documents */ $largestock = $collection->find()->sort(“quantity desc")->limit(3)->execute();

CRUD Operations in php NoSQL/Document

/* UPDATE a document*/ $collection->modify("name like 'SuperDent'")->set(“quantity", 15)->execute(); $collection->modify("name like :pr_name")->bind(["pr_name" => $productName])-> set(“quantity", 15)->execute(); $collection->modify(“quantity < 10")->set("comment", "order more")->execute(); /* DELETE documents */ $collection->remove("name like 'Super%'")->execute();

DBMS or NoSQL ?

Why not both ?

Copyright @ 2018 Oracle and/or its affiliates. All rights reserved.

Basic Concepts

Basic Concepts SQL & NoSQL (joining the worlds)

Agenda

• MySQL “Status Quo Nov 2018”

• Key SQL updates in MySQL 8

• The new MySQL Shell

• NoSQL – No SQL in MySQL anymore?

• MySQL in 6 months … what you can expect

• Powerful union of MySQL Enterprise Edition & Oracle Cloud Infrastructure – Amazing underlying software and high-performance hardware

• Self-managed & Easy to use – Automates most common DBA tasks: backups, patches,

updates, replication configuration, etc.

– Web console, REST API, CLI, SDKs, and seamless integration

• Elasticity & High Availability – Replication across different physical locations

• Security & Compliance – Ready-to-use advanced security options and compliant with

the most demanding Enterprise regulations

MySQL as a Service

Oracle Responsibility

• Backup and recovery

• High-availability with automatic fail-over

• Vertical and horizontal scaling

• Database and OS patching

• Monitoring and log handling

• Security with advanced options available in MySQL Enterprise Edition

User Responsibility

• Logical schema modeling

• Query design and optimization

• Define data access and retention policies

54

Managed MySQL

A Joint Venture

RAPID

MySQL Oracle Labs

• Ground breaking technology from Oracle Labs

• Extreme Performance with In-Memory Analytics

• Integrated with MySQL – Can instantiate a new service or add to a running MySQL service

• Support for Real-Time SQL Analytics – Analytics provided over the same transactional store

• Architected for Oracle Cloud Infrastructure – Designed for Massive Elasticity

• Supports data on premise or in the cloud

MySQL as a Service

Analytics

Architecture

MySQL Server

InnoDB RAPID Client RAPID Server

Client/App

Dat

a Lo

ad

SQL

Qu

ery

Qu

ery

Res

ult

Town State Population

Ontario CA 158007

Salinas CA 151060

Rockford IL 150115

Pomona CA 149474

Lakewood CO 144126

Hampton VA 146437

• MySQL

• Fastest OLTP

• RAPID

• Fastest OLAP

• Columnar

• In memory

• Vectorized

• Distributed

• Initial LOAD direct from InnoDB to RAPID

• Real-time updates for subsequent writes

RAPID MySQL Integration Architecture

MySQL Server

RAPID Cluster

Manager

RAPID Load

Module

RAPID Exec

Client

RAPID Compiler

& Optimizer

Global State DS Network Layer

RAPID

RAPID Server Node 1

RAPID Server Node 2

RAPID Server Node N

Storage Engine

(InnoDB) Single System for OLTP & Analytics

Provides Real-Time Analytics Support

59

Carsten Thalheimer

Carsten.Thalheimer@Oracle.com // M: +49 172 / 8833386 You can meet me @DOAG 2018 (320)