informix sql & nosql: putting it all together

IBM INFORMIX putting it all together

John Miller, Shawn Moe,

Keshav MurthyIBM Informix Development

2

Explosion of mobile

devices – gaming and social apps

Advertising: serving ads

and real-time bidding

Social networking,

online communities

E-commerce, social

commerce

Machine data and real-time operational decisions

Smart Devices

Internet of Things

Internet of Things

3

Explosion of mobile

devices – gaming and social apps

Advertising: serving ads

and real-time bidding

Social networking,

online communities

E-commerce, social

commerce

Machine data and real-time operational decisions

Smart Devices

Internet of Data, really

Internet of Things

SQLSQL SQL, {JSON}, Spatial

SQL, {JSON}, Spatial

{JSON}, TimeSeri

es

{JSON}, TimeSeri

es

SQL, {JSON}

SQL, {JSON}

Simple, {JSON}, Timeseri

es

Simple, {JSON}, Timeseri

es

SQL, {JSON}

SQL, {JSON}

4

IoT Applications – IBM Reference Architecture

Gateway Operational Zone Warehouse/Mart Analytics Services and Contents

Shared Operational Information

Rule Engine

ETL

Real-TimeData Store

Hadoop VideoAnalytics

Big DataExplorer

AnalyticTools

Connected Device

Analyzed Data

Map

Red

uce

HDFS/GPFS

DeviceManagement

:

PredictiveMaintenance

TrafficOptimization

Driving Behavior

IncidentAnalysis

InfotainmentService

Raw Data

SummarizedData

Notification AnalyticReport

B2C/B2BPortal

AdminConsole

OperatorConsole

Loca

l Int

ellig

ence

Net

wor

k S

uppo

rt

StreamProcessing

ET

L

RD

B

Dat

a M

art

SOE Data

VideoManagement

Asset DataManagement

Master DataManagement

ReferenceData Hub

Video Data

..

EnvironmentData, etc.

Other Data

Loca

lD

atab

ase

© 2014 IBM Corporation5

IBM Cloud: Think it. Build it. Tap into it.IoT Solutions, an architecture.

Collection of data for all sensors

Data from other kinds of

sensors

Consumer / Business

Sensors in the home

Informix TimeSeries Service

NoSQL, Relational, Timeseries & Spatial storage & analytics

Informix Warehouse Accelerator

SPSS/Cognos

MessageSight / MQTT

SoftLayer / BlueMix

BigInsights

Gateways for data consolidation

Infosphere Streams

(no gateway)

= IBM products = IBM Informix Relational Database

In-memory analytics

Predictive analytics and dashboard

Cloud infrastructure

Hadoop

Publish / Subscribe

Real-time analytics

• Individual Car Recognition in the parking zone

•Composite sensors to transmit license image

•Cloud service to recognize the car plate number

•Location tracking

• Individual Car Recognition in the parking zone

•Composite sensors to transmit license image

•Cloud service to recognize the car plate number

•Location tracking

IBM Informix developer edition. Download Now:

http://www-03.ibm.com/software/products/en/infodeveedit

IBM Informix developer edition. Download Now:


Handling Big Data

IBM Bluemix

10

SQL {NoSQL:JSON}Define Schema first Write the program first

Relational Key-value, Document, column family, graph and text

Changing schema is hard

Assumes dynamic schema

Scale-up Scale-out

ACID consistency BASE consistency

Transactions No Transactions

SQL Proprietary API; Sometimes has the “spirit” of SQL

11

SQL TimeseriesDefine Schema first Create Timeseries Row

Type

Relational Timeseries Optimized with projection to relational

Changing schema is hard

Changing schema is hard; Change is easy with Timeseries({JSON})

Scale-up Scale-up & Scale-out

ACID consistency ACID consistency

SQL SQL extensions.

Data Management: devices to Cloud

Enterprise replication + Flexible Grid

App Server

JDBC

App ServerMongo Driver

Listener

Informix/1Primary

Informix/1SDS/HDR

Informix/1RSS

Informix/2Primary

Informix/2SDS/HDR

Informix/2RSS

Informix/3Primary

Informix/3SDS/HDR

Informix/3RSS

Informix/4Primary

Informix/4SDS/HDR

Informix/4RSS

Informix/5Primary

Informix/5SDS/HDR

Informix/5RSS

Informix/6Primary

Informix/6SDS/HDR

Informix/6RSS

Mongo APIMongo API

Node.JSNode.JS

Express.JSExpress.JS

AngularJSAngularJS

REST APIsREST APIs

NoSQL

NoSQL

SQLSQL

Device

sDevice

s Gatew

ay

Gatew

ay

Cloud

Cloud

Informix warehouse Accelerator

Informix: All Together Now!

13

SQL Tables

JSON CollectionsTimeSeries

MQ Series

SQL APIsJDBC, ODBC

SQL APIsJDBC, ODBC

Informix

IWA – BLU ACCELERATION

GENBSON: SQL to {BSON}

MongoDB Drivers

MongoDB Drivers

TEXT SEARCH

SPATIALTIME SERIES

{BSON}

Data Model Should

NOT restrict Data

Access

Data Model Should

NOT restrict Data

Access

Universal Schema

Seamless Access

Universal Schema

Seamless Access

SQL APISQL API

MongoDB API

(NoSQL)

MongoDB API

(NoSQL)

Relational TableRelational Table JSON CollectionsJSON Collections

Standard ODBC, JDBC, .NET, OData, etc.Language SQL.


Mongo APIs for Java,

Javascript, C++, C#,...



Direct SQL Access.Dynamic Views

Row types


Row types

Mongo APIs for Java, Javascript, C++, C#,...Mongo APIs for Java,


Hybrid Access: SQL & JSON

SQL APISQL API

MongoDB API

(NoSQL)

MongoDB API

(NoSQL)

Relational TableRelational Table JSON CollectionsJSON Collections








Row types


Row types



JSON CollectionsJSON CollectionsJSON CollectionsJSON Collections

Standard SQL/extJDBC/ODBC

JSON Support


JSON Support

Virtual TableJSON supportVirtual TableJSON support

TimeseriesTimeseriesJSON CollectionsJSON Collections TimeseriesTimeseriesRelational TableRelational TableJSON CollectionsJSON CollectionsTimeseriesTimeseries

Hybrid Access:SQL, JSON & Timeseries


JSON Support


JSON Support

TimeseriesTimeseries


JSON Support


JSON Support




JSON Support


JSON Support




JSON Support


JSON Support




JSON Support


JSON Support




JSON Support


JSON Support


SQL APISQL API

Mongo API

(NoSQL)

Mongo API

(NoSQL)

Relational TableRelational TableJSON CollectionsJSON Collections








Row types


Row types



JSON CollectionsJSON CollectionsJSON CollectionsJSON Collections


JSON Support


JSON Support


TimeseriesTimeseriesJSON CollectionsJSON Collections TimeseriesTimeseriesRelational TableRelational Table JSONJSON TimeseriesTimeseriesSpatial

Text

SpatialText

Standard SQLJDBC/ODBC

JSON Support

Standard SQLJDBC/ODBC

JSON Support

JSON SupportJSON Support

Hybrid Access:SQL, JSON, Timeseries & Spatial

Mapping A JSON To A SQL Table

CREATE TABLE photos(data BSON);

SELECT data.GPSLatitude::lvarchar as GPSLatitude,

data.GPSLongitude::lvarchar as GPSLongitude,

data.Make::varchar(64) as Make,

data.oateTimeoriginal::datetime year to day as dt,

data.exposuretime::int

data.pixelxdimension::float,

data.pixelydimension::float

FROM photos;

20

Timeseries on JSON

CREATE ROW TYPE info( stime datetime year to fraction(5),

jdata bson);

CREATE TABLE iotdata(id int primary key, tsdata timeseries(info) );

INSERT INTO iotdata VALUES(472,'origin(2014-04-23 00:00:00.00000), …, regular,[({“temp":78, “wind":7.2, “loc":“Miami-1 "})]');

INSERT INTO iotdata values(384,'origin(2014-04-21 00:00:00.00000), …, regular,[({“sleep": 380, “steps":7423, “name":"Joe "})]');

SELECT GetFirstElem(tsdata,0)::row(timestamp datetime year to fraction(5), jdata json) FRONM tj;

(expression) ROW('2014-04-21 00:00:00.00000','{“sleep":380,“steps":7423,“name":"Joe "}')

…

21

Timeseries on JSON

CREATE TABLE iotvti(id INT PRIMARY KEY,

stime DATETIME YEAR TO FRACTION(5)),

jdata BSON);

SELECT id,

jdata.temp::int,

jdata.loc::varchar(32)

FROM iotvti WHERE jdata.temp > 75;

db.iotvti.find({“jdata.temp”:{$gt:75}, {jdata:1}, {jdata:1});{“temp":75, “wind":7.2, “loc":“Miami-1 "}

22

Informix REST API

REpresentational State Transferhttp://<hostname>[:<port#>]/<db>/<collection>

Integrated into Informix

GET /demo/people?sort={age:-1}&fields={_id:0,lastName:0}RESPONSE: [{"firstName":"Anakin","age":49},

{"firstName":"Padme","age":47},{"firstName":"Luke","age":31},{"firstName":"Leia","age":31}]

GET /stores_demo/ts_data_v?query={loc_esi_id:"4727354321046021"}

23

A va ilab le M e thodsM e th o d P a th D e scrip tio n

P O S T / C re a te a n e w d a ta b a se

P O S T /d b C re a te a n e w co lle c tio n

P O S T /d b /co lle c tio n C re a te s a n e w d o cu m e n t

G E T / D a ta b a se lis tin g

G E T /d b C o lle c tio n lis tin g

G E T /d b /co lle c tio n Q u e ry th e co lle c tio n

D E L E T E / D ro p a ll d a ta b a se s

D E L E T E /d b D ro p a d a ta b a se

D E L E T E /d b /co lle c tio n D ro p a co lle c tio n

D E L E T E /d b /co lle c tio n ? q u e ry= {...} D e le te d o cu m e n ts th a t sa tis fy th e q u e ry fro m a co lle c tio n

P U T /d b /co lle c tio n U p d a te a d o cu m e n t

INFORMIX REST API

ODBC, JDBC connections

Informix Dynamic Server

Tables

Tables

Relational Tables and views


JSON CollectionsJSON Collections

{Customer}{Customer}

partnerspartners

SQL & BI Applications


{Orders}{Orders}

CRMCRM

InventoryInventory

TablesTimeseries TablesTimeseries Tables

{mobile/devices}{mobile/devices}

Analytics


Informix Database Server


BI Applications

Data mart Tools

Ready

IBM Smart Analytics Studio


Tables

Tables





partnerspartners



{Orders}{Orders}

CRMCRM

InventoryInventory


{Orders}{Orders}

Text index (BTS) spatial indicesText index (BTS) spatial indices

Informix Warehouse Accelerator – In-Memory Query Engine


SQL Apps/Tools

MongoDB Drivers

NoSQL Apps/Tools

Mongo clientMongo client

Node.JSNode.JS

Express.JSExpress.JS

AngularJSAngularJS

IWA: Complex Data Analysis


Informix Warehouse AcceleratorBI Applications


Factdim1

Dim4 - View

dim3

dim2

dim2Informix

IoT ApplicationsLoB Apps

IoT Applications

BI Applications

Mobile Apps

Informix

IWA: Complex Data Analysis




SQL Table

SQL View

{JSON}

SQL Table

SQL TableInformix

LoB Apps

IoT Applications

BI Applications

Mobile Apps

Informix

Timeseries

{JSON}

{JSON}Cognos

SQL Table



Tables

Tables





partnerspartners



{Orders}{Orders}

CRMCRM

InventoryInventory


{mobile/devices}{mobile/devices}

Analytics


Hybrid Power for IoT AppsRight type for right data – SQL, JSON, Timeseris,

Spatial, TextVariety of APIs: REST, JDBC, ODBCPlatform & device optionsRight size for the right problem – Customize What’s good for embed is good good for the cloudContinuous availabilityAccelerate with Informix Warehouse AcceleratorMix it in with BluemixScale-up, Scale-out

IBM Informix developer edition.


IBM Informix developer edition.


Download

Now!Download

Now!

Thank you

informix sql & nosql: putting it all together

Software