OCCIwareOne Cloud API To Rule Them All

Marc Dutoo, Open Wide

Etienne Juliot, Obeo

EclipseCon France, ToulouseJune, 09th 2016

OverviewSpeaker

– Marc Dutoo, Head of R&D Dept. at Open Wide, a Smile group company● OCCIware coordinator, Data / Cloud expert

– Etienne Juliot, Obeo Founder● Eclipse Sirius expert

Schedule

– 7' the need for a Cloud consumer platform– How did we get there ?

● 7' OCCI(ware) introduction● 7' Eclipse Modeling approach● Demo : 7' tooling, 7' runtime

Once upon a time...

Once upon a time...

• In a hole in the ground there lived a hobbit.

Once upon a time...

• In a hole in the ground there lived a hobbit.• On his shoulders the fate of the world came to rest.• In order to save creation, he had to cross the world and drop a ring in a volcano to rid it of evil.

Wait a minute.

Err... sorry, this would have been easy peasy, but that's folktales. Let me try again :

• In order to save creation, he had to monitor energy consumption everywhere on Earth to overturn climate change.

Now we're in the real world here.

Monitor energy consumption

• Not only per user, or per utility provider company,• but per city, region, country,

- and per activity, usage, number of children, amount of hair on feet would most probably help also.

Multi-point of view data... that calls for an open world approach of data – that's Linked Data.

Linked Data & Datacore

• Linked Open Data ? That's Open Data sets that can be cross-queried because they have been semantically reconciled together• Enter Ozwillo Datacore :

- it holds data that is shared between applications of the Ozwillo app store : geographical elements, organizations, reusable app business data…- it stores it in a shardable replicated MongoDB and is built in Apache CXF / Spring & Java- it provides it through a REST API that is both web-friendly and semantic web-compatible thanks to JSON-LD, and helps developers with a Playground.

Linked Data, as a Service

Linked Data as a Service (LDaaS)

But not your dad's Linked Data. If you want it to save the world, it has to scale up the whole way :

• at Infrastructure level - that's IaaS : provision enough CPU & storage, for each city• at Platform level - that's PaaS : deploy java code and mongo replicated cluster shards on each of them• at Software level - that's SaaS : configure Linked Data governance i.e. models and kinds of use :

- high write, non-robust data collection (home energy consumption sensor notifications)- read/query-heavy data analysis (aggregation per energy consumer, provider, city, region, country)

Cloud Computing - The solution

• Luckily for our hobbit, there are well-known standards, solutions and tools.

The solution The problem

• Oops… too many actually.

Cloud Computing – the problem

Cloud Computing – more problems

And that's not like choosing one single provider for all layers would solve the problem :• even if there are a few PaaS+SaaS providers (G..., C... F...),• nowadays hybrid cloud is not a question anymore but the rule (save for too critical or not at all critical stuff),• (which brings the interesting question of multi Cloud monitoring)• and our hobbit still has to do the SaaS layer and the whole LDaaS assembly himself.

From a single Cloud provider...

… to a noodle plate

Cloud Computing – down the hobbit hole

But our hobbit just wants to be a Cloud consumer !• he doesn't want to become a Cloud provider ! there are already plentiful expert ones out there, he doesn't want to deploy Cloud provider solutions, nor use the specific expert tool required for each of them.

- He just wants to smoke a good pipe !

• He'd need his own tools, but he doesn't want to write them.

=> Isn't there a Cloud consumer platform out there ? Hello ?

A light of hope

• But in this fragmented landscape of layers, APIs and implementations, each coming with its own tools and paradigm,•there is hope to be able to control them.•to bind them and rule them all.•(did I mix up the script here ?)

A light of hope

• One ring one Cloud API to rule them all.

Cloud Studio, with Docker

Custom Studio for LdaaS (Linked Data as a Service)

Docker Studio

IaaS - Virtual Box machine

Docker Studio

IaaS - Open Stack machine

Docker Studio

… started !

Linked Data Studio

SaaS - Linked Data with dedicated analytics entry point

Linked Data Studio

… using a specific mongodb replica within its cluster !

How did we get there ?- 1. OCCI(ware) introduction

OCCIware Product

OCCIware Studio

Design, Verify, Simulate & DevelopEverything as a Service

OCCIware Runtime

Deploy, Execute & ManageEverything as a Service

OCCI specifications

OCCI 101OCCI Core (metamodel)

Network

Container

Environment

Compute- memory- started- start()

Database

Application

Storage

Router

Deployable

NetworkLink

DatabaseLink

EnvironmentLink

Everything is Resource or Link, be it at …

Platformlevel

Infrastructurelevel

Applicationlevel

OCCI Extensions (models)

OCCIware Factsheet

• 72 man year, 5,6m€ budget, sponsored by French ministry of Industry over 2015-2018• 3 academics, 5 companies, 2 associations• To lower Cloud Computing adoption costs and break up barriers between its various implementations, layers, domains

- Especially Data Center, deployment, Big Data, Linked Data

• By bringing to OGF's Open Cloud Computing Interface (OCCI) the power of formal languages and model driven engineering (MDE)

OCCIware Objective

Managing Everything as a Service in the clouds

Software as a Service (SaaS)

Big Data as a Service (BDaaS)

Linked Data as a Service (LDaaS)

Platform as a Service (PaaS)

Container as a Service (CaaS)

Infrastructure as a Service (IaaS)

DataCenter as a Service (DCaaS)

Network as a Service (NaaS)

How did we get there ?- 2. Eclipse Modeling approach

Cloud Designer – modeleur de configuration générique

Occi tools creation with EMF

Ecore

Your Domain Model

Your Model

EClass, EReference

Occi standard

Occi Extension

Occi Configuration

Entity, Kind, Link

Docker, Infrastructure

Server 1, VM Foo

Occi tools creation with EMF

Occi standard

Occi Extension

Occi Configuration

Entity, Kind, Link

Docker, Infrastructure

Server 1, VM Foo

Ecore

Your Domain Model

Your Model

?

EClass, EReferenceEcore

EClass, EReference

Domain Model « Promotion »

Occi.ecore

MyExtension.occi

Entity, Kind, Link

Docker, Infrastructure

EcoreEClass, EReference

Ecore

MyExtension.ecore

MyConfiguration.docker

EClass, EReference

Server 1, VM Foo

Docker

OCCI vs Ecore

● OCCI is a meta-meta model but we don't want to reinvent the wheel :

1) Creation of your OCCI Extension conformed to OCCI.ecore

2) Automatic generation of yourextension.ecore domain model !!

3) Creation of your OCCI Configuration conformed to yourextension.ecore

OCCIware Studio Factory

<<E>>DesignerSkeleton

e.ecore

<<E>>Extension

e.genmodel

e.odesign

Occie2Ecore

Ecore2Sirius EMF genmodel

EMF Java code

EMF Java code

EMF Java code

<<E>>Configuration

<<E>>Configuration

<<E>>Configuration

Making this process generic

● Generate a metamodel (ecore, genmodel, code) from an OCCI Extension

● Generate the skeletton of a Sirius designer, ready to be customized

Making this process generic

Making this process generic

How did we get there ?- 3. Step-by-step demo

- File > New > Other…

- Cloud Designer > OCCI Extension Project

New Linked Data OCCI Extension - wizard

OCCI Extension designer

Linked Data Extension – adding kind

Linked Data Extension – adding attribute

Linked Data Extension – adding parent

Linked Data – 3 target scenarii

3.2.1.

Linked Data Extension – modeling configuration of all 3 target scenarii

1. 2.3.

Linked Data Extension – alternate DSL editor

Linked Data Extension – generate...

Linked Data Extension – generated files

Linked Data Extension – generated doc

Linked Data Extension – generated runtime configuration

Linked Data Extension – register

.ecore > generate Linked Data Designer

Linked Data Designer – Sirius editor

Linked Data Designer – adding Drag'n'Drop from another Cloud layer configuration - 1

Linked Data Designer – adding Drag'n'Drop from another Cloud layer configuration - 2

Running Linked Data Designer

Linked Data Designer – original LDProjects

Linked Data Designer – adding analytics-optimized LDProject

Linked Data Demo – Docker Studio

Linked Data Demo – configuring 1 java node & a 3-node mongodb cluster in Docker Studio

Linked Data Demo – Docker OCCI configuration

Linked Data Designer – drag'n'dropping Infrastructure configuration from other file

Linked Data Designer – drag'n'dropped Virtual Machine from Infrastructure config.

Linked Data Designer – selecting secondary mongo to use for analytics instead of primary

- File > New > Other…

- Cloud Designer > Java Connector Project

Linked Data Connector – code setup of Datacore API's Java client

Linked Data Connector – code action by calling Datacore API using its client

OCCI attributes and links are auto injected :

Linked Data Demo – Datacore before update

Data project geo_analytics_1 doesn't exist :

Debugging a query shows that data is accessed through the mongo cluster's primary :

Linked Data Demo – doing « update » action

Linked Data Demo – Datacore after update

Data project geo_analytics_1 has been created :

Debugging a query shows that data is accessed through the configured mongo secondary :

Upcoming in OCCIware

- Studio : more generators & connectors, improve existing ones, integrate simulator, decision-making tool… - Complete erocci-dbus-java bridge & HTTP bridge, so that Studio-generated connectors can be deployed within the Studio, on erocci or standalone indiscriminately - Runtime : complete OCCIfication of ActiveEon ProActive Multi-IaaS connector and Roboconf PaaS manager - Develop specified OCCI monitoring solution and OCCI administration console's live Playground - Develop use cases : Datacenter, Big Data, Deployment, Linked Data - … and contribute back to OCCI 2.0 standard !

Any questions ?Thanks for your attention !

Contact : http://www.occiware.org - marc.dutoo at openwide.fr

Source : https://github.com/occiware

Partners :

Sponsors :DGE (PIA) & System@tic, SCS, Images & Réseaux, PICOM, Minalogic clusters