slide guss 2014 events hub - future of integration

34
#JSS2014 Les journées SQL Server 2014 Un événement organisé par GUSS

Upload: radoine-douhou

Post on 25-Jul-2015

223 views

Category:

Software


0 download

TRANSCRIPT

Page 1: Slide guss 2014   events hub - future of integration

#JSS2014

Les journées

SQL Server 2014

Un événement organisé par GUSS

Page 2: Slide guss 2014   events hub - future of integration

#JSS2014

Les journées

SQL Server 2014

Un événement organisé par GUSS

Session « Le futur de l’intégration »

Michel Hubert – MVP Azure

Radoine Douhou

Page 3: Slide guss 2014   events hub - future of integration

#JSS2014

• Etat des lieux.

• Futur de l’intégration.

• Solutions d’intégration orientées Cloud.

• Solutions d’intégration orientées Events & IoT.

• Démo.

Agenda

Page 4: Slide guss 2014   events hub - future of integration

#JSS2014

Evolution of Integration

Once upon a Time

Point to point (“Spaghetti”)

Integration broker (EAI/B2B)

Enterprise Service Bus

HR

CRM E-commerce

ERP

Business partner

HR

CRM E-commerce

ERP

Business partner

HR

CRM E-commerce

ERP

Page 5: Slide guss 2014   events hub - future of integration

#JSS2014

BizTalk Server

Partners

Evolution of Integration

Microsoft BizTalk Server

Page 6: Slide guss 2014   events hub - future of integration

#JSS2014

BizTalk Server ESB toolkit

e-Commerce

Suppliers

EDI

EDIFACT / AS2

WS

RESTFul

Market place

warehouse

Magasins

SI Back Office

Orchestration B2B

BI

SSIS

SSRS

SSAS

Microsoft BizTalk Server

Common scenario

Page 7: Slide guss 2014   events hub - future of integration

#JSS2014

FUTUR DE L’INTEGRATION

Nouveaux usages

Page 8: Slide guss 2014   events hub - future of integration

#JSS2014

Point to point (“Spaghetti”)

Integration broker (EAI/B2B)

Enterprise Service Bus What’s next?

?HR

CRM E-commerce

ERP

Business partner

HR

CRM E-commerce

ERP

Business partner

HR

CRM E-commerce

ERP

Future of Integration

What’s next ?

Page 9: Slide guss 2014   events hub - future of integration

#JSS2014

Future of

Integration

Mobiles

Events

Messaging

EAI/ESB

Business Process Automation

Telemetry

Analytics

BizTalk Server

BizTalk Services

Service Bus

Queuing

Events Hub

Stream Analytics

Hybrid Connection

B2B

Future of Integration

Integration = Cloud + Mobile + IoT

IoT

SaaS

Page 10: Slide guss 2014   events hub - future of integration

#JSS2014

SOLUTIONS D’INTEGRATION ORIENTEES

CLOUD

BizTalk Server / BizTalk Services

Page 11: Slide guss 2014   events hub - future of integration

#JSS2014

Upgrade to BizTalk Server 2013 R2

with confidence

Accelerate vertical solutions with

standards and platform support

Extend existing solutions to Azure

Service Bus Adapters

RESTFul/Json Adapter

BizTalk Server 2013 R2

Cloud Ready

Page 12: Slide guss 2014   events hub - future of integration

#JSS2014

BizTalk Server

Microsoft Azure

PartnersBizTalk Services

Service Bus

Future of Integration

Federated Enterprise Service Bus

Page 13: Slide guss 2014   events hub - future of integration

#JSS2014

Upgrade to BizTalk Server 2013 R2

with confidence

Accelerate vertical solutions with

standards and platform support

Extend existing solutions to Azure

Develop and test integration

solutions in the cloud without re-

writes

Clear up on-premises infrastructure

capacity and provision in minutes

instead of weeks

Service Bus Adapters

RESTFul/Json Adapter

BizTalk Server 2013 Virtual Machines

BizTalk Server 2013 R2

Cloud Ready

Page 14: Slide guss 2014   events hub - future of integration

#JSS2014

App owner Datacenter admin

Customer data center

Microsoft Azure data center

LOB app Active Directory

BizTalk LOB app

SQL

Agility for app owners Control for IT pros

BizTalk Server 2013 R2

Develop in Azure

Page 15: Slide guss 2014   events hub - future of integration

#JSS2014

Upgrade to BizTalk Server 2013 R2

with confidence

Accelerate vertical solutions with

standards and platform support

Extend existing solutions to Azure

Develop and test integration

solutions in the cloud without re-

writes

Clear up on-premises infrastructure

capacity and provision in minutes

instead of weeks

Enable hybrid extension of current

on-premises BizTalk Server

deployments with Azure BizTalk

Services

Power new cloud-hosted business-

to-business, Internet of Things, and

EDI capabilities

Service Bus Adapters

RESTFul/Json Adapter

BizTalk Server 2013 Virtual Machines

Microsoft Azure BizTalk Services

BizTalk Server 2013 R2

Cloud Ready

Page 16: Slide guss 2014   events hub - future of integration

#JSS2014

Microsoft Azure BizTalk Services (MABS)

Page 17: Slide guss 2014   events hub - future of integration

#JSS2014

Equivalent to BizTalk ESB itinerary Transforms

Enrich

Validation XML and xsd schema

MABS

Bridge principle

Page 18: Slide guss 2014   events hub - future of integration

#JSS2014

Hybrid Connections

Web Sites

Mobile Services

Corporate network

Microsoft SQL Server

Hybrid Connection

Other published

resourcesHybrid Connection

Manager

Page 19: Slide guss 2014   events hub - future of integration

#JSS2014

Use Cases

Microsoft Azure

On-premises

AS2 SFTP HTTP

? !

Home office Branch office Regional office

Page 20: Slide guss 2014   events hub - future of integration

#JSS2014

SOLUTIONS D’INTEGRATION ORIENTEES

EVENTS / IOT

Service Bus, Event Hub, Stream Analytics

Page 21: Slide guss 2014   events hub - future of integration

#JSS2014

Azure Service Bus

Relay

Queue

Topic

Notification Hub

Event Hub

NAT and Firewall Traversal Service

Request/Response Services

Unbuffered with TCP Throttling

Many publishers and many consumers to

communicate over a FIFO like channel.

(Competing consumers and Queue-based

Load leveling scenarios)

Pub / Sub communication channel. Each

Consumer subscribes to a copy of message

High-scale notification distribution

Most mobile push notification services

Millions of notification targets

Azure Service Bus

Cloud Messaging Broker

Page 22: Slide guss 2014   events hub - future of integration

#JSS2014

Service Bus Events Hub

Event Producers

Azure Event Hub

> 1M Producers> 1GB/sec

Aggregate Throughput

Up to 32 partitions via portal, more on

request

Part

itio

ns

Direct

PartitionKey

Hash

Throughput Units:

• 1 ≤ TUs ≤ Partition Count

• TU: 1 MB/s writes, 2 MB/s reads

Consumer Group(s)

Receivers

AMQP 1.0

Credit-based flow control

Client-side cursors

Offset by Id or Timestamp

Page 23: Slide guss 2014   events hub - future of integration

#JSS2014

Event Producers

> 1M Producers> 1GB/sec

Aggregate Throughput

Direct

PartitionKey

Hash

• Consistent security model with Service Bus,

extended by publisher policies

• Publisher protocols :

– HTTPS : Short lived, low throughput :

– AMQP : Long lived, High throughput :

• Publish modes :

– Directely to a PartitionId.

– Automatic hash-based distribution by

PartitionKey or Publisher Identity

• Stream or batch

Event Hub

Producers

Page 24: Slide guss 2014   events hub - future of integration

#JSS2014

• 1 Partition : ordered sequence of events

that is held. A partition can be thought of

as a “commit log.”

• Segmentation of the event stream for

scale-out :

– Parallelism on both Publishing and Polling

sides.

• Default 16, minimum 8, self-service maximum

32 :

– 1024 partitions via Azure Support.

Azure Event HubPart

itio

ns

Event Hub

Partition principle

Page 25: Slide guss 2014   events hub - future of integration

#JSS2014

• view (state, position, or offset) of an entire

Event Hub

• Publish/subscribe mechanism of Event

Hubs is enabled through consumer groups.

• Enable multiple consuming applications to

each have a separate view of the event

stream, and to read the stream

independently at its own pace and with its

own offsets

Consumer Group(s)

Event Hub

Consumer Group

Page 26: Slide guss 2014   events hub - future of integration

#JSS2014

• Receive from partitions via consumer

groups

• Client-side cursors allowing to freely

process and repeatedly reprocess the

available retained event stream based on

offsets or timestamps.

• Using .NET API or using generic AMQP 1.0

client (e.g. Apache Proton-C/J)

ID, Time, [Data]

ID, Time, [Data]

ID, Time, [Data]

Time

Id

Event Hub

Consumers

Page 27: Slide guss 2014   events hub - future of integration

#JSS2014

Stream Analytics

Data Source

Collect Process

Consume

Deliver

Event Inputs- Event Hub

- Azure Blob

Transform- Temporal joins

- Filter

- Aggregates

- Projections

- Windows

- Etc.

Enrich

Correlate

Outputs- SQL Azure

- Azure Blobs

- Event Hub

BI

Dashboards

Predictive

Analytics

Azure

Storage

• Temporal Semantics

• Guaranteed delivery

• Guaranteed up time

Azure Stream Analytics

Reference Data- Azure Blob

Page 28: Slide guss 2014   events hub - future of integration

#JSS2014

• Easily filter, project, aggregate, join streams, add static data with streaming data, detect patterns or lack of patterns with a few lines of SQL

• Development and debugging experience through Azure Portal

• Built-in monitoring trough Azure Portal

• No hardware acquisition and maintenance

• Up and running in a few clicks (and within minutes) => Bypasses deployment expertise

• Elasticity of Azure for scale up or scale down

• Distributed, scale-out architecture

• Integrated with highly-

scalable publish-

subscriber Events Hub.

• Transform, augment,

correlate, temporal

operations.

• Detect patterns

and anomalies in

streaming data

• Correlate streaming with

reference data

Stream Analytics

Value added

Page 29: Slide guss 2014   events hub - future of integration

#JSS2014

CLICK STREAM ANALYSIS

Page 30: Slide guss 2014   events hub - future of integration

#JSS2014

Demo architecture

e-Commerce

web site

Log file

Events Producer

ETW

Azure Event Hub

Stream Analytics

Tumbling window

Collect

Transform

SQL Azure

Deliver

Power View

Real Time

analysis

AMQP Client

Page 31: Slide guss 2014   events hub - future of integration

#JSS2014

Merci à nos sponsors

Page 32: Slide guss 2014   events hub - future of integration

#JSS2014#JSS2014

Les évaluations des sessions,

c’est important !!

…Et en plus on peut gagner

des cadeaux

Page 33: Slide guss 2014   events hub - future of integration

#JSS2014

http://GUSS.Pro/jss

Page 34: Slide guss 2014   events hub - future of integration

#JSS2014#JSS2014