a new platform for a new era - sdjug · title: springoneplat - data microservices created date:...
TRANSCRIPT
![Page 1: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/1.jpg)
A NEW PLATFORM FOR A NEW ERA
![Page 2: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/2.jpg)
© Copyright 2014 Pivotal. All rights reserved.© Copyright 2014 Pivotal. All rights reserved.
Data Microservices with Spring
2
Breaking the enterprise integration monolith
Fred Melo @fredmelo_br
![Page 3: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/3.jpg)
© Copyright 2014 Pivotal. All rights reserved.© Copyright 2014 Pivotal. All rights reserved. 3
About Me
@fredmelo_br
Fred Melo
Mountain View, CA
https://www.linkedin.com/in/melofred
![Page 4: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/4.jpg)
© Copyright 2014 Pivotal. All rights reserved.
![Page 5: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/5.jpg)
© Copyright 2014 Pivotal. All rights reserved.
![Page 6: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/6.jpg)
© Copyright 2014 Pivotal. All rights reserved.
User Mgmt
Order Mgmt
Shipping
Inventory
Payment
Payment
Order Mgmt
Inventory
User Mgmt
Shipping
Contract
Contract
Contract
ContractContract
Conway’s Law and Microservices
Going Agile - Breaking the monolith
Dev Team
![Page 7: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/7.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Deploy
> cf push
Cloud Controller
BBS (message bus)
RouterApp Deployed
Order Mgmt
Contract
Automating through the Platform
![Page 8: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/8.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Scale
Re-deploy
Migrate
Distribute
Upgrade UpdateOrder Mgmt
Contract
User Mgmt
Contract
Pivotal Cloud Foundry
![Page 9: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/9.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Integration Bus / ESB
System A
System BUser Mgmt
Order Mgmt
Shipping
Inventory
Payment
![Page 10: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/10.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Integration Bus / ESB
System A
System BOrder Mgmt
Contract
InventoryContract
ShippingContractUser
Mgmt
Contract
![Page 11: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/11.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Integration Bus / ESB
System A
System B
ETL
Data Warehouse
Order Mgmt
Contract
InventoryContract
ShippingContractUser
Mgmt
Contract
![Page 12: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/12.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Integration Bus / ESB
System A
System BOrder Mgmt
Contract
InventoryContract
ShippingContractUser
Mgmt
Contract
ETL
Data Warehouse
???
![Page 13: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/13.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Anatomy of a typical Integration Pipeline
Source
Pro
cess
ing
Ste
p
Pro
cess
ing
Ste
p
Pro
cess
ing
Ste
p
Pro
cess
ing
Ste
p
Data Data
Monolith
Destination
![Page 14: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/14.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Integration Pipeline
Scale
Re-deploy
Upgrade
Migrate
Distribute
Update
Source Destination
Pro
cess
ing
Ste
p
Pro
cess
ing
Ste
p
Pro
cess
ing
Ste
p
Pro
cess
ing
Ste
p
![Page 15: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/15.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Integration Pipeline
Source Destination
Pro
cess
ing
Ste
p
Pro
cess
ing
Ste
p
Pro
cess
ing
Ste
p
Pro
cess
ing
Ste
p
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
![Page 16: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/16.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Binding Abstraction Layer
Transport Options
Integration Pipeline
Data Messaging Microservices
Spring Cloud Stream
Data Data
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
![Page 17: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/17.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Spring Cloud DataFlowOrchestrate composable Data Microservices on modern runtimes
Auto Scaling
Auto Healing
Aggregated Logging
Integrated Metrics
Transport Transparency
Infrastructure Transparency
Data Pipeline Visual Design
Integrated Monitoring
Pivotal Cloud Foundry
Data Microservice
SpringBoot app
Contract
SpringBoot app
Contract
Data Microservice
Data Microservice
Data Microservice
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SourcesDestinations
Processors
HTTP
Greenplum Database
S3
![Page 18: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/18.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Backing Services
Transport Options
Pivotal Cloud Foundry
Auto Scaling
Auto Healing
Aggregated Logging
Integrated Metrics
Transport Transparency
Infrastructure Transparency
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
Data Pipeline Visual Design
Integrated Monitoring
![Page 19: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/19.jpg)
© Copyright 2014 Pivotal. All rights reserved.
JSON Filter Transform Enrich
CustomHTTP
Sample pipeline
Deploy
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
Pivotal Cloud Foundry
Spring Cloud Data Flow
![Page 20: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/20.jpg)
© Copyright 2014 Pivotal. All rights reserved.
DEMO
“Fast Data"
Machine Learning
Normal
High Risk
![Page 21: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/21.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Rest / HTTP
Transactions
Browser
Machine Learning
?GemFire
Greenplum
Analytics
![Page 22: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/22.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Machine Learning (Clustering)
GemFire
Greenplum
Analytics
Train
![Page 23: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/23.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Machine Learning (Clustering)
GemFire
Greenplum
Analytics
Traindistance from home location
tran
sact
ion
valu
e
Medium RiskLow Risk
High Risk
![Page 24: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/24.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Machine Learning (Clustering)
GemFire
Greenplum
Analytics
TrainPMML
+ Distance from home
ML Evaluation
high risk transactions
Suspects
![Page 25: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/25.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Rest / HTTP
Transactions
Browser
High Risk Alerts
GemFire
Greenplum
+ Distance from home
![Page 26: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/26.jpg)
© Copyright 2014 Pivotal. All rights reserved.
Order Mgmt
Contract
InventoryContract
ShippingContract
PaymentsContract
Pivotal Cloud Foundry
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
SpringBoot app
Contract
Data Warehouse
PMML Model
External services
Actionable insightsRecommendation
Engine
Business Microservices Data Microservices
![Page 27: A NEW PLATFORM FOR A NEW ERA - SDJUG · Title: SpringOnePlat - Data Microservices Created Date: 8/22/2016 5:53:23 PM](https://reader033.vdocuments.mx/reader033/viewer/2022060400/5f0e0bef7e708231d43d5a65/html5/thumbnails/27.jpg)
A NEW PLATFORM FOR A NEW ERA