devops at ing one analytics - homepage - devops pro moscow...poechali!” cit. ing is a top...
TRANSCRIPT
![Page 1: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/1.jpg)
DevOps at ING one Analytics
Giuseppe d’Alessio
Combining data engineering with data operations
Moscow, 16 November 2017
![Page 2: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/2.jpg)
Bio• Made in Italy
• Living in the Netherlands
• SW Engineering, Machine Learning & Pattern Recognition
Work• Data Engineer @ ING Fast Data NL• Analytics @ ING Direct Italy
• Digital Channels / Security @ ING WBS NL
Giuseppe d’Alessio
https://nl.linkedin.com/in/giuseppedalessio@peppeweb
“Поехали! Poechali!” cit.
![Page 3: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/3.jpg)
ING is a top financial enterprise, operating since 1881
Customers
33 MillionPrivate, Corporate and Institutional Customers
Countries
41 In Europe, Asia,
Australia, North and South America
Employees
52,000
Market leaders Benelux
Growth markets
Commercial Banking
Challengers
3
![Page 4: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/4.jpg)
ING’s Think Forward strategy requires a data-driven approach
4
![Page 5: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/5.jpg)
Continuous Analytics
![Page 6: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/6.jpg)
In ING’s one way of working, ‘business’ and ‘IT’ go hand-in-hand
(Biz)DevOps Continuous Delivery
![Page 7: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/7.jpg)
Squad
In ING’s one way of working, ‘business’ and ‘IT’ go hand-in-hand
SquadSquad
Chapter
Guild
DS
DA
OpsOps
CJE
DS
DA
DS
CJE
DS
Ops
DS
CJE
Dev DevDevDevDev
Tribe
AgileCoach
![Page 8: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/8.jpg)
DevOps and Continuous Delivery journey
8
Engineering Culture
ContinuousDelivery
Team commitment
Enterprise commitment
ContinuousDelivery as a Service
2010
![Page 9: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/9.jpg)
Continuous Delivery as a Service (video)
9
![Page 10: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/10.jpg)
Continuous Delivery as a Service
10
The pipeline provides:
• Standardization
• Automation
• Deliver and collect metrics
to improve it.
• Best practices
![Page 11: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/11.jpg)
Release & Acceptance criteria
11
• Toll gate
• Collect all the evidences to assure:
• Code Quality
• SCAT performance
• Security
• Change management process
• IT Risk under control
![Page 12: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/12.jpg)
Actionable Insights
![Page 13: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/13.jpg)
Streaming Analytics enables us to detect patterns in real time, and respond to events for customers’ benefit
13
Secure and reliable
Relevant
Personal
Omnichannel
Predictive Actionable
![Page 14: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/14.jpg)
![Page 15: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/15.jpg)
Credit card payment
15
![Page 16: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/16.jpg)
Travel Insurance
16
![Page 17: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/17.jpg)
Technology
![Page 18: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/18.jpg)
Streaming Data Platform
18
CEP Engine Machine Learning Engine Post-Processor
RawEvent
BusinessEvent
NotificationEvent
“detect pattern”
“determinerelevant
notifications”
“produce notification”
Application Flow :
Kafka Events:
Data storage:
Business Flow:
CustomerProfiles
NotificationDefinitions
Models
GetCustomer
Profile
ApplySelectionCriteria
Score Notificatio
ns
DetectPattern
Get Intermed
Event
Get RawEvents
SendOutput
Business
Users
System configuration: Machine Learning
Environment
ConfigurationGUI
(future scope) Data
Scientists
CreateIntermed
Event
GetBusiness
Event
CreateBusiness
Event
IntermediateEvent
Format Event
Data Lake
Models
Feedback loop
![Page 19: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/19.jpg)
Flink:• is an open source framework for distributed, in-memory (big) data analytics• Likes Java, Scala, and (a bit) Python• Several APIs for streams (DataStream), batch (DataSet), and relational data (Table API)
Benefits:• true streaming: per-event processing, no micro-batching• high volumes, low latency
Features:• state management and fault tolerancy: savepointing, checkpointing exactly-once
semanctics• time windowing: event time, flexible (e.g. sum all transactions in the past minute)• Complex Event Processing, Machine Learning, SQL (at various levels of maturity)
What is Flink, and why use it?
19
![Page 20: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/20.jpg)
Complex Event Processing with FlinkCEP
20
• Allows detection of event patterns in data streams
• In contrast to traditional DBMSs where a query is executed on stored data, CEP executes
data on a stored query.
![Page 21: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/21.jpg)
Conclusions
![Page 22: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/22.jpg)
22
Questions
![Page 23: DevOps at ING one Analytics - Homepage - DevOps Pro Moscow...Poechali!” cit. ING is a top financial enterprise, operating since 1881 Customers 33 Million Private, Corporate and Institutional](https://reader036.vdocuments.mx/reader036/viewer/2022071500/611f106d35b0850ab43a8939/html5/thumbnails/23.jpg)
Follow us to stay a step ahead
ING.com
YouTube.com/ING
SlideShare.net/ING@ING_News LinkedIn.com/company/ING
Flickr.com/INGGroupFacebook.com/ING