model-driven continuous deployment for quality...
TRANSCRIPT
![Page 1: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/1.jpg)
DICE
Horizon 2020 Research & Innovation Action
Grant Agreement no. 644869
http://www.dice-h2020.eu
Funded by the Horizon 2020Framework Programme of the European Union
Model-Driven Continuous Deployment for Quality DevOps
D. A. Tamburri, E. Di Nitto, M. Guerriero, M. Artač, T. Borovsak
![Page 2: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/2.jpg)
Roadmap
o Intro: The Rise of Big Data
o Research Solution: Project DICE
o DICE Meta-models
o M2M and TOSCA in DICE
2
![Page 3: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/3.jpg)
Intro: The Rise of Big Data
3
o Software market rapidlyshifting to Big data
Expected 32% compoundannual growth rate in EUthrough 2016
Just 35% of Big dataprojects are successful[CapGemini’15]
![Page 4: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/4.jpg)
Intro: Building blocks for DIAs today
4
Coordinator (Kafka)
Orchestrator (Hadoop Cluster)
Data Store
Batch Layer
Speed Layer
Serving LayerServing LayerServing Layer
Data Source
Data Source
Distributedcomputation
Data streaming
HDFS
Distributed storage
Lambda architectureCloud infrastructure
…
…
…
![Page 5: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/5.jpg)
Intro: example challenges!
Legacy Systems and Processes
5
Big Data AnalyticsAdoption
Learning curves
Initial prototype
Risk of failure
(+ others…)
![Page 6: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/6.jpg)
DICE Objectives
o Tackling skill shortage and steep learning curves High-degree of MDE automation via DICE tools
o Shorten time to market for DIAs Push out new products, without sacrificing quality
o Decrease development and testing costs Fast iterative definition of application prototypes
o Reduce number and severity of quality incidents DevOps-fashioned Iterative refinement of application design
6
![Page 7: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/7.jpg)
DICE incremental modeling and analysis[6]
7
DICE Platform Independent Model (DPIM)
DICE Technology Specific Model (DTSM)
DICE Deployment Specific Model (DDSM)
is implemented by
is deployed onto
TOSCA blueprint
Analysis
Analysis
Analysis & Optimization
M2M transformation
M2M transformation
M2T transformation
o To provide soundabstractions and UMLProfile for DIA
o To provide a soundmethod for:
o DICE-basedContinuous DIAArchitecting
o DICE-basedContinuous DIADeployment
![Page 8: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/8.jpg)
DICE incremental modeling and analysis[6]
8
DICE Platform Independent Model (DPIM)
DICE Technology Specific Model (DTSM)
DICE Deployment Specific Model (DDSM)
is implemented by
is deployed onto
TOSCA blueprint
Analysis
Analysis
Analysis & Optimization
M2M transformation
M2M transformation
M2T transformation
o To provide soundabstractions and UMLProfile for DIA
o To provide a soundmethod for:
o DICE-basedContinuous DIAArchitecting
o DICE-basedContinuous DIADeployment
In a DevOps Fashion!
Architect
Deploy
Monitor
Analyse
Re-Deploy
![Page 9: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/9.jpg)
Where does this come from?Dev Area Views (Development structure, business logic, quality verification)
Ops Area Views (Deployment, process, monitoring)
![Page 10: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/10.jpg)
Where does this come from? DICE!
10Featuring the DICE H2020 EU Project
Ops Area Views (Deployment,…)
Dev Area Views (Development structure, business logic,…)
![Page 11: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/11.jpg)
Where does this come from? DICE!
11Featuring the DICE H2020 EU Project
Deployment and Operations area of our architecture framework
![Page 12: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/12.jpg)
Where does this come from? DICE!
12Featuring the DICE H2020 EU Project
…
![Page 13: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/13.jpg)
Where does this come from? DICE!
13Featuring the DICE H2020 EU Project
…
![Page 14: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/14.jpg)
DICE Meta-models - DTSM
14
Storm Package
HadoopMR Package
SparkPackage<<Uses>>
<<Uses>>
Oryx 2Package
<<Uses>>
Core DTSM Package
<<Uses>>
...DICE extensibility
*Metamodels here are illustrative mockups – You can think of using ADOxx to reproduce them!
![Page 15: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/15.jpg)
DICE Meta-models* - DTSM Core
1. Common to all DICE tech-packs;2. Distinguishing analysable elements;3. Distinguishing properties;4. Simplified property definition;
15*Metamodels here are illustrative mockups – You can think of using ADOxx to reproduce them!
![Page 16: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/16.jpg)
DICE Meta-models – DDSM[5]
16
MODACloudsML Specification
TOSCA4DICESpecification
Automated transformation
MODA4DICE::HadoopMR
MODA4DICE::Oryx2
MODACloudsML4DICE
TOSCA Blueprint *.yaml
...
DDSM Metamodels
DDSM Metamodel
Automated transformation
DICER M2T Tool
![Page 17: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/17.jpg)
Where does this come from? DICE!
17Featuring the DICE H2020 EU Project
…
![Page 18: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/18.jpg)
DICE Meta-models – DDSMTOSCA
18
“Topology and Orchestration Specification for Cloud Applications”
1. OASIS Standard;
2. Cloud Infrastructure-as-code language;
1. Type-strength Intentmodeling;
2. In DICE? Big-DataTOSCA “profile”;
Service template
![Page 19: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/19.jpg)
DICE Meta-models – DDSMTOSCA
19
Tier (Group Type)
“TOSCA is used to describe the deployment view for cloud applications”
source_resource
Node_Type_A
target_resource
Node_Type_B
Requirement
connect_relationship
ConnectsTo
Capability
Nodes - are the resources or components that will be materialized or consumed in the deployment topology
Relationshipsexpress the dependencies between the nodes (not the traffic flow)
Requirement - CapabilityRelationships can be customized to match specific source requirements to target capabilities
GroupsCreate Logical, Management or Policy groups (1 or more nodes)
Node templates to describe components in thetopology structure
Relationship templates to describe connections,dependencies, deployment ordering
![Page 20: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/20.jpg)
With M2M a TOSCA blueprint happens!
20
……….."storm_nimbus_host": {
"relationships": [{"type": "cloudify.openstack.server_connected_to_floating_ip", "target": "storm_floating_ip"
}, {"type": "cloudify.openstack.server_connected_to_security_group", "target": "storm_security_group"
}], "type": "dice.medium_host"},
"zookeeper": {
"relationships": [
{
"type": "cloudify.relationships.contained_in",
"target": "zookeeper_host",
"source_interfaces": {
"cloudify.interfaces.relationship_lifecycle": {
"preconfigure": "scripts/connect_zookeeper_servers.sh"
} } } ],
………..
![Page 21: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/21.jpg)
And after that… DevOps!
21
TOSCA!
TOSCADeployment!
In a DevOps Fashion!
Architect
Deploy
Monitor
Analyse
Re-Deploy
![Page 22: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/22.jpg)
Back to basics… DICE Views and Tools!4. Deploy improved-by-verificationarchitecture
![Page 23: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/23.jpg)
5. Monitor and continuouslyenhance/architect/deploy DIA
Back to basics… DICE Views and Tools!
![Page 24: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/24.jpg)
Conclusions, take-home messages
o MDE+TOSCA = DevOps Synergy!
o Big-Data and TOSCA are fit for each other…
but technologies are not mature, yet!
24
![Page 25: Model-Driven Continuous Deployment for Quality …wp.doc.ic.ac.uk/dice-h2020/wp-content/uploads/sites/75/...Model-Driven Continuous Deployment for Quality DevOps D. A. Tamburri, E](https://reader030.vdocuments.mx/reader030/viewer/2022041021/5ed1894be769293795113c69/html5/thumbnails/25.jpg)
Bibliographyo [1] A. Rajbhoj, V. Kulkarni, and N. Bellarykar. Early experience with model-driven development of mapreduce
based big data application. In Software Engineering Conference (APSEC), 2014 21st Asia-Pacific, volume 1,pages 94–97, Dec 2014.
o [2] S. Santurkar, A. Arora, and K. Chandrasekaran. Stormgen - a domain specific language to create ad-hocstorm topologies. In Computer Science and Information Systems (FedCSIS), 2014 Federated Conference on,pages 1621–1628, Sept 2014.
o [3] Abel Gomez, Jose Merseguer, Elisabetta Di Nitto, Damian A. Tamburri “Towards a UML Profile for DataIntensive Applications”, Proceedings of 2nd “Quality Aware DevOps” (QUDOS) workshop, to appear
o [4] Elisabetta Di Nitto, Pooyan Jamshidi, Michele Guerriero, Ilias Spais, Damian A. Tamburri “A SoftwareArchitecture Framework for Quality-Aware DevOps”, Proceedings of 2nd “Quality Aware DevOps” (QUDOS)workshop, to appear
o [5] Matej Artacˇ, Tadej Borovsak, Elisabetta Di Nitto, Michele Guerriero, Damian A. Tamburri, “Model-DrivenContinuous Deployment for Quality DevOps”, Proceedings of 2nd “Quality Aware DevOps” (QUDOS) workshop,to appear
o [6] Guerriero, Michele; Tajfar, Saeed; Tamburri, Damian A. & Di Nitto, Elisabetta (2016), “Towards a model-driven design tool for big data architectures.”, in 'BIGDSE@ICSE' , ACM, , pp. 37-43 .
25