EVENTUAL CONSISTENCY

06/25/2015, ALBRECHT STÄBLER, SHAREHOLDER NOVATEC HOLDING GMBH

� Eventual Consistency: New Paradigms are needed for Sof tware-Development, -Engineering

� -Processes and -Architectures to execute the digital Tran sformation

� Albrecht Stäbler, Shareholder of NovaTec Holding GmbH

� albrecht.staebler@gmx.net

� Best Practices and trends from projects, national and international

� „Diskussionskreis Cloud, Forschung und Technologie“ together with the companies IBM, RedHat, DB-Systel, BG-Phoenics, University of Stuttgart and others

� Many thanks especially to the ICO- and BlueMix-teams from IBM

About the author / albrecht.staebler@gmx.net

Albrecht Stäbler is teaching Cloud native Architectures EA/EAM, Software-Engineering & -Architectures, Operating System Software, Middleware and Workflow Management at universities domestically and abroad

In strategic customer projects Albrecht Stäbler provided his experiences and his know-how for audits and crisis management

As CEO Albrecht Stäbler was responsible for the corporate strategy

Within the last years Albrecht Stäbler was strategically dealing with solution concepts for digital business transformations according to architectural and methodological aspects like cloud native architectures for new important business fields like IoT, BigData, Industry 4.0 and others

7/9/2015

Albrecht Stäbler, born in 1965, until Mai 2015 chairman of the board of NovaTec Holding GmbH, is holding a university degree as Diplom-Ingenieur (University of Stuttgart)

2

Disruption meets fields like technology, methods and user behaviour and habit

7/9/2015

Examples to the traditional thinking

• Pollution: calculation average for Baden Württemberg

• Traditional thinking (ACID based)

• Missing sensors are not allowed

• Completeness

• No fault tolerance

• New thinking

• Mass data � statistical approach

• Missing sensor not relevant

• Fault tolerance, one missing sensor does not influence the overall statement

• Culture of the „second chance“

• To strand as an entrepreneur is not an issue, it‘s a feature to improve the second challenge

3

Disruption meets fields like technology, methods and user behaviour and habit

7/9/2015

Examples according to the new globalized approach

Pillars of Israels startup success story

• Society: immigration enhances with multi culturalaspects

• Culture: Chutzpah – The word is sometimesinterpreted - particularly in business parlance - asmeaning the

• amount of courage

• mettle or ardor that an individual has

• No hierarchy

• Political: army dominated new economy

• Famous Unit 8200 is an Israeli IntelligenceCorps unit responsible for collecting signalintelligence (SIGINT) and code decryption

4

Different speed in digital business transformation in different countries

7/9/20155

• Kingdom of Saudi Arabia: highest usage of twitter and facebook in average per habitant worldwide

• Political changes: Arabic spring was organized using messaging services

• Sometimes better infrastructure in so called „developping countries“

• Greenfield approach

• Investment protection

• Different culture and ages according to pay with own data

• Stability and obsolescenses

• Robustness inside systemic world / robustness / fault tolerances inside systems

Traditional Approach

7/9/2015

BusinessGoals

IT / ISStrategy

Enterprise ITArchitecture

IT Projects

IT Projects

IT Projects

BusinessExploration /

Discovery

Business Transformation

DesignBusinessChange

Programme

“Traditional” Business Strategyand Transformation Process

“Traditional” IT Strategy andArchitecture Process

Start

IT gets engaged late in Technology

Implementation Projects after ability to

collaborate has largely passed

“BusinessArchitecture”

(Defined)

Portfolio and Solution

Architecture

Business Transformation

Initiatives are started here,

outside of the IT strategy

process

6

Consequences to the traditional approach

7/9/2015

• Business and IT often remain poorly aligned/connected

• IT Departments are often still seen as a cost

• Business and IT find it hard to communicate effectively

• This lack of alignment often results in:• Costs of IT being significantly larger than necessary• Strategic investments being under-utilised

• Service Oriented Architecture makes this even more critical• Technical and methodological evolution (lessons learned): micro services

7

Why is this approach unsustainable

7/9/2015

Need for Business Change

Government Prominent Role

Disruptive Technology Changes

Generation Y

Leadership Excellence Vision

Regulation

“Executives need to stop looking at IT projects as

technology installations and start looking at them as periods of organizational

change that they have the responsibility to manage”

Harvard Business Review, November 2006 by Andrew McAfee, Associate Professor, Harvard Business School

Too many external factors affecting technological competencies

8

Perception of IT is often falling behind expectations to find answers to new demands

Source: gardeviance.org

GAP

Smart Devices

Real-Time BI / Mobile

BigData

7/9/20159

We see an emergence of Systems of Engagement and Internet of Things co-existing with Systems of Record

Systems of EngagementSystems of Record

• Data & Transactions

• App Infrastructure

• Virtualized Resources

• Mobile

• Social Networking

• BigData and AnalyticsNext Generation Architectures

• Sensors

• Embedded intelligence

• Connected devices

Internet of Things

CRM ERP

Systems of Discovery

Insight

Signal from noise

7/9/201510

• The growing complexity of IT systems and soon a trillion connected things demand that sprawling processes become standardized services that are efficient, secure and easy to access.

• A Service Management System will provide visibility, control and automation across IT and business services to ensure consistent delivery.

• “Cloud Computing” describes a new consumption and delivery model for IT services

• “Tosca” describes a declarative and imperative topology and orchestration model for the cloud

IT also needs to become smarter … about delivering “services”(continuous, automated delivery)

7/9/201511

OASIS Topology and Orchestration Specification for Cloud Applications (TOSCA) TC

Shifting Trends in the IT EnvironmentCEOs in the meanwhile see IT as a business enabler, no longer as a cost factor only

From monolithic applications � � To dynamic services

From static infrastructure � To cloud services

From programmed systems � To cognitive systems

From structured data at rest � To unstructured data in motion

From stable well-defined workloads � To unpredictable workloads

From standard devices � To a variety of devices

From proprietary standards � To open innovation

From Corporate owned IT � To Infrastructure as a Service

7/9/201512

Standardization; OPEX savings; faster time to value

Networking

Storage

Servers

Virtualization

O/S

Middleware

Runtime

Data

Applications

Traditional On-Premises

Networking

Storage

Servers

Virtualization

O/S

Middleware

Runtime

Data

Applications

Platformas a Service

Networking

Storage

Servers

Virtualization

O/S

Middleware

Runtime

Data

Applications

Softwareas a Service

Networking

Storage

Servers

Virtualization

Middleware

Runtime

Data

Applications

Infrastructureas a Service

O/S

Vendor Manages in CloudClient Manages

IaaS, PaaS, SaaS – who manages what?

7/9/201513

Modular, non-monolithic architecture(“micro services”)

• System consists of many small, self-contained components• Each component / service developed and operated by a self-

contained team

Built-in Multi-tenancy • The higher the level of multi-tenancy, the more efficient the app is• Built from the beginning to be operated as a (very) large, shared

distributed system – avoiding dedicated SW installations

Horizontal Scalability • Each component can be scaled horizontally

Elastic • Can flexibility grow and shrink depending on load or other needs

Clean Separation of stateful from stateless components

• Stateless components can be treated more efficiently than statefulones

Loosely coupled • Non-blocking communication • Components can come up at any point in time• Since anything can fail anytime (and be restarted elsewhere),

avoid startup dependencies

Attributes of cloud-native software (1)

7/9/201514

Design for failure • The larger a distributed app is, the more likely it is that a failure occurs for one of its components

• Avoidance of single points of failure• Code assumes any node can fail at any point in time• All application components should be resilient to reboots

Granular failure • In case one component of an application fails, the remaining ones should continue to be available

Accessible via idempotent APIs • API-based accessibility for an app• Both usage and lifecycle mgmt API

Developed in a “hosted-first” model • Development and operations covered by a single team, owning the application e2e – from dev to ops

• Typically very small teams• Updates are applied very frequently (vs. big shipments every 6-9

months)• Management concerns need to be addressed by the application

from the beginning• Granular rollout – features are continuously added to the code (no

large feature branches), gradually opened up to select audiences

Easy-of-use / self-service • In today’s cloud-native world it’s critical to allow achieving early success very quickly – without requiring the user to read lots of documentation

Attributes of cloud-native software (2)

7/9/201515

Multi-tenancy Options –Degrees of Isolation and Sharing

SharingIsolation

Hardware

OS

AppServer & Database

Multi-TenantApplication

Tenant Tenant

Hardware

OS

AppServer & Database

ApplicationInstance

Tenant

ApplicationInstance

Tenant

Hardware

AppServer & Database

ApplicationInstance

Tenant

AppServer & Database

ApplicationInstance

Tenant

OS

Hardware

AppServer & Database

ApplicationInstance

Tenant

AppServer & Database

ApplicationInstance

Tenant

OS OS

shared hardware shared OS shared middleware shared application

Shared AppShared Hardware Shared OS Shared Middleware

7/9/201516

What workloads are we seeing move to Cloud delivery?

• Single virtual appliance workloads

• Test and Pre-production systems

• Mature packaged offerings like email and collaboration

• Software development environments

• Batch processing jobs with limited security requirements

• Isolated workloads where latency between components is not an issue

• Storage solutions / storage as a service

• Backup solutions / backup and restore as a service

• Some data intensive workloads, if the provider has a cloud storage offering tied to the cloud compute offering

• Workloads which depend on sensitive data normally restricted to the enterprise

• e.g. employee information, health care records

• Workloads composed of multiple, co-dependent services

• High throughput online transaction processing

• Workloads requiring a high level of auditability, accountability

• e.g. workloads subject to Sarbanes-Oxley

• Workloads based on 3rd party software which does not have a virtualization or cloud aware licensing strategy

• Workloads requiring detailed chargeback or utilization measurement as required for capacity planning or departmental level billing

• Workloads requiring customization (e.g. customized SaaS

What workloads may not be ready for Cloud delivery today?

Considerations for moving Workloads to the Cloud

7/9/201517

Cloud Services Spectrum

18

Cloud Enabled Workloads

Cloud Centric Workloads

Scalable

Virtualized

Elastic

Multi-tenant

Standardized InfrastructureHeterogeneous Infrastructure

Existing Middleware Workloads

EmergingPlatform Workloads

Automated LIfecycle Integrated Lifecycle

Compatibility with existing systems Exploitation of new environments

7/9/201518

A New Operating Model for Cloud Centric Applications

Capabilities and User Experience Today Emerging

Primary Workload Types Systems of Record(Transactional)

Systems of Engagement (+ Record)(BigData, Analytics, Mobile/Social Channels)

Delivery Model Planned Incremental (DevOps)

Development and OperationsTeam Sizes 100s and Costly 10s with built-in DevOps automation

Release Frequency Months to Years Days to Weeks, based on business opportunity

Integration Frequency Weeks Continuous

Infrastructure Deployment Days Minutes

Time to Value Planned Opportunistic

Operational Model Systems Management Built in to application, Recovery Oriented Computing, Continuous Availability

Service Sourcing Develop Consume and Assemble(Public and Private)

7/9/201519

Workloads and Applications

ClientApplications

Analytics

MobileAccess

CustomApplications

Database

Cloud CentricSystems of

Engagement

Cloud EnabledSystems of

Record

• BPM• WAS/J2EE• CICS/IMS

• Portal• …

• Hadoop• PHP

• Cassandra• Nginx• Munin

• …

Existing Workloads

New workloads

Social & Collaboration

Packaged Apps(IBM, SAP, Oracle)

Middleware(J2EE, Transactions)

+

Driven by market needs vs. technology

Optimized for agility and velocity for variable workloads, scale, dynamic composition, multiple programming

models & services

New scalable runtime focusing on progressive composition with

loosely coupled delegated models integrating development,

application services, operations, and infrastructure

Optimized for reducing OPEX for existing patterns and integrating

with existing operational and service management processes

Significant body of automation and integration content for existing workloads (SAP, Oracle etc.)

Still a large opportunity as customers implement virtualization

and evolve to standardization

+

7/9/201520

Five Emerging Cloud Architectures / new DevOps- and SE processes according to

1. Virtualized Traditional - Extensions of Java Application Servers, Support for ‘Traditional’ Transactional Workloads (Cloud enabled)

• Moving existing workloads to the cloud

• Requires best practices, patterns, tooling

2. Database Centric - data driven + small computation on small data• With multi-tenancy attractive for enterprise and service providers

3. Content Centric - computation needs to be close to data + large computation on large data

• Data Mining, Analytics, Data Warehouse,

4. Loosely Coupled - computation and data are separate• Can be addressed by existing middleware, but ‘relaxed consistency’ models emerging

5. Storage Analytics - Data and Storage Integration

Composable software development, integrated continuous delivery as a service, IDE as a service, … i.E. BlueMix

7/9/201521

Investment protection, non greenfield (standard situation):Manage cloud native architectures / Cloud enabling strategiesExisting approaches for Enterprise Architecture Management are too limited

Oversized

Do-it-yourself• High training effort due to strong), level

of detail results in extensive data gathering upfront

• Tools, methods and vizualisations suitable for few experts only (IT Architects)

• Hard to keep data current due to lacking engagement of the rest of the organization

• High initial invest

• „Re-invent the wheel“ when defining own Architecture Frameworks and data models

• Effort to customize generic tools such as SharePoint or Excel

• Manual creation of appealing vizual reports of the IT landscape

• Cluttered user interfaces which are hard to use

7/9/201522

More than 50% of companies report increased speed to access internal experts, reduce communication cost and increased speed to access knowledge. 3)

A different approach with more social interaction promises greater benefits for Architecture Management

Source: 1) 3) McKinsey Global Survey, 2) IBM Global CIO Study, 4) Detecon, 5) Nucleus Research

15% of applications can be shut down if identified through transparency 4)

7/9/201523

The need for new approaches was born in daily projects and roles in a Logistics company

Application & Technology

Lifecycle Management

Business & IT Transformation

IT Integration after M&A

IT Complexity Reduction

~ 1000 applications in 90 countries

IT Integration in> 80 countries

Cloud Roadmap for >

100 applications

Legacy IT landscape in > 100

business units

7/9/201524

collaboration focusses on information that can be turned into value for the company

IT Inventory Interactive Reporting Collaboration Plat form

• Capture the knowledge of distributed users in a single source of truth without training effort

• Fast and easy access to essential information for everybody (Smart Search)

• Faster communication and engage your employees to collaborate across organizational boundaries

• Establish an efficient information exchange better than email Ping-Pong and cluttered file-shares

• Harmonize the IT portfolio to optimize IT cost and reduce IT complexity

• Increase the use of information by reports tailored to the users needs (Filters & Views)

7/9/201525

smart data model covers all central building blocks of TOGAF architecture management

Source: TOGAF

TOGAF

UserGroup

BusinessCapability

IT Component

Tech.Stack

Data Object Application

Recommendation

Attribute in Fact Sheets

Project

����

Should not be covered in a structured way , because:• Understand text format is easier• Link to related objects difficult to

achieve

Good coverage of all essential information in repository• All concepts in Business-,

Information-Systems and Technology-Architecture can be reflected

• To ensure high data quality and usage of a broader audience not all details will be stored (avoid data silos)

Planning & Collaboration supports transformation management • Lifecycle, successor and

visualization of roadmaps• Management of the IT project

poretfolio to achieve target architecture

7/9/201526

Interaction and collaboration betweendifferent stakeholders make handling IT topics more easy

Understandable Terms

Self-explanatory fields

Easy customization

Inline-Forms

Interactive Reporting

Collaboration Platform

CIO, CFO, CEO

Business Unit IT

Business Manager &

Project Leaders

Finance & Procurement

IT Infrastruktur &

Service Managers

IT Architects

7/9/201527

THANK YOU FOR YOUR ATTENTION

06/25/2015, ALBRECHT STÄBLER, HERMAN HOLLERITH ZENTRUM