microservices, your no1 priority asd2zmdbbm9feqrf.cloudfront.net/2016/usa/pdf/brkspv-2200.pdf ·...

MicroServices, your No1 priority as a software organization, how not to get left behind

Nermeen Ismail – Principal Engineer

BRKSPV-2200

• What does it take to compete in today’s dynamic landscape?

• MicroServices: How can they help win the competition and what are they anyway?

• Developing, deploying & operating microservices

Agenda

What does it take to compete and innovate in a dynamic business landscape?

© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public

Klaus SchwabFounder and Executive ChairmanWorld Economic Forum

“In the new world, it is not the big fish that eats the small fish, it is the fast fish that eats the slow fish”

BRKSPV-2200 5


• Innovate

• React

Speed wins

Business Priorities For Service and Content Providers

• Cost to purchase

• Cost to deploy and operateLow cost wins

• High quality product

• Resilient service

Quality wins

BRKSPV-2200 6


Did Someone Say Scale?

Big Bang

BRKSPV-2200 7


Brave new world!

• Waterfall

• Less automation

• Traditional release cycle

• Proprietary SW

• Dedicated HW

• On-Prem ownership

• SW as a product

• Agile

• More automation

• CI/CD pipelines

• Open source

• Virtualization

• Cloud on-demand consumption

• IaaS, PaaS, SaaS

BRKSPV-2200 8

What about SW

architecture?


The Monolith Challenge

• Slow inter-dependent development

• Rollouts require cross-team coordination

• Slow ramp up for new developers

• Technology updates are very risky

• Failure isolation is tricky

• No independent scaling

• Monolith is good till it is not

Speed, Resiliency, Cost Management, Scale => not all at once

BRKSPV-2200 9


Need for MicroServicesMarket leaders in video are at the pinnacle of innovation:

• Netflix, Amazon leading the way in microservices and video streaming/delivery

• MicroServices combined with Continuous Delivery (CD) and Platforms as a Service

(PAAS) have allowed them to dominate field

BRKSPV-2200 10

What are microservices and how they can help me win?


“MicroServices”: Software architecture that builds systems from granular, independent collaborating services.

Microservice B Microservice CMicroservice D

Microservice A

BRKSPV-2200 12

© 2016 Cisco and/or its affiliates. All rights reserved. Cisco Public 13BRKSPV-2200

Benefit: Minimize Change Delta & Risk Across Deployments

• Granular

• Single responsibility principal

• Alignment to business capabilities

• Independent

• Develop, test, deploy

Smaller, faster releases -> Nimble service that can respond quickly to a dynamic market

User story involves minimal number of

microservices

Fast development, testing and deployment

Reduced risk for overall system


Challenge: It is a Distributed System!

Svc 1

Svc 1

Svc 1

Svc 2

Svc 2

Svc 2DB DB

Svc n

Svc n

Svc n

IP Network

Model

Communicate & Independent

Service Discovery

Load Balancing

Multiple data management

State synchronization

Latency

Timeouts

Cascaded errors

Deploy & upgrade

Configure

E2E testing

Monitor and troubleshoot

BRKSPV-2200 14


To MicroService or not to MicroService?

• Complex

• Fast changes are required

• Upfront investment is possible

• Agile teams• Empowered, cross-functional

• DevOps• Automation is core

• Operation is core

• Decentralized governance• Architectural framework/service blueprint

• Trusted, seasoned teams

• Active cross-team collaboration

SPVSS is gradually embracing MSA across multiple projects (RPIL, Infinite Video, Infinite Home)

BRKSPV-2200 15

Developing, deploying and operating microservices

Basic development


Modeling MicroServicesExample: Video Control System

Video Content

ManagementContent Playback User/Accts

Mgmt

Video

Content

Ingestion

Video

Content

Mgmt

Continuous

Watching

Playback

Session

Mgmt

Scalability

External I/F

Scalability

Business capabilities

Well defined I/F

High cohesion

CollaborationStart coarse grained

Business capabilities

Basic

Dev.

BRKSPV-2200 18


Communication via Common Database

• Independence compromised

• Loose coupling is lost

• Technology agnostic is lost

• Transitional mechanism

Video

Content

Mgmt

Continuous

Watching

Content DB

Content ID: MetaData

Content ID: MetaDataUser ID: Content Index: vPos

Content Index: vPos

Basic

Dev.

BRKSPV-2200 19


Communication via Request/ResponseMultiple Databases

• Service independence maintained

• Careful design for fault isolation

• Transactions can get complex

• Latency could be an issue

getMetaData

MetaData

Video

Content

Mgmt

Continuous

Watching

DB

1

contentID: MetaData

DB

2

userID: contentID, MetaData

Basic

Dev.

BRKSPV-2200 20


Event-based Communication

• Asynchronous communication

• Low latency

• Natural fault isolation

• Scales well

• Distributed choreography

• Eventual consistency

• Good monitoring is a must

• Recovery: retries, rollback, apologies

deleteEvent

Subscribe

MESSAGE

BUS

deleteEvent

Basic

Dev.

Video

Content

Mgmt

Continuous

Watching

Notify

Service

Subscribe

deleteEvent

getMetaData

MetaData

BRKSPV-2200 21

Resiliency


Sam NewmanBuilding Micro services

“Everything can fail and will fail! Plan for it”

BRKSPV-2200 23


Redundant Deployments

• Resilience + Performance

• Horizontal scaling of services

• Scalability group : multiple instances

• Load balancing

• Stateless symmetric services

• Minimum 2 instances for resiliency• 3 is the sweet spot

• DB and infrastructure redundancy

Resiliency

BRKSPV-2200 24

Svc m

Scalability Group m

D

B

Load

BalancerLoad

Balancer


Multiple Data Centers

• Geographic affinity

• Resiliency

• Simplicity through symmetry

• Minimum 2 centers for resiliency

• 3 is the sweet spot

• Capacity planning should accommodate failover

East coast

Resiliency

West coast

BRKSPV-2200 25

East coast

Svc m

Scalability Group m

D

B

Load

BalancerLoad

Balancer

D

BSvc n

Scalability Group n

Load

BalancerLoad

Balancer

Svc m

Scalability Group m

D

B

Load

BalancerLoad

Balancer

D

BSvc n

Scalability Group n

Load

BalancerLoad

Balancer

West coast


East coast

West coast

East coast

Svc m

Scalability Group m

D

B

Load

BalancerLoad

Balancer

D

BSvc n

Scalability Group n

Load

BalancerLoad

Balancer

Svc m

Scalability Group m

D

B

Load

BalancerLoad

Balancer

D

BSvc n

Scalability Group n

Load

BalancerLoad

Balancer

West coast

Slow 500 ERROR

Failure Injection: What If?

Resiliency

Failure Injection

• Do not find out when “real” failures happen

• Inject failure in a predictable manner

• Make failure part of normal deployment

• Create “developing for resiliency” culture

BRKSPV-2200 26


Developing for Resiliency

• Failure isolation

• Promote failure recovery

• Failure plan

Resiliency

Service A

Service B Service C

Connx

stackConnx

stack

BRKSPV-2200 27

• Separate connection stacks

• Retry to a different instance

• Circuit Breakers


Circuit Breakers

Service A

Service 1Circuit

Breaker Request

Logic

Tripped Circuit

Service 1

Service A

Circuit Retrial

Circuit

Breaker Request

Logic

Service 1

Service A

Circuit

Breaker Request

Logic

Sunny Day

• Fast failure

• Load shedding

Resiliency

Service A

Service 1Circuit

Breaker Request

Logic

Timeout/Errors

BRKSPV-2200 28

Monitoring


Monitoring: why?

• Multiple microservices

• Multiple instances

• Multiple Databases

• Replicated

• Infrastructure services

• Multiple instances

• Multiple regions

• Sometimes Zones

• All networked Svc 1

Svc 2D

B

D

B

SD

Svc n

LB1

Network

Region n

Monitoring

Svc 1

Svc 2D

B

D

B

SD

Svc n

LB1

Network

Region 1

BRKSPV-2200 30

Monitoring is not optional!


What should monitoring tell us?

• Is service x working?

• Service health checks

• Are there pattern changes?

• Metrics

• Troubleshooting

• Logs

• Infrastructure services

Svc 2Svc 2

Svc 2

Svc 1Svc 1

Svc 1D

B

Svc nSvc n

Svc nLB1

LB1Who done it?

Monitoring

Svc 1

CPU 60%Svc 1

CPU 90%????

BRKSPV-2200 31


Host 1

Monitoring Example

• Instance level

• Generate• Health checks

• Metrics: API response times, fatal errors, circuit breakers status

• Logs

• Host level

• Monitoring Client

• Metrics generation: CPU, Memory

• Health check & metrics: collect & transport

• Log client

• Logs collection, translation & transport

Monitoring

ServerLogging Server

Monitoring

Dashboard

Logging

Dashboard

Metrics

Collect

Log

files

Svc X instance

Health

checksMetrics

Log

files

Log

clientMonitoring

client

Monitoring

BRKSPV-2200 32

Aggregate & Alert

Sensu

Search and Visualize

ELK


Logs

• Standard log format

• Across services

• Easy to index & search

• End to end request correlation

• Request correlation ID

Generate Request Context

corID = 9876

reqSrc = user1-iPAD

reqSvcID = entitles.inst1

reqInfo : getEntitledCont

timeStamp : zzzz.yyyy.xxxx

corID : 9876

reqSRC : user1-iPAD

mySvcID : enittles.inst1

reqPrevHop : user1-iPAD

VCM

inst 2

usrEnt

inst 1

entitles

inst 1

getEntitledContent

getContent

reqContext

getEntitlements

reqContext

reqInfo : getContent

timeStamp : zzzz.xxxx.yyyy

corID : 9876

reqSRC : user1-iPAD

mySvcID : vcm.inst2

reqPrevHop : entitles.inst1

reqInfo : getEntitlement

timeStamp : zzzz.xxxx.yyyy

corID : 9876

reqSRC : user1-iPAD

mySvcID : usrEnt.inst1

reqPrevHop : entitles.inst1

Monitoring

BRKSPV-2200 33

Testing: a balancing act

Quality Speed


Integration

testsDeploy

Integration Pipeline

35BRKSPV-2200

CI/CD PipelinesTesting

Service X CI/CD pipeline

Check in Build Unit testsService

testsDeploy

Service B CI/CD pipeline


testsDeploy

Service A CI/CD pipeline


testsDeploy

• Lots of unit tests

• Lots of service tests

• Minimize integration tests

• Smart upgrades and rollbacks

• Smoke tests

• Monitoring

Deployment


Container

Scheduling

Layer

Automation

• Automate all layers

• Containers are your best friend

• Support multiple deployment environments

• Externalize configuration

Node

1

Node

n

Region 1

Node

1

Node

n

Region m

Infrastructure Base Layer

Deployment

Management Layer

Services

Dashboard

Logging

Dashboard

Monitoring

Dashboard

Dscvr

SvcLB

LB Confi

g

Montr

Svc

Log

Svc

Infrastructure Service Layer

Svc 1 Svc n

Business Layer

DB \Cach

e

Svc

BRKSPV-2200 37


Upgrades

• Bug fix or new behavior

• New feature

• Modified APIs

• Support rollbacks

Deploy new microservice versions with zero downtime

• API versioning

• interface.feature.bugfix

• Rule-based routing

• Traffic percentage to API version

• Client type to API version

• Database compatibility with new and old versions of microservices

Canary Release & Monitor, Monitor, Monitor

Deployment

BRKSPV-2200 38


Production Service

B

Service

X

Service

A+

90%

10%

Service

A

Canary Release

Production Service

B

Service

XService

A

Service

A+

40%

60%

Deployment

BRKSPV-2200 39

Production Service

B

Service

X

100%

Service

A+


Upgrades in action

• Bug fix/Behavior change

• Deploy new service, test and monitor

• Pass -> upgrade old instances

• Features

• Clients with new feature go to the new service• Rule-based routing

• Test old clients with the new service

• Pass -> upgrade old instances

• Modified interface

• New clients go to new service

• Old clients go to old service

• Old clients upgraded -> upgrade old instances

Deployment

BRKSPV-2200 40


Conclusion: Winning with MicroServices

• Organization

• Agile, DevOps culture, Decentralized

• Model: Business capabilities

• Development:

• Independent, Stateless and Resilient

• Service blueprint. CI/CD pipelines

• Containerized

• Automatic redundant deployment

• Simplicity through symmetry

• Failure injection

• Canary upgrades

• Monitoring

• Smaller Faster Releases

• Resilient Service with Zero Down Time & Low Risk Upgrades

• Independent Scalability and Controllable Cost

Nimble, high quality, scalable service that fosters innovation and market leadership

BRKSPV-2200 41


Complete Your Online Session Evaluation

Don’t forget: Cisco Live sessions will be available for viewing on-demand after the event at CiscoLive.com/Online

• Give us your feedback to be entered into a Daily Survey Drawing. A daily winner will receive a $750 Amazon gift card.

• Complete your session surveys through the Cisco Live mobile app or from the Session Catalog on CiscoLive.com/us.

BRKSPV-2200 42

CiscoLive.com/Online

http://ciscolive.com/Online

http://ciscolive.com/Online

http://ciscolive.com/us

http://ciscolive.com/us

Please join us for the Service Provider Innovation Talk featuring:

Yvette Kanouff | Senior Vice President and General Manager, SP Business

Joe Cozzolino | Senior Vice President, Cisco Services

Thursday, July 14th, 2016

11:30 am - 12:30 pm, In the Oceanside A room

What to expect from this innovation talk

• Insights on market trends and forecasts

• Preview of key technologies and capabilities

• Innovative demonstrations of the latest and greatest products

• Better understanding of how Cisco can help you succeed

Register to attend the session live now or

watch the broadcast on cisco.com


Continue Your Education

• Demos in the Cisco campus

• Walk-in Self-Paced Labs

• Lunch & Learn

• Meet the Engineer 1:1 meetings

• Related sessions

• Delivering great multi-device video experiences from the cloud • BRKSPV-1501 – Thu 8:30 a.m.

• Infinite Video – beating Netflix at their own game • PSOSPV-1501 – Wed 9:30 a.m.

• Shipped & Mantl – The Business Case for Using an Integrated Cloud Dev Platform• BRKDEV-1003 – Wed 1:30 p.m.

BRKSPV-2200 44

• Larry Mitchel: Cisco RPIL architect

• First SPVSS project to embrace micro services architecture (MSA)

Acknowledgements

• Sam Newman: “Building micro services”

• http://samnewman.io/books/building_microservices/

• The New Stack: Microservices - Four Essential Checklists when Getting Started

• http://thenewstack.io/microservices-four-essential-checklists-getting-started/

• Mantl: The Bedrock of Hassle Free MicroServices Infrastructure

• https://mantl.io

References

http://samnewman.io/books/building_microservices/

http://thenewstack.io/microservices-four-essential-checklists-getting-started/

http://mantl.io

• Netflix Simian Army – Resiliency and Failure Injection

• http://techblog.netflix.com/2011/07/netflix-simian-army.html

• https://github.com/Netflix/SimianArmy

• Introduction to ELK – Log collection, processing & visualization

• https://www.elastic.co/webinars/introduction-elk-stack

• Introduction to Sensu - Monitoring

• https://sensuapp.org

References

https://github.com/Netflix/SimianArmy

https://github.com/Netflix/SimianArmy

https://www.elastic.co/webinars/introduction-elk-stack

https://sensuapp.org

Thank you

microservices, your no1 priority asd2zmdbbm9feqrf.cloudfront.net/2016/usa/pdf/brkspv-2200.pdf ·...

Documents