a macro perspective on the toolbox for microservices · 2020-06-10 · spring cloud service a...
TRANSCRIPT
CADEC 2020.01.23 & 2020.01.29 | CALLISTAENTERPRISE.SE
SPRING CLOUD + KUBERNETES + ISTIO = ?
A MACRO PERSPECTIVE ONTHE TOOLBOX FOR MICROSERVICES
MAGNUS LARSSON
AGENDA
• Why?
• Challenges
• Open Source to the rescue!
• Overlaps
• Demo
• Summary
WHY?
?• Easier to scale• Faster releases
• Requires• Autonomous components• Share nothing architecture
• Forms a distributed system!
CHALLENGES
?
CHALLENGES
WHERE ARE THE SERVICES?WHICH SERVICE TO CALL?
DISCOVERY SERVER
A-1
?B-2
B-3B-1
B
CHALLENGES
WHERE ARE THE SERVICES?WHICH SERVICE TO CALL?
DISCOVERY SERVER
HOW TO HIDE PRIVATE SERVICES?HOW TO PROTECT PUBLIC SERVICES?
EDGE SERVER
Service A
Client
Edge Server
Service CService B
CHALLENGES
WHERE ARE THE SERVICES?WHICH SERVICE TO CALL?
DISCOVERY SERVER
WHERE IS MY CONFIGURATION?ARE ALL SERVICES CONFIGURATION UP TO DATE?
CENTRALIZED CONFIGURATION
HOW TO HIDE PRIVATE SERVICES?HOW TO PROTECT PUBLIC SERVICES?
EDGE SERVER
CONFIG
CONFIG
CONFIG
CONFIG
CONFIG
CONFIG
CHALLENGES
WHERE ARE THE SERVICES?WHICH SERVICE TO CALL?
DISCOVERY SERVER
WHERE ARE THE LOGS?
HOW TO CORRELATE LOGSFROM DIFFERENT SERVICES?
LOG ANALYSIS
WHERE IS MY CONFIGURATION?ARE ALL SERVICES CONFIGURATION UP TO DATE?
CENTRALIZED CONFIGURATION
HOW TO HIDE PRIVATE SERVICES?HOW TO PROTECT PUBLIC SERVICES?
EDGE SERVER
CHALLENGES
WHERE ARE THE SERVICES?WHICH SERVICE TO CALL?
DISCOVERY SERVER
WHERE ARE THE LOGS?
HOW TO CORRELATE LOGSFROM DIFFERENT SERVICES?
LOG ANALYSIS
WHERE IS MY CONFIGURATION?ARE ALL SERVICES CONFIGURATION UP TO DATE?
CENTRALIZED CONFIGURATION
HOW TO • DEPLOY SERVICES?• SCALE SERVICES?• UPGRADE SERVICES?• RESTART FAILING SERVICES?
SERVICE MANAGEMENT
HOW TO HIDE PRIVATE SERVICES?HOW TO PROTECT PUBLIC SERVICES?
EDGE SERVER
OBSERVE
COMPAREACT
CURRENT STATE è DESIRED STATE
CHALLENGES
WHERE ARE THE SERVICES?WHICH SERVICE TO CALL?
DISCOVERY SERVER
WHERE ARE THE LOGS?
HOW TO CORRELATE LOGSFROM DIFFERENT SERVICES?
LOG ANALYSIS
WHERE IS MY CONFIGURATION?ARE ALL SERVICES CONFIGURATION UP TO DATE?
CENTRALIZED CONFIGURATION
WHAT HARDWARE RESOURCES ARE USED?MONITORING
HOW TO • DEPLOY SERVICES?• SCALE SERVICES?• UPGRADE SERVICES?• RESTART FAILING SERVICES?
SERVICE MANAGEMENT
HOW TO HIDE PRIVATE SERVICES?HOW TO PROTECT PUBLIC SERVICES?
EDGE SERVER
CHALLENGES
WHERE ARE THE SERVICES?WHICH SERVICE TO CALL?
DISCOVERY SERVER
WHERE ARE THE LOGS?
HOW TO CORRELATE LOGSFROM DIFFERENT SERVICES?
LOG ANALYSIS
WHERE IS MY CONFIGURATION?ARE ALL SERVICES CONFIGURATION UP TO DATE?
CENTRALIZED CONFIGURATION
WHAT HARDWARE RESOURCES ARE USED?MONITORING
HOW TO • DEPLOY SERVICES?• SCALE SERVICES?• UPGRADE SERVICES?• RESTART FAILING SERVICES?
SERVICE MANAGEMENT
HOW ARE MY SERVICES PERFORMING?
OBSERVABILITY
HOW TO HIDE PRIVATE SERVICES?HOW TO PROTECT PUBLIC SERVICES?
EDGE SERVER
CHALLENGES
WHERE ARE THE SERVICES?WHICH SERVICE TO CALL?
DISCOVERY SERVER
WHERE ARE THE LOGS?
HOW TO CORRELATE LOGSFROM DIFFERENT SERVICES?
LOG ANALYSIS
WHERE IS MY CONFIGURATION?ARE ALL SERVICES CONFIGURATION UP TO DATE?
CENTRALIZED CONFIGURATION
WHAT HARDWARE RESOURCES ARE USED?MONITORING
HOW TO • DEPLOY SERVICES?• SCALE SERVICES?• UPGRADE SERVICES?• RESTART FAILING SERVICES?
SERVICE MANAGEMENT
HOW ARE MY SERVICES PERFORMING?
OBSERVABILITY
HOW TO HIDE PRIVATE SERVICES?HOW TO PROTECT PUBLIC SERVICES?
EDGE SERVER
HOW TO CONTROL ROUTING?• RATE LIMITING• CANARY & BLUE/GREEN UPGRADES
TRAFFIC MANAGMENT
CHALLENGES
WHERE ARE THE SERVICES?WHICH SERVICE TO CALL?
DISCOVERY SERVER
WHERE ARE THE LOGS?
HOW TO CORRELATE LOGSFROM DIFFERENT SERVICES?
LOG ANALYSIS
WHERE IS MY CONFIGURATION?ARE ALL SERVICES CONFIGURATION UP TO DATE?
CENTRALIZED CONFIGURATION
WHO IS CALLING WHO?DISTRIBUTED TRACING
WHAT HARDWARE RESOURCES ARE USED?MONITORING
HOW TO • DEPLOY SERVICES?• SCALE SERVICES?• UPGRADE SERVICES?• RESTART FAILING SERVICES?
SERVICE MANAGEMENT
HOW ARE MY SERVICES PERFORMING?
OBSERVABILITY
HOW TO HIDE PRIVATE SERVICES?HOW TO PROTECT PUBLIC SERVICES?
EDGE SERVER
HOW TO CONTROL ROUTING?• RATE LIMITING• CANARY & BLUE/GREEN UPGRADES
TRAFFIC MANAGMENT
CHALLENGES
WHERE ARE THE SERVICES?WHICH SERVICE TO CALL?
DISCOVERY SERVER
WHERE ARE THE LOGS?
HOW TO CORRELATE LOGSFROM DIFFERENT SERVICES?
LOG ANALYSIS
WHERE IS MY CONFIGURATION?ARE ALL SERVICES CONFIGURATION UP TO DATE?
CENTRALIZED CONFIGURATION
WHO IS CALLING WHO?DISTRIBUTED TRACING
WHAT HARDWARE RESOURCES ARE USED?MONITORING
HOW TO • DEPLOY SERVICES?• SCALE SERVICES?• UPGRADE SERVICES?• RESTART FAILING SERVICES?
SERVICE MANAGEMENT
HOW ARE MY SERVICES PERFORMING?
OBSERVABILITY
HOW TO HIDE PRIVATE SERVICES?HOW TO PROTECT PUBLIC SERVICES?
EDGE SERVER
HOW TO CONTROL ROUTING?• RATE LIMITING• CANARY & BLUE/GREEN UPGRADES
TRAFFIC MANAGMENT
HOW TO HANDLE FAULTS?• SLOW OR NO RESPONSE• TEMPORARY FAULTS• OVERLOAD
RESILIENCE
REQUIRED CAPABILITIES!
WHERE ARE THE SERVICES?WHICH SERVICE TO CALL?
DISCOVERY SERVER
WHERE ARE THE LOGS?
HOW TO CORRELATE LOGSFROM DIFFERENT SERVICES?
LOG ANALYSIS
WHERE IS MY CONFIGURATION?ARE ALL SERVICES CONFIGURATION UP TO DATE?
CENTRALIZED CONFIGURATION
WHO IS CALLING WHO?DISTRIBUTED TRACING
WHAT HARDWARE RESOURCES ARE USED?MONITORING
HOW TO • DEPLOY SERVICES?• SCALE SERVICES?• UPGRADE SERVICES?• RESTART FAILING SERVICES?
SERVICE MANAGEMENT
HOW ARE MY SERVICES PERFORMING?
OBSERVABILITY
HOW TO HIDE PRIVATE SERVICES?HOW TO PROTECT PUBLIC SERVICES?
EDGE SERVER
HOW TO CONTROL ROUTING?• RATE LIMITING• CANARY & BLUE/GREEN UPGRADES
TRAFFIC MANAGMENT
HOW TO HANDLE FAULTS?• SLOW OR NO RESPONSE• TEMPORARY FAULTS• OVERLOAD
RESILIENCE
Service D
Service A Service B Service C
WHERE ARE WE?• Why?
• Challenges
• Open Source to the rescue!
• Overlaps
• Demo
• Summary
THE EVOLUTION
2014
SPRINGCLOUD
Client
Service D
Spring Cloud
Service B
Spring Cloud
Service C
Spring Cloud
Service A
Spring Cloud
Spring Cloud = Application libraries + Services
Discovery Service
(Netflix Eureka)
Circuit BreakerDashboard
(Netflix Turbine + Hystrix Dashboard)
OAuthAuthorization
Server(spring-security)
Config Server(spring-cloud-config
+ GitHub)
Edge server (Netflix Zuul)
Event Bus(RabbitMQ)
Limited to microservices based on Java and Spring
THE EVOLUTION
2013
GOOGLEKUBERNETES
DOCKER
2014
SPRINGCLOUD
Server
Docker engine
Container C Container D
Container A Container B
THE EVOLUTION
2013
GOOGLEKUBERNETES
DOCKER
2014
SPRINGCLOUD
Server
Docker engine
Container C Container D
Container A Container B
>1 server required in production
• High Availability• Scalability
THE EVOLUTION
Master Node Worker Node
Container Runtime
Container C Container D
Observe
Analyze
Act
The control loop
Desired StateStorage
Operator
Updates theDesired state
Reads theDesired state
Reads theActual state
Update the
Actual state
Container A Container B
Kubernetes: A Container Orchestrator
• A cluster of servers running Docker engine acting as one big server
• Enforces actual state = desired state
THE EVOLUTION
2015
K8S V1.0CNCF
SERVICE MESH
ISTIO
20142013
GOOGLEKUBERNETES
DOCKERSPRINGCLOUD
2016 2017 2018
REDHATOPENSHIFT 3.0
GOOGLEGKE
AMAZONEKS
AZUREAKS
VMWAREPKS
2019
Service Mesh Control plane
Ingress Gateway
Service Mesh Proxy
Egress Gateway
Service Mesh Proxy
Microservice B
Service Mesh Proxy
Microservice A
Service Mesh Proxy
Service Mesh Data plane
ISTIO: Service mesh
Observability, Security, Resilience and Traffic Management
THE EVOLUTION
2015
K8S V1.0CNCF
20142013
GOOGLEKUBERNETES
DOCKERSPRINGCLOUD
2016 2017 2018
REDHATOPENSHIFT 3.0
GOOGLEGKE
AMAZONEKS
AZUREAKS
VMWAREPKS
2019100% OPEN SOURCE
SERVICE MESH
ISTIO
CAPABILITY MAPPING
HOW TO HANDLE FAULTS?• SLOW OR NO RESPONSE• TEMPORARY FAULTS• OVERLOAD
WHERE ARE THE SERVICES?WHICH SERVICE TO CALL?
WHERE ARE THE LOGS?
HOW TO HIDE PRIVATE SERVICES?HOW TO PROTECT PUBLIC SERVICES?
HOW TO • DEPLOY SERVICES?• SCALE SERVICES?• UPGRADE SERVICES?• RESTART FAILING SERVICES?
WHERE IS MY CONFIGURATION?ARE ALL SERVICES CONFIGURATION UP TO DATE?
RESILIENCEDISCOVERY SERVER
HOW TO CORRELATE LOGSFROM DIFFERENT SERVICES?
LOG ANALYSIS
CENTRALIZED CONFIGURATION
WHAT HARDWARE RESOURCES ARE USED?
WHO IS CALLING WHO?DISTRIBUTED TRACING
MONITORING
SERVICE MANAGEMENTOBSERVABILITYHOW ARE MY SERVICES PERFORMING?
EDGE SERVER
Service D
Service A Service B Service C
SPRING CLOUD
ISTIO
KUBERNETES
EFK
HOW TO CONTROL ROUTING?• RATE LIMITING• CANARY & BLUE/GREEN UPGRADES
TRAFFIC MANAGMENT
WHERE ARE WE?• Why?
• Challenges
• Open Source to the rescue!
• Overlaps
• Demo
• Summary
OVERLAPS
Capability Spring Cloud Kubernetes Istio
Service Discovery X XCentral Configuration X XEdge Server X X XDistributed Tracing X XResilience X X
FEATURE COMPLETENESS, E.G. FOR AN EDGE SERVER
Feature Spring CloudGateway
KubernetesIngress Controller
IstioIngress Gateway
Security
- OAuth 2.0 & OIDC X X X- Automated provisioning and
renewal of certificatesX X
Routing
- URL path based X X X- Header based X XObservability XTraffic Management X
OVERLAPS - HOW TO CHOOSE?
• Prefer platform over application library- Independence of microservice implementations
» E.g. language or frameworks
• Exceptions, i.e. use application library for
1. Managing trace ids in a microservice» Setting inbound trace id on outbound requests
2. Resilience mechanisms, e.g. timeout, retry and circuit breakers» Fine tuning often depends on business logic
Note: Platform based resilience is much better than none at al…
OVERLAPS - SELECTIONS
Capability Spring Cloud Kubernetes Istio
Service Discovery X XCentral Configuration X XEdge Server X X XDistributed Tracing X XResilience X X
OVERLAPS – SELECTIONS
Capability Spring Cloud Kubernetes Istio
Service Discovery Netflix EurekaSpring Cloud Load Balancer
Kube Proxy & Service objects
Central Configuration Spring Cloud Config server Config Maps & Secrets
Edge Server Spring Cloud Gateway Ingress Controller Ingress Gateway
Distributed Tracing • Spring Cloud Sleuth• Zipkin
• Jaeger• Zipkin
Resilience Resilience4J Timeout, Retries &Outlier Detection
• Spring Cloud Sleuth • Jaeger
CAPABILITY MAPPING
HOW TO HANDLE FAULTS?• SLOW OR NO RESPONSE• TEMPORARY FAULTS• OVERLOAD
WHERE ARE THE SERVICES?WHICH SERVICE TO CALL?
WHERE ARE THE LOGS?
HOW TO HIDE PRIVATE SERVICES?HOW TO PROTECT PUBLIC SERVICES?
HOW TO • DEPLOY SERVICES?• SCALE SERVICES?• UPGRADE SERVICES?• RESTART FAILING SERVICES?
WHERE IS MY CONFIGURATION?ARE ALL SERVICES CONFIGURATION UP TO DATE?
RESILIENCEDISCOVERY SERVER
HOW TO CORRELATE LOGSFROM DIFFERENT SERVICES?
LOG ANALYSIS
CENTRALIZED CONFIGURATION
WHAT HARDWARE RESOURCES ARE USED?
WHO IS CALLING WHO?DISTRIBUTED TRACING
MONITORING
SERVICE MANAGEMENTOBSERVABILITYHOW ARE MY SERVICES PERFORMING?
EDGE SERVER
Service D
Service A Service B Service C
SPRING CLOUD
ISTIO
KUBERNETES
EFK
HOW TO CONTROL ROUTING?• RATE LIMITING• CANARY & BLUE/GREEN UPGRADES
TRAFFIC MANAGMENT
Prometheus
Grafana
• Monitoring
SPRING CLOUD + KUBERNETES + ISTIO
MongoDB
MongoDB
MySQLReview
Product
RecommendationProduct Composite
• Service Management• Service Discovery• Configuration
Kubernetes
Resilience4J
• Resilience
Fluentd
KibanaElastic-search
• Log analysis
Istio Control Plane
Istio Proxy
Istio Proxy
Istio ProxyIstio ProxyIstio Proxy
Kiali
• Traffic Management• Security
• Observability• Distributed tracing Jaeger
Istio Ingress Gateway
• Edge Server
Istio ProxySpring Cloud Sleuth
• Distributed tracing
WHERE ARE WE?
• Why?• Challenges• Open Source to the rescue!• Overlaps
• Demo- Observability- Logging- Tracing- Monitoring- Resilience
• Summary
DEMO - OBSERVABILITY
DEMO - CENTRALIZED LOGGING
DEMO - CENTRALIZED LOGGING
DEMO - DISTRIBUTED TRACING
DEMO - DISTRIBUTED TRACING
DEMO - MONITORING
DEMO - RESILIENCE
DEMO - RESILIENCE
DEMO - RESILIENCE
SUMMARY
• Microservices promise- Easier to scale- Faster release cycles
• Cooperating microservices è Distributed System- Inherent complexity- Can be managed with Open Source
» Application library, e.g. Spring Cloud» Container orchestrators, e.g. Kubernetes» Service mesh, e.g. Istio
• Handle overlaps
• Works great together!- …if used correctly
RECOMMENDED READING
• Book – Hands-on microserviceshttps://www.packtpub.com/web-development/hands-on-microservices-with-spring-boot-and-spring-cloud
• Blog series – Java & GO based microservices• https://callistaenterprise.se/blogg/teknik/
2015/05/20/blog-series-building-microservices/