quo vadis netflix stack? - java forum stuttgart 2020 · 04/07/2019 · netflix oss – well known...
TRANSCRIPT
Quo VadisNetflix Stack?
// Baris Cubukcuoglu & Fabian Keller
Photo by Ajeet Mestry on Unsplash
Baris CubukcuogluSoftware EngineerPhotographer@bariscubuk__
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 2
Fabian KellerSoftware Engineer
Woodworker@_fabiankeller
Netflix OSS – Well known Projects
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 4
ZuulAPI Gateway
EurekaService Registry
ArchaiusCentralized Config
Microservice AHystrix
Ribbon
Microservice BHystrix
Ribbon
Feign Feign
Eureka, Feign & Ribbon (with Spring Cloud)
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 5
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
Hystrix (with Spring Cloud)
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 6
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
Zuul (with Spring Cloud)
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 7
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
Archaius (with Spring Cloud)
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 8
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
9
Maintenance Mode
Photo by Pandu Agus Wismoyo on Unsplash
ArchaiusCentralized Config
ZuulAPI Gateway
EurekaService Registry
Microservice AHystrix
Ribbon
Microservice BHystrix
Ribbon
Feign Feign
Which Projects are affected?
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 10
Maintenance Mode
Maintenance Mode
Are there any alternatives?TL;DR: yes!
For Hystrix:Resilience4j, Sentinel, Spring Retry
For Hystrix Dashboard / Turbine:Micrometer + Monitoring System (e.g. Prometheus etc.)
For Archaius:Spring Cloud Config, HashiCorp Vault
For Ribbon:Spring Cloud Load Balancer, gRPC
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 11
Fallacies of Distributed Computingsee https://bit.ly/1IEpFC0
The network is reliableLatency is zeroBandwidth is infiniteThe network is secure Topology doesn’t changeThere is one administratorTransport cost is zeroThe network is homogeneous
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 12
Patterns
• Stability• Capability• Transparency
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 13
Photo by chuttersnap on Unsplash
15% Internet Traffic
Service Discovery
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 15
Client
Registry
Service AService A
Service A
Service AService A
Service B
Load balance
Register
Lookup
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
Load Balancing
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 16
Client
Service Service
Availability Filtering
Client
Service Service
Round Robin
Client
Service 0.7
Service 0.3
Weighted Response Time
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
Circuit Breaker
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 17
Execute Command
Run
Fallback
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
Bulkhead
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 18
Thread pool
Service X
Service Z
Resource Call
Resource Call
Resource Call
DB Y
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
Bulkhead
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 19
Service X
Service Z
Resource Call
Resource Call
Resource Call
DB Y
Pool X Pool Y Pool Z
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
API Gateway
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 20
Client
Resource
Resource
Resource
Resource
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
API Gateway
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 21
Client
Resource
Resource
Resource
Resource
API Gateway
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 22
The Modern Cloud Stack
Photo by Pero Kalimero on Unsplash
Where do we begin with?A homogeneous microservice landscape, yeah right…
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 23
NetflixOSS
NetflixOSS
NetflixOSS
NetflixOSS
NetflixOSS
NetflixOSS
NetflixOSS
NetflixOSS
?!
So how do we solve all these challenges?Using sidecars instead of libraries
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 24
Application Layer
Cross-cutting concerns
Infrastructure Layer
Cross-cutting concerns
Application NetflixOSS
Application
Egre
ss
Ingr
ess
Egre
ss
Ingr
ess
Egre
ss
Ingr
ess
Egre
ss
Ingr
ess
Egre
ss
Ingr
ess
Egre
ss
Ingr
ess
Egre
ss
Ingr
ess
Egre
ss
Ingr
essEg
ress
Ingr
ess
Adding the sidecars to all containersWe can have the exact same architecture as before
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 25
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 26Photo by Erdenebayar Bayansan on Pixabay
Cloud FoundryWhat is Cloud Foundry?
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 27
Open Source Faster to iterate Scalable platform
cf push -p target/spring-music.jar
Container Network
Cloud Foundry Service Discoverya.k.a. Container Networking. Also enables client-side load balancing via DNS
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 28
moments.apps.internal10.254.40.156
media.apps.internal10.254.40.148
media.apps.internal10.254.40.123
media.apps.internal10.254.40.42
myapp.com10.254.40.156
moments.apps.internal10.254.40.148
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
DNS? moments.apps.internalA 10.254.40.148A 10.254.40.156
Cloud Foundry Service DiscoveryHow to setup CF container network
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 29
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
Cloud Foundry Route ServicesTaking care of ingress traffic
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 30
Cloud Foundry
CF Router AppLoad
BalancrClient
Route Service
Service Broker
• Security
• Rate Limiting
• Caching
• Tracing
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
Cloud Foundry Dynamic App Config
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 31
CF SET-ENV CF RESTART
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 32
Kubernetes Istio
Photo by Bobby Burch on Unsplash
Kubernetes Service DiscoveryDeployments and Services
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 33
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
Environment
moments172.10.40.156
media172.10. 40.148
media172.10.40.123
media172.10.40.42
Cluster IP10.254.40.156
moments172.10.40.148
DNS? momentsA 10.254.40.148
Kind: k8s/DeploymentKind: k8s/Service
moments.my-namespace.svc.cluster.local10.98.61.186
Kubernetes Service DiscoveryExposing a service
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 34
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
new RestTemplate().exchange(„http://moments/api/v1/timeline“, …)
Pod 1 moments-pod172.10.40.148
Pod 2 moments-pod172.10.40.156
Istio ArchitectureControl Plane and Data Plane
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 35
Control Plane
Data Plane
Pilot Citadel Mixer
Pod 1
Envoy
App 1
Pod 2
Envoy
App 2
Istio VirtualServiceAdding a VirtualService layer to intelligently route traffic
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 36
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
Kind: k8s/DeploymentKind: k8s/Service
moments-v110.98.61.186
Pod 1 moments-v1-pod
Pod 2 moments-v1-pod
Kind: istio/VirtualService
moments
Match routing rules:/api/v1 à moments-v1/api/v2 à moments-v2
Kind: k8s/Deployment
Pod 1 moments-v2-pod
Kind: k8s/Service
moments-v210.98.61.193
Istio Destination RulesDestination rules apply after routing rules are evaluated
Apply various policies to traffic:
• Load Balancer Policy (RR, Random)
• Client TLS
• Circuit Breaker
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 37
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
K8s/Istio Dynamic App Config
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 38
VIM KUBE-APP.YAML KUBECTL APPLY
Service Discovery Load Balancing Circuit Breaker API Gateway Dynamic Config
Istio Additional FeaturesThis was just the beginning…
• Automatic Retries• Rate Limiting• mTLS• Policy Enforcement• Observability & Monitoring• Distributed Tracing• Fault Injection
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 39
Platform Support
Service Discovery Eureka Container Networking Kubernetes Services
Load Balancing Ribbon Container Networking Kubernetes Services
Circuit Breaker Hystrix / Turbine – Istio Destination Rule
API Gateway Zuul Route ServicesCanary Deployments
Istio Virtual ServiceIstio Gateway
Dynamic Config Archaius cf set-envcf restart
vim kube-app.yamlkubectl apply
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 40
LimitationsThere is always a flipside
• We‘re now hiring for a full-time YAML engineer
• Sometimes infrastructure shouldn‘t take care (e.g. retries)
• Yet another markup language stack to learn
• People will start to write microservices in other languages
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian Keller 41
Thanks!kubectl apply –f questions.yaml
Come visit our booth!
2019-07-04 / Java Forum Stuttgart / Baris Cubukcuoglu & Fabian KellerPhoto by Ajeet Mestry on Unsplash