@boisematt cradlepoint’s distinguished engineer journey to ... · salt docker consul/vault d1, d2...
Post on 29-Jun-2020
9 Views
Preview:
TRANSCRIPT
1
Cradlepoint’s Journey to KubernetesMatt MessingerDistinguished Engineer
@BoiseMatt
2© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
tech
3© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
Kubernetes!
4© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
Mobile NetworksSD-WAN | WiFi | Telematics
IoT NetworksInternet of Things | Edge
NetCloud ManagerNetwork Management | Analytics | Security
Branch NetworksSD-WAN | Edge Security | WLAN
Industry leader in 4G/LTE network solutions and moving to 5G
5© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
NetCloud Manager - the early days.
2013 - 2015
6© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
The problem with success.
2016 - 2017
7© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
The problem of flexibility.
8© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
Too many ways to deploy codeService(s) CI Artifact Deployment Scripts
(master → AWS)OS Config on EC2(app depend.)
Service Install on EC2 App Config on EC2(environment & secrets)
A1, A2 Docker debian "stack-builder" tool Chef Native Chef
B1 Docker Docker & jar custom bash + CloudFormation
Chef Native Chef
C1, C2, C3, C4, C5, C6
Docker Docker custom bash + CloudFormation
Salt Docker Consul/Vault
D1, D2 Docker Docker & jar custom bash + CloudFormation
Chef Native Chef
E1, E2, E3, E4, E5 Docker Docker & deb "marshal" tool Salt Kubernetes Consul/Vault
F1, F2, F3 Docker Docker & jar "marshal" tool Chef Native Chef
G1, G2 Docker tar "stackctl" tool Salt Native Salt
H1, H2, H3, H4 Docker debian "stackctl" tool Salt Native Salt
I1 None tar custom bash + CloudFormation
CloudFormation Native S3/KMS
9© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
Productivity Inhibitors• Difficult to change teams
• Microservice maintenance challenges
• Local development was hard
• Creating new microservices was hard
• Too many deployment Jenkins jobs
10© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
R&D ProductivityNumber of Developers × Developer Efficiency
11© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
The campaign for change.
12© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
Kube Squad
13© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
What are we going to build?
14© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
Design Objectives• Build a fully automated system test pipeline
• Simplify local development
• Simplify deploying services
• Simplify microservice bringup
15© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
How did we build it?
16© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
Old WayAd hoc and Manual
New WayStandardized and Automated
Container Standardization Metaphor
vs.
17© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
microservicecode
microservicedependencies
Image
18© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
microservicecode
microservicedependencies
Container
Pod
Container
Container
19© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
microservicecode
microservicedependencies
Container
Pod
Container
Container
Pod Pod
ReplicaSet
Deployment
20© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
microservicecode
microservicedependencies
Container
Pod
Container
Container
ReplicaSet
Deployment
Service
Pod Pod
21© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
microservicecode
microservicedependencies
Container
Pod
Container
Container
ReplicaSet
Deployment
Service
ConfigMap
:
Pod Pod
ConfigMap
ConfigMap
Secret
Secret
22© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
microservicecode
microservicedependencies
Container
Pod
Container
Container
ReplicaSet
Deployment
Service
ConfigMap
:
Pod Pod
ConfigMap
ConfigMap
Secret
Secret
service.yaml
deployment.yaml
configmaps.yaml
secrets.yaml
23© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
Helm Chart
24© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
accounts-1.1164.434.tgz
Chart Repository
package
upload
Helm Chart
25© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
Helm Umbrella Chart “Manifest”
26© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
service: B version:123
service: A version: 45
service: Bversion: 123
service: Cversion: 456
service: Dversion: 789
service: A version: 45
service: Bversion: 122
service: Cversion: 456
service: Dversion: 789
L3 Manifest“golden manifest”
L3 Candidate
L3 Gate
pass L3 testing
L3 Quality Gate
27
A
B
C
A
B
C
A
B
C
A
B
C
B local
Demos Router Development
Team Branches
Telepresence
...
MyStack: Single-click Developer Stack in AWS
28
More about MyStacks• Creation takes about 10 minutes
• Auto-deletes if not used for 7 days
• Default is small but can opt-in for more services
• Developer dashboard for control and custom urls
29© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
Simple deployment to any stack
30© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
service: A version: 45
service: Bversion: 122
service: Cversion: 456
service: Dversion: 789
L3 Manifest“golden manifest”
L3 Gate
pass L3 testing
auto deployment
Continuous Deployment to Test StacksA
B
C
A
B
C
A
B
C
Load
Perf
Salesforce
...
31© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
Full chart visibility in every deployment
32© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
GitLab Project Template
33© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
Achieved Design ObjectivesBuild a fully automated system test pipeline - L3
Simplify local development - MyStacks
Simplify deploying services - Auto Promotion
Simplify microservice bringup - ~ 1 hour
34© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
Productive Engineers are Happy Engineers
35© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
Lessons Learned• Build a Kubernetes cluster pipeline
• Helm 2 has some warts
• L3 is complex and takes investment
• Microservices are still hard
36© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
Future• Complete last few charts on production
• Service Mesh
• Helm 3
• kops to EKS
• Canary Deployments
37© Cradlepoint Inc. | All Rights Reserved | Information Subject To Change Without Notice
Thank You!• Come visit the Cradlepoint table in the lobby!
• Twitter @BoiseMatt
• R&D Blog @ https://cradlepoint.com/blog
top related