euan finlay microservices to kubernetes switching horses ... · switching horses midstream the...

66
Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Upload: others

Post on 30-May-2020

17 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Switching HorsesMidstreamThe challenge of migrating 150microservices to Kubernetes

Euan Finlay@efinlay24

Page 2: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

@efinlay24

Page 3: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

@efinlay24

Page 4: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

/usr/bin/whoami

@efinlay24

Page 5: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

/usr/bin/whodoiworkfor

No such file or directory.

@efinlay24

Page 6: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

https://www.ft.com

Page 7: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

https://www.ft.com https://www.iquestgroup.com

Page 8: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

@efinlay24

Page 9: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

@efinlay24

Page 10: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

@efinlay24

Page 11: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

@efinlay24

Running highly availableservices is complicated.

Page 12: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

2015 > Docker in production.201620172018

Page 13: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

https://blog.gardeviance.org/2014/03/on-mapping-and-evolution-axis.html

Page 14: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Spend your innovation tokens wisely.

http://mcfunley.com/choose-boring-technology

Page 15: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

For us, the benefitsoutweighed the risks.

@efinlay24

Page 16: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Our AWS costs droppedby around 40%.

@efinlay24

Page 17: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

It was much easier to buildand deploy new services.

@efinlay24

Page 18: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

However, supporting a home-built platform can be difficult.

@efinlay24

Page 19: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

It's even harder when you don'tfully understand how it works.

@efinlay24

Page 20: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Choose boring technology.

http://mcfunley.com/choose-boring-technology

Page 21: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

20152016 > Tools started maturing.20172018

Page 22: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

https://blog.gardeviance.org/2014/03/on-mapping-and-evolution-axis.html

Page 23: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

We are not a clusterorchestration company.

@efinlay24

Page 24: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

In late 2016, we started toinvestigate alternatives.

@efinlay24

Page 25: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Metric for success #1:Reduce the amount of time spent keeping production healthy.

@efinlay24

Page 26: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Metric for success #2:Reduce the number ofsarcastic comments on Slack.

@efinlay24

Page 27: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

@efinlay24

Page 28: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

We chose Kubernetes.

@efinlay24

Page 29: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

https://twitter.com/lizrice/status/828872836777385984

Page 30: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Using leading edge technologies requires you to be comfortablewith change.

@efinlay24

Page 31: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Switching horses midstreamimage?

Page 32: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

201520162017 > Kubernetes migration begins.2018

Page 33: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Lots of other work going on at the same time

@efinlay24

Page 34: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Running in parallelcomplicates things further.

@efinlay24

Page 35: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

During our parallel run, therewere over 2000 code releases.

@efinlay24

Page 36: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

https://helm.sh/

Page 37: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Integrating the service into a templated jenkins pipeline

Page 38: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Each individual change wasn't huge,but 150 small changes add up quickly.

@efinlay24

Page 39: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Unfortunately, we discovereda lot of broken things...

@efinlay24

Page 40: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Some services hadn't been builtfor a very long time.

@efinlay24

Page 41: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Nightly builds can help,even if you don’t deploy them.

@efinlay24

Page 42: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Not all of our service healthendpoints worked correctly.

@efinlay24

Page 43: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Some of our servicesdidn't restart gracefully.

@efinlay24

Page 44: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

We always want to improve things.

@efinlay24

Page 45: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

We had to get everyone involved.

@efinlay24

Page 46: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Feedback from our teamswas essential.

@efinlay24

Page 47: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

We should have swarmedon the work for longer.

@efinlay24

Page 48: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Running in parallel increasedour release overheads.

@efinlay24

Page 49: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

...and also increasedour AWS costs.

@efinlay24

Page 50: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Not just AWS costs either

@efinlay24

Page 51: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

https://www.youtube.com/watch?v=sJx_emIiABk

Page 52: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Platform migration is a marathon,not a sprint.

@efinlay24

Page 53: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Don't underestimate the timeand resources required tomigrate to Kubernetes.

@efinlay24

Page 54: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Our iQuest colleagues were essential to making this successful.

Thank you, Tommy and Sorin. :)

@efinlay24

Page 55: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

2015201620172018 > Kubernetes go-live.

Page 56: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Everything went smoothly.

@efinlay24

Page 57: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Was it worth it?

@efinlay24

Page 58: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

We have a far more stable platform.

@efinlay24

Page 59: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

We have happier developers.

@efinlay24

Page 60: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

We can learn from others.

And we can share our knowledge.

@efinlay24

Page 61: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

@efinlay24

Page 62: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

@efinlay24

Nearly the end.

(don't clap yet)

Page 63: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

@efinlay24

Page 64: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

Sarah Wells@sarahjwells

https://www.youtube.com/watch?v=H06qrNmGqyE

Page 65: Euan Finlay microservices to Kubernetes Switching Horses ... · Switching Horses Midstream The challenge of migrating 150 microservices to Kubernetes Euan Finlay @efinlay24

The end.

(please clap)