Spinnaker and Microservices @ Datawire.io

Trials and TribulationsPhilip Lombardi

Engineer

datawire.io

Background...

1. Philip Lombardi @ Datawire.io (twitter: @TheBigLombowski)

2. Datawire.io is building a Microservices Development Kit to enable developers to build resilient microservice applications.

3. Check us out after meetup: datawire.io

2

datawire.io 3

What is a Microservice?

datawire.io

Common Microservice Definitions

It’s a service that is...

● Small● Self contained● Narrow in scope● Bounded context● Independent● Loosely coupled

Sort of…

All these things describe attributes

4

datawire.io

A (simpler) Microservices Definition

A Microservice is a unit of business logic.

A Microservice application is a distributed composition of business logic via services.

5

datawire.io

Microservice Apps tend to look like this...

6

“Death Star Topology”

datawire.io

Microservices Benefits and Tradeoffs

● Easier to reason about the individual components that make up the system.

● Easy to add new biz logic.

● More difficult to operate.

● More difficult to deploy than a classic monolith.

7

datawire.io

A Traditional Architecture...

8

Client

DNS

Load Balancer

Server

Data

datawire.io

Traditional Architecture Deployment Options

● Relatively well-known patterns for updating a Monolithic App.

● Blue - Green (or Red - Black if you’re Netflix)

● Canary

● Shutdown and Replace

9

datawire.io

But… Microservices Apps are a bit more Complex

● Definitely more than 1 tier of servers processing biz logic.

● Likely more than 1 data storage solution in place.

● Possibly polyglot programming stack.

● Services probably have dependencies between each other…

● … If you’re lucky the dependencies aren’t circular.

10

datawire.io

A Microservices Architecture...

11

Client

DNS

Load Balancer

API Server

Accounts

Votes

Comments

Favorites

Mystery Feature

datawire.io

Good Luck Deploying That without Help

● Simply too many moving parts…

● Brain starts to hurt as you think about concurrent versions…

● Gotta keep the whole system up and running because downtime = $$$.

12

datawire.io

Fortunately…

13

datawire.io

At Datawire...

1. We have a handful of services that are composed to build the backend of our product.

2. Basically two backend developers.

3. All but 1 of those services is deployed via Spinnaker (it’s being ported soon because the old deploy is painful).

4. Spinnaker allows us to deploy but it hasn’t been a silver bullet. There is definitely pain for a small adopting team which we’ve learned about.

14

datawire.io

The Good

1. The claims Spinnaker makes are actually true and it does work!

2. Once you learn how to navigate around the UI and how pipelines are constructed it’s fairly easy to use.

3. Getting Started is painless. Run a pre-baked AMI.

4. Really easy to define custom workflows for putting services into production.

15

datawire.io

The Bad

1. The UIa. Does have some peculiarities around element behaviors.b. Definitely does some odd things at times (the Applications list doesn’t update sometimes).

c. Lacks clarity around some form elements.

2. Deployment of Spinnakera. The pre-baked AMI’s are good for limited use but quickly you run into customization issues and

complexities.

b. Had to bake our own custom AMI for use in VPN etc.

3. Developers Complained it was Too Slowa. Not really a Spinnaker fault per-se, but developers wanted something faster than AMI bakes.

16

datawire.io

The Ugly

1. UI issues scared and scarred my coworkersa. “It’s too complex”b. “It doesn’t work”

c. The unstated goal of Spinnaker is “CD Pipelines Made Easy” but it’s hard to get people to buy that

when their first experiences are painful.

2. Re-configuration is a nightmarea. You need to modify YAML docs on the FS and restart Spinnaker.

i. Lots of stress around restarting Spinnaker when there is lots of deployments in progress.b. No good answer for backing up all that config.

i. S3 seems to be the answer nowadays but back when we adopted Cassandra was the only

option.

3. Documentationa. No docs about what a “production” quality setup looks like.

b. No docs about how to upgrade Spinnaker safely.17

datawire.io

Some Reflections on Spinnaker

1. Personally, I still like the tool...

2. Jury is still deliberating on whether we continue with it at Datawire...a. We’re planning to containerize and move to Kubernetes in the not so distant future.

b. We will likely explore the k8s integration and make a decision then.

3. I would probably pay some amount of money for managed and easy to configure Spinnaker (*hint hint*).

4. Might have just picked Spinnaker too early for a team our size. a. You can count the number of server-side devs @ Datawire on a single hand.

b. Complexity of our apps microservices is low enough that a complex workflow mgmt solution is probably not what we needed.

c. I could totally see Spinnaker being a lot more valuable at a previous gig where the team was much larger and there was a relatively complex workflow.

18

datawire.io

Places Where Improvement is Badly Needed

1. Deployment.

2. UI.

3. Configuration and Configuration Reloading. a. Already having nightmares about this if we use Spinnaker w/ Kubernetes since you need to

constantly update the dockerRegistries config in Cloud Driver.

19

datawire.io

Closing Thoughts

● Spinnaker solves a relatively complex problem: Managing CD Pipelines and Deployment Orchestration

● Might be overkill for a small team especially if you have limited engineering budget to manage Spinnaker OR train users on it.

● Properly vet whether your development team is ready for the tool before adopting.

20

datawire.io

To learn more

Contact me:

● plombardi@datawire.io● Twitter: @TheBigLombowski

Jobs

● https://www.datawire.io/careers/○ Java|Scala|Kotlin, Python, Go and Kubernetes Developers Wanted!○ Hiring for DevOps | SRE role!

Try

● https://github.com/datawire/mdk● https://www.datawire.io

21