taking a saas shop from no ops to noops, pavel suchman

No ops to NoOps

A story of a journey...

github.com/flypunk

● In IT since '99, Unix SysAdmin, OPS@XIV, cloud/devops since 07 (@SeekingAlpha)

● Cloud migration and deployment automation in Amadesa

I am a freelancer specializing in Web Applications Delivery

About me

Not an easy thing to describe - see 'NoOps' Debate Grows Heated@AppFog Carlson - Use 3rd party PaaS

@Netflix Cockroft - Outsource messing with hardware to AWS and build your own PaaS

@Etsy Allspaw - We do it the same way as Netflix, but I HATE the term, especially the word 'NO' in it ;)

WTF is NoOps?!

Culture, software architecture, methodologies and tools that enable a (SaaS) company run their business without a dedicated Operations team.

What do I mean by NoOps

Y U NO OPS!?

Waste - Any activity not contributing to customer value (Agile)

While you have No Nops you have...

No DelaysNo Communications overheadNo Dev vs. Ops fingers pointing - Dev is always to blame ;)

NoOps - Eliminating Waste

Study what is needed

Build what is missing

Get the hell out!

If I did my job well, the company shouldfigure out production/operations on its own.

My goal as a (NoOps) consultant

Today - www.massiveimpact.com

End to end solution for ads on cellular

Conversions, not clicks

The client - MassiveImpact

Hired new people

Written new code

Had to start a new production site

The company pivoted from a different product

.Net on the front end

Big data (hive) on backend

MassiveImpact technology stack:

The head of QA with some Ops experience

I need to ensure that he and his bosses could sleep well at night...

Is it possible for MassiveImpact to continuegrowing without hiring a dedicated Ops employee?

I think - yes!

Who is responsible for ops in the new environment?

Amazon VPC: Harder to set up, more secure, closer to collocated network setup

Let's make everything organized: Chef, Cloud Formation

Btw, we have 2 weeks to launch!

Start manually, add automation later

Requirements for the new environment

ServerDensity for both machine and service monitoring

● Has built in alerts and graphs● Machines have to be able to connect to

the internet to use it● Used the NAT-PAT instance inside VPC +

static IPs to achieve it.

NoOps advantage: A cloud solution.

Monitoring

Written a naming/access policy doc

Created IAM users, groups, policies and resources in accordance with the doc.

Pro tip: don't use MixedCaseBucketNames - hive won't work with them.

Another tip: EMR with IAM keys and policies is not trivial

AWS IAM credentials

Why EMR failed

Set up a server that will work as binaries repository

Put artifacts into sane default places

Will be used later for CI and deployment scripts

Deployment

It (kinda;) works!

You can bootstrap and manage nodes with WinRM!

NoOps advantages:

Configuration as codeNo need to manage AMIs (It's a nightmare)

Chef on Windows

Not everything works

First time in my career able to manage M$ servers and install packages from CLI. == Awesome!!!

Easy way to manage users in centralized manner without AD

Chef on Windows - continue

The system is fully automatic and is run by Cron

Notifications using SNS and email

NoOps advantage: It is fully data driven(Just put the new files to a specified S3 bucket)

Data crunching - Gurobi Cloud

Monitoring - Implemented

Chef - in development

Cloud Formation - planned

CI/CD - planned

Ops/business console (GeckoBoard) - planned

Current status

People, then methodologies, last tools

Architect for no ops!Change is hard, but engineers will accept new tech when shown that it worksMake the changes backwards compatible (Libraries for AWS calls)Use external services as a default choice

Takeaways

BTW, you can get the presentation here:

Q&A