experience in teaching devops

Experience in teaching DevOps

Len Bass

Topics

• What is DevOps

• Class offering

• Student knowledge shortcomings

© Len Bass 2015 2

Over the wall development

copyright 2015 Len Bass 3

Board has idea

Developers implement

Operators place in production

Time

Where Does the Time Go?

• As Software Engineers our view is that there are the following activities in software development

• Requirements

• Design

• Implementation

• Test

• Code Complete

• Different methodologies will organize these activities in different ways.

• Agile focuses on getting to Code Complete faster than with other methods.

4

Developers

implement

copyright 2015 Len Bass

What is wrong?

• Code Complete Code in Production

• Between the completion of the code and the placing of the code into production is a step called: Deployment

•Deploying completed code can be very time consuming because of concern about errors that could occur.

5copyright 2015 Len Bass

What is the work flow for code from a multiteam development effort

• You develop and test your code in isolation

• Your code is integrated with code developed by other teams to see if an executable can be constructed.

• The built system is tested for correctness

• The built system is tested for performance and other qualities (staging)

• The built system is placed into production


Errors can occur at every stage

• Every error must either be corrected or prevented.

• Preventing errors can be done through some combination of

• Process

• Architecture

• Tooling

• Coordination among teams.

• Coordination takes time.

• Correcting errors takes time

© Len Bass 2015 7

How much time?

• Historically, releases are scheduled for once a quarter or once a year to give time to coordinate and adequately test.

• This means there may be months delay before a new concept or feature is added to a system.

• This delay has become more and more unacceptable.

• Weekly or daily releases are becoming the norm.


Goal of DevOps

• The goal of DevOps is to reduce the time to market without compromising quality by• Reducing errors during deployment

• Speeding up incident handling after deployment

• Speeding up deployment


Example for reducing errors during deployment

• Use microservice architecture

• Each team makes its own choices of technology for the service it is developing

• Avoids inconsistencies in versions and technology choices that are discovered during deployment process

© Len Bass 2015 10

Example for speeding up incident handling

• Developers carry pagers and are the first respondents to an incident

• When an incident occurs developers do not need to wait for the incident to be escalated through a ticketing system

• Gives developers cradle to grave responsibility for a service

© Len Bass 2015 11

Example for speeding up Deployment (Deployment pipeline)

• Developer creates and tests code on local machine.

• Checks code into a version control system

• Continuous integration server (CI) builds the system and runs a series of integration tests.

• After passing the tests, the system is promoted to a staging environment where it undergoes more tests including performance, security, and user acceptance tests.

• After passing those tests, the system is promoted to provisional production where it undergoes even more tests.


Pre-commit tests

XBuild Image and Perform Integration

tests

UAT / staging / performance

tests

Deploy to production

Commit

...Pre-commit

testsCommit

Developers

promote to normal

production

Just to summarize

• Placing a system into production can be error prone and time consuming

• Supporting a system after it gets into production can be time consuming and error prone

• DevOps is a set of practices intended to speed up placing a robust system into production

© Len Bass 2015 13

Topics

• What is DevOps

• Class offering


© Len Bass 2015 14

Context for DevOps course

• I teach a DevOps course as a portion of the CMU MSE professional master’s program

• Students are graduate students at CMU. Some have industrial experience, some right out of undergraduate school

• Currently in the fourth offering of the course

© Len Bass 2015

Course structure

• Half theory

• Lectures on video to be seen prior to class

• Textbook and other readings

• In class discussion about lectures and readings

• Half assignments

• Assignments involve a variety of open source tools

• Evaluation based on theory and assignments

© Len Bass 2015

Discussion examples

• What was your takeaway from the videos and readings?

• Each student is called on

• Generate a design of a relevant tool, e.g. Continuous Integration server

• Done in groups

• Time in discussion matches time in video

• Balances out of class reading/video with inclass time spent discussing

© Len Bass 2015

Assignment

• Sample assignment:

• Create two virtual machines

• Install SSH so that one machine can securely connect to another without being prompted for a password

• One page discussion: Discuss the pluses and minuses of having “known hosts” in SSH

• Each assignment takes roughly half a day

© Len Bass 2015 18

Syllabus

• You can retrieve the syllabus from

https://github.com/devopseducator/2016workshop

© Len Bass 2015

Major topics

• DevOps overview

• Networking

• Cloud

• Deployment pipeline

• Architecture

• Deployment

• Security

• Monitoring

© Len Bass 2015 20

Elaborating on the syllabus - networking

• it is amazing what Masters students in CS at CMU do not know. A reflection of their undergraduate curriculum.

• Virtualization

• Virtual machines

• Virtual machine images

• Networks

• IP addresses

• DNS servers and TTL (time to live)

• Ports

• Load Balancers

© Len Bass 2015

Elaborating on the syllabus –cloud

• Failures in the cloud

• Distributed coordination – Zookeeper, Consul

• Scalability

© Len Bass 2015

Deployment pipeline

• Introduction to a deployment pipeline

• What are the roles and responsibilities of various teams with respect to the pipeline

• Case study in business continuity

• Environments of the pipeline in more detail with types of tests

• Development

• Build

• Staging

• Production

• Case study of company that sets up continuous deployment pipelines

© Len Bass 2015

Elaborating on the syllabus –architecture and advanced

packaging

• Microservice architecture

• Case study in migrating to a microservice architecture

• Containers

© Len Bass 2015

Elaborating on the syllabus –deployment

• Designing for deployment

• Use of feature toggles

• Backward and forward compatability

• Managing schemas

© Len Bass 2015

Elaborating on the syllabus –security

• Basic security

• Credential management

• Securing the network

• Securing the pipeline

• Securing an app

© Len Bass 2015

Elaborating on the syllabus –monitoring

• What to monitor

• How to monitor

• Interpreting monitoring data

© Len Bass 2015

Assignments – all open source except AWS

• VirtualBox - virtualization

• SSH – secure connection between VMs

• Vagrant – tool for creating and provisioning a single VM

• Jenkins – continuous integration server

• Chef – configuration management

• Docker - containers

• AWS EC2 Container Service

• Chef Vault – secure configuration management

• Nagios - monitoring

© Len Bass 2015

Student knowledge shortcomings – 1

• Networking 101

• URL - > IP

• Ports

• TCP/IP

• Networking 102

• Subnets

• Firewalls

© Len Bass 2015 30

Proposal

• Establish an “infrastructure” minor to the Software Engineering degree. It could have the following courses:

• Operating systems

• Networking

• Operations/Security

• The cloud

• DevOps

© Len Bass 2015 33

Summary

• DevOps is a set of practices to

• Shorten time to deployment

• Reduce errors during deployment process

• Shorten time to respond to incidents

• A course on DevOps is attractive to students

• There are gaps in knowledge from standard undergraduate CS or SE curricula

© Len Bass 2015 34

More information

• Textbook

• https://github.com/devopseducator/2016workshop

• https://github.com/devopseducator/2017workshop

© Len Bass 2015 35

https://github.com/devopseducator/2016workshop