taking the long view: how the oslo program reduces technical debt

Taking the Long View: How the Oslo Program

Reduces Technical Debt

Doug Hellmann and Mark McLoughlin Kilo Design Summit

November, 2014

Mission

To produce a set of python libraries containing code shared by OpenStack projects. The APIs provided by these libraries should be high quality, stable, consistent, documented and generally applicable.

Project Motivation

• Consistency for deployers

• Consistency for developers

• Identify and foster common patterns

Our Motivation"I cannot imagine the future, but I care about it. I know I am a part of a story that starts long before I can remember and continues long beyond when anyone will remember me. I sense that I am alive at a time of important change, and I feel a responsibility to make sure that the change comes out well. I plant my acorns knowing that I will never live to harvest the oaks.”

http://longnow.org/clock/

Early Days

• Aug 2010 - Jay Pipes registers “openstack-common” on launchpad

• Nov 2010 - Todd Wiley adds plugin handler code to Bexar

• Dec 2010 - Jay Pipes adds early configuration library

• Summer 2011 - Brian Lamar starts database modules

• Oct 2011 - First summit session at Essex summit

Essex - Folsom

• Nov 2011 - Jason Koelker creates openstack-common repository; Mark McLoughlin proposes configuration management module

• Essex Release - Common libraries included in release notes

• Folsom Summit - "managed copy and paste" concepts; work on pbr library begins

• Aug 2012 - Project granted PTL for Grizzly

Grizzly• Mark McLoughlin elected the team’s first PTL

• Adopt name “Oslo”

• Team grows

• Define our mission statement

• Release oslo.config library

• Rename openstack-common repository to oslo-incubator

Havana - Juno

• Havana - Release oslo.messaging; improve incubation tools

• Icehouse - oslo.messaging adoption; evolution of tools and processes; oslo.db team forms

• Juno - Major graduation push results in 7 new libraries; Liaison program started

The Oslo Team

• Generalist code reviewers

• Specialist API maintainers

From Scratch

• oslosphinx - documentation theme

• pbr - packaging

Adopted

• cliff - command line apps

• stevedore - dynamic code loading

• taskflow - job workflow

• pylockfile - inter-process synchronization

Graduation Changes

• Clean up configuration options

• Provide configuration-free API?

• Plug leaky APIs

• Expand/contract API to meet needs of consumers

• Name the library

funcutils

importutils

pycadf

oslo.config

oslo.messaging

authutils

cliff

cliutils

audit

middleware/context

cache

lockutils

timeutils memorycache

apiclient

strutils

gettextutils

fileutils

processutils

cfgfilter

config crypto

db

hooksstevedore

imageutils

excutils

local

log

jsonutils

sslutils

network_utils

log_handler

notifier

rpc

context

service

versionutils

zmq

middleware

policy

quota

reports

rootwrap

xmlutils

eventlet_backdoor

loopingcall

periodic_task

requestutils

threadgroup

fixture

test

units

uuidutils

oslo.version

oslo.vmware

pbr

scheduler

taskflow

taskflow

oslo.serialization

oslo.i18n

oslo.utils

stevedoreoslo.server

oslo.concurrency

oslo.configoslo.io

oslo.crypto

oslo.reports oslo.policy

oslo.middleware

oslo.messaging

oslo.image

oslo.cache

oslo.quota

oslo.vmware

oslo.client

PyCADF

oslo.log

oslo.db

oslo.versionutils

oslo.hooks

New in Juno

Graduated Libraries

• oslo.config

• oslo.db

• oslo.i18n

• oslo.messaging

• oslo.middleware

• oslo.rootwrap

• oslo.serialization

• oslotest

• oslo.utils

Beyond OpenStack

• Upstream work: Bug fixes & Requirements

• Outside contributors

• Adoptions

Oslo & Python 3

• Python 2.7 is receiving bug fixes, but no new features

• Targeting Python 3.4

• Bottom-up approach

oslo.context

taskflow

oslo.serialization

oslo.i18n

oslo.utils

stevedoreoslo.server

oslo.concurrency

oslo.configoslo.io

oslo.crypto

oslo.reportsoslo.policy

oslo.middleware

oslo.messaging

oslo.image

oslo.cache

oslo.log oslo.vmware

oslo.client

PyCADF

oslo.quota oslo.db

oslo.versionutils

oslo.hooks

Plans for Kilo

Getting Involved

• Reviews!

• Look for patterns in projects

Questions?