software maintenance pyconpl 2016

1

So you "want" to maintain a Python legacy code basePyConPL 2016César Cardenas Desales

2

About me

● Software Architect at Webrepublic AG

● Python user since v.2.0

● Co-organizer of the Swiss Python Summit (i.e. PyCon Switzerland) and the Zurich Python User Group

3

Agenda

❖ Introduction

❖ Maintenance “framework”

1. Understand (your system)

2. Safety net

3. Improve

4

Introduction

6

Why discuss maintenance?

● 50% software cost - Barry Boehm

● 40 to 80% software costs - Robert L. Glass

7

Computer programming

8

Software maintenance

10

Software maintenance?

12

Automotive Industry

● Warranty? 5 years or 100,000 KM● Life-time: 100,000 to 200,000 KM (Avg. 11.4 years)

13

Rail Industry

● Short cycle maintenance: 80 years

14

Rail Industry

● Short cycle maintenance: 80 years● Long cycle maintenance: 150 years

15

IT Industry?

● Linux kernel: 1991, 25 years ● LaTeX: 1985, 31 years● U.S. Treasury Department - Individual Master File: 56

years

16

IT Industry?


years ● Probably many Fortran or Cobol systems

17

IT Industry?



COBOL programmer wanted

- 5 years experience life expectancy

18

IT Industry?



● Mine WAS 4 years● Yours?

19

Maintenance: Raffle of a Tiger

20

The life of a maintainer (i.e. me)

1. “Our code is messy”2. “Everyone’s code is messy”

○ 3k LOC class○ A single switch/case

3. “Somebody’s code is messy”

21

1 Understand

22

Motto

“Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.”

23

1 Understand - Context

● Code doesn’t live in a void

● Ask users● THE business case● Interview power users● Create personas● Read use cases● Write use cases

24

1 Understand - Architecture

● Understand the architecture

● Create diagrams

25

1 Understand - Architecture

● Print critical code● Read the code● Do code walkthroughs● Rubber duck debugging

26

1 Understand - Docs

● Read the docs● Write the docs● Comment the code

● Docstrings● Doxygen● Sphingx

27

1 Understand - Debug it

● Observe program behaviour● For the sake of it

● pdb● pudb● ipdb● Bugjar● Winpdb● Your IDE

28

1 Understand - Profile it

● Does 80/20 hold?

● cProfile, profile● timeit● heapy● memory_profiler● Your IDE● top, htop● IPython SnakeViz

29

1 Understand - Break it

● Scalability?● What breaks?● Why does it break?● Does it matter?● Recovery?

● nose, pytest● Apache Benchmark● Locust● JMeter● Adhoc scripts

30

2 Safety net

31

2 Safety net - Automated tests

● Regression tests● Acceptance tests● Data driven tests

● nose, pytest● coverage● selenium● tox

32

2 Safety net - Fail often and early

● Neutral/Nightly build● Continuous Integration

● Broken build =>

Not safe to release

33

2 Safety net - The build

● Github protected branches

● Jenkins● Buildout● Buildbot● Travis CI

34

2 Safety net - Monitoring

● What errors are current?● Opbeat

35

2 Safety net - Monitoring

● Visualize and filter log entries● Logstash + Kibana

36

2 Safety net - Automate it all

● Because humans make mistakes

● Provisioning deployment

● Ansible● Salt● Puppet● Chef

37

3 Improve

38

3 Improve - The code

● Leave the code cleaner than you found it

● pep8 -> pycodestyle● Flake8 (PyFlakes + pycodestyle)● pylint● radon● Your IDE

39


● Radon○ LOC, SLOC○ Cyclomatic Complexity○ Maintainability Index○ Halstead Metrics

41


$ radon cc -o SCORE -s -a processing/tasks.py

42


$ pylint bot.py

43

3 Improve - The people

● TDD if possible● At least on new features

44


● TDD if possible● Do Code Reviews

3 Improve - People - Code reviews

45

The Ego Factor

● No “quick and dirty” fixes “Your reputation at stake with every commit”


46

Timeless

● Being here for a while (Fagan, 1976)

● Not going away any time soon

● Technology agnostic

● Not a fad (Like EJB, Java Applets, Web CGI, AOP, Visual Programming, Big Data, SVN)


47

Spread knowledge

● Beginners learn from the experts’ code

● The experienced get to know other modules or systems

● Increases “Bus Factor”


48

Increase quality of software

● Reviewed solutions tend to be better

● Defects are found and fixed early

Source: IBM Systems Sciences Institute


49

Assign Reviewers

● Everyone participates

● Works well with couples

● ¿How long? ~1 sprint or week


50

Assign reviewers

● 50%

● 100%


51

5252

Rule #1

It’s not personal


53

Basic Rules

● Respond (ACK) and solve in a timely manner

● Not too little, not too much (400 loc. max.)

● Ask before overwriting somebody’s code

● Use checklists

54


● Code Reviews○ Like washing your hands

after going to the toilet

55

Let’s make it happen.Thank you.

@ccdesales