software maintenance pyconuk 2016

1

So you "want" to maintain a Python legacy code basePyConUK 2016César Cardenas Desales

2

About me

● Software Architect at Webrepublic AG

● Python user since v.2.0

● Co-organizer of the Swiss Python Summit (i.e. PyCon Switzerland) and the Zurich Python User Group

3

Agenda

❖ Introduction

❖ Maintenance “framework”

1. Understand (your system)

2. Safety net

3. Improve

4

Introduction

5

Why discuss maintenance?

● 50% software cost - Barry Boehm

● 40 to 80% software costs - Robert L. Glass

6

Computer programming

7

Software maintenance

8

9

Software maintenance?

10

11

Automotive Industry

● Warranty? 5 years or 100,000 KM● Life-time: 100,000 to 200,000 KM

12

Rail Industry

● Short cycle maintenance: 80 years

13

Rail Industry

● Short cycle maintenance: 80 years● Long cycle maintenance: 150 years

14

IT Industry?

● Linux kernel: 1991, 25 years ● LaTeX: 1985, 31 years● U.S. Treasury Department - Individual Master File: 56

years

15

IT Industry?

● Linux kernel: 1991, 25 years ● LaTeX: 1985, 31 years● U.S. Treasury Department - Individual Master File: 56

years ● Probably many Fortran or Cobol systems

COBOL programmer wanted

- 5 years experience life expectancy

16

IT Industry?

● Linux kernel: 1991, 25 years ● LaTeX: 1985, 31 years● U.S. Treasury Department - Individual Master File: 56

years ● Probably many Fortran or Cobol systems

● Mine WAS 4 years● Yours?

17

Maintenance: Raffle of a Tiger

18

The life of a maintainer (i.e. me)

1. “Our code is messy”2. “Everyone’s code is messy” (3k LOC class)3. “Somebody’s code is messy”

19

1 Understand

20

1 Understand - Context

● Code doesn’t live in a void

● Ask users● THE business case● Interview power users● Create personas● Read use cases● Write use cases

21

1 Understand - Architecture

● Understand the architecture

● Create diagrams

22

1 Understand - Architecture

● Print critical code● Read the code● Do code walkthroughs● Rubber duck debugging

23

1 Understand - Docs

● Read the docs● Write the docs● Comment the code

● Docstrings● Doxygen● Sphingx

24

1 Understand - Debug it

● Observe program behaviour● For the sake of it

● pdb● pudb● ipdb● Bugjar● Winpdb● Your IDE

25

1 Understand - Profile it

● Does 80/20 hold?

● cProfile, profile● timeit● heapy● memory_profiler● Your IDE● top, htop● IPython SnakeViz

26

1 Understand - Break it

● Scalability?● What breaks?● Why does it break?● Does it matter?● Recovery?

● nose, pytest● Apache Benchmark● Locust● JMeter● Adhoc scripts

27

2 Safety net

28

2 Safety net - Automated tests

● Regression tests● Acceptance tests● Data driven tests

● nose, pytest● coverage● selenium● tox

29

2 Safety net - Fail often and early

● Neutral/Nightly build● Continuous Integration

● Broken build =>

Not safe to release

30

2 Safety net - The build

● Github protected branches

● Jenkins● Buildout● Buildbot● Travis CI

31

2 Safety net - Monitoring

● What errors are current?● Opbeat

32

2 Safety net - Monitoring

● Visualize and filter log entries● Logstash + Kibana

33

2 Safety net - Automate it all

● Because humans make mistakes

● Provisioning deployment

● Ansible● Salt● Puppet● Chef

34

3 Improve

35

3 Improve - The code

● Leave the code cleaner than you found it

● pep8 -> pycodestyle● Flake8 (PyFlakes + pycodestyle)● pylint● radon● Your IDE

36

3 Improve - The code

● Radon○ LOC, SLOC○ Cyclomatic Complexity○ Maintainability Index○ Halstead Metrics

37

38

3 Improve - The code

$ radon cc -o SCORE -s -a processing/tasks.py

39

3 Improve - The code

$ pylint bot.py

40

3 Improve - The people

● TDD if possible● Do Code Reviews

41

3 Improve - The people

● Code Reviews○ Spread knowledge○ The Ego Factor○ Increase quality of software

“Your reputation at stake with every commit”

42

3 Improve - The people

● Code Reviews○ Easily done with Github○ Like washing your hands

after going to the toilet

43

Ask me

● Ascend a Swiss (Python) summit

● Why can you buy cornish pasties in Mexico

44

Let’s make it happen.Thank you.

@ccdesales