Download - Software maintenance PyConPL 2016
![Page 1: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/1.jpg)
1
So you "want" to maintain a Python legacy code basePyConPL 2016César Cardenas Desales
![Page 2: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/2.jpg)
2
About me
● Software Architect at Webrepublic AG
● Python user since v.2.0
● Co-organizer of the Swiss Python Summit (i.e. PyCon Switzerland) and the Zurich Python User Group
![Page 3: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/3.jpg)
3
Agenda
❖ Introduction
❖ Maintenance “framework”
1. Understand (your system)
2. Safety net
3. Improve
![Page 4: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/4.jpg)
4
Introduction
![Page 5: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/5.jpg)
5
![Page 6: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/6.jpg)
6
Why discuss maintenance?
● 50% software cost - Barry Boehm
● 40 to 80% software costs - Robert L. Glass
![Page 7: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/7.jpg)
7
Computer programming
![Page 8: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/8.jpg)
8
Software maintenance
![Page 9: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/9.jpg)
9
![Page 10: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/10.jpg)
10
Software maintenance?
![Page 11: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/11.jpg)
11
![Page 12: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/12.jpg)
12
Automotive Industry
● Warranty? 5 years or 100,000 KM● Life-time: 100,000 to 200,000 KM (Avg. 11.4 years)
![Page 13: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/13.jpg)
13
Rail Industry
● Short cycle maintenance: 80 years
![Page 14: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/14.jpg)
14
Rail Industry
● Short cycle maintenance: 80 years● Long cycle maintenance: 150 years
![Page 15: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/15.jpg)
15
IT Industry?
● Linux kernel: 1991, 25 years ● LaTeX: 1985, 31 years● U.S. Treasury Department - Individual Master File: 56
years
![Page 16: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/16.jpg)
16
IT Industry?
● Linux kernel: 1991, 25 years ● LaTeX: 1985, 31 years● U.S. Treasury Department - Individual Master File: 56
years ● Probably many Fortran or Cobol systems
![Page 17: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/17.jpg)
17
IT Industry?
● Linux kernel: 1991, 25 years ● LaTeX: 1985, 31 years● U.S. Treasury Department - Individual Master File: 56
years ● Probably many Fortran or Cobol systems
COBOL programmer wanted
- 5 years experience life expectancy
![Page 18: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/18.jpg)
18
IT Industry?
● Linux kernel: 1991, 25 years ● LaTeX: 1985, 31 years● U.S. Treasury Department - Individual Master File: 56
years ● Probably many Fortran or Cobol systems
● Mine WAS 4 years● Yours?
![Page 19: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/19.jpg)
19
Maintenance: Raffle of a Tiger
![Page 20: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/20.jpg)
20
The life of a maintainer (i.e. me)
1. “Our code is messy”2. “Everyone’s code is messy”
○ 3k LOC class○ A single switch/case
3. “Somebody’s code is messy”
![Page 21: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/21.jpg)
21
1 Understand
![Page 22: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/22.jpg)
22
Motto
“Always code as if the guy who ends up maintaining your code will be a violent psychopath who knows where you live.”
![Page 23: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/23.jpg)
23
1 Understand - Context
● Code doesn’t live in a void
● Ask users● THE business case● Interview power users● Create personas● Read use cases● Write use cases
![Page 24: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/24.jpg)
24
1 Understand - Architecture
● Understand the architecture
● Create diagrams
![Page 25: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/25.jpg)
25
1 Understand - Architecture
● Print critical code● Read the code● Do code walkthroughs● Rubber duck debugging
![Page 26: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/26.jpg)
26
1 Understand - Docs
● Read the docs● Write the docs● Comment the code
● Docstrings● Doxygen● Sphingx
![Page 27: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/27.jpg)
27
1 Understand - Debug it
● Observe program behaviour● For the sake of it
● pdb● pudb● ipdb● Bugjar● Winpdb● Your IDE
![Page 28: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/28.jpg)
28
1 Understand - Profile it
● Does 80/20 hold?
● cProfile, profile● timeit● heapy● memory_profiler● Your IDE● top, htop● IPython SnakeViz
![Page 29: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/29.jpg)
29
1 Understand - Break it
● Scalability?● What breaks?● Why does it break?● Does it matter?● Recovery?
● nose, pytest● Apache Benchmark● Locust● JMeter● Adhoc scripts
![Page 30: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/30.jpg)
30
2 Safety net
![Page 31: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/31.jpg)
31
2 Safety net - Automated tests
● Regression tests● Acceptance tests● Data driven tests
● nose, pytest● coverage● selenium● tox
![Page 32: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/32.jpg)
32
2 Safety net - Fail often and early
● Neutral/Nightly build● Continuous Integration
● Broken build =>
Not safe to release
![Page 33: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/33.jpg)
33
2 Safety net - The build
● Github protected branches
● Jenkins● Buildout● Buildbot● Travis CI
![Page 34: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/34.jpg)
34
2 Safety net - Monitoring
● What errors are current?● Opbeat
![Page 35: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/35.jpg)
35
2 Safety net - Monitoring
● Visualize and filter log entries● Logstash + Kibana
![Page 36: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/36.jpg)
36
2 Safety net - Automate it all
● Because humans make mistakes
● Provisioning deployment
● Ansible● Salt● Puppet● Chef
![Page 37: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/37.jpg)
37
3 Improve
![Page 38: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/38.jpg)
38
3 Improve - The code
● Leave the code cleaner than you found it
● pep8 -> pycodestyle● Flake8 (PyFlakes + pycodestyle)● pylint● radon● Your IDE
![Page 39: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/39.jpg)
39
3 Improve - The code
● Radon○ LOC, SLOC○ Cyclomatic Complexity○ Maintainability Index○ Halstead Metrics
![Page 40: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/40.jpg)
40
![Page 41: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/41.jpg)
41
3 Improve - The code
$ radon cc -o SCORE -s -a processing/tasks.py
![Page 42: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/42.jpg)
42
3 Improve - The code
$ pylint bot.py
![Page 43: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/43.jpg)
43
3 Improve - The people
● TDD if possible● At least on new features
![Page 44: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/44.jpg)
44
3 Improve - The people
● TDD if possible● Do Code Reviews
![Page 45: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/45.jpg)
3 Improve - People - Code reviews
45
The Ego Factor
● No “quick and dirty” fixes “Your reputation at stake with every commit”
![Page 46: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/46.jpg)
3 Improve - People - Code reviews
46
Timeless
● Being here for a while (Fagan, 1976)
● Not going away any time soon
● Technology agnostic
● Not a fad (Like EJB, Java Applets, Web CGI, AOP, Visual Programming, Big Data, SVN)
![Page 47: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/47.jpg)
3 Improve - People - Code reviews
47
Spread knowledge
● Beginners learn from the experts’ code
● The experienced get to know other modules or systems
● Increases “Bus Factor”
![Page 48: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/48.jpg)
3 Improve - People - Code reviews
48
Increase quality of software
● Reviewed solutions tend to be better
● Defects are found and fixed early
Source: IBM Systems Sciences Institute
![Page 49: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/49.jpg)
3 Improve - People - Code reviews
49
Assign Reviewers
● Everyone participates
● Works well with couples
● ¿How long? ~1 sprint or week
![Page 50: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/50.jpg)
3 Improve - People - Code reviews
50
Assign reviewers
● 50%
● 100%
![Page 51: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/51.jpg)
3 Improve - People - Code reviews
51
![Page 52: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/52.jpg)
5252
Rule #1
It’s not personal
![Page 53: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/53.jpg)
3 Improve - People - Code reviews
53
Basic Rules
● Respond (ACK) and solve in a timely manner
● Not too little, not too much (400 loc. max.)
● Ask before overwriting somebody’s code
● Use checklists
![Page 54: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/54.jpg)
54
3 Improve - The people
● Code Reviews○ Like washing your hands
after going to the toilet
![Page 55: Software maintenance PyConPL 2016](https://reader034.vdocuments.mx/reader034/viewer/2022042723/5875bc611a28ab33128b496d/html5/thumbnails/55.jpg)
55
Let’s make it happen.Thank you.
@ccdesales