introducing wsgi_lineprof / pycon jp 2017 lt
TRANSCRIPT
![Page 1: Introducing wsgi_lineprof / PyCon JP 2017 LT](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a6e91837f8b9a70728b565d/html5/thumbnails/1.jpg)
Introducing wsgi_lineprof
Yusuke Miyazaki 2017-09-08
PyCon JP 2017
![Page 2: Introducing wsgi_lineprof / PyCon JP 2017 LT](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a6e91837f8b9a70728b565d/html5/thumbnails/2.jpg)
Yusuke Miyazaki @ymyzk• Master’s course student of Kyoto University
• Programming languages / Type system
• Member of CAMPHOR- in Kyoto
• Community for student engineers and designers
• Intern of a start-up in Tokyo (Working Remotely)
• Uses Node.js (TypeScript & JavaScript) mainly
• We’re hiring!
Visit ymyzk.com for more details
![Page 3: Introducing wsgi_lineprof / PyCon JP 2017 LT](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a6e91837f8b9a70728b565d/html5/thumbnails/3.jpg)
Do you like building web applications?
🙋🙋🙋
![Page 4: Introducing wsgi_lineprof / PyCon JP 2017 LT](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a6e91837f8b9a70728b565d/html5/thumbnails/4.jpg)
Do you like building"fast"
web applications?"""
![Page 5: Introducing wsgi_lineprof / PyCon JP 2017 LT](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a6e91837f8b9a70728b565d/html5/thumbnails/5.jpg)
Do you know ISUCON?🙋"🙋"
![Page 6: Introducing wsgi_lineprof / PyCon JP 2017 LT](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a6e91837f8b9a70728b565d/html5/thumbnails/6.jpg)
ISUCON = Iikanjini Speed Up Contest
• Team: 2 or 3 people
• Objective: Make the given web application faster
• Modify configuration of OS or middleware
• Add or replace middleware
• Rewrite applications…
• Rule: If a benchmark score (≈ number of requests) is high, then you win!!
![Page 7: Introducing wsgi_lineprof / PyCon JP 2017 LT](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a6e91837f8b9a70728b565d/html5/thumbnails/7.jpg)
ISUCON = Iikanjini Speed Up Contest
• I participated in ISUCON in 2015 and 2016…
• But I lost in qualification rounds
• It is not easy to speed up existing applications
![Page 8: Introducing wsgi_lineprof / PyCon JP 2017 LT](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a6e91837f8b9a70728b565d/html5/thumbnails/8.jpg)
How can we speed up applications?
🤔🤔🤔
![Page 9: Introducing wsgi_lineprof / PyCon JP 2017 LT](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a6e91837f8b9a70728b565d/html5/thumbnails/9.jpg)
— Robert C. Pike "Notes on Programming in C"
“Measure. Don't tune for speed until you've measured, and even then don't unless one
part of the code overwhelms the rest.”
![Page 10: Introducing wsgi_lineprof / PyCon JP 2017 LT](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a6e91837f8b9a70728b565d/html5/thumbnails/10.jpg)
How can we find bottlenecks? 🤔🤔🤔
![Page 11: Introducing wsgi_lineprof / PyCon JP 2017 LT](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a6e91837f8b9a70728b565d/html5/thumbnails/11.jpg)
Use Profilers• Python
• profile / cProfile
• timeit
• line_profiler
• memory_profiler
• WSGI
• Werkzurg
• linesman
They works well. But I want more simple one for webapp
(especially for ISUCON)
![Page 12: Introducing wsgi_lineprof / PyCon JP 2017 LT](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a6e91837f8b9a70728b565d/html5/thumbnails/12.jpg)
I create a new profiler for web applications
💪💪💪
![Page 13: Introducing wsgi_lineprof / PyCon JP 2017 LT](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a6e91837f8b9a70728b565d/html5/thumbnails/13.jpg)
🎉 wsgi_lineprof 🎉
![Page 14: Introducing wsgi_lineprof / PyCon JP 2017 LT](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a6e91837f8b9a70728b565d/html5/thumbnails/14.jpg)
wsgi_lineprof• WSGI middleware
• Works well with WAFs: Django, Flask, Bottle…
• Easy to integrate with existing applications without changing existing code of app.
• Provides line-by-line profiling
• Records every function call during a request
• Records the followings for every lines:
• Number of times the line was executed
• Amount of time spent to execute the line
![Page 15: Introducing wsgi_lineprof / PyCon JP 2017 LT](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a6e91837f8b9a70728b565d/html5/thumbnails/15.jpg)
How to use# Assume that WSGI app is defined as "app"
server = make_server('127.0.0.1', 8000, app) server.serve_forever()
# wsgi_lineprof filters = [ FilenameFilter("main.py"), TotalTimeSorter(), ] app = LineProfilerMiddleware(app, filters=filters)
![Page 16: Introducing wsgi_lineprof / PyCon JP 2017 LT](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a6e91837f8b9a70728b565d/html5/thumbnails/16.jpg)
Demo
![Page 17: Introducing wsgi_lineprof / PyCon JP 2017 LT](https://reader035.vdocuments.mx/reader035/viewer/2022062504/5a6e91837f8b9a70728b565d/html5/thumbnails/17.jpg)
Conclusion• wsgi_lineprof is a simple WSGI middleware for line-
by-line profiling
github.com/ymyzk/wsgi_lineprof
• Proposals, feedback, and PRs are welcome!
• Let’s participate in ISUCON using Python and win a victory! 🏆🏆🏆
Thank you for Listening!!