Download - a quick Introduction to PyPy
![Page 1: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/1.jpg)
PyPy
Compiling Dynamic Language Implementations
![Page 2: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/2.jpg)
The Python Situation
![Page 3: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/3.jpg)
The Python Situation
• lots of implementationso CPython - Posixo Jython - JVM o IronPython - .NET
• specialized projectso Stackless Pythono unladen-swallowo Psyco
![Page 4: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/4.jpg)
What PyPy is
1.Python Implementation2.Translation Framework
![Page 5: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/5.jpg)
What PyPy is
1. Python implemented in Python
![Page 6: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/6.jpg)
What PyPy is
1. Python implemented in Python
• looks like this:
$> python py.py
![Page 7: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/7.jpg)
What PyPy is
1. Python implemented in Python
• looks like this:
• runs on top of CPython
$> python py.py
![Page 8: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/8.jpg)
What PyPy is
1. Python implemented in Python
• looks like this:
• runs on top of CPython• about 2000 times slower than CPython
$> python py.py
![Page 9: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/9.jpg)
What PyPy is
2. Translation Framework
$> python py.py
$> ./pypy-c $> ./pypy-jvm$> ./pypy-cli
![Page 10: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/10.jpg)
some PyPy facts
• founded in 2003 by Holger Krekel and Armin Rigo• 2004 - 2007 EU Project - Sixth Framework• since 2007 Open Source Project• occasional Funding by Google• Sprint-Driven Development
![Page 11: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/11.jpg)
PyPy Components
1.Architecture2.RPython3.Interpreter 4.Translation Framework
![Page 12: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/12.jpg)
PyPy componentsArchitektur
![Page 13: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/13.jpg)
PyPy componentsRPython• RPython = Restricted/Reduced Python
![Page 14: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/14.jpg)
PyPy componentsRPython• RPython = Restricted/Reduced Python
o Restricted = most possible static subset of Python
![Page 15: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/15.jpg)
PyPy componentsRPython• RPython = Restricted/Reduced Python
o Restricted = most possible static subset of Python o required to perform Type Inference
![Page 16: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/16.jpg)
PyPy componentsRPython• RPython = Restricted/Reduced Python
o Restricted = most possible static subset of Python o required to perform Type Inference o input to the Translation Framework
![Page 17: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/17.jpg)
PyPy componentsRPython• RPython = Restricted/Reduced Python
o Restricted = most possible static subset of Python o required to perform Type Inference o input to the Translation Frameworko no real specification - just some hints
![Page 18: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/18.jpg)
PyPy componentsRPython• RPython = Restricted/Reduced Python
o Restricted = most possible static subset of Python o required to perform Type Inference o input to the Translation Frameworko no real specification - just some hintso used for writing interpreters
![Page 19: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/19.jpg)
PyPy componentsRPython• RPython = Restricted/Reduced Python
o Restricted = most possible static subset of Python o required to perform Type Inference o input to the Translation Frameworko no real specification - just some hintso used for writing interpreterso can be used for writing extensions as well
![Page 20: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/20.jpg)
PyPy componentsInterpreter
![Page 21: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/21.jpg)
PyPy componentsInterpreter• written in RPython
![Page 22: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/22.jpg)
PyPy componentsInterpreter• written in RPython• Internals:
o bytecode compiler -> generates bytecode
![Page 23: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/23.jpg)
PyPy componentsInterpreter• written in RPython• Internals:
o bytecode compiler -> generates bytecodeo bytecode evaluator -> interprets bytecode
![Page 24: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/24.jpg)
PyPy componentsInterpreter• written in RPython• Internals:
o bytecode compiler -> generates bytecodeo bytecode evaluator -> interprets bytecode o object space -> handles operations on objects
![Page 25: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/25.jpg)
Python Interpreter
![Page 26: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/26.jpg)
PyPy componentsTranslation Framework
![Page 27: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/27.jpg)
PyPy componentsTranslation Framework
![Page 28: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/28.jpg)
PyPy componentsTranslation Framework
![Page 29: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/29.jpg)
PyPy componentsTranslation Framework
![Page 30: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/30.jpg)
![Page 31: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/31.jpg)
PyPy > CPython
1.High Level Language Implementation
2.JIT Generation3.Object Spaces4.Stackless
![Page 32: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/32.jpg)
PyPy > CPythonHigh Level Language Implementation• easy to experiment with
![Page 33: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/33.jpg)
PyPy > CPythonHigh Level Language Implementation• easy to experiment with • easy to implement new features
![Page 34: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/34.jpg)
PyPy > CPythonHigh Level Language Implementation• easy to experiment with • easy to implement new features
o lazily computed objects and functions
![Page 35: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/35.jpg)
PyPy > CPythonHigh Level Language Implementation• easy to experiment with • easy to implement new features
o lazily computed objects and functionso plug-able garbage-collection
![Page 36: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/36.jpg)
PyPy > CPythonHigh Level Language Implementation• easy to experiment with • easy to implement new features
o lazily computed objects and functionso plug-able garbage-collection o runtime replacement of live-objects
![Page 37: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/37.jpg)
PyPy > CPythonHigh Level Language Implementation• easy to experiment with • easy to implement new features
o lazily computed objects and functionso plug-able garbage-collection o runtime replacement of live-objectso stackless concurrency
![Page 38: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/38.jpg)
PyPy > CPythonJIT Generation• based on Partial Evaluation via
Yoshihiko Futamura (1971). "Partial Evaluation of Computation Process – An Approach to a Compiler-Compiler"
![Page 39: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/39.jpg)
PyPy > CPythonJIT Generation• based on Partial Evaluation via
Yoshihiko Futamura (1971). "Partial Evaluation of Computation Process – An Approach to a Compiler-Compiler"
• says: it's possible to take an interpreter and transform it into a compiler
![Page 40: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/40.jpg)
PyPy > CPythonJIT Generation• based on Partial Evaluation via
Yoshihiko Futamura (1971). "Partial Evaluation of Computation Process – An Approach to a Compiler-Compiler"
• says: it's possible to take an interpreter and transform it into a compiler
• means:having such a thing is pretty cool because writing interpreters is a lot easier than writing compilers.
![Page 41: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/41.jpg)
PyPy > CPythonJIT Generation
![Page 42: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/42.jpg)
PyPy > CPythonObject Spaces• Flow ObjSpace• Thunk ObjSpace• Taint ObjSpace• Dump ObjSpace• Transparent Proxies
![Page 43: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/43.jpg)
Thunk Object Space
• provides:1.lazily computed objects • lazily computed functions• globally replaceable objects
![Page 44: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/44.jpg)
Thunk Object Space
• provides:1.lazily computed objects • lazily computed functions• globally replaceable objects
• is implemented in about 200 lines of code
![Page 45: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/45.jpg)
Thunk Object Space
• provides:1.lazily computed objects • lazily computed functions• globally replaceable objects
• is implemented in about 200 lines of code
• Example:
>>>> a = "hello">>>> b = "world">>>> a + b'helloworld'
>>>> become(a,b)>>>> a + b'worldworld'
![Page 46: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/46.jpg)
Taint Object Space
• provides protection for:o sensitive data that should not leak
![Page 47: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/47.jpg)
Taint Object Space
• provides protection for:o sensitive data that should not leak
• provides protection from:o untrusted data that needs to be validated
![Page 48: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/48.jpg)
Taint Object Space
• provides protection for:o sensitive data that should not leak
• provides protection from:o untrusted data that needs to be validated
• Example:>>>> password = "secret">>>> password'secret'
>>>> password = taint("secret")>>>> passwordTraceback (application-level): File "<inline>", line 1 in <interactive> passwordTaintError
![Page 49: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/49.jpg)
PyPy > CPythonStackless• infinite Recursion
o CPython limited to depth=1000o regular PyPy limited to depth=5000
![Page 50: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/50.jpg)
PyPy > CPythonStackless• infinite Recursion
o CPython limited to depth=1000o regular PyPy limited to depth=5000
• Microthreadso Coroutineso Tasklets and Channelso Greenlets
![Page 51: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/51.jpg)
PyPy > CPythonStackless• infinite Recursion
o CPython limited to depth=1000o regular PyPy limited to depth=5000
• Microthreadso Coroutineso Tasklets and Channelso Greenlets
• Coroutine Pickling• Coroutine Cloning
![Page 52: a quick Introduction to PyPy](https://reader034.vdocuments.mx/reader034/viewer/2022042509/554a2a60b4c90542548b4ec5/html5/thumbnails/52.jpg)
Demo
Lazy functions and objectsTainted objectsTranslator Shell