Type Annotations in Python: Whats, Whys & Wows

Andreas Dewes (@japh44)

Europython 2017 - Rimini

Outline

Explain why type annotations areinteresting and how they cameto be.

Show you howyou can usethem in yourown code base.

Analyze howpeople actuallyuse them and show you whatelse you can do with them.

Motivation: When We Discover Bugs

Formal Proof ofCorrectness

Design Review

Unit / Integration / … Testing

Compiler Errors via Type Checker

Code Reviews

Static Analysis

We Don‘t, OurCustomer Does

Position in Software Life Cycle

Enter: Type Hinting

The Python Way: Gradual Typing

Unannotated code

Annotated code

external code

our code

external type information

History of Type Hints in Python

https://www.python.org/dev/peps/****

PEP 482 – LiteratureOverview

PEP 483 – Theory ofType Hints

PEP 484 – Type Hints

2014

3.5

PEP 3107 – FunctionAnnotations

2006

3.0

DraftInformal Accepted / Final

PEP 544 – Protocols: Structural Subtyping

PEP 526 – Variable Annotations

2016

3.6

2017

Annotation Syntax in Python

Return type annotation

Argument annotation

Variable annotation(Python 3.6 only)

Architecture of Type Hinting in Python

Python Interpreter

Just stores type annotations in a special__annotations__ variable.No runtime effectsotherwise!

The „typing“ module

Allows us to specify the types that wewant to annotate our code with (requiredfor non-standard types and advanced usecases)

External tools(e.g. mypy)

Uses annotations toperform type checking(or other functionality)

Getting StartedWith Type Hints: MyPy

• Originally Written by Jukka Lehtosalo, now also stronglypushed by Dropbox and Guido van Rossum

Easy to install & use

> pip install mypy> mypy [file to check] …

http://mypy-lang.org/

Example Code Base: Flor

Our initial code

• Small but functionalcodebase: Less than200 lines of code

• No external dependencies: Goodas an example

• Not many „exotic“ types: Easy toannotate

• Compatible withPython 2+3: Good fortesting different approaches

A Test Script

Adding Type Hints

• Go through the code functionby function, adding hints to all arguments and return types

• Possibly add hints toambiguous variable initializations (if needed)

• Import and use necessarytypes from the typing module

• Try to make mypy happy

I brokemypy!

Quick fix:bytes → _bytes

Running MyPy: It (Finally) Works!

Argument 1 to „BloomFilter“ has incompatible type „float“; expected „int“ (…)

Argument 1 to „add“ of„BloomFilter“ has incompatibletype „str“; expected „bytes“

Unsupported operandtypes for + (List[int] and „str“)

But … now we lost Python 2 compatibility

Second Approach:Type Comments

• Instead of writinghints as code, we addthem as comments

• A special syntax tellsmypy to treat themas type hints

• We still need toimport the „typing“ module (there is a workaround though)

Nice! But what about code we can‘t change?

Stub files (.pyi)

Mypy will look for Stubfiles in several places(search path, currentdirectory, typeshed, …). If it finds a .pyi file and a .py file, it will onlyload the .pyi!

Third Approach: Stub Files

Building a Stub File

• As before, start withthe code

• Remove all actualvalues and functionbodies, just leavingthe signatures

• Use ellipsis (…) toindicate missing parts

• Add type hints

• Think „header files“ for Python

ComparingOur Approaches

https://travis-ci.org/DCSO/flor

commentedinline

https://stackoverflow.com/questions/43516780/adding-type-information-without-dependency-on-typing-module

Pros & Cons

Inline+ The „canonical“ way

of using type hints

+ Easy to read

+ Code and hints arekept in one place

- Only compatible withPython ≥ 3.3 (or ≥ 3.6 if using variable annotations)

Type Comments+ Keeps code

compatible withPython ≥ 2.7

+ Allows variable annotations regardlessof Python version

- Ugly (?)

- Still requires importingthe „typing“ module(but there is a workaround)

Stub Files+ Does not modify

original source at all

+ Allows use of latestfeatures regardless ofPython version

- Duplicatesmaintenance effort

- Does not (yet) allowchecking of the actualcode against the stubs

There‘s(Much) More ToType Hints!

• Generics• Type variables• Classes• Generators• …

Are People Actually Using This? Let‘s Check!

We download code from the top 1000 Python repositories on Github and check it for any kindof type hinting: Inline annotations, type comments and stubs.

https://github.com/adewes/type-annotations-in-the-wild

103 projects with at least 1 type annotation

53 projects with at least 10

24 projects with at least 100

inline

comments

pyi files

Top Python Repositories with Type Hints

Function & Variable Annotations:Not Only For Type Hinting

> foo.__annotations__{‘x‘ : int,‘return‘ : bool}

Example: Contracts in Python

(not saying this is a good idea, just that it works…)

https://gist.github.com/adewes/b87c8adc95e768ebf6366130ad6d85a7

Summary

Type hintingworks & makesyour code morerobust (for youand for others)

You can use itnow already(regardless ifyou use Python 2 or 3)

Annotationscan do morethan type hinting (but think about ittwice)

Thanks!

Slides:

https://slideshare.com/japh44

Me:

@japh44

andreas@7scientists.com

License:

CC BY-NC 3.0: https://creativecommons.org/licenses/by-nc/3.0/en/