PythonEssentials

AsimJalis

PythonEssentials i

Contents

1 Introduction 1

1.1 Instructor . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.2 Introductions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 1

1.3 Hands-OnLearning . . . . . . . . . . . . . . . . . . . . . . . . . . 1

2 HistoryofPython 2

2.1 GuidovanRossum . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2.2 Python’sGoals . . . . . . . . . . . . . . . . . . . . . . . . . . . . 2

2.3 VersionsandTimeline . . . . . . . . . . . . . . . . . . . . . . . . . 2

2.4 Python2.7VersusPython3.0 . . . . . . . . . . . . . . . . . . . . . 3

2.5 Python2.7and3.0Differences . . . . . . . . . . . . . . . . . . . . 4

3 Introduction 4

3.1 WhatisPython . . . . . . . . . . . . . . . . . . . . . . . . . . . . 4

3.2 PythonvsPerl, Ruby, PHP . . . . . . . . . . . . . . . . . . . . . . . 5

3.3 StartupsUsingPython . . . . . . . . . . . . . . . . . . . . . . . . . 5

3.4 JobsinDifferentLanguages . . . . . . . . . . . . . . . . . . . . . . 5

3.5 InstallingPython . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3.6 PythonScripts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 6

3.7 PythonBAT Files . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.8 Documentation . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

3.9 InteractiveShell . . . . . . . . . . . . . . . . . . . . . . . . . . . . 8

4 Numbers 9

4.1 IntegersandFloating-PointNumbers . . . . . . . . . . . . . . . . . 9

4.2 Variables . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

4.3 PythonConventions . . . . . . . . . . . . . . . . . . . . . . . . . . 11

PythonEssentials ii

4.4 ArithmeticOperators . . . . . . . . . . . . . . . . . . . . . . . . . 11

4.5 AssignmentOperators . . . . . . . . . . . . . . . . . . . . . . . . . 12

4.6 IncrementingandDecrementing . . . . . . . . . . . . . . . . . . . 14

4.7 TipCalculator . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

5 Strings 14

5.1 Strings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 14

5.2 Single-QuotesandDouble-Quotes . . . . . . . . . . . . . . . . . . 15

5.3 RawStrings . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

5.4 TripleQuotes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

5.5 ConcatenatingStrings . . . . . . . . . . . . . . . . . . . . . . . . . 17

5.6 ConvertingBetweenNumbersandStrings . . . . . . . . . . . . . . . 18

5.7 StringFormatting . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

5.8 PercentStringFormatting . . . . . . . . . . . . . . . . . . . . . . . 20

5.9 FormatSyntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

5.10 ConversionSpecifiers . . . . . . . . . . . . . . . . . . . . . . . . . 21

5.11 ConversionFlags . . . . . . . . . . . . . . . . . . . . . . . . . . . 21

5.12 ConversionTypes . . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5.13 NumericConverter . . . . . . . . . . . . . . . . . . . . . . . . . . 22

5.14 FormattingAddendum . . . . . . . . . . . . . . . . . . . . . . . . 22

5.15 ReadingInput . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

5.16 PopulationCalculator . . . . . . . . . . . . . . . . . . . . . . . . . 23

6 Functions 24

6.1 CallingFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . 24

6.2 DefiningFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . 26

6.3 VariablesInFunction . . . . . . . . . . . . . . . . . . . . . . . . . 27

6.4 DefaultArgumentValues . . . . . . . . . . . . . . . . . . . . . . . 28

PythonEssentials iii

6.5 KeywordArguments . . . . . . . . . . . . . . . . . . . . . . . . . . 28

6.6 MultipleReturnValues . . . . . . . . . . . . . . . . . . . . . . . . 29

6.7 ScopingRules . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

6.8 Globals . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

6.9 CreatingLibrariesofFunctions . . . . . . . . . . . . . . . . . . . . 32

7 Documentation 32

7.1 FunctionDocstrings . . . . . . . . . . . . . . . . . . . . . . . . . . 32

7.2 ModuleDocstrings . . . . . . . . . . . . . . . . . . . . . . . . . . 33

8 Objects 33

8.1 InteractingwithObjects . . . . . . . . . . . . . . . . . . . . . . . . 33

9 GeneratingReports 34

9.1 FileReadingandWriting . . . . . . . . . . . . . . . . . . . . . . . 34

9.2 HTML Report . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

10 Booleans 38

10.1 BooleanType . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

10.2 BooleansFromComparisons . . . . . . . . . . . . . . . . . . . . . 38

10.3 ComparisonOperators . . . . . . . . . . . . . . . . . . . . . . . . 39

10.4 ConvertingBooleans . . . . . . . . . . . . . . . . . . . . . . . . . 40

10.5 BooleanOperators . . . . . . . . . . . . . . . . . . . . . . . . . . 42

11 ControlFlow: Branching 43

11.1 IfandElse . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

11.2 If, Elif, andElse . . . . . . . . . . . . . . . . . . . . . . . . . . . . 43

11.3 If-ElseandBooleanOperators . . . . . . . . . . . . . . . . . . . . . 44

11.4 If-ElseOperator . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

PythonEssentials iv

11.5 ValidatingNumericInputs . . . . . . . . . . . . . . . . . . . . . . . 46

11.6 CatchingControl-C . . . . . . . . . . . . . . . . . . . . . . . . . . 47

11.7 ThrowingExceptions . . . . . . . . . . . . . . . . . . . . . . . . . 48

12 ControlFlow: Looping 49

12.1 LoopingwithWhile . . . . . . . . . . . . . . . . . . . . . . . . . . 49

12.2 LoopingwithFor . . . . . . . . . . . . . . . . . . . . . . . . . . . 51

13 Collections 52

13.1 DefiningLists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 52

13.2 AccessingListElements . . . . . . . . . . . . . . . . . . . . . . . . 53

13.3 ListSurgery . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 53

13.4 CreatingLists . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 54

13.5 ListOperations . . . . . . . . . . . . . . . . . . . . . . . . . . . . 55

13.6 CommandLineArguments . . . . . . . . . . . . . . . . . . . . . . 56

13.7 SequencesandFunctions . . . . . . . . . . . . . . . . . . . . . . . 56

13.8 Lambda . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

13.9 Map . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 57

13.10Filter . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 58

13.11Dictionaries . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 59

13.12Tuples . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

13.13TuplesvsListPerf . . . . . . . . . . . . . . . . . . . . . . . . . . . 61

13.14Sets . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

13.15SetSyntax . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 63

14 OperatingSystem 63

14.1 SystemEnvironment . . . . . . . . . . . . . . . . . . . . . . . . . . 63

14.2 RunningShellCommands . . . . . . . . . . . . . . . . . . . . . . . 64

14.3 OtherProcessConstructs . . . . . . . . . . . . . . . . . . . . . . . 64

PythonEssentials v

15 Third-PartyModules 65

15.1 PipvsEasy_Install . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

15.2 WindowsPipInstall . . . . . . . . . . . . . . . . . . . . . . . . . . 66

15.3 Unix/MacPipInstall . . . . . . . . . . . . . . . . . . . . . . . . . 66

15.4 Demjson . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

15.5 IPython . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68

15.6 WebServices . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

16 Objects 70

16.1 DefiningObjects . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

16.2 WhyObjects . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

16.3 Inheritance . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 72

16.4 ImplicitMethods . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

16.5 StrandLen . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

16.6 StrvsRepr . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

16.7 OperatorOverloading . . . . . . . . . . . . . . . . . . . . . . . . . 74

16.8 NewStyleandOldStyleObjects . . . . . . . . . . . . . . . . . . . 74

16.9 OperatorPrecedence . . . . . . . . . . . . . . . . . . . . . . . . . 75

17 RegexandWebScraping 76

17.1 Regex . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

17.2 SpecialCharacters . . . . . . . . . . . . . . . . . . . . . . . . . . . 77

17.3 AdvancedSpecialCharacters . . . . . . . . . . . . . . . . . . . . . 78

17.4 SpecialSequences . . . . . . . . . . . . . . . . . . . . . . . . . . 78

17.5 RegexFunctions . . . . . . . . . . . . . . . . . . . . . . . . . . . . 79

17.6 RegexFlags . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 80

17.7 RegexExercises . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

17.8 WebScraping . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 81

PythonEssentials vi

18 CSV Files 83

18.1 ReadingCSV Files . . . . . . . . . . . . . . . . . . . . . . . . . . . 83

18.2 WritingCSV Files . . . . . . . . . . . . . . . . . . . . . . . . . . . 84

19 Database 86

19.1 CreatingTables . . . . . . . . . . . . . . . . . . . . . . . . . . . . 86

19.2 InsertingRecords . . . . . . . . . . . . . . . . . . . . . . . . . . . 87

19.3 BatchInserts . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

19.4 SelectingRecords . . . . . . . . . . . . . . . . . . . . . . . . . . . 88

19.5 CursorMethods . . . . . . . . . . . . . . . . . . . . . . . . . . . . 89

19.6 Querying . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 90

19.7 MySQL andPostgreSQL . . . . . . . . . . . . . . . . . . . . . . . . 90

20 NoseTests 91

20.1 UnitTesting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 91

20.2 NoseTests . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

20.3 NoseInstall . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 93

20.4 NoseTestExample . . . . . . . . . . . . . . . . . . . . . . . . . . 93

20.5 NoseSetUp . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 95

21 XML Parsing 96

21.1 xmltodict . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 96

21.2 ElementTree . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 98

22 WebApplicationsUsingFlask 99

22.1 SettingUpFlask . . . . . . . . . . . . . . . . . . . . . . . . . . . . 99

22.2 HelloWorldWebApp . . . . . . . . . . . . . . . . . . . . . . . . 100

22.3 TwoPageWebApp . . . . . . . . . . . . . . . . . . . . . . . . . . 100

22.4 TipCalculatorWebApp . . . . . . . . . . . . . . . . . . . . . . . . 102

PythonEssentials vii

22.5 DeployingonPythonAnywhere . . . . . . . . . . . . . . . . . . . . 105

22.6 MVC Application . . . . . . . . . . . . . . . . . . . . . . . . . . . 106

23 ListComprehensions 111

23.1 ListComprehensions . . . . . . . . . . . . . . . . . . . . . . . . . 111

23.2 ListsofDictionaries . . . . . . . . . . . . . . . . . . . . . . . . . . 112

24 Pexpect 112

24.1 Pexpect . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 112

24.2 PexpectMethods . . . . . . . . . . . . . . . . . . . . . . . . . . . 114

25 Threading 115

25.1 Threading . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 115

26 MiscTopics 117

26.1 ArgsandKWArgs . . . . . . . . . . . . . . . . . . . . . . . . . . . 117

26.2 ClassmethodandStaticmethod . . . . . . . . . . . . . . . . . . . . 118

26.3 Closures . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 119

26.4 Decorators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 120

26.5 Memoizer . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

26.6 Iterators . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 121

27 Next 123

27.1 NextSteps . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 123

PythonEssentials 1

1 Introduction

1.1 Instructor

• AsimJalis

• HasworkedassoftwareengineeratMicrosoft, Hewlett-Packard, andSalesforce.

• http://linkedin.com/in/asimjalis

1.2 Introductions

• Whatisyourname? Whatdoyoudo?

• Howareyouplanningtousewhatyoulearnhere?

• Whatisyourperfectoutcome?

1.3 Hands-OnLearning

• Howwillthiscoursework?

– Hands-onclass.

– Learnbydoing.

• Whyhands-on?

– Helpsyougetmostoutofclass.

– Youinteractwithmaterialmoredeeply, learn.

– Encouragessmallmistakes, fasterlearning.

– Helpsgetissuesresolvedhere, now.

– Afterwardsyouretaintheexperience.

PythonEssentials 2

2 HistoryofPython

2.1 GuidovanRossum

WhowrotePython?

PythonwaswrittenbyGuidovanRossum, alsoknownasPython’sBenevolentDictatorforLife(BDFL).

Whatdoeshelooklike?

Here isapicturefrom2006.

2.2 Python’sGoals

Whatledhimtowriteit?

Accordingto Guido, ”[In]December1989, I waslookingfora’hobby’programmingprojectthatwouldkeepmeoccupiedduringtheweekaroundChristmas. [...] I de-cidedtowriteaninterpreterforthenewscriptinglanguageI hadbeenthinkingaboutlately. [...] I chosePythonasaworkingtitlefortheproject, beinginaslightlyirrever-entmood(andabigfanofMontyPython’sFlyingCircus).”

Whatweresomeofitsgoals?

Ina1999DARPA fundingproposal, vanRossumdefinedPython’sgoalsasfollows:

• aneasyandintuitivelanguagejustaspowerfulasmajorcompetitors• opensource, soanyonecancontributetoitsdevelopment• codethatisasunderstandableasplainEnglish• suitabilityforeverydaytasks, allowingforshortdevelopmenttimes

2.3 VersionsandTimeline

WhathasbeenPython’stimeline?

PythonVersion Released Changes

PythonEssentials 3

0.9.0 Feb1991 ReleasedbyGuidoVanRossumtoalt.sources

1.0 Jan1994 Added lambda, map, filter, reduce

1.5 Dec1997 Bugfixes

1.6 Sep2000 Licensechanges

2.0 Oct2000 Listcomprehensions

2.1 Apr2001 Generators

2.2 Dec2001

2.3 Jul2003

2.4 Nov2004

2.5 Sep2006

2.6 Oct2008

2.7 Jul2010

3.0 Dec2008 Compatibilitybreakingchanges

3.1 Jun2009

3.2 Feb2011

3.3 Sep2012

2.4 Python2.7VersusPython3.0

ShouldI usePython2.7orPython3.0?

YoushouldusePython2.7because:

• Manylibrarieshavenotbeenportedoverto3.0

• 2.7shipswithMacandothersystemsbydefault

• Youwillencountercodewrittenin2.7

Someofthechangesin3.0areunderthesurfaceanddonotaffectthesyntax. Othersdo. Wewilldiscussimportantdifferencesastheycomeup.

PythonEssentials 4

2.5 Python2.7and3.0Differences

Differencesin print

• Use print("Hello world") not print "Hello world"• Use print("Hello world", end=" ") not print "Hello world",• Use print() not print• Use print("Error", file=sys.stderr) not print>>sys.stderr,

"error"• Use print((a, b)) not print (a,b)

Differencesinintegerdivision

• In2.7 3/2 isintegerdivisionandsoitequals 1.• In3.0 3/2 isfloatdivisionandsoitequals 1.5.• Inboth2.7and3.0 3//2 isintegerdivision.• Whenyouwantintegerdivisionuse 3//2 inbothtobesafe.

3 Introduction

3.1 WhatisPython

WhatisPython?

• Pythonisahigh-levelprogramminglanguage.

• Fastedit-compile-runcycle. Fast, interactive, programmingexperience.

• Greatforwritingquickprograms, thatovertimeevolveintobigprograms.

• Usefulforautomatingsysadmintasks, builds, websites.

• ShipswithMacandUnixmachinesbydefault. Easytoinstall. Everywhere.

• Object-orientedandthecodedoesnotbuckleoverasitgrowsovertime.

PythonEssentials 5

3.2 PythonvsPerl, Ruby, PHP

All of these languages are scripting languages. The code canbe run immediatelywithoutwaiting for thecompiler. This leads toa fast interactive funprogrammingexperience.

Feature Python Perl Ruby PHP

Nocompilationneeded X X X X

Webframework Django, Flask Catalyst, Dancer Rails CakePHP,CodeIgniter

Forcommand-lineutilities X X X

Object-orientedbaked-in X X

Scalesforlargeapps X X

3.3 StartupsUsingPython

WhataresomeapplicationsandstartupsthatusePython?

• BitTorrent waswrittenbyBramCoheninPython. HestartedinApril2001, andreleaseditinJuly2001.

• eGroups waswritten in200,000 linesof Pythonaccording to founder ScottHassan. ItwaslateracquiredbyYahoo! andturnedintoYahoo! Groups.

• Yelp iswritteninPython.

• Reddit waswritteninLispandthenrewritteninPythoninoneweekend.

3.4 JobsinDifferentLanguages

HowdoesPythoncomparewiththeotherlanguagesinjobs?

Here istherelativenumberofjobsbasedondatafrom.

PythonEssentials 6

3.5 InstallingPython

Exercise: InstallPython.

Solution:

• OnWindowsinstallPython2.7.5from http://www.python.org/getit/

• OnMacPython2.7comespreinstalled.

• Verifythatyouhave python.

python --version

3.6 PythonScripts

Exercise: Writeascriptthatprints Hello, world.

Solution:

• Savethisinatextfilecalled hello.py usinganeditorlikeSublimeText, Tex-tWrangler, NotePad++, orTextPad.

#!/usr/bin/env python

print "Hello, world."

• Type python hello.py torunit.

• Ortype chmod 755 hello.py andthentype ./hello.py torunit.

Notes:

• Itisconventionaltoputthe .py extensiononPythonscripts. However, itisnotrequired. Wecouldhavecalledtheprogram hello insteadof hello.py.

• WhitespaceissignificantinPython. Theindentationofthestatementindicateswhatblockitisinaswewillseelater. Forthisprogramtowork, thestatementmusthavezeroindentation.

PythonEssentials 7

• Nosemicolonisrequiredtoendthestatement.

• Printautomaticallyputsanewlineaftertheoutput.

• Ifyouwanttoprintoutputwithoutanewlineputacommaafterit.

• Thiswillnotprintanewline.

#!/usr/bin/env python

print "Hello, world.",

3.7 PythonBAT Files

Exercise: CreateaWindowsBAT filethatprintshelloworldusingPython.

Solution:

• OnWindowssavethisto file.bat andthenyoucanrunitfromthecommandline.

@echo off & python.exe -x "%~f0" %* & goto :EOF

print "Hello, world."

• OryoucansavePythonin file.py andthenusethis file.bat.

@echo offpython file.py

3.8 Documentation

Exercise: Findthedocumentationforthe raw_input function.

Solution:

• InthePythonconsoletype help(raw_input).

• help canbeusedwithPythonfunctionsandobjects.

PythonEssentials 8

3.9 InteractiveShell

Exercise: Calculate 3 + 4 ontheinteractiveshell.

Solution:

• Type python.

• AtthePythonprompt, enter 3 + 4.

Notes:

• NotethedifferencebetweenbeinginthePythonshellandintheterminalshell.Itislike Inception--shellinsideashell.

• Pythonhasastrongopiniononeverything, includinghowyoushouldexittheshell. Toseethistryexitingtheshellbytyping exit or quit.

Exercise: WhatistheZenofPython?

Solution:

• Type python.

• AtthePythonprompt, enter import this.

Exercise: RunascriptfromthePythonshell.

Solution: YoucanrunaPythonscriptfromthePythonshellusing execfile asfol-lows.

execfile(r'/path/to/script/hello.py')

Notes:

• ComparethedifferencebetweenrunningscriptsfromthePythonshellversusrunningthescriptfromtheterminalorthecmdshell.

PythonEssentials 9

4 Numbers

4.1 IntegersandFloating-PointNumbers

ThemostbasicdatatypeinPythonisthenumber. Pythonhasbothintegersaswellasfloatingpointnumbers. Youcanenternumbersliterallyintotheshelltoseetheirvalues.

Exercise: Guessthevaluesandtypesofthesenumberliterals: 123, 0x10, 010, 0, 1.1,1.1e3, 0.0

Solution:

Literal Value Type

123 123 integer

0x10 16 integer

010 8 integer

0 0 integer

1.1 1.1 floating-point

1.1e3 1100.0 floating-point

0.0 0.0 floating-point

Notes:

• Youcandeterminethetypeofavaluebytyping type(123) or type(123.0).

• Thefunction type returnsthetypeasastring.

4.2 Variables

Exercise: Writeaprogramthatdividesarestaurantcheckof$43between3friends.

Solution:

PythonEssentials 10

# Divide restaurant check of $43 between 3 people.

persons = 3amount = 43.0amount_per_person = \

amount / personsprint "Amount per person:", amount_per_person

Notes:

• Variablesareslotsinmemorywheredatasuchasnumbersarestored.

• A variableacquiresavalueafteritisassigned.

• The = operatorassignsthevaluefromtherighttothevariableonitsleft. Itslefthandsidecanonlycontainasinglevariable.

• Variablesandnumberscanbecombinedontherighthandsideinarithmeticexpressions.

SyntacticNotes:

• Statementsareexecutedinsequence.

• Statementsendwithnewline.

• Whitespaceandindentationisimportant.

• Statementsmustoccurononeline.

• Ifastatementmustcontinueonthefollowingline, endthelinewitha \.

• Commentsstartwith # andendattheendofline.

• Pythonprograms areminimal and have very little syntactic noise: no semi-colons, no $ beforevariablenames, nomysterioussymbols.

PythonEssentials 11

4.3 PythonConventions

• Variablenamesaremadeupofaletterorunderscore, followedbyletters, un-derscores, ornumbers.

• Python convention is to use snake_case rather than camelCase orPascalCase forvariableandfunctionnames.

• Pythonclassnamesuse PascalCase.

• Pythonmodulenameswhicharethesameasfilenamesalsouse snake_case.

• NamingconventionsandstyleguidelinesarediscussedinmoredetailinPEP 8http://www.python.org/dev/peps/pep-0008.

• PEP standsfor PythonEnhancementProposal.

4.4 ArithmeticOperators

WhatarithmeticoperatorsdoesPythonhave?

Expression Result

a + b Adding a and b

a - b Subtracting b from a

a * b Multiplying a and b

a / b Dividing a by b (producesfractionin3.0)

a // b Dividing a by b (integerdivision)

a b Ra ising a tothepower b

a % b Remainderofdividing a by b

Notes:

• Theseoperatorsdon’tchangethevalueofthevariablesintheexpression.

• Theyproduceanewvaluewhichcanbeassignedtoavariable.

PythonEssentials 12

• In x = a + b thevalueof a and b arenotchanged. Onlythevalueof x. Onlythevariabletotheleftof = changes.

4.5 AssignmentOperators

Exercise: A large14”pizzaat ExtremePizza costs$14.45. Each topping is$1.70.Supposewewantjalapenos, olives, artichokehearts, andsun-driedtomatoes. Howmuchwillthetotalbe?

Solution1:

crust = 14.45topping = 1.70

price = crustprice = price + topping # jalapenosprice = price + topping # olivesprice = price + topping # artichoke heartsprice = price + topping # sun-dried tomatoes

print "Pizza Price:", price

Notes:

• A variablecanbeassignedtomultipletimes.

• A variablecanrecycleitsownpreviousvalueontheleftof =.

• = isnotmathematicalequality. So price = price + topping isnotaparadox.

• = issimplyassignment.

Solution2:

crust = 14.45topping = 1.70

PythonEssentials 13

price = crustprice += topping # basilprice += topping # olivesprice += topping # cilantroprice += topping # pesto

print "Pizza Price:", price

Notes:

• Becausethepattern x = x + a occursalotthereisashort-handforit: x +=a.

• Readthisas: modify x byadding a toit.

• += iscalledanassignmentoperator.

• Itisarelativeof =.

• Ittoochangesthevalueofthevariableonlyonitslefthandside.

Whatarethedifferentassignmentoperators?

Assignment Meaning

a = b Modify a bysettingitto b

a += b Modify a byadding b toit

a -= b Modify a bysubtracting b fromit

a *= b Modify a bymultiplying b withit

a /= b Modify a bydividingitby b

a = b Mo dify a byraisingittothepower b

a %= b Modify a bysettingittotheremainderofdividingitby b

PythonEssentials 14

4.6 IncrementingandDecrementing

Exercise: Writethestatementforincrementingthevalueof a by 1.Solution:

a += 1

Exercise: Writethestatementfordecrementingthevalueof a by 1.Solution:

a -= 1

4.7 TipCalculator

Exercise: Writeaprogramthatdividesarestaurantcheckof$43between3friendsandaddsatipaswell.

Solution:

persons = 3amount = 43.0tip_rate = 0.15tip = amount * tip_rateamount_per_person = amount / personstip_per_person = tip / personsamount_per_person += tip_per_personprint "Amount per person:", amount_per_person

5 Strings

5.1 Strings

Besidesintegersandfloating-pointnumbersyoucanalsousestringsasadatatype.Stringrepresenttext.

Stringsaremarkedbydouble-quotesorsingle-quotes. Thesequoteshavethesamemeaning.

PythonEssentials 15

message1 = 'Hello'message2 = 'Goodbye'print message1print message2

5.2 Single-QuotesandDouble-Quotes

What’sthedifferencebetweensingle-quotesanddouble-quotes?

• ThereisnodifferenceinPython.

• Youcanencloseasingle-quoteeasilyinadouble-quotedstring, andadouble-quoteinasingle-quotedstring.

• Escape sequences like \n work in both double-quotes, aswell as in single-quotes.

Whatotherescapesequencesarethere?

Sequence Value

\n Newline

\t Tab

\a Bell

\' Single-quote

\" Double-quote

\\ Backslash

Exercise: Print Jim's Garage.

Solution1:

print 'Jim\'s Garage'

Solution2:

PythonEssentials 16

print "Jim's Garage"

5.3 RawStrings

Exercise: Print c:\temp\dir\file.txt.

Solution1::

print 'c:\\temp\\dir\\file.txt'

Solution2:

print r'c:\temp\dir\file.txt'

Solution3:

print r"c:\temp\dir\file.txt"

Notes:

• The r prefixbeforethestringturnsitintoarawstring.

• Whatyouseeiswhatyouget.

• Thebackslashesarenolongerescapecharacters. Insteadtheyare justback-slashes.

• Therawstringislikeaziplockbag. Thestringispreservedexactlyasyousaveit.

• Youcanputthe r prefixbeforeasingle-quotedstring, adouble-quotedstring,oratriple-quotedstring. Ithasthesameeffectinallcases.

PythonEssentials 17

5.4 TripleQuotes

Pythonalsohastriple-quotesforstringsthatspanmultiplelines.

Exercise: Define usage_text foratipcalculator.

Solution:

usage_text= '''TIP CALCULATOR

USAGEpython tip.py AMOUNT PERSONS TIP_RATE

NOTESPrints out the total amount due per person including tip.'''

Notes:

• Theopening ''' havetobeimmediatelybeforethefirstlinetoavoidablanklineatthebeginning.

• Theclosing ''' havetobeimmediatelyafterthelastlinetoavoidanewlineattheend.

5.5 ConcatenatingStrings

Stringscanbeconcatenatedusingthe + operator.

Exercise: Combineafirstnameandlastnamewithaspaceinthemiddle.

Solution1:

first_name = 'Dmitri'last_name = 'Hayward'full_name = first_name + ' ' + last_nameprint full_name

Solution2:

PythonEssentials 18

name = ''name += 'Dmitri'name += ' 'name += 'Hayward'print name

Notes:

• Wecouldhavewritten name += 'Dmitri' as name = name + 'Dmitri'aswell.

• Theassignmentoperator += forstringsmeansmodifythevariableonthelefthandsidebyappendingtheexpressionontherighthandsidetoit.

5.6 ConvertingBetweenNumbersandStrings

Exercise: Convertastring "43.0" toafloating-pointnumber.

Solution:

amount_string = "43.0"amount = float(amount_string)

Exercise: Convertastring "3" toaninteger.

Solution:

people_string = "3"people = int(people_string)

Exercise: Convertanumber 14.33 toastring.

Solution:

amount_per_person = 14.33amount_per_person_string = str(14.33)

Notes:

PythonEssentials 19

• Thefunction int convertsalldatatypestointegers.

• Similarly, thefunction float convertsitsinputtofloating-pointnumber.

• Andthefunction str convertanydatatypetoastring.

Exercise: Checkwhathappensifyoucall float("hello").

5.7 StringFormatting

Exercise: Writeaprogramthatdividesarestaurantcheckof$43between3friends.Printtheanswerformattednicelywithadollarsign.

Solution:

persons = 3amount = 43.0amount_per_person = amount / personsprint "Amount per person: ${0}".format(amount_per_person)

Notes:

• The .format functionisappliedtoastringandreturnsastringwhichisprintedby print.

• Wecouldhavesaveditasavariableaswell.

output = "Amount per person: ${0}".format(amount_per_person)print output

Exercise: Trimtheamounttotwodecimalplaces.

Solution:

persons = 3amount = 43.0amount_per_person = amount / personsprint "Amount per person: ${0:.2f}".format(amount_per_person)

PythonEssentials 20

Exercise: Whatwillbetheoutputoftheseformatstrings?

print "{0}, {1}, {2}".format("a", "b", "c")print "{1}, {2}, {0}".format("a", "b", "c")print "{2}, {2}, {2}".format("a", "b", "c")

Solution:

a, b, cb, c, ac, c, c

Notes:

• Thenumberintheparenthesesreferstothepositionoftheargumentofformat.

• Youcanuseanargumentasmanytimesasyouwant. Andinanyorder.

5.8 PercentStringFormatting

Exercise: Writeaprogramthatdividesarestaurantcheckof$43between3friends.Printtheanswerformattednicelywithadollarsign.

Solution:

persons = 3amount = 43.0amount_per_person = amount / personsprint "Amount per person: %.2f" % (amount_per_person,)

Notes:

• The % operatorislikethe format function, exceptitusestraditionalC printfformatsyntax.

• ThisisdeprecatedinPython3.0and format isthepreferredwayofformattingoutput.

• However, thisiscommonlyusedandyouwillseethisincodeoften.

PythonEssentials 21

5.9 FormatSyntax

Syntax Meaning

"%d %s" % (10, "hello") Substitutevaluesfromtuple

"%(k)d %(k2)s" % {"k1":10, "k2":"hello"} Substitutevaluesfromdictionary

5.10 ConversionSpecifiers

Conversionspecifiersarestringsofform %d and %s. Hereistheirfullsyntax.

• The’%’character, whichmarksthestartofthespecifier.• Optionalmappingkey, ofform (key). Formatargumentisdictionaryinstead

oftuple.• Optional conversionflags (seebelow), whichaffecttheresultofsomeconver-

siontypes.• Minimumfieldwidth(optional). Ifspecifiedas * valuereadfromtuple.• Optionalprecision. Givenas . followedbyprecision. Ifspecifiedas * value

readfromtuple.• Conversiontype (seebelow).

5.11 ConversionFlags

Flag Meaning

# Put 0 beforeoctal, 0X or 0x beforehexadecimal, anddecimalpointinfloatingpointalways

0 Zeropadnumericvalues

- Leftadjustconvertedvalue(overrides 0 conversionifbothgiven)

‘‘ Blankinsertedbeforepositivenumber(oremptystring)

+ Signcharacter(+ or -)precedesconversion(overridesblankflag)

PythonEssentials 22

5.12 ConversionTypes

Conversion Meaning

d Signedinteger

i Signedinteger

o Signedoctal

x Signedhexadecimal(lowercase)

X Signedhexadecimal(uppercase)

e Floatingpointexponential(lowercase)

E Floatingpointexponential(uppercase)

f Floatingpoint

F Floatingpoint

g Floatingpoint. Useslowercaseexponentialformatifexponent<-4

G Floatingpoint. Usesuppercaseexponentialformatifexponent<-4

c Singlecharacter

r String(convertsobjectusing repr())

s String(convertsobjectusing str())

% The % character

5.13 NumericConverter

Exercise: Writeaprogramthatconvertsdecimalintegerstooctalandhexadecimal.

Exercise: WriteaprogramthatgeneratesanHTML pagefromtitleandbodystrings.

5.14 FormattingAddendum

Formattingusing .format ismostlylike % formatting. However, therearesomedif-ferences.

PythonEssentials 23

http://docs.python.org/2/library/string.html

Pythonintroducesa 0o13 asanewnotationforoctalinPython3.0. "{0:#o}".format(x)outputswiththe 0o prefix. Thismakesoctalsimilartohexadecimalwhichuses 0x.Also 0b isusedforbinary.

5.15 ReadingInput

Exercise: Writeaprogramthatasksauserforhisnameandthensayshellotohimorher.

Solution:

import readlineuser_name = raw_input('What is your name? ')print 'Hello, {0}!'.format(user_name)

Notes:

• Youcanput import readline atthetopofthefile. ThislineenablesUnixreadlineeditingoftheinput.

• InPython3.0 raw_input hasbeenrenamedto input.

5.16 PopulationCalculator

Exercise: EstimatethepopulationofCaliforniain2021. Thepopulationin2012was38,000,000andthegrowthratewas1%. Thepopulationcanbeestimatedas

pop = pop_initial * (1 + growth_rate)**years

Here years isthenumberofyearssince2012, pop_initialisthepopulationin2012.

Solution:

PythonEssentials 24

pop_initial = 38 * 1000 * 1000year_initial = 2012year_final = 2021year_count = year_final - year_initialpop_rate = 0.01pop_final = pop_initial * ((1 + pop_rate) ** year_count)print "The population of California in year {0} will be {1:,.0f}".format(

year_final, pop_final)

Notes:

• {1:,.0f} printsafloatingpointnumberwithzerodecimalplacesandwithcommas.

Exercise: Are theparenthesesaround (1 + $pop_rate) ** $year_count re-quired?

Solution:

• Precedencefromhighertoloweris: ** * / + -

• Sostrictlyparenthesesarenotneeded.

• However, it is safer touseparenthesesandnotguessprecendence if it’snotclear.

Exercise: Generalizethisprogramtotakeanyyearasinput.

6 Functions

6.1 CallingFunctions

Exercise: Writeaprogramthatthrowsadice.

Solution1:

PythonEssentials 25

import randomdice = random.randint(1,6)print 'The dice was {0}'.format(dice)

Solution2:

from random import *dice = randint(1,6)print 'The dice was {0}'.format(dice)

Solution3:

import random as rdice = r.randint(1,6)print 'The dice was {0}'.format(dice)

Notes:

• import pullsinfunctionsfromotherpackages.

• Besidesbuilt-infunctions, allfunctionsandclassesrequire import.

• Usually import isputatthebeginningofthefile, butitcanoccuranywherebeforethefunctioncall.

• import random importsfunctionsandvariablesintothe random namespace.

• from random import * imports functionsandvariables into thecurrentnamespace.

• import random as r importsfunctionsandvariablesintothenamespace r.Thisisusefulifyouwanttoabbreviatealongnamespacename.

PythonEssentials 26

6.2 DefiningFunctions

Exercise: Writeafunctionthattakesanameandreturnsagreeting, andthencallit.

Solution:

name = 'Jim'

def greet(name):greeting = "Hello, " + name + "!"return greeting

print greet(name)print greet('Dmitri')print greet('Alice')print greet('Jim')

Notes:

• Theindentationisimportant. Pythondetermineswherethefunctionendsbasedontheindentation.

• Thefunctionparameter name containsacopyofthevaluethatispassedintothefunctioncall. Ifyoumodify name inthefunctionthatwillnotaffectthecaller’svariablevalue.

• Ifafunctionhasnobodyyoucansimplyput pass init.

• Functionscannotbecalledbeforetheyaredefined.

Exercise: Writetwofunctions. ThefirstconvertsCelsiustoFahrenheit. ThesecondconvertsFahrenheittoCelsius.

Solution:

def c2f(c):f = (c * 9./5.) + 32.return f

PythonEssentials 27

def f2c(f):c = (f - 32.) * 5./9.return c

Notes:

• Allvariablesintroducedinthefunctionforexample f and c disappearassoonasthefunctionexits.

• Theydonothaveanyconnectiontovariablesof thesamenameoutsidethefunction.

6.3 VariablesInFunction

Exercise: Verifythatafunctioncannotaffectthevariablesoutsideitsscope.

Solution:

var1 = "hello"

def f(arg1):# New unrelated var1 is created.# New var1 will go away with scope.var1 = "goodbye"

# Arg1 value is overwritten.# Does not affect calling parameter.arg1 = "new"

# Access local var1.print var1

# Call will not change global var1 value.f(var1)

PythonEssentials 28

6.4 DefaultArgumentValues

Exercise: Createatipcalculatorfunctionthatassumesadefaulttiprateof15%.

Solution:

def calculate_tip(amount, tip_rate = 0.15):tip = amount * float(tip_rate)return tip

print calculate_tip(10, 0.20)print calculate_tip(10, 0.15)print calculate_tip(10)

6.5 KeywordArguments

Exercise: Createatipcalculatorfunctionthatcalculatesthetotalamountperpersonincludingtip. Itassumesadefaulttiprateof15%andassumesthatbydefaultthereisonlyoneperson.

Solution:

def calculate_tip(amount, persons=1, tip_rate=0.15):tip = amount * float(tip_rate)amount += tipamount_per_person = amount / personsreturn amount_per_person

print calculate_tip(43, 3, 0.15)print calculate_tip(43, persons=3, tip_rate=0.20)print calculate_tip(amount=43, persons=3, tip_rate=0.20)print calculate_tip(amount=43, tip_rate=0.20)print calculate_tip(amount=43, tip_rate=0.20, persons=3)print calculate_tip(tip_rate=0.20, persons=3, amount=43)

Notes:

• Youcancombinepositionargumentsandkeywordarguments.

PythonEssentials 29

• However, onceyoustartusingkeywordsargumentsyoucannotgobacktopo-sitionalargumentsinthatfunctioncall.

6.6 MultipleReturnValues

Exercise: Writeafunctionthatpromptstheuserforamount, people, andtiprate, andthenreturnsallthreevalues.

Solution1:

def get_input():amount = float(raw_input('Amount: '))people = int(raw_input('People: '))tip_rate = float(raw_input('Tip Rate: '))return amount, people, tip_rate

amount, people, tip_rate = get_input()

print amount, people, tip_rate

Solution2:

def get_input():amount = float(raw_input('Amount: '))people = int(raw_input('People: '))tip_rate = float(raw_input('Tip Rate: '))return (amount, people, tip_rate)

(amount, people, tip_rate) = get_input()

print amount, people, tip_rate

Notes:

• Ineffectthetwosolutionsareequivalent.

PythonEssentials 30

• Inbothcasesthethreevariables amount, people, and tip_rate willgetthevaluestheuserentered.

• UnderthehoodSolution1returnsatuple. Solution2makesthatexplicitbyusingthetuplenotation. Wewilldiscusstuplesinafuturesection.

• Solution1ismoreidiomaticandiscommonlyusedinPythoncode.

6.7 ScopingRules

Python’sscopingrulescanberememberedusingthismneumonic: LEGB.

Scope Meaning

Local Definedinthecurrent def

Enclosed Definedintheenclosing def

Global Definedinthemodule

Builtin Pythonbuiltin

Notethatthismeansthatyoucanoverridebuiltinvariables.

Exercise: Redefine raw_input tothrowanexception.

Solution:

def raw_input(message):raise Exception(

"raw_input: not implemented")

name = raw_input("What is your name? ")print "Hello " + name + "!"

Notes:

• Whenavariable is read it is lookedupfirst in the local scope, and then inenclosed, thenglobal, thenbuiltin. Ifitisnotfoundinanyofthemanerrorisraised.

PythonEssentials 31

• Whena variable is assigned to it becomesattached to the scopewhere theassignmenthappens.

• A morelocalvariablecanhidealesslocalvariablewithinitsscope.

• Newscopesareintroducedinfunctiondefinitions.

• Ingeneralafunctioncanreadthevariablesdefinedinouterscopes, butcannotwritetothem.

6.8 Globals

Exercise: Defineamodulevariableusing VERBOSE = True. Thenresetitto Falseinafunctioncalled disableVerbose().

Solution:

VERBOSE = True

def disableVerbose():global VERBOSEVERBOSE = False

disableVerbose()print 'Verbose: ' + str(VERBOSE)

Notes:

• The global keywordputsvariablesinthemoduleorglobalscopewithinthescopeinwhichitisused.

• Withoutthe global keywordthe VERBOSE inside disableVerbose willberedefinedasalocalvariable, andsettingitto False willhavenoeffectontheVERBOSE intheglobalscope.

• If global isusedinthemoduleorglobalscopeithasnoeffect.

PythonEssentials 32

6.9 CreatingLibrariesofFunctions

Exercise: Createa util.py fileandthenimportitintoyourprogram.

Solution:

In util.py, type:

def greet():print "Hello, world"

InyourPythonfiletype:

import utilutil.greet()

Notes:

Ifthefileisnotinthesamedirectoryaddtheselinesfirst.

import sysdir = '/path/to/dir-containing-file'sys.path.append(dir)

7 Documentation

7.1 FunctionDocstrings

Youcandocumentafunctionsimplybyinsertingadocumentationstringor docstringimmediatelyafterstartingthefunction.

Exercise: Createahelloworldfunctionthatdocumentsitself.

Solution1:

def hello():'Prints hello world'print "Hello world!"

PythonEssentials 33

Solution2:

def hello():'''Prints hello world'''print "Hello world!"

Notes:

• Thetwosolutionshaveidenticaleffect.

• UseSolution2ifyouhavedocumentationthatislongerthanoneline.

Exercise: Use help(hello) toviewthedocumentation.

Exercise: Lookat hello.__doc__ toviewtherawdocstring.

Notes:

• Docstringsareattachedtotheobjectstheyaredefinedwith.

7.2 ModuleDocstrings

Exercise: Add a docstring to util.py. Observe that it is visible in help and as__doc__.

8 Objects

8.1 InteractingwithObjects

Howdoyouinteractwithobjects?

• InPythoneverythingisanobject.

• Hereishowyouinteractwithobjects.

obj.method(arg1, arg2, arg3)

PythonEssentials 34

• Everyobjecthasmethodsonit, whicharelikefunctions, excepttheyarespe-cializedtothatobject.

• Forexample, acarandacomputerbothhavea start function, buttheymeanspecificthingsforthatobject.

Whyisthepointofallthis?

• Objectsallowthecodetobemodularandmoreorganized.

• Eachpartofthesystemonlyknowsaboutitself.

• Objectsareintroverts.

• Sochangestoanobject’sinsidesarelesslikelytobreakthesystem.

Exercise: Look at the string functionson thePython console and thenfindawaytouppercaseastring. Youcanfindthefunctionsforanobjectusingthe dir(obj)function.

Exercise: Findoutifastringendswithaquestionmark.

Exercise: Findthe format functiononthestring.

9 GeneratingReports

9.1 FileReadingandWriting

Howcanyouread, write, andappendtoafile?

# Read.f = open('tmp_file.txt', 'r')contents = f.read()f.close()

# Write.f = open('tmp_file.txt', 'w')f.write(contents)

PythonEssentials 35

f.close()

# Append.f = open('tmp_file.txt', 'a')f.write(contents)f.close()

Exercise: Writeaprogramthatlogsthecurrentdateandtimeinafilecalled time.log.

Hint: Usethiscodetogetthecurrentdateandtimeasastring.

import datetimetime = str(datetime.datetime.now())

Solution1:

# Get current time.import datetimetime = str(datetime.datetime.now())

# Compose log message.event = 'Program executed'message = time + "\t" + event + "\n"

# Write to log file.log_file = open('tmp.log', 'a')log_file.write(message)log_file.close()

Solution2:

# Get current time.import datetimetime = str(datetime.datetime.now())

# Compose log message.event = 'Program executed'

PythonEssentials 36

message = time + "\t" + event + "\n"

# Write to log file.with open('tmp.log', 'a') as log_file:

log_file.write(message)

Notes:

• with isamoreconcisewayofdoingfileoperations.

• with automaticallyclosesthefileaftertheblockofcodeisexecute.

• Thiswayyoudon’thavetoremembertoclosethefile.

• Alsoincaseofanerroryoudon’thavetoworryaboutthefilenotgettingclosed.

• Closingafileisimportantbecause:

– Onwritessometimesthesystembufferstheoutputandwritesittothefilewhenthefileisclosed.

– Sometimesthereisalimitonhowmanyfilescanbeopenatonetime.

Howcanyouread, write, andappendtoafileusing with?

# Read.with open('tmp_file.txt', 'r') as f:

contents = f.read()

# Write.with open('tmp_file.txt', 'w') as f:

f.write(contents)

# Append.with open('tmp_file.txt', 'a') as f:

f.write(contents)

PythonEssentials 37

9.2 HTML Report

Exercise: GenerateHTML expensereportfora3-daybusinesstriptoPittsburg, PA.Theairfarewas$512, thehotelwas$63pernight, therentalcarwas$30daily.

Solution:

# Define HTML template.REPORT_TEMPLATE = '''\<h1>Expense Report</h1><table border="1"><tr>

<th>Description</th><th>Amount</th>

</tr><tr>

<td>Airfare</td><td>${0:.2f}</td>

</tr><tr>

<td>Hotel</td><td>${1:.2f}</td>

</tr><tr>

<td>Rental Car</td><td>${2:.2f}</td>

</tr></table>'''

# Calculate amounts.hotel_rate = 63car_rate = 30.0days = 3airfare = 512.0hotel = hotel_rate * dayscar = car_rate * days

PythonEssentials 38

# Generate report HTML.report_html = REPORT_TEMPLATE.format(

airfare, hotel, car)

# Save report to file.report_file = 'report.html'with open(report_file, 'w') as f:

f.write(report_html)

Notes:

• Thetemplateislaidoutsoitlooksassimilartotheactualfileaspossible.

• Thetemplateisgeneric.

• Thesametemplatecanbeusedwithdifferentamountvalues.

10 Booleans

10.1 BooleanType

Whatisabooleantype?

• Variablesofbooleantypeholdoneoftwovalues: True or False.

10.2 BooleansFromComparisons

Exercise: GuessthebooleanvalueoftheseexpressionsandthenverifyonthePythonconsole.

# Numeric equality.3 == 33 == 03.0 == 33.00 == 3

PythonEssentials 39

3.0 == 3.0

# String equality."hello" == "hello""hello" == "world""hello" == "HELLO"

# Boolean equality.True == TrueFalse == FalseTrue == False

# Inequality.3 != 33 != 4"hello" != "world"

# Numeric ordering.1 < 21 <= 23 >= 43 > 43 >= 3

# String ordering."hello" > "world""hello" < "world""hello" >= "hello""hello" > "0"

# Boolean ordering.True < FalseFalse < True

10.3 ComparisonOperators

Whatarethedifferentcomparisonoperators?

PythonEssentials 40

ComparisonOperators IsTrueWhen

a < b a islessthan b

a <= b a islessthanorequalto b

a > b a isgreatthan b

a >= b a isgreatthanorequalto b

a == b a isequalto b

a != b a isnotequalto b

a <> b a isnotequalto b (removedinPython3.0)

a is b a and b pointtothesameobjectinmemory

a is not b a and b pointtodifferentobjectsinmemory

Notes:

• Rememberthat = isassignmentandnotequality. Equalityis ==.

• Everyonetripsonthisseveraltimes.

• Forstrings, thecomparisonsaredoneindictionaryorder.

• Forindividualcharacters, "0" < "A" and "A" < "a"

10.4 ConvertingBooleans

Exercise: Guesswhetherthesecallsto bool(x) produce True or False andthenverifyonthePythonconsole.

# String to boolean.bool("True")bool("true")bool("TRUE")bool("False")

PythonEssentials 41

bool("")bool("1")bool("0")

# Numbers to boolean.bool(0)bool(1)bool(2)bool(-1)bool(-2)bool(0.0)bool(1.0)bool(2.0)bool(300.0)bool(-1.0)

# None to boolean.bool(None)

Notes:

• Allstringsaretrue, excepttheemptystringwhichisfalse.

• Thestring "False" istrue. Wrapyourheadaroundthatone.

• Allnumbersconverttotrueexcept 0 and 0.0.

• None isthezeroforobjects. Itisalsofalse.

Exercise: Guesswhatthe str() functiondoesto True and False andthenverifyonthePythonconsole.

Solution:

str(True)str(False)

Notes:

• True and False arecase-sensitive.

• Trytyping true and false intothePythonconsoleandseewhathappens.

PythonEssentials 42

10.5 BooleanOperators

Whatarebooleanoperators?

• Pythonprovidesthesebooleanoperators: and, or, not.

• Thesecanbeusedtocombinebooleanvaluestocreatecomplexlogicalexpres-sions.

Exercise: Basedonwhether is_raining and has_umbrella calculatethevalueofshould_eat_out.

Solution:

is_raining = Truehas_umbrella = Falseshould_eat_out = (not is_raining) or has_umbrella

Whatarethedifferentbooleanoperators?

Operator IsTrueWhen

a and b a istrueand b istrue

a or b a istrueor b istrue

not a a isfalse

Notes:

• The and and or operatorsshort-circuit. Chainsof and and or areonlyevalu-atedasfarasneededtodeterminetheirvalue. Forexampleifan and expressioncontainsa False evaluationstopsandtheresultis False. Ifan or expressioncontainsa True evaluationstopsandtheresultis True.

PythonEssentials 43

11 ControlFlow: Branching

11.1 IfandElse

Exercise: Writeacoinflippingappthatprints”heads”whenitgetsahead, and”tails”whenitgetsatail. Assume0istailsand1isheads.

Solution:

import random

flip = random.randint(0,1)if flip == 0:

result = "tails"else:

result = "heads"

print "Coin flip:", result

Notes:

• The if/else constructrepresentsabranchin theroad. Theprogramhas tochoosebetweentwodoors: eitherthe if orthe else.

• Iftheconditioninthe if istrueitexecutesthecodeinsidethe if block. Oth-erwiseitexecutesthecodeinthe else block.

11.2 If, Elif, andElse

Exercise: Writeaprogramthatcalculatesalettergradebasedonastudent’sGPA.

Grade GPA Minimum

A 3.5

B 2.5

C 1.5

D 1.0

F 0.0

PythonEssentials 44

Solution:

gpa = float(raw_input("Enter GPA: "))

if gpa >= 3.5:grade = "A"

elif gpa >= 2.5:grade = "B"

elif gpa >= 1.5:grade = "C"

elif gpa >= 1.0:grade = "D"

else:grade = "F"

print "Grade: " + grade

Notes:

• The if/elif/else constructrepresentsaforkintheroad.

• Theprogramevaluateseachconditionandtakesthefirstbranchwhosecondi-tionistrue.

• Ifnoconditionistruethenittakesthe else clause.

• The elif andthe else areoptional.

• Soifnoconditionistrue, andthereisno else clausethenitignoresthewholeconstruct.

11.3 If-ElseandBooleanOperators

Exercise: Writeacoinflippingappthatflipsthecointhreetimesanddeclaresawinnerbasedonwhichsidedidbetter.

Solution:

PythonEssentials 45

import random

def flip_to_str(flip):if flip == 0:

return "Tails"else:

return "Heads"

# Flip coins 3 times.flip1 = random.randint(0,1)flip2 = random.randint(0,1)flip3 = random.randint(0,1)

# Print result of all coin flips.print \

"Result: " + \flip_to_str(flip1) + ", " + \flip_to_str(flip2) + ", " + \flip_to_str(flip3)

# Figure out which side won.flips = flip1 + flip2 + flip3if flips == 2 or flips == 3:

winner = "Heads"else:

winner = "Tails"

# Print which side won.print "Winner: " + winnerprint "Good job, " + winner + "!"

Notes:

• Insteadofcheckingfor flips == 2 or flips == 3 youcancheckfor flips>= 2.

PythonEssentials 46

11.4 If-ElseOperator

Pythonhasanif-elseoperatorforshortconditionalstatements.

Exercise: Createafunctionthatproduces "Heads" ifitispassed 1 and "Tails" ifitispassed 0.

Solution:

def coin_to_string(coin):return "Heads" if coin == 1 else "Tails"

11.5 ValidatingNumericInputs

Exercise: Writefunctionstovalidatestringasvalidnumbers.

Solution1 :

def is_int(x):try:

value = int(x)is_valid = True

except:is_valid = False

return is_valid

amount = raw_input("Amount: ")if not is_int(amount):

print "Amount has to be a number."

Solution2 :

def is_int(x):try:

int(x)except:

return Falsereturn True

PythonEssentials 47

amount = raw_input("Amount: ")if not is_int(age):

print "Amount has to be a number."

Notes:

• Thetwosolutionsareequivalent.

• Solution2ismoreconcise.

• Youcanalsocatchtheexceptioninyourcodedirectly.

• Thecodeinthe except blockexecutesonlyifanexceptionisthrown. Other-wiseitisnotrun.

• Assoonasanexceptionoccursthecodeinthe try blockbailsout. Itdoesnotcontinueprocessingstatementsinthatblock.

11.6 CatchingControl-C

Exercise: Writefunctionstovalidatestringasvalidnumbers. EndtheprogramiftheuserpressesControl-C.

Solution: Oneoftheproblemswiththeprevioussolutionsisthattheyeatallexceptionsincludingkeyboardinterrupts. Soitisdifficulttoterminatetheprograms.

def is_int(x):try:

int(x)except Exception as e:

# print type(e)return False

return True

try:amount = raw_input("Amount: ")

except KeyboardInterrupt:

PythonEssentials 48

exit(1)if not is_int(amount):

print "Amount has to be a number."

Notes:

• Thefixistocatch KeyboardInterrupt.

• Alsothegenericexceptioniscaughtastype Exception andtheexceptioncanbecapturedinthevariable e. Thisletsyouexamineitstypeincaseadifferentexceptiongetsthrown.

• Discusstheprosandconstocatchingallexceptionsversuscatchingspecificexceptions.

• Youcandeclaretheexceptionvariablein except ifyouneedtousetheinfor-mationfromtheexception.

11.7 ThrowingExceptions

Exercise: Terminateaprogrambythrowinganexception.

Solution:

def die_now():raise Exception("Game over")

Exercise: Writetoafileguaranteeingthatitgetsclosedevenifanexceptionisraisedin write.

Solution1:

f = open("file.txt", "w")try:

f.write("Hello world\n")finally:

f.close()

PythonEssentials 49

Solution2:

with open("file.txt", "w") as f:f.write("Hello world\n")

Notes:

• Thecodeinthe finally blockexecutesbothwhenthereisanexceptionandwhenthereisnotanexception. Itisusefulforclosinghandles, andfortakingcareofrequiredcleanup.

• Solution2uses with toautomaticallyclosethefile.

12 ControlFlow: Looping

12.1 LoopingwithWhile

Exercise: Pickanumberfrom1to10andthenprompttheusertoguessit.

Solution:

import randomnumber = random.randint(1,10)guess = -1print "I picked a number."while number != guess:

guess = raw_input("Guess what number I picked: ")guess = int(guess)if number == guess:

print "Yes, you guessed right."else:

print "You were wrong. Try again."

Notes:

• Theprogramkeepsrunningthecodein the while blockuntil theconditionbecomesfalse.

PythonEssentials 50

Exercise: Theprogramshouldgivetheuserahintandmentionifthenumberishigherorlower.

Solution:

import randomnumber = random.randint(1,10)guess = -1print "I picked a number."while number != guess:

guess = raw_input("Guess what number I picked: ")guess = int(guess)if number == guess:

print "Yes, you guessed right."elif guess < number:

print "Too low. Try again."else:

print "Too high. Try again."

Exercise: Rewritetheprogramasaninfiniteloop. Terminatetheprogramwhentheuserguessescorrectlyorwhentheusertypes q.

Solution:

import randomnumber = random.randint(1,10)print "I picked a number."while True:

guess = raw_input("Guess what number I picked: ")if guess == "q":

breakguess = int(guess)if number == guess:

print "Yes, you guessed right."break

elif guess < number:print "Too low. Try again."

else:print "Too high. Try again."

PythonEssentials 51

Notes:

• Use break tobreakoutofaloopearly.

• Use continue tostartfromthebeginningoftheblockagain.

• With the while statementwearenowcapableof lockingupamachinebycreatingaloopthatneverterminates.

12.2 LoopingwithFor

Exercise: Printthesquaresofthenumbersfrom0through19.

Solution1:

i = 0while i < 20:

square = i ** 2print "square of {0} = {1}".format(

i, square)i += 1

Solution2: Loopsforgoingthroughsequencesofnumbersaresocommonlyusedthatthereisaspecialconstructforit: the for loop.

for i in range(20):square = i ** 2print "square of {0} = {1}".format(

i, square)

Solution3: xrange isjustlike range exceptitproducesalazysequenceinsteadofalist.

for i in xrange(20):square = i ** 2print "square of {0} = {1}".format(

i, square)

PythonEssentials 52

Notes:

• xrange useslessmemorythan range.

Exercise: Rewrite thetipcalculatorwithinputvalidation. Incasetheusersubmitsinvalidinputuseawhilelooptokeeppromptingtheuserforvalidinput. Theloopcanendwhentheusersubmitsvalidinput.

Exercise: WriteanN-diceappwhichrollsN dice, X times, andthenreportsthetotalandaveragedicevalue.

Exercise: Writeabot. Whenitseesaquestionitsays”I don’tknow.”Andwhenitseesastatementitshouldsay, ”I totallyagree.”

13 Collections

13.1 DefiningLists

Exercise: PrintthelistofcountiesintheSanFranciscoBayArea.

Solution:

# Build a list.counties = [

"Alameda County","Contra Costa County","Marin County","Napa County","San Francisco County","San Mateo County","Santa Clara County","Solano County","Sonoma County",

]

# Loop through the list.for county in counties:

print county

PythonEssentials 53

13.2 AccessingListElements

Exercise: Yahoo! provideshistoricalstockdataasCSV.Hereisasampleline.

Date,Open,High,Low,Close,Volume,Adj Close2013-03-28,23.63,23.77,23.45,23.53,17611900,23.53

Writeaprogramthatprintsthedateandtheclosingprice.

Solution:

# Date,Open,High,Low,Close,Volume,Adj Closeline = "2013-03-28,23.63,23.77,23.45,23.53,17611900,23.53"fields = line.split(',')date = fields[0]close = fields[4]print "Date: " + dateprint "Close: " + close

Notes:

• TheYahoodataisavailablefromthisURL http://finance.yahoo.com/q/hp?s=YHOO.

• The .split(',') functionwhichappliestostringobjects, splitsthestringtocreatealist.

• Hereishowyousplitastringoncommas.

tokens = string.split(',')

• Listindexingiszero-based.

• Thei-thelementofthelistisaccessibleas fields[i]

13.3 ListSurgery

Exercise: Whatarethedifferentwaysofaccessingelementsinthelist?

Solution:

Inthefollowing i isapositiveintegeror 0.

PythonEssentials 54

Expression Meaning

list[0] Firstelementof list

list[1] Secondelementof list

list[i] i+1thelementof list

list[-1] Lastelementof list

list[-i] ith-lastelementof list

list[1:3] Sublistof2ndand3rdelementsof list

list[0:2] Sublistoffirst2elementsof list

list[:2] Sublistoffirst2elementsof list

list[2:] Sublistoffirst3rdandfollowingelementsof list

13.4 CreatingLists

Whatarethedifferentwaysofcreatingalist?

• Youcandefinealistdirectly.

words = ["a","b","c"]

• Youcangetalistfromafunctionlike .split().

words = "a b c".split(' ')

• Youcanappendtoanexistinglist.

words = []words.append("a")words.append("b")words.append("c")

PythonEssentials 55

13.5 ListOperations

Whataresomeusefuloperationsavailableonalist?

• Hereishowyougetasortedlistfromanunsortedlist.

sorted_words = sorted(words)

• Hereishowyousortalist. Thischangesthelistitself.

words.sort()

• Hereishowyoureversealist.

words.reverse()

• Hereishowyoufindthelengthofalist.

print len(words)

Exercise: Generateasequenceof20randomnumbersfrom1through10, storetheminalist, calculatetheiraverage, minimum, andmaximum, andprintthemout.

Solution:

import randomnumbers = []for i in range(20):

numbers.append(random.randint(1,10))average = float(sum(numbers)) / len(numbers)print "Avg: " + str(average)print "Min: " + str(min(numbers))print "Max: " + str(max(numbers))

PythonEssentials 56

13.6 CommandLineArguments

Exercise: Writea tipcalculator that takes theamount, people, and tip rateon thecommandline.

Hint:

import sysscript_name = sys.argv[0]amount = float(sys.argv[1])people = int(sys.argv[2])tip_rate = float(sys.argv[3])

Notes:

• sys.argv[0] containsthenameofthescript.

• sys.argv[1] containsthefirstargument, sys.argv[2] containsthesecondargument, andsoon.

13.7 SequencesandFunctions

Expression Meaning

len(seq) Numberofelementsin seq

sorted(seq) Sortedcopyof seq

sorted(seq,f) Sortedcopyof seq sortedaccordingto f

reversed(seq) Reversedcopyof seq

map(f,seq) Copyof seq with f appliedtoeachelement

filter(f,seq) Subsequenceof seq containingelementsforwhich f istrue

seq[i] ithelementof seq

seq[i:j] Subsequenceof seq from i until j-1

seq[i:] Subsequenceof seq from i untiltheend

PythonEssentials 57

seq[:j] Subsequenceof seq fromthebeginningto i-1

seq[:] Copyof seq

list1 + list2 Copyofconcatenationof list1 and list2

list1 * n n copiesof list1 concatenatedtogether

str1 + str2 Copyofconcatenationof str1 and str2

str1 * n n copiesof str1 concatenatedtogether

Exercise: Createaprogramthatasksauserforanumberandthenprintsabirthdaycakewiththatmanycandles.

13.8 Lambda

Exercise: Createaonelinefunctionthatreturnstrueifanumberiseven.

Solution:

is_positive = lambda x: x % 2 == 0

Notes:

• Lambdaprovidesasimplewayofcreatinganonymousone-linefunctions.

• Theyareparticularlyusefulwith functions that takeother functions as argu-ments, suchas map, filter, andsort.

Exercise: Listalltheevennumbersbetween0and100.

13.9 Map

Exercise: Writeafunctionthatsquaresalistofnumbers.

Solution:

def square(list):map(lambda x: x*x, list)

PythonEssentials 58

13.10 Filter

Exercise: WriteaprogramthatasksforahostnameandprintsoutitsIP addressusingnslookup.

Solution1:

import os

hostName = raw_input("Host: ")output = os.popen('nslookup ' + hostName).read()output_lines = output.split('\n')

addresses = []for line in output_lines:

if line.startswith('Address:'):addresses.append(line)

key,value = addresses[-1].split(' ')print value

Solution2:

import os

hostName = raw_input("Host: ")output = os.popen('nslookup ' + hostName).read()output_lines = output.split('\n')

addresses = filter(lambda line: line.startswith('Address:'),output_lines)

key,value = addresses[-1].split(' ')print value

Notes:

PythonEssentials 59

• Solution1usesthetraditionalloopandifconstruct.

• Solution2uses filter.

13.11 Dictionaries

Exercise: ConvertthefinancialdatainCSV formatintoadictionarymappingcolumnnamestovalues. Thenprintoutthedateandtheclose.

Date,Open,High,Low,Close,Volume,Adj Close2013-03-28,23.63,23.77,23.45,23.53,17611900,23.53

Solution:

# Date,Open,High,Low,Close,Volume,Adj Closeline = "2013-03-28,23.63,23.77,23.45,23.53,17611900,23.53"row = line.split(',')record = {

"date" : row[0],"open" : row[1],"high" : row[2],"low" : row[3],"close" : row[4],"volume" : row[5],"adj" : row[6],

}

print "Date: " + record['date']print "Close: " + record['close']

Notes:

• Hereishowyoudefineadictionary.

d = {"key1" : "value1","key2" : "value2","key3" : "value3",

}

PythonEssentials 60

• Hereishowyoureadavaluefromadictionary.

print d["key1"]

• Hereishowyouaddanewvaluetoadictionary.

d["key4"] = "value4"

• Hereishowyouupdateavalueinadictionary.

d["key1"] = "new-value1"

• Hereishowyoufindthelengthofadictionary.

print len(d)

• Hereishowyoucheckifakeyexists.

"key1" in d

• Hereishowyoudeleteakeyfromadictionary.

del d["key1"]

• Hereishowyouloopthroughadictionary.

for key in d.keys():print d[key]

13.12 Tuples

Herearesomeexamplesofhowtocreatetuples. Tuplesarejustlikelists, excepttheyareimmutable.

# Defining tuplest1 = (1,2,3)t2 = 1,2,3print t1print t2

PythonEssentials 61

# Emptyt3 = ()print t3

# Singletont4 = (1,)print t4

# Tuple unpackinga,b,c = t1print a,b,c

Exercise: Writeafunctionthatfindsthedistancebetweentwopointsrepresentedattuples. Testthefunction.

13.13 TuplesvsListPerf

• Hereishowtomeasuretheperformancefortupleversuslistcreation.

/Users/asimjalis> python -m timeit "x=[1,2,3,4,5,6,7,8]"1000000 loops, best of 3: 0.244 usec per loop/Users/asimjalis> python -m timeit "x=(1,2,3,4,5,6,7,8)"10000000 loops, best of 3: 0.0328 usec per loop

• Hereishowtomeasuretheperformancefortupleversuslistaccess.

/Users/asimjalis> python -m timeit -s "x=(1,2,3,4,5,6,7,8)" "y=x[3]"10000000 loops, best of 3: 0.07 usec per loop/Users/asimjalis> python -m timeit -s "x=[1,2,3,4,5,6,7,8]" "y=x[3]"10000000 loops, best of 3: 0.054 usec per loop

• -s takesasetupstatementwhichisexecutedonce.

• Hereishowtomeasuretheperformanceofafunctioncalled func inthe Fooclassinyourcode.

python -m timeit -s "import mylib1, mylib2; a = Foo()" "a.func()"

Exercise: Measuretheperformanceofusing .format versus % forformatting.

PythonEssentials 62

13.14 Sets

Exercise: Createsetofengineers, programmers, managers. Definethesetofengineer-ingmanagementandfull-timemanagement. Printallthesetsout.

Solution:

# Define setsengineers = { 'John', 'Jane', 'Jack', 'Janice' }programmers = { 'Jack', 'Sam', 'Susan', 'Janice' }managers = { 'Jane', 'Jack', 'Susan', 'Zack' }

# Unionemployees = engineers | programmers | managers

# Intersectionengineering_management = engineers & managers

# Differencefulltime_management = managers - engineers - programmers

# Add elementengineers.add('Marvin')print engineers

# Superset testemployees.issuperset(engineers)

# Updateemployees.update(engineers)employees.issuperset(engineers)

# Remove elementfor group in [engineers, programmers, managers, employees]:

group.discard('Susan')print group

# Alternative syntax for defining sets.engineers2 = set(['John', 'Jane', 'Jack', 'Janice'])

PythonEssentials 63

13.15 SetSyntax

Expression Meaning

{ "e1", "e2", "e3" } Createaset

set([ "e1", "e2", "e3" ]) Createaset

a & b Setofelementsinboth a and b (intersection)

a | b Setofelementsineither a or b (union)

a - b Setofelementsin a butnotin b

k in a Element k isinset a

Exercise: Createaprogramthatchecksifanaddressislocalhost.

Hint: Createasetofvariationsof localhost (localhost, 127.0.0.1), andcheckuserinputagainstit.

14 OperatingSystem

14.1 SystemEnvironment

Exercise: Printthe HOME directory.

Hint: ThesystemenvironmentisavailableinPythonasadictionary.

Solution:

import osprint os.environ['HOME']

Exercise: Printthecontentsof os.environ.

Exercise: Getthe PATH variable, thensplititintoalist, andprintitoutonepathperline.

PythonEssentials 64

14.2 RunningShellCommands

Exercise: Runthepingcommandandprintitsoutput.

Solution1: OnMacandUnixrunpingwiththe -c 1 option. Thiswillpingtheserveronce.

import subprocessoutput = subprocess.check_output('ping -c 1 google.com', shell=True)print output

Solution2: OnWindowsthe -c optionisnotsupported. Sorunthiscommandwiththe -c 1 option.

import subprocessoutput = subprocess.check_output('ping google.com', shell=True)print output

Notes:

• check_output returnstheoutputoftheshellcommand.

• Iftheshellcommandfails, check_output willthrowanexception.

14.3 OtherProcessConstructs

Exercise: Runaprocesswithoutcapturingoutput.

Solution:

os.system("echo 'hello world'")

os.system("echo 'hello world' > /dev/null")

Exercise: Runaprocessandcapturetheoutputanderror.

Solution1:

PythonEssentials 65

# Stdout can be captured.print os.popen("echo 'hello world'").read()

# Stderr not captured.print os.popen("bad 'hello world'").read()

# Capture stderr by combining with stdout.print os.popen("bad 'hello world' 2>&1").read()

Solution2:

import subprocess

proc = subprocess.Popen("bad 'hello world'",stderr=subprocess.PIPE,stdout=subprocess.PIPE,shell=True)

print proc.stdout.read()print proc.stderr.read()

15 Third-PartyModules

15.1 PipvsEasy_Install

Whatare pip and easy_install?

• Therearetwowaysofinstallingpackages: pipandeasy_install.

• Pipisthe”newhotness”andisrecommended.

• Ithasbetterdocumentationandrecoversinmorecases.

• Theseprogramsgrab themodulesyouwantandall theircorrectly-versioneddependencies.

• Soyoucanbelikeakidinacandystore. Whateveryoupointatisyours.

PythonEssentials 66

Whatis PyPI?

• PyPI isthePythonPackageIndex.

• Itisacollectionoflibrariesthatyoucandownloadusingpip.

• TobrowsePyPI goto https://pypi.python.org andclickon BrowsePack-ages.

15.2 WindowsPipInstall

Exercise: InstallpiponWindows.

Solution:

• Installsetuptoolsfrom http://www.lfd.uci.edu/~gohlke/pythonlibs/#setuptools

• Installpipfrom http://www.lfd.uci.edu/~gohlke/pythonlibs/#pip

• IneachcasemakesureyoupicktheversionthatmatchesyourprocessorandtheversionofPythonthatyouareusingonyourmachine.

• Pipshouldgetinstalledinalocationsimilarto C:\Python27\Scripts\pip.exe.Youcaneitheraddthe Scripts foldertoyourPATH orrunitdirectlyfromthislocation.

15.3 Unix/MacPipInstall

Exercise: InstallpiponUnixorMac.

Solution:

sudo easy_install pip

PythonEssentials 67

15.4 Demjson

Exercise: Install demjson.

Solution:

• CheckthatitisnotinstalledbytypingthisinthePythonconsole.

import demjson

• Lookatpip’sdocumentation.

pip help

• Findfullnameof demjson.

pip search demjson

• Install demjson.

sudo pip install demjson

• OnWindowsyouwon’tneed sudo butyoumightneedtoruninacmdconsolewindowthatisrunningasadministrator.

• Checkthatthemoduleisinstalled. Thisshouldnownotthrowerrors.

import demjson

• Youcanuse demjson toconvertPythondatastructurestoJSON strings.

import demjsonrecord = {

"date" : "2012-10-10","open" : 10,"close" : 11,

}json = demjson.encode(record)print json

PythonEssentials 68

• YoucanalsouseittoconvertJSON stringstoPythondatastructures.

import demjsonjson = '[1, 2, 3, 4]'data = demjson.decode(json)print data

15.5 IPython

Exercise: InstallIPython.

Solution:

sudo pip install ipython

# Optional for autocompletesudo pip install readline

Notes:

• Ifpipinstallfailstryinstallingipythonandreadlinewitheasy_install.

• IPythongivesyouautocomplete.

• Itprettyprintsthedatastructures.

• Also itprovidesaneatworkflowwhere itautomatically reloadsall themod-ules for you each time you call a function. See http://ipython.org/ipython-doc/dev/config/extensions/autoreload.html fordetails.

Exercise: EnableautoreloadinIPython.

Solution:

%load_ext autoreload%autoreload 2

Notes:

• Testthisoutbyimportingamodule, invokingafunctiononit, andthenchangingthemodule.

PythonEssentials 69

15.6 WebServices

Exercise: FindouthowmanyUS dollarsareinaBritishpound, usingthisJSON webservice http://www.google.com/ig/calculator?hl=en&q=1GBP=?USD.

Solution:

import urllib2import demjson

url = 'http://www.google.com' \+ '/ig/calculator' \+ '?hl=en&q=1GBP=?USD'

json = urllib2.urlopen(url).read()data = demjson.decode(json)dollars = data['rhs'].split(' ')[0]dollars = float(dollars)print "1 GBP = {0} USD".format(

dollars)

Notes:

• urllib2.urlopen(url).read() gets thecontentsat url andisasimplewaytoaccesswebservices.

• Pythonshipswithajsonencoderanddecoderaswell. Hereisanexampleofhowtouseit.

import json

# Convert JSON to Python data.data = json.loads('{"a" : 1, "b" : 2}')print type(data)print data

# Convert Python data to JSON.json = json.dumps({"a" : 1, "b" : 2})print type(json)print json

PythonEssentials 70

• However, demjsonismoreresilientinthefaceofbadorlazyinput.

• Forexample, Python’sjsonmodulefailsiftheJSON usesunquotedkeys, whiledemjsonisabletoparsethiscorrectly.

16 Objects

16.1 DefiningObjects

Exercise: Createanobjectthatsplitsrestaurantbills.

Solution:

class CheckSplitter:

def __init__(self, tip_rate):self.tip_rate = tip_rate

def split(self, amount, people):amount = \

amount + amount * self.tip_ratesplit_amount = amount / float(people)return split_amount

check_splitter = CheckSplitter(0.15)split_amount = check_splitter.split(43, 3)print "Split amount: ${0:.2f}".format(split_amount)

Notes:

• CheckSplitter isa class, and check_splitter isan object oran instance.

• Classesare likePlatonic ideals. Andobjectsor instancesare their concretemanifestations.

• Objectsarelikefunctions. Excepttheyhavememory. Theyrememberstate.

PythonEssentials 71

• Forexample, check_splitter holdsapermanentmemoryof tip_rate. Itkeepsthisinside self.tip_rate.

• Objectfunctionsareknownasmethods.

• Withinamethodtheparametersandlocalvariablesareaccessedjustlikeinnormalfunctions, withtheirnames.

• Withinamethodinstancevariables(forexmaple tip_rate)areaccessedusingself.tip_rate.

NamingConventions:

• Classnamesstartwithupper-caseletters. Objectnamesstartwithlower-caseletters.

• Classnamesarespelledwithmixedcaseinsteadofunderscores. Objectandmethodnamesarespelledusingunderscores.

16.2 WhyObjects

Whyuseobjects?

• Thinkofobjectsassmallblackboxesinsideaprogramthathavesomeinternalstate.

• Youcanuseanobjecttodousefulthingswithoutunderstandinghowitisdoingthem. Itislikeaprogramwithinaprogram.

• Objectswerefirstinventedinsimulationsoftware.

• Intuitivelytheyarelikeobjectsinreallife.

• Whenyouapproachalargeproblemyoucandivideitintoobjectsandmakeiteasiertosolve.

• Objectsformakindofknowledgebarrier. Theyknownothingabouthowtheyarebeingused. Andthecallerknowsnothingabouthowtheobjectisdoingwhatitisdoing.

PythonEssentials 72

• Theyhaveaminimalcontract--definedbythemethods.

Thisisgreat. Butwhyhaveobjects?

• Objectsmakeiteasierforhumanstothinkaboutprograms.

• Objectsareawayofbreakingupthecomplexlogicofaprogramintosmallbite-sizedpieces.

• Computershavenoneed forobjects. Theyareperfectlyhappy if thewholeprogramwasonebigfilewithnofunctionsorobjects.

Whyisthedifferencebetweenfunctionsandobjects? Whycan’tweusefunctionstobreakupaprogramintopieces?

• Functionshavenostate. Theyhavenomemoryofwhatjusthappened. Theyareusefulforstate-lesscomputations.

• Whenyouwantcomputationswithstateyouneedobjects.

• Bothfunctionsandobjectsmakeiteasiertotestapplications.

16.3 Inheritance

Python supports inheritance. However, it is generallynotusedverymuch. Sinceeverythingisanobjectitiseasyparameterizeonclassfunctionstoachievethesameeffect.

However, inheritance isuseful in frameworks to indicatea relationship. Also it isusefulforbuildinguphierarchiesofexceptions.

Exercise: CreateasubclassofExceptioncalled AppError forcustomerapplicationexceptions.

Solution:

class AppError(Exception):def __init__(self, message):

super(AppError, self).__init__(message)

PythonEssentials 73

Notes:

• Youcantestthisusing raise AppError('hi') andthenobservingtheex-ceptiongettingthrown.

16.4 ImplicitMethods

Pythonobjectshavesomespecialmethodsthatcanbeoverriddentochangethebe-havioroftheunderlyingclass.

Exercise: Createa Point objectwithtwofields, x and y. Changetheobjectsothatstr(p) displaysthecoordinatesofthepoint.

Hint: Overridethe __str__ methodoftheobject.

Exercise: Createageneralsolutionthatworksforanyobject.

Hint: Usethe __dict__ propertyinthe __str__ method.

16.5 StrandLen

Functionslike str and len relyonimplicitfunctionsattachedtoallobjects.

Exercise: Cleanuptheoutputof str of Point byreturning str(self.__dict__).

Exercise: Make len returnthedistanceofapointfromtheorigin.

16.6 StrvsRepr

__repr__ isthestringthatisusedintheconsolewhenyouinspectanobject.

• __repr__ goalistobeunambiguous

• __str__ goalistobereadable

• Container’s __str__ usescontainedobjects’ __repr__

• If __repr__ isprovideditisalsousedfor __str__

• If __str__ isprovideditisnotusedfor __repr__

PythonEssentials 74

Notes:

• A commonoutputfor __repr__ is

"%s(%r)" % (self.__class__, self.__dict__)

• Pythondoesnotdothisbydefaultbecauseitcouldleadtoinfiniterecursion.

16.7 OperatorOverloading

Exercise: Allowtwopointstobeaddedtocreateathirdpoint.

Hint: Overridethe __add__(self, new_point) method.

Operator Function

a + b a.__add__(b)

a * b a.__mul__(b)

a & b a.__and__(b)

a | b a.__or__(b)

Exercise: Define * tocomputethedot-productofthetwovectorscorrespondingtothepoints.

Exercise: Define * tocomputethedot-productif theotherelementisapoint, andscalethepointifitisan int ora float.Hint: Usethesemethodcallstofigureoutthetypeof p:

instanceof(p, Point)instanceof(p, int)instanceof(p, float)

16.8 NewStyleandOldStyleObjects

Youwillnoticethat type doesnotworkonobjectsoftype Point. Thereisawaytofixthis.

PythonEssentials 75

Exercise: Make type workwith Point.

Solution:

• Declare Point as class Point(object) insteadof class Point.

• Python2.*hastwokindsofobjects: old-styleobjectsandnew-styleobjects.

• Ifyouinheritfrom object yougetnew-styleobjectsthatworkwith type. Oth-erwiseyougetold-styleobjectsthatdon’tworkwithtype.

• Python3cleansupthisareaandgivesyouthesamekindofobjectregardlessofhowyoudeclareyourclass.

16.9 OperatorPrecedence

Operatorshavefixedprecedencedescribedbythefollowingtable.

Description Operator

Lambdaexpression lambda

Conditionalexpression if – else

BooleanOR or

BooleanAND and

BooleanNOT not x

Comparisons in, not in, is, is not, <, <=, >, >=, <>, !=, ==

BitwiseOR |

BitwiseXOR ˆ

BitwiseAND &

Shifts <<, >>

Additionandsubtraction +, -

Multiplication, division, remainder *, /, //, %

Positive, negative, bitwiseNOT +x, -x, ~x

PythonEssentials 76

Exponentiation[9] **

Arrayandobjectaccess x[index], x[index:index], x(arguments...), x.attribute

Tuple, list, dictionary (a,b,c), [a,b,c], {key: value}

17 RegexandWebScraping

17.1 Regex

Exercise: ThisistheresponsefromCraigslistonsearchingforPythonjobs. Extractthenumberinthestring.

'>found 524 postings<'

Solution:

import retext = '>found 524 postings<'found = re.findall(r'>found (\d+)', text)count = int(found[0])

Notes:

• Thefirstargumentto findall isaregularexpression.

• findall returnsalistcontainingjust 524.

• Withoutparenthesesintheregular findall wouldreturntheentirestring.

• Whatwill re.findall(r'\d+', input) return?

PythonEssentials 77

17.2 SpecialCharacters

Character Matches

. Anycharacterexceptanewline

ˆ Startofstring

$ Endofstringorjustbeforenewlineatendofstring

X* 0ormore(greedy)repetitionsofX

X+ 1ormore(greedy)repetitionsofX

X? 0or1(greedy)ofX

X*?,X+?,X?? Non-greedyversionsofpreviousspecialcharacters

X{m} m repetitionsofX

X{m,n} From m to n repetitionsofX

X{m,n}? Non-greedyversionofabove

X{m,} m ormorerepetitionsofX

X{m,}? Non-greedyversionofabove

X{,m} m orfewerrepetitionsofX

X{,m}? Non-greedyversionofabove

\\ Backslash

[abcd] a through d

[a-z] Lowercaseletters

[-a-z] Hyphenandlowercaseletters

[A-Z] Uppercaseletters

[a-zA-Z] Lowercaseanduppercaseletters

[ˆa-d] Not a through d

X|Y Either X or Y

(X) Match X andcaptureinregister 1, 2 etc.

(?iLmsux) SettheI,L,M,S,U,orX flagfortheRE

PythonEssentials 78

Exercise: Writearegexthatmatchesaninteger.

Exercise: Writearegexthatmatchesafloatingpointnumber.

Exercise: WritearegexthatmatchesanXML tag.

17.3 AdvancedSpecialCharacters

Character Matches

(?:X) Match X withoutcapturingit

(?P<foo>X) Match X andcaptureinregister foo

(?P=foo) Matchvalueinregister foo

(?#comment) A comment; ignored

(?=X) Matchesif X matchesnext, withoutconsumingstring

(?!X) Matchesif X doesn’tmatchnext

(?<=X) Matchesifprecededby X (mustbefixedlength)

(?<!X) Matchesifnotprecededby X (mustbefixedlength)

(?(foo)X|Y) Matches X if foo wascaptured, matches Y otherwise

(?(1)X|Y) Matches X if 1 wascaptured, matches Y otherwise

Exercise: Writearegexthatmatchesall q followedbya u. Thenwriteonethatfindsall q notfollowedbya u.

17.4 SpecialSequences

Sequence Matches

\1 Contentsofregister 1

\12 Contentsofregister 12

\g<1> Contentsofregister 1 (onlyinreplacement)

PythonEssentials 79

\g<foo> Contentsofregister foo (onlyinreplacement)

\A Startofstring

\Z Endofthestring

\b Iswordboundary

\B Isnotwordboundary

\d Decimaldigitcharacter; equivalentto [0-9]

\D Complementof \d

\s Whitespacecharacter; equivalentto [ \t\n\r\f\v]

\S Complementof \s

\w Alphanumericcharacter; equivalentto [a-zA-Z0-9_]

\W Complementof \w

\\ Literalbackslash

Exercise: Writearegexthatmatchesanemailaddress.

17.5 RegexFunctions

Function Meaning

match() Returnmatchobjectifpatternoccursatbeginningofstring

search() Returnmatchobjectifpatternoccursanywhereinstring

sub() Substitutepattern, returnnewstring

subn() Substitutepattern, returnnewstring, substitutioncount

split() Splitstringbyoccurrencesofpattern

findall() Returnlistofpatternoccurrences

finditer() Returniteratorofmatchobjects

compile() Compilepatterninto RegexObject

purge() Clearregexcache

escape() Backslashescapeallnon-alphanumericsinpattern

PythonEssentials 80

Function Signature

match() match(pattern, string, flags=0)

search() search(pattern, string, flags=0)

sub() sub(pattern, repl, string, count=0, flags=0)

subn() subn(pattern, repl, string, count=0, flags=0)

split() split(pattern, string, maxsplit=0, flags=0)

findall() findall(pattern, string, flags=0)

finditer() finditer(pattern, string, flags=0)

compile() compile(pattern, flags=0)

purge() purge()

escape() escape(pattern)

Notes:

• Youcanaccesscaptureregistersusing m.group(1) and m.group('foo')

• Here m isthematchobjectreturnedby match, search, and finditer.

17.6 RegexFlags

Short Long Meaning

I IGNORECASE Usecase-insensitivematching.

L LOCALE Uselocalefor \w, \W, \b, \B

S DOTALL . matchesallcharactersincludingnewline.

M MULTILINE ˆ matchesbeginningofeachlineandstring

$ matchesendofeachlineandstring

X VERBOSE Ignorewhitespaceandcomments

U UNICODE UseUnicodelocalefor \w, \W, \b, \B

PythonEssentials 81

17.7 RegexExercises

Exercise: Writeagrep-likeprogramthatprintseachlineinitsinputfilesthatcontainsaregexmatch.

Exercise: Writeased-likeprogramthattakesapattern, replacement, andalistoffiles,andprintsouttheresultafterthereplacements.

17.8 WebScraping

Exercise: WriteaprogramthattakesakeywordandreturnsthenumberofjobsonCraigslistcontainingthatkeyword.

Solution:

import reimport urllib2

keyword = 'python'

# Compose the URL.url = 'http://sfbay.craigslist.org' + \

'/search/jjj?query=' + keyword

# Get contents.contents = urllib2.urlopen(url).read()

# Extract count.found = re.findall(r'>found (\d+)', contents)count = int(found[0])

# Print result.print '{keyword} jobs: {count}'.format(

keyword=keyword, count=count)

Exercise: Writeaprogram that takesa listofkeywordson thecommand lineandgeneratesareport

Solution:

PythonEssentials 82

import reimport urllib2import sys

for keyword in sys.argv[1:]:url = 'http://sfbay.craigslist.org' + \

'/search/jjj?query=' + keywordcontents = urllib2.urlopen(url).read()found = re.findall(r'>found (\d+)',

contents)count = int(found[0])

print '{keyword} jobs: {count}'.format(keyword=keyword, count=count)

Exercise: WritethesameprogramanddisplaytheresultsinanHTML page.

Solution: ThisprogramwilloutputHTML.Tosaveittoafileredirectitwhenrunningitusing python program.py > output.html.

import reimport urllib2import sys

# Check args.program = sys.argv[0]if len(sys.argv) <= 1:

print 'Craiglist SF Job Tracker'print 'Usage: ' + program + ' ' + \

'KEYWORD1 [KEYWORD2] ... > OUT.HTML'sys.exit(1)

# Print heading.print '<h1>SF Jobs By Keywords</h1>'

# Print table header row.print '<table border="1">'print '<tr>'

PythonEssentials 83

print '<th>Keyword</th>'print '<th>Jobs</th>'print '</tr>'

# Find jobs for all languages.for keyword in sys.argv[1:]:

# Get keyword count from Craigslist.url = 'http://sfbay.craigslist.org' + \

'/search/jjj?query=' + keywordcontents = urllib2.urlopen(url).read()found = re.findall(r'>found (\d+)',

contents)count = found[0]

# Print row.print '<tr>'print '<td>' + keyword + '</td>'print '<td>' + count + '</td>'print '</tr>'

# End table.print '</table>'

18 CSV Files

18.1 ReadingCSV Files

Exercise: CreateaprogramthatreadsandprintshistoricalstockpricedataforAppleavailableasCSV fromYahoo!.

Solution1:

Tofindout thehistoricalprices forApple (AAPL) go to http://finance.yahoo.com/q/hp?s=AAPL. Onthebottomofthepageclickon DownloadtoSpreadsheet.Savethisas table.csv.

In demo.py:

PythonEssentials 84

# Read CSV rows as lists.import csvwith open('table.csv', 'rb') as csvfile:

reader = csv.reader(csvfile)for row in reader:

print row

Solution2:

SetupthefilethesameasSolution1. ThiswillreadtheCSV fileasasequenceofdictionaries.

# Read CSV rows as dictionaries.import csvwith open('table.csv', 'rb') as csvfile:

reader = csv.DictReader(csvfile)for row in reader:

print row

18.2 WritingCSV Files

Exercise: CreateaCSV filecontainingthefollowingsalesdata: wehadsalesof$300,$350, $250ondays1, 2, 3.

Solution1:

In demo.py:

import csv

# Define sales data.sales_data = [

{ "day" : 1, "sales" : 300 },{ "day" : 2, "sales" : 350 },{ "day" : 3, "sales" : 250 },

]

# Write CSV file.

PythonEssentials 85

import csvwith open('output.csv', 'wb') as f:

writer = csv.DictWriter(f,fieldnames=('day','sales',))

writer.writeheader()writer.writerows(sales_data)

Notes:

• Thecommand writer.writeheader() willputaCSV headeronthefile.

• Thecommand writer.writerows() willoutput the sales_data asCSVintothefile.

Solution2:

In demo.py:

import csv

# Define sales data.sales_data = [

( 1, 300, ),( 2, 350, ),( 3, 250, ),

]

# Write CSV file.import csvwith open('output.csv', 'wb') as f:

writer = csv.writer(f)writer.writerow(('day','sales'))writer.writerows(sales_data)

Notes:

• Whenyouusearegular writer (asopposedtoa DictWriter)thedatahastobeasequenceoflistsortuples.

• Alsoyouhavetowritetheheaderrowyourself.

PythonEssentials 86

19 Database

19.1 CreatingTables

Exercise: Createadatabasetableforholdingstocktransactions.

Solution:

import sqlite3

with sqlite3.connect('test.db') as connection:cursor = connection.cursor()

# Create table.cursor.executescript(

"DROP TABLE " +"IF EXISTS stocks;" +

"CREATE TABLE stocks ( " +"date text, " +"action text, " +"symbol text, " +"quantity real, " +"price real " + ")")

# Commit changes.connection.commit()

Notes:

• executescript canbeusedtorunmultipleSQL statementstogether.

• Oryoucanuse execute toruneachstatementseparately.

• commit ensuresthatthechangestothedatabasearecommittedandnotthrownaway.

• with ensuresthatthedatabaseconnectionisclosedevenincaseofexceptions.

PythonEssentials 87

• Ifanexceptionoccursin executescript, thecodewilldropoutofthe withand commit willnotgetcalled.

• Closingdatabaseconnectionsisimportant, becauseotherwisetheconnectioncankeepconsumingsystemresources.

• Youcanalsoexplicitlyclosetheconnection, usingthissyntax:

importsqlite3connection=sqlite3.connect(’test.db’)asconnection: cursor=connection.cursor()#RunSQL throughcursor. connection.commit()connec-tion.close()

• Thisisdangerousbecauseifanexceptionoccurstheconnectionwillnotclose.

• ThisisusefulthoughifyouareinteractingwiththedatabasethroughthePythonconsole.

19.2 InsertingRecords

Exercise: Insertasinglerowofdata.

Solution:

import sqlite3

with sqlite3.connect('test.db') as connection:cursor = connection.cursor()cursor.execute(

"INSERT INTO stocks VALUES (?,?,?,?,?)",('2006-01-05','BUY','RHAT',200,40.14))

cursor.commit()

Notes:

• UsingbindvariablesismoresecurethanembeddingvaluesintotheSQL be-causethispreventsSQL injectionattacks.

• Usingbindvariablesalsohasbetterperformance.

• Youcanalsouse UPDATE and DELETE usingsimilarsyntax.

PythonEssentials 88

19.3 BatchInserts

Exercise: Insertmultiplerecordsinonecall.

Solution:

purchases = [('2006-03-28', 'BUY', 'IBM', 1000, 45.00),('2006-04-05', 'BUY', 'MSFT', 1000, 72.00),('2006-04-06', 'SELL', 'IBM', 500, 53.00), ]

cursor.executemany('INSERT INTO stocks VALUES (?,?,?,?,?)',purchases)

Notes:

• Insteadofalistofrecordsyoucanalsoinsertanarbitrarysequenceofrecords.

19.4 SelectingRecords

Exercise: Selectallrowsofdata.

Solution:

import sqlite3

with sqlite3.connect('test.db') as connection:cursor = connection.cursor()cursor.execute(

"SELECT * FROM stocks WHERE action=?",('BUY',))

for row in cursor:print row

Notes:

• Allinteractionisthroughthe cursor object.

PythonEssentials 89

• The cursor objectletsyouiterateoveralltherowsjustselected.

• Besidesiteratingovertherowsyoucanalsofetchasingleroworfetchmultiplerowsfromthe cursor object.

# Fetch one row.print cursor.fetchone()

# Fetch all rows as a list.print cursor.fetchall()

# Fetch 3 rows.print cursor.fetchmany(3)

• However, iteratingovertherowsisthemostscalablewayofdoingselects.

• Thiswaythedatabasedoesnothavetoloadlargevolumesofdatainmemory.

19.5 CursorMethods

CursorMethod Meaning

execute(sql) ExecuteSQL withoutvars

execute(sql, (v1,v2,)) ExecuteSQL onceforvartuple

executemany(sql, ExecuteSQL repeatedlyforeachvartuple

[(v1,v2,),(v11,v22,)]

executescript(sqls) ExecuteSQL statementsseparatedby ;

fetchone() Fetchonerowastuple

fetchmany(size=2) Fetch size rowsastuplesequence

fetchall() Fetchallrowsastuplesequence

lastrowid LastrowID forinsert

rowcount Rowsaffectedbyinsert, update, delete

PythonEssentials 90

19.6 Querying

Exercise: Createacommandlinetoolwiththeseusages.

# List all tables in database.dbtool database

# List all columns in tabledbtool database table

# List table row countdbtool database table count

Hint:

• Hereishowtogetalltablenames.

cursor.execute("SELECT name FROM sqlite_master " +"WHERE type='table'")

• Hereishowtogetalltablecolumns.

cursor.execute('PRAGMA TABLE_INFO(stocks)')

• Hereishowtogetthenumberofrowsinatable.

cursor.execute('SELECT COUNT(*) FROM stocks')

19.7 MySQL andPostgreSQL

Note: TheSQL API weusedforSqlitewillworkwithotherdatabases, withoneimpor-tantdifference:

• InMySQL andPostgreSQL insteadof ? youuse %s asthebindvariableplaceholderinqueries.

PythonEssentials 91

• Hereishowthequeriesdiffer:

– QueryinSqlite:

cursor.execute("SELECT * FROM stocks WHERE symbol=?", ('T',))

– QueryinMySQL andPostgreSQL:

cursor.execute("SELECT * FROM stocks WHERE symbol=%s", ('T',))

20 NoseTests

20.1 UnitTesting

Exercise: Writeasimpleclasscalled TipCalc whichcalculatesthetipforagivenamount. Writeaunittestthatchecksthatthetipon$100is$15.

Solution:

In tip.py:

import sys, unittest

class TipCalc:def __init__(self, tip_rate):

self.tip_rate = tip_rate

def compute(self, amount):return amount * self.tip_rate

class TipCalcTest(unittest.TestCase):def test_computeTip(self):

tip_calc = TipCalc(0.15)assert 15 == tip_calc.compute(100)

def main(argv):# Main program logic goes here.

PythonEssentials 92

# Command line args available in argv.print "Now in tip.py main()"print "argv = " + str(argv)

def start():print '__name__: ' + __name__if __name__ == '__main__':

if len(sys.argv) > 1 and sys.argv[1] in \["-test", "/test", "--test"]:

sys.argv.pop(0)unittest.main()

else:main(sys.argv)

start()

In demo.py:

import foo

print "Now in demo.py"

Runtheprogramas:

python demo.py arg1 arg2

Runthemaininthemodule tip.py as:

python tip.py arg1 arg2

Runthetestsinfooas:

python tip.py --test

Notes:

PythonEssentials 93

• Towriteunittestsyouhavetoimport unitttest.

• Testshavetoliveinsideaclassthatextends unittest.TestCase.

• Testsliveinsidetestmethods.

• Everytestmethodnamemuststartwith test.

• The start() functionandthecallto start() areboilerplatecodethatyouneedtoincludeeverytimeyouhaveunittestsandproductioncodeinthesamefile. Thisfunctiondecideswhethertorunthemaincodeorthetestcode.

20.2 NoseTests

Nose testsmakeiteasiertorununittests. Youdon’thavetomodifythecodewithboilerplatecode.

20.3 NoseInstall

Exercise: Install nose.

sudo pip install nose

20.4 NoseTestExample

Exercise: Writeasimpleclasscalled TipCalc whichcalculatesthetipforagivenamount. Writeaunittestthatchecksthatthetipon$100is$15.

Solution

In tip.py:

import sys, unittest

class TipCalc:def __init__(self, tip_rate):

self.tip_rate = tip_rate

PythonEssentials 94

def compute(self, amount):return amount * self.tip_rate

class TipCalcTest(unittest.TestCase):def test_computeTip(self):

tip_calc = TipCalc(0.15)assert 15 == tip_calc.compute(100)

def multiply(a,b):return a * b

def test_multiply():assert 12 == multiply(3,4)

Runthetestsas:

# Quietnosetests tip.py

# Verbosenosetests -v tip.py

# Another syntax for verbose.nosetests --verbose tip.py

# Don't capture output.nosetests --nocapture tip.py

Notes:

• Nosemakesiteasytoruntests.

• Itrunsteststhatliveinsubclassesof unittest, forexample, TipCalcTest.

• It also runs tests that are in methods that start with test, for example,test_multiply.

PythonEssentials 95

• Bydefaultnosecapturesthestandardoutputtomaketestsquiet. Todisablethisfeaturefordebugging, runitusing --nocapture.

• Nosemakesiteasytowriteconcisetests.

• Noseistherecommendedwayofwritingandmaintainingunittests.

20.5 NoseSetUp

Exercise: Runsetupbeforerunningthetests. Runteardownafterrunningthetests.Setupisusefulforsettingupsomecontextforthetest. Teardownisusefulforcleaningupthecontext.

In demo.py:

import unittest

from nose import with_setup # optional

def setup_module(module):print ("") # this is to get a newline after the dotsprint ("setup_module before anything in this file")

def teardown_module(module):print ("teardown_module after everything in this file")

def my_setup_function():print ("my_setup_function")

def my_teardown_function():print ("my_teardown_function")

@with_setup(my_setup_function, my_teardown_function)def test_multiply():

print 'test_multiply'assert 3*4 == 12

class TestStuff(unittest.TestCase):

PythonEssentials 96

def setUp(self):print ("TestStuff:setUp() before each test method")

def tearDown(self):print ("TestStuff:tearDown() after each test method")

@classmethoddef setUpClass(cls):

print ("setup_class() before any methods in this class")

@classmethoddef tearDownClass(cls):

print ("teardown_class() after any methods in this class")

def test_multiply(self):print 'test_multiply()'assert 5*6 == 30

Runthiswith:

nosetests --nocapture demo.py

Notes:

• Noseprovidesdifferentwaysofdoingsetupandteardown.

• Youcanrunsetuponceforthewholefile, oryoucanrunsetupbeforeeachtest, oryoucanrunsetupbeforeeachtestinatestclass.

• Theabovecodeillustratesallthesedifferentscenarios.

21 XML Parsing

21.1 xmltodict

Exercise: Findthetemperatureforagivenzipcode.

PythonEssentials 97

Hint:

Generalizethefollowingcodetouseazipcodefrom sys.argv.

import urllibimport urllib2import xmltodict

zip = '27709'print "zip =", zip

# Get WOEID.wid_query = \

"select woeid" \+ " from geo.places" \+ " where text='" + zip + "' limit 1"

wid_url = \'http://query.yahooapis.com' \+ '/v1/public/yql' \+ '?format=xml&q=' \+ urllib.quote_plus(wid_query)

wid_xml = urllib2.urlopen(wid_url).read()

data = xmltodict.parse(wid_xml)wid = data['query']['results']['place']['woeid']

# Get temperature.weather_url = \

"http://weather.yahooapis.com" \+ "/forecastrss" \+ "?format=xml" \+ "&w=" + wid \+ "&u=f"

weather_xml = urllib2.urlopen(weather_url).read()

weather_data = xmltodict.parse(weather_xml)temp = weather_data['rss']['channel']['item']['yweather:condition']['@temp']

PythonEssentials 98

print "temp = ", temp

Notes:

• xmltodict.parse convertsXML to a nested structure of Pythondicts andarrays.

• FormorelowlevelaccesstoXML takealookatthismodule.

import xml.etree.ElementTree

21.2 ElementTree

Exercise: FindthenameoftheairportwithcodeSFO.

Solution:

import xml.etree.ElementTree as ElementTree

xml = '''\<?xml version="1.0"?><airports>

<airport code="SFO"><name>San Francisco</name><terminals>4</terminals>

</airport><airport code="SJC">

<name>San Jose</name><terminals>3</terminals>

</airport><airport code="OAK">

<name>Oakland</name><terminals>2</terminals>

</airport></airports>'''

PythonEssentials 99

# Parse XML.data = ElementTree.fromstring(xml)

# Find all elements called name.data.findall('.//name')

# Find name of airport with code SFO.data.findall(".//airport[@code='SFO']/name")[0].text

# Get first child of root node.data.getchildren()[0]

# Get inner text of node.data.text

# Get attributes of node.data.attrib

# Print out XML.ElementTree.tostring(data)

22 WebApplicationsUsingFlask

22.1 SettingUpFlask

Exercise: Install flask onyourmachine.

Solution:

• Checkifflaskisalreadyinstalledonyourmachine: OpenthePythonconsoleandtype import flask.

• Ifyougetanerrorthenitisnotinstalled.

• Toinstallitgrabitusingpip.

sudo pip install Flask

PythonEssentials 100

22.2 HelloWorldWebApp

Exercise: Writeahelloworldwebapp.

Solution:

from flask import Flask, requestapp = Flask(__name__)

INDEX_HTML = '''\<h1>Hello World</h1>Hello world!'''

@app.route('/')def index():

return INDEX_HTML

if __name__ == '__main__':app.debug = Trueapp.run()

Notes:

• app.debug = True ishelpfulduringdevelopment. Ithotswapsthecodeifyoumakeachangewhileitisrunning. Alsoitprintsbettererrormessagesiftherearefailures.

• The route attachesaURL pathtothisspecificmethod.

• ThemethodhastocomposeandreturntheHTML thatthebrowserwilldisplay.

22.3 TwoPageWebApp

Exercise: Writeatwo-pagewebsite.

Solution:

PythonEssentials 101

from flask import Flask, requestapp = Flask(__name__)

HOME_HTML = '''\<h1>Hello World</h1>[ <a href='/home'>Home</a> |

<a href='/about'>About</a> ]<br><br>

Hello world!'''

ABOUT_HTML = '''\<h1>About</h1>[ <a href='/home'>Home</a> |

<a href='/about'>About</a> ]<br><br>

This is the about page.'''

@app.route('/')@app.route('/home')def home():

return HOME_HTML

@app.route('/about')def about():

return ABOUT_HTML

if __name__ == '__main__':app.debug = Trueapp.run()

Notes:

• Multipleroutescanbeattachedtoasinglemethod.

• DependingontheURL theappropriatemethodisinvoked.

PythonEssentials 102

22.4 TipCalculatorWebApp

Exercise: Writeaweb-basedtipcalculatorapp.

Solution:

from flask import Flask, requestapp = Flask(__name__)

def parse_float(x):try:

return float(x)except:

return None

def get_error(x, name):x_value = parse_float(x)if x_value == None:

return name + ' "' + x + \'" is not a number.<br>'

return ''

class Calc:def __init__(self, amount, people, tip_rate):

self.result = ''self.error = ''

# Build error.self.error += get_error(amount, 'Amount')self.error += get_error(people, 'People')self.error += get_error(tip_rate, 'Tip rate')

# Build result.amount = parse_float(amount)people = parse_float(people)tip_rate = parse_float(tip_rate)

PythonEssentials 103

if amount == None: returnif people == None: returnif tip_rate == None: return

tip = amount * (tip_rate / 100.0)amount_per_person = (amount + tip) / peopleself.result = \

'Amount per person: ${0}'.format(amount_per_person)

def get_error(self):return self.error

def get_result(self):return self.result

HOME_HTML='''\<h1>Tip Calculator</h1>

<div style='color:red'>{error}</div>

<br>

<form>

Amount<br><input type="text" name="amount" value="{amount}"><br><br>

People<br><input type="text" name="people" value="{people}"><br><br>

Tip Rate<br><input type="text" name="tip_rate" value="{tip_rate}"><br><br>

PythonEssentials 104

<button name="button">Calculate</button>

</form>

{result}'''

@app.route('/', methods=['GET','POST'])def index():

amount = request.values.get('amount','')people = request.values.get('people','')tip_rate = request.values.get('tip_rate','')button = request.values.get('button', None)

# First pass.if button == None:

return HOME_HTML.format(amount=amount,people=people,tip_rate=tip_rate,error="",result="")

# Second pass.calc = Calc(amount, people, tip_rate)html = HOME_HTML.format(

amount=amount,people=people,tip_rate=tip_rate,error=calc.get_error(),result=calc.get_result())

return html

if __name__ == '__main__':app.debug = Trueapp.run('localhost',8000)

PythonEssentials 105

Notes:

• Youwillnoticethat@app.route hasanadditionalparameterinit: methods=['GET','POST'].This indicates toFlask that themethodcanhandlebothGET andPOST.ThisisneededbecausetheformsubmitsitsinputsusingPOST.

• Thismightseemlikealotofcode. Thisisnormalasourtipcalculatorgraduallyturnsintoarealprogram. Handlingallthedetailsmakestheprogramlarger.

22.5 DeployingonPythonAnywhere

Exercise: Deploythisapponthecloudusing PythonAnywhere.

Solution:

• Goto https://www.pythonanywhere.com andsignupforafree N00b ac-countunder Pricing&signup.

• Makesureyouareloggedin.

• Clickonthe Web tab.

• Clickon Addanewwebapp --> Next --> Flask

• Notethepathwhereyourappwillbedeployed.

• Clickon Files --> mysite --> flask_app.py

• Thiswillputyouintheeditor.

• Copyandpasteyourtipcalculatorcodehere.

• Clickon Save.

• Clickon mysite --> Web --> Reloadapp.

• Clickonthelinktoyourappnextto Youcanseeyourwebappat.

• Nowtryvisitingyourappwithyourmobilephone.

PythonEssentials 106

22.6 MVC Application

Exercise: Createasimpleblogapplicationwithadatabasebackend.

Solution:

• The following solution isbasedon the flaskr applicationdescribed in theFlaskdocumentation.

• Create a directory called flaskr. Create two subdirectories in it called:templates and static.

• Create flaskr/schema.sql withthesecontents:

drop table if exists entries;create table entries (id integer primary key autoincrement,title text not null,text text not null

);

• Create flaskr/flaskr.py withthesecontents:

import sqlite3from flask import Flask, request, \

session, g, redirect, url_for, abort, \render_template, flash

# configurationDATABASE = '/tmp/flaskr.db'DEBUG = TrueSECRET_KEY = 'development key'USERNAME = 'admin'PASSWORD = 'default'

# create our little application :)app = Flask(__name__)app.config.from_object(__name__)app.config.from_envvar('FLASKR_SETTINGS', silent=True)

PythonEssentials 107

def connect_db():return sqlite3.connect(app.config['DATABASE'])

from contextlib import closingdef init_db():

with closing(connect_db()) as db:with app.open_resource('schema.sql', mode='r') as f:

db.cursor().executescript(f.read())db.commit()

@app.before_requestdef before_request():

g.db = connect_db()

@app.teardown_requestdef teardown_request(exception):

db = getattr(g, 'db', None)if db is not None:

db.close()

@app.route('/')def show_entries():

cur = g.db.execute('select title, text from entries order by id desc')entries = [dict(title=row[0], text=row[1]) for row in cur.fetchall()]return render_template('show_entries.html', entries=entries)

@app.route('/add', methods=['POST'])def add_entry():

if not session.get('logged_in'):abort(401)

g.db.execute('insert into entries (title, text) values (?, ?)',[request.form['title'], request.form['text']])

g.db.commit()flash('New entry was successfully posted')return redirect(url_for('show_entries'))

@app.route('/login', methods=['GET', 'POST'])

PythonEssentials 108

def login():error = Noneif request.method == 'POST':

if request.form['username'] != app.config['USERNAME']:error = 'Invalid username'

elif request.form['password'] != app.config['PASSWORD']:error = 'Invalid password'

else:session['logged_in'] = Trueflash('You were logged in')return redirect(url_for('show_entries'))

return render_template('login.html', error=error)

@app.route('/logout')def logout():

session.pop('logged_in', None)flash('You were logged out')return redirect(url_for('show_entries'))

if __name__ == '__main__':app.run()

• Create static/style.css withthesecontents:

body {font-family: sans-serif;background: #eee;

}a, h1, h2 {

color: #377ba8;}h1, h2 {

font-family: 'Georgia', serif;margin: 0;

}h1 { border-bottom: 2px solid #eee; }h2 { font-size: 1.2em; }.page {

PythonEssentials 109

margin: 2em auto;width: 35em;border: 5px solid #ccc;padding: 0.8em;background: white;

}.entries {

list-style: none;margin: 0;padding: 0;

}.entries li { margin: 0.8em 1.2em; }.entries li h2 { margin-left: -1em; }.add-entry {

font-size: 0.9em;border-bottom: 1px solid #ccc;

}.add-entry dl { font-weight: bold; }.metanav {

text-align: right;font-size: 0.8em; padding: 0.3em;margin-bottom: 1em;background: #fafafa;

}.flash {

background: #cee5F5;padding: 0.5em;border: 1px solid #aacbe2;

}.error {

background: #f0d6d6;padding: 0.5em;

}

• Create template/layout.html withthesecontents:

<!doctype html><title>Flaskr</title>

PythonEssentials 110

<link rel=stylesheet type=text/csshref="{{ url_for('static', filename='style.css') }}">

<div class=page><h1>Flaskr</h1><div class=metanav>{% if not session.logged_in %}

<a href="{{ url_for('login') }}">log in</a>{% else %}

<a href="{{ url_for('logout') }}">log out</a>{% endif %}</div>{% for message in get_flashed_messages() %}

<div class=flash>{{ message }}</div>{% endfor %}{% block body %}{% endblock %}

</div>

• Create template/login.html withthesecontents:

{% extends "layout.html" %}{% block body %}<h2>Login</h2>{% if error %}<p class=error><strong>Error:</strong>{{ error }}{% endif %}<form action="{{ url_for('login') }}" method=post>

<dl><dt>Username:<dd><input type=text name=username><dt>Password:<dd><input type=password name=password><dd><input type=submit value=Login>

</dl></form>

{% endblock %}

• Create templates/show_entries.html withthesecontents:

PythonEssentials 111

{% extends "layout.html" %}{% block body %}{% if session.logged_in %}

<form action="{{ url_for('add_entry') }}"method=post class=add-entry>

<dl><dt>Title:<dd><input type=text size=30 name=title><dt>Text:<dd><textarea name=text rows=5 cols=40></textarea><dd><input type=submit value=Share>

</dl></form>

{% endif %}<ul class=entries>{% for entry in entries %}

<li><h2>{{ entry.title }}</h2>{{ entry.text|safe }}{% else %}

<li><em>Unbelievable. No entries here so far</em>{% endfor %}</ul>

{% endblock %}

• Runthisas python flaskr.py.

• ThentestitbyvisitingtheURL thatitprintsout.

23 ListComprehensions

23.1 ListComprehensions

Exercise: Generateallthepowersof2from1through6.

Solution1:

powers_of_two = [2**n for n in range(1, 6)]

PythonEssentials 112

Solution2:

powers_of_two = (2**n for n in range(1, 6))

Notes:

• Listcomprehensionsgeneratelistsusinganotationthatissimilartothe forallinmathematics.

• Solution1producesalist.

• Solution2producesaniterator. Thisisbasicallylikealist. Youcanuseitinsidea for. However, itislazyandthereforemoreefficient. Itdoesnotdotheworkuntilyoutrytoconsumethiswitha for.

23.2 ListsofDictionaries

Exercise: Createa listofdictionaries, containingkeys i and square. Here i isanumberfrom0to9and square isitssquare.

Solution: d=[dict(i=i,square=i*i)foriinxrange(10)]

Notes:

• Wearecreatingthedictionariesusinglistcomprehensions.

• dict(k1=v1,k2=v2,k3=v3) isanotherwaytocreatedictionaries.

• Wecouldhaveused {} hereinsteadaswell.

[{"i":i,"square":i*i} for i in xrange(10)]

24 Pexpect

24.1 Pexpect

Exercise: SimulateHTTP accessusingtelnet. WriteapexpectscripttoaccessGoogleandprintouttheresponsestatus.

Solution:

PythonEssentials 113

import pexpectimport sys

host = 'google.com'

# Start telnet.proc = pexpect.spawn(

'telnet ' + host + ' 80')

# Echo to stdout.proc.logfile = sys.stdout

# Wait for connectionproc.expect('Trying [\d\.]+')proc.expect('Connected to ' + host)proc.expect("Escape character is .*")

# Send HTTP request.proc.sendline ('GET / HTTP/1.1')proc.sendline ('host: ' + host)proc.sendline ('')

# Wait for status line.index = proc.expect([

'HTTP/1.1 301','HTTP/1.1 404','HTTP/1.1 200',pexpect.EOF,pexpect.TIMEOUT,

])

# Report status.if index == 0: r = "301"elif index == 1: r = "404"elif index == 2: r = "200"elif index == 3: r = "EOF"elif index == 4: r = "TIMEOUT"else: r = "UNKNOWN"

PythonEssentials 114

print "Result: " + r

# Give control to user.proc.interact()

Notes:

• Insteadof logfile youcanalsoset logfile_send and logfile_read tologwhatissentandwhatisreadfromtheprocess.

• Insteadofsettingthelogfilesto sys.stdout youcanalsosetthemtoafileobjectthatyougetthrough open.

24.2 PexpectMethods

Inthefollowingmethods, p istheobjectreturnedby pexpected.spawn(command)

Method Meaning

spawn(command) Startsanewprocess

p.sendline(line) Sendlinewithlinefeed

p.expect(re) Waitsforregexmatchon re

p.expect([re1,re2,]) Returnsindexofmatchingregex

p.interact() Givecontroltouser

p.wait() Waituntilprocessexits

p.terminate() Sends SIGHUP, SIGINT,then SIGKILL

PythonEssentials 115

25 Threading

25.1 Threading

Exercise: Createtwoqueuesandtwosetsofthreads. Thefirstsetpicksintegersoffthefirstqueueandputsthemonthesecondqueue. Thesecondremovesthemfromthesecondqueue. Eachthreadlogsthedatathatitreceivesinalogqueue.

Solution:

import Queueimport threadingimport sysimport time

# Create two queues and a log queue.queue1 = Queue.Queue()queue2 = Queue.Queue()log_queue = Queue.Queue()

# Thread to move from queue1 to queue2.class Thread1(threading.Thread):

def __init__(self, in_queue, out_queue):threading.Thread.__init__(self)self.in_queue = in_queueself.out_queue = out_queue

def run(self):while True:

# Grab item from in queue.item = self.in_queue.get()

# Log it on log queue.log_queue.put("1:"+str(item))

# Put it on out queue.self.out_queue.put(item)

PythonEssentials 116

# Inform in queue we are done.self.in_queue.task_done()

# Thread to consume from queue2.class Thread2(threading.Thread):

def __init__(self, in_queue):threading.Thread.__init__(self)self.in_queue = in_queue

def run(self):while True:

# Grab item from queue.item = self.in_queue.get()

# Log it on log queue.log_queue.put("2:"+str(item))

# Inform in queue we are done.self.in_queue.task_done()

# Remember when we started.start = time.time()

# How many threads to run.thread_count = 10

# Start all Thread1.for i in range(thread_count):

t = Thread1(queue1, queue2)t.setDaemon(True)t.start()

# Start all Thread2.for i in range(thread_count):

t = Thread2(queue2)t.setDaemon(True)

PythonEssentials 117

t.start()

# Fill up queue1.for x in range(10):

queue1.put(x)

# Wait for queues to empty.queue1.join()queue2.join()

# Print out log queue contents.while not log_queue.empty():

x = log_queue.get()print x

print "Elapsed Time: %s" % (time.time() - start)

Notes:

• Python’ssynchronizationmechanismis Queue.Queue.

• Threadsshouldnotmodifyshareddatastructures. Insteadtheyshouldcommu-nicatethroughqueues.

• Thinkofthreadsasworkersontheassemblyline.

• Eachthreadpicksupanitemfromtheincomingqueue, worksonit, andplacesitsoutputontheoutgoingqueue.

26 MiscTopics

26.1 ArgsandKWArgs

Exercise: HowdoyouwriteavariableargfunctioninPython?

Solution:

PythonEssentials 118

def f(*args,**kwargs): print "f:", args,kwargsdef g1(*args,**kwargs): f(*args,**kwargs)def g2(*args,**kwargs): f(args,kwargs)

g1(1,2,3,a=10,b=20)g2(1,2,3,a=10,b=20)

Notes:

• g1 isavariableargsfunction. Itreceivesitspositionalargumentsin args anditskeywordargumentsin kwargs.

• Ifyouwanttocallanothervariableargfunctionfrom g1 youhavetopassontheargumentsas *args and **kwargs.

• Ifyouwanttoconsumethemin g1 youaccessthemdirectlyas args (whichwillgiveyoualist)and kwargs (whichwillgiveyouadictionary).

26.2 ClassmethodandStaticmethod

Exercise: HowdoyouputclassmethodsinPython?

Solution:

class A(object):@classmethoddef class_foo(cls,x):

print "executing class_foo(%s,%s)"%(cls,x)@staticmethoddef static_foo(x):

print "executing static_foo(%s)"%xdef foo(self,x):

print "executing foo(%s,%s)"%(self,x)

a=A()

a.foo("Hello")a.class_foo("Hello")a.static_foo("Hello")

PythonEssentials 119

Notes:

• Youcanaddclassmethodsusing @classmethod.

• Youcanaddstaticmethodsusing @staticmethod.

• Staticmethodsdonotknowwhatclasstheyareinandonlyhaveaccesstotheargumentstheyarepassed. Theyarelikemethodsthatareoutsideaclass.

• Classmethodsarepassedtheclassnameoftheclasstheywerecalledon.

• Staticmethodsaremostlyanorganizationalconvenience. Itisawayofputtingutilityfunctionsonclasses.

26.3 Closures

Exercise: Createafunctionthatapproximatesthederivativeof f usinganintervalofdx.

Solution:

def make_derivative(f, dx):def derivative(x):

return (f(x + dx) - f(x)) / dxreturn derivative

import mathff = make_derivative(math.sin, 0.01)print ff(0)

Notes:

• The make_derivative functionisafunctionfactory. Ittakessomeparametersandhard-wiresthemintothefunctionthatitproduces.

• Theinnerfunctioncalled derivative herekeepsreferencestotheparametersthatwereinitscontextwhenitwascreated. Inthiscaseitholdsreferencestof and dx.

PythonEssentials 120

• Whenwecall ff withaparticularvalueof x itusesthevaluesof f and dx thatwerehard-wiredintoitwhenitwascreated.

• A closureisthiscontextwheretheoriginalvaluesof f and dx arestored. Itrepresentsthehard-wiredvaluesthatthecreatedfunctionrememberswhenitisinvokedlater.

26.4 Decorators

Exercise: Create a custom decorator that can be applied to functions using@my_decorator. Thedecoratorshouldprint”Callingdecoratedfunction”whenitsfunctionisinvoked.

Solution:

from functools import wrapsdef my_decorator(f):

@wraps(f)def wrapper(*args, **kwds):

print 'Calling decorated function'return f(*args, **kwds)

return wrapper

@my_decoratordef example():

"""Docstring"""print 'Called example function'

example()example.__name__example.__doc__

Notes:

• A decoratorallowsyoutoexecutecodebeforeandafteradecoratedfunction.

• functools.wraps isaconveniencefunctionthatensuresthatthewrappedfunctionretainsitsnameanddocstring.

PythonEssentials 121

26.5 Memoizer

Exercise: Createamemoizerdecorator. A memoizercachesthevalueitproducedforagivensetofarguments. Thenthenexttimeitiscalleditreturnstheresultfromitscache. Ifitiscalledwithnewarguments, itcomputesanewvalue, andthensavesthatinthecacheaswell.

Solution:

import functools

def memoize(obj):cache = obj.cache = {}

@functools.wraps(obj)def memoizer(*args, **kwargs):

key = str(args) + str(kwargs)if key not in cache:

cache[key] = obj(*args, **kwargs)return cache[key]

return memoizer

@memoizedef hello(name):

print "Computing value"return "Hello, " + name

# First time it computes the value.print hello('Jim')

# Second time it returns cached value.print hello('Jim')

26.6 Iterators

Exercise: Createafunctionthatuppercasesthecharactersofastringandthenreturnsaniteratorforiteratingthroughthem. Aniteratorisanobjectyoucanconsumeina

PythonEssentials 122

for loop. Itislikealistexceptitis lazy andonlyproducesitsresultswhenthe forloopdemandsit. Iteratorsaremoreefficientmemory-wisethanlists. Theycanalsobefasterbecausetheyonlydotheminimumworknecessary.

Solution1:

# Generatordef upper_case_gen(text):

for char in text:yield char.upper()

for c in upper_case_gen("hello"): print c

Solution2:

# Generator Expressiondef upper_case_genexp(text):

return (char.upper() for char in text)

for c in upper_case_genexp("hello"): print c

Solution3:

# Iteratorclass upper_case_iter():

def __init__(self, text):self.text = textself.index = 0

def __iter__(self):return self

def next(self):try:

result = self.text[self.index].upper()except IndexError:

PythonEssentials 123

raise StopIterationself.index += 1return result

for c in upper_case_iter("hello"): print c

Solution4:

# Getitemclass upper_case_getitem():

def __init__(self, text):self.text = text

def __getitem__(self, index):result = self.text[index].upper()return result

for c in upper_case_getitem("hello"): print c

Notes:

• Hereare4differentwaysofdoingthis.

• OftheseSolutions1and2arethepreferredones.

27 Next

27.1 NextSteps

Congratulations. Youhavecompletedthecourse.

Whatshouldyoudonext?

Oneofthebestwaystolearnprogrammingistotypecodemanuallyfromexamplesandmake itwork. Youcanuse this techniquewithexamples from thisbookandexamplesfromotherplaces.

OnceyoufeelcomfortablewithPython, considertakingonasmallproject. Picksome-thingthatcanbedoneinadayorso. Iftheprojectistoolargeitdelaysexperiencing

PythonEssentials 124

thegratificationofgettingdone. Thisisnotasmuchfunassomethingthatgetsdonequickly.

Whenitfinallyworksthatwillproduceasweetfeelingofgratificationandaccomplish-mentthatishardtobeat.

Goodluckonyourapps, andI hopeyouenjoyyournewexperienceswithPython.