introduction to python and web programming

88
Intro to Python & Web Programming For Beginners San Diego Python Users Group Kendall Chuang, David Fischer, Trey Hunner, David Neiss Monday, January 28, 13

Upload: david-neiss

Post on 03-Sep-2014

2.178 views

Category:

Documents


3 download

DESCRIPTION

Slides created for the San Diego Python Users Group to teach our "Intro To Python" class. We have held the class twice already and will continue to do so as there continues to be interest in learning Python.

TRANSCRIPT

Page 1: Introduction to Python and Web Programming

Intro to Python &Web Programming

For BeginnersSan Diego Python Users Group

Kendall Chuang, David Fischer, Trey Hunner, David Neiss

Monday, January 28, 13

Page 2: Introduction to Python and Web Programming

Introductions

• Your name

• Your Python experience and software background

• What you would like to do with Python?

This helps us better focus the training for the next iteration

Monday, January 28, 13

Page 3: Introduction to Python and Web Programming

Thanks to Sponsors:

• Ansir Innovation Center = big thanks for the space

• Python Software Foundation = $ for food

• San Diego Python Users Group = volunteers

Monday, January 28, 13

Page 4: Introduction to Python and Web Programming

SD Python Users Group

• Started by Kendall Chuang and David Fischer

• Appx 300 people

• Free, monthly meet-ups. Usually last Thursday

• 7:30, appx 1.5 hours, @Ansir

• Usually one or two speakers

• All experience levels

• Listed on sdtechscene.org and meetup.com

• (optional) after meeting @ pub

Monday, January 28, 13

Page 5: Introduction to Python and Web Programming

Today’s Overview

• Morning learn & do

• Lunch break around 12

• Afternoon, learn & do

• Do project

• Bathroom locations, wifi login/pw, parking reminder

Monday, January 28, 13

Page 6: Introduction to Python and Web Programming

Today’s Goals1. Breadth (not depth) coverage of Python

2. Reading Python, but not necessarily writing it

3. Learn how to figure out Python for yourself:

a. try it in the Python shell

b. help() in the Python shell

c. google “The Python Tutorial” in Python docs

d. google/stack overflow

Monday, January 28, 13

Page 7: Introduction to Python and Web Programming

The Python Interpreter

• The “python” executable

• Compiles Python source code into Python byte codes (the .pyc files you will see in your dirs)

• Can be used to run entire programs or interactively (the “shell” mode)

Monday, January 28, 13

Page 8: Introduction to Python and Web Programming

Python Shell• Fosters “exploratory/interactive programming”

• Prints expression values automatically

• >>> is the shell prompt

• help() - enters into a help mode

• dir() - prints names in module’s namespace

• Use up arrow to retrieve last command

• Can be used as a calculator, for example, 1*2*3*4*5 to compute 5!

• quit via exit()

Monday, January 28, 13

Page 9: Introduction to Python and Web Programming

Do• Startup the Python shell

• help() to enter the help system

• Type “list” to get help on Python’s Lists

• “quit” to exit help mode

• Alternatively, you can just type help(list)

• dir() to get a dump of names in current scope

• quit() or CTRL-D to exit the shell

• print ‘hello world’ to print the string “hello world”

• Compute the number of seconds in a day using the shell interactive mode

Monday, January 28, 13

Page 10: Introduction to Python and Web Programming

Print Statement

• Before we do anything, we need a way to look at values using the print command.

• print <expression>

• <expression> can be a literal, variable, or expression:

print “hello world” # this is a literal

print 1+1 # 1+1 is an expression

myTeam = “Chargers” # myTeam is a variable

print myTeam # print out the value of myTeam

Monday, January 28, 13

Page 11: Introduction to Python and Web Programming

Print Formatting• For string literals, you can use single or double quotes

• For substitutions into a string, use %s for strings and ints

# Dynamic HTML generation in Python!

print “<title>%s</title>” % “MyWebSite”

print “size=%d” % 10

• For multiple values, surround with parenthesis (called a tuple)

print ‘<img src=”%s” alt=”%s”>’ % (“www.foo.com/x.png”, “A picture”)

Monday, January 28, 13

Page 12: Introduction to Python and Web Programming

Do

• Print your name

• Print your name and house number using print formatting string “I am %s, and my house address number is %s” and a tuple

Monday, January 28, 13

Page 13: Introduction to Python and Web Programming

User Input

• raw_input() is a built in function

• Use raw_input(“user prompt”)

• Prints the “user prompt” and then waits for user input followed by return key

• raw_input() returns what user entered

• Assign raw_input() to a variable, val = raw_input()

Monday, January 28, 13

Page 14: Introduction to Python and Web Programming

Do

• Ask the user to enter their name, by typing the following into the Python shell, raw_input(“Hi sexy, what’s your name?”)

• Next, print “Hey <yourName>, you are kinda cute” and substitute the user input for <yourName>.

Monday, January 28, 13

Page 15: Introduction to Python and Web Programming

Variables

• Created using an arbitrary name (certain rules apply, must start with a letter, case sensitive, not be a reserved word,...) For example, “tweet” or “this_URL” or “jsonData”

• Dynamically bound to objects of specific data types at assignment time. For example, errCode=404 (an int), url=‘www.cnn.com’ (a string), pi=3.1415 (a float)

• No need to declare types. type is inferred from object. In the previous case, we would have an int, string, and float type

• Type can be determined using type(variable)

Monday, January 28, 13

Page 16: Introduction to Python and Web Programming

Variables...

• When a variable isn’t bound to anything, its type is “NoneType”

• If you assign a variable to a function that has no return value, you get None

• to test a variable for “none-ness”, use “var is None”, which is a bool expression

Monday, January 28, 13

Page 17: Introduction to Python and Web Programming

Do

• Create a variable using your name and assign an integer of your birth month to it

• Print the variable and print its type

• Assign a string of your birth month to the same variable

• Now print the variable and print its type

• Notice that the type of the variable changes with the data

Monday, January 28, 13

Page 18: Introduction to Python and Web Programming

Boolean (bool)

• two values, True or False

• case sensitive

• “and” does boolean AND operation. Only True of both are True

• “or” does boolean OR operation. True if either are True

• “not” does boolean inversion. not True == False

• == for comparison (not single equals, that’s assignment)

Monday, January 28, 13

Page 19: Introduction to Python and Web Programming

Integers and Floats

• Usual operators +, -, *, /, **

• +=, *=, /=, -= shortened forms. No ++ or -- operators as in C/Java

• Integers have arbitrary precision(!), try 2**1000. Floats don’t (try 2.1**1000)

• Floating point number must have decimal point

• Note difference between 5/2 and 5.0/2

• use int(value) and float(value) to convert into that type

Monday, January 28, 13

Page 20: Introduction to Python and Web Programming

Strings• Use quotes around text

• Single or double quotes, but must match

• If string must contain quotes, escape it with backslash. For instance “he said \”Wow!\””

• Immutable

• len() for length

• type() == str

• Use str() to convert a type to a string. str(2) == “2”

• Use triple quoting to continue text after an end-of-line (which normally terminates a line)

Monday, January 28, 13

Page 21: Introduction to Python and Web Programming

String Operations• Concatenation via +, returns new string combining both

• Concatenation via *, repeats string N times

• string.upper(), converts string to upper case

• string.swapcase(), swaps case of chars in string

• string.split(), splits string and returns list of words

• string.strip(), removes leading and trailing white space, returns new string

• string.count(“whatToCount”)

• string.replace(“this”, ”withThat”), returns new string with changes

• string.find(“whatToFind”, fromIndex) - returns index of whatToFind in string starting at position fromIndex, -1 if not found

• Many more, look in docs

Monday, January 28, 13

Page 22: Introduction to Python and Web Programming

Do• Create two variables, one assigned to “http://www.” and another to

“cnn.com”

• Concatenate (join) the two into a third variable so you get a complete URL

• Create another variable with the complete URL repeated 10 times, but with a comma between the URLs

• Split the string “<h1> This is an HTML header </h1>” into its space delimited words

• Count the number of occurrences of “jpg” in this string “a.jpg, b.gif, c.jpg, d.png”

• From the URL string “www.facebook.com” create a new string and replacing “facebook” with “twitter”.

Monday, January 28, 13

Page 23: Introduction to Python and Web Programming

Splicing• 0 indexed/based - as per C/Java/C#

• Examples:

• S = ‘hello world’

• S[0] = ‘h’

• S[1] = ‘e’

• S[-1] = ‘d’

• S[1:3] = ‘el’

• S[:-2] = ‘hello wor’

• S[2:] = ‘llo world’

Monday, January 28, 13

Page 24: Introduction to Python and Web Programming

Do• Crate a string contain the following JSON encoded

information: [ {color : ”red” , value : ”#f00” } ]

• Use .replace(this,withThat) to remove all whitespace from the string

• Find the index to the first quote, use .find(“\””)

• Find the index to the second quote, use .find(“\””,firstIndex+1)

• Extract the value mapped to color (in this case red) which is between the first two quotes, using the splicing operator and the indexes found in the 2 previous steps

Monday, January 28, 13

Page 25: Introduction to Python and Web Programming

Lists• Create a new list with [ ] or list()

• Lists are arrays that can change size, elements can be changed, added or removed

• Elements or a list can be of mixed types, including other lists

• Supports splice operations

• For example

rssTags = [“<channel>”,”<title>”,”<pubDate>”, ”<rss>”]

print type(rssTags) # type is ‘list’

print rssTags[1:3] # supports slicing

rssTags[0] = “<link>” # mutable

print len(rssTags) # returns length of list

Monday, January 28, 13

Page 26: Introduction to Python and Web Programming

Lists...• list.sort() # sorts the list in place (doesn’t return sorted

list)

• list.reverse() #reverses list in place

• list.append(item) # appends item to end of list

• list.remove(item) # removes item from list

• list.count(item) # counts number of items in list

• del list[index] # removes elements, can use splice indexing

• bounds checked, throws exception

Monday, January 28, 13

Page 27: Introduction to Python and Web Programming

Do• Create a list of 5 strings of URLs

• Using splicing, print the second and third elements of the list

• Print the length of the list

• Sort the list

• Reverse the list

• Remove the first entry in the list

• Append the number 100 to the list. Notice that the list contains strings and numbers.

• Try to access the 100th element in the list and explain what happened

Monday, January 28, 13

Page 28: Introduction to Python and Web Programming

Dictionaries• Can be created via dict(), {}, or {key1:val1, key2:val2, ...}

• A set of key:value pairs

• Unordered - probably should really be called a map not a dictionary (actual ordering is based on hash of key, so key must be hashable and constant)

• Can grow and shrink

• Specific values are accessed via [key]

• Add new values via [newKey] = newValue

• Remove values via del dict[key]

• Use dict.has_key(key) (or, “key in dict”) to determine if key is contained

• Accessing undefined values generates an exception

• len() returns number of key/value pairs in dict

Monday, January 28, 13

Page 29: Introduction to Python and Web Programming

Do• Create an empty dictionary to keep track of user preferences as

key/value pairs

• Add in 3 key/value pairs mapping a user preference string to a value for the following pairs, “userName”-> some string, “autoLogout” -> a bool, and “maxLoginTime” -> any integer

• Print the contents of the dictionary. What order are the entries? Why?

• Add in a new key/value pair “homePage” -> “www.cnn.com”

• Query the dict to return the value for the key “userName”

• Change the value associated with “userName” to a different name

• Delete the “homePage” key/value pair from the dict

Monday, January 28, 13

Page 30: Introduction to Python and Web Programming

Sets• Contains a set of data where there are no duplicates

• Created by or {values,...} or set([1,2,3]) (but not {}, which creates an empty dict)

• Supports union, intersection, difference operations

• set.add(value) to add a value

• set.remove(value) to remove value

• set.intersection(anotherSet) to find the elements in common to both

• set.union(anotherSet) to combine anotherSet into this set, removes duplicates

• set.difference(anotherSet) removes anotherSet members from this set

• element in set - in operator tests for inclusion in set

Monday, January 28, 13

Page 31: Introduction to Python and Web Programming

Do• Create two small sets of URLs you visited today and yesterday

• Check if “www.cnn.com” is in either set

• Create a union of the two sets. Are there are duplicates in the union? Why not?

• Lets say you visited “www.xyz.com” today. Add that to your set representing today’s URLs

• Print the set of all the URLs that you visited on both days (the union)

• Print only the URLs that you visited on both today and yesterday (the intersection)

• Print only the URLs that you visited today but not yesterday (the difference)

• See how handy sets can be?

Monday, January 28, 13

Page 32: Introduction to Python and Web Programming

Aliasing• Variables refer to underlying objects

• Assigning one variable to another makes both point to the same object

• If both point to same object, one can appear to change the other

a = [1,2,3]

b = a

id(a) == id(b) # true

a[0] = 99

print b # notice that b got changed

• Use copy operations to make copies/deep copies

b = a[:]

id(a) == id(b) # false

a = [1, [2,3], 4]

b = copy.deepcopy(a) # import copy first

id(a[1]) == id(b[1]) # false

Monday, January 28, 13

Page 33: Introduction to Python and Web Programming

if/elif/else statements• Used to conditionally perform operations

• Formatted as “if expression:”

• Note the trailing colon which is required - will trip up C/C++ programmers

• Conditional block must be indented, either spaces or tabs. All code in block must have same indentation

For example:

a = 11

if a > 10:

print “a is greater than 10”

elif a>= 10 and a <= 20:

print “a is between 10 and 20”

else:

print “a is greater than 20”

Monday, January 28, 13

Page 34: Introduction to Python and Web Programming

for loops

• Used to iterate over a sequence, such as a list or set

for searchTerm in [“PERL”,”Python”,”C”]:

googleFor(searchTerm)

• Use range(low, high, step) to generate a range of numbers to iterate over

for b in range(1,10,2):

print b # odd numbers from 1 to 9

Monday, January 28, 13

Page 35: Introduction to Python and Web Programming

for loops• continue - continues with the next cycle of the loop., skipping rest of code in

block. For example:

for i in range(1,1000):

if someSpecialCondition(i):

continue

doSomethingWith(i)

• break - breaks out of the innermost loop. For example:

while True:

if someCondition():

break

Monday, January 28, 13

Page 36: Introduction to Python and Web Programming

Do

Compute the product of all numbers that are a multiple of 5 between 5 and 100, inclusive (e.g., 5, 10, 15,.... 100)

Monday, January 28, 13

Page 37: Introduction to Python and Web Programming

Functions

• why? abstraction, program decomposition, code reuse/reduce duplication

• declare functions with “def name(argList):”

• function name must follow naming rules (and not use a reserved word as its name)

• function body (line following def) must be indented

• end the function with a return statement if you are returning a value; otherwise, no return statement is necessary. Note that function doesn’t declare its return type. Functions that don't have return values effectively return “None”

Monday, January 28, 13

Page 38: Introduction to Python and Web Programming

Arguments

• argList is a comma separated list of argument names. You select the argument names that make sense for the function

• as usual in Python, argument types are not declared explicitly

• arguments are passed “by reference”, so a function could change their values (if they are mutable types)

• If you don’t want function to change values passed in by reference, make deep/copy or use immutable type (tuple)

• Advanced - Python supports default argument values, named arguments, and variable number of arguments

Monday, January 28, 13

Page 39: Introduction to Python and Web Programming

Variable scopes• Variables defined outside a function are

global to that file

• Variables defined within a function are local to that function

• Objects can be returned from the function via the “return” statement

• Variables, functions, classes in other modules may be accessed in multiple ways (will be discussed in upcoming Modules slides)

Monday, January 28, 13

Page 40: Introduction to Python and Web Programming

Example• Create a function that accepts a list of URL strings and return a dictionary

mapping the URL to the number of times it was visited

sites = ["www.cnn.com","www.yahoo.com","www.facebook.com", "www.cnn.com","www.yelp.com", "www.facebook.com"]

def countVisits(sitesVisited):

numVisited = dict()

for site in sitesVisited:

if numVisited.has_key(site):

numVisited[site] = numVisited[site] + 1

else:

numVisited[site] = 1

return numVisited;

print countVisits(sites)

Monday, January 28, 13

Page 41: Introduction to Python and Web Programming

Do

• Write a function that accepts a string containing the name of a site and returns a proper HTTP URL for that site

For example for “yelp”, return “http://www.yelp.com”, so prefix it with “http://www.” and suffix it with “.com”

Monday, January 28, 13

Page 42: Introduction to Python and Web Programming

Exceptions

• Exceptions indicate an error condition

• “throw” means that the exception has occurred

• Python can throw exceptions or you can throw them

• Used to simplify error handling since error handing need not be locally coded

• An advanced topic, but not complicated

Monday, January 28, 13

Page 43: Introduction to Python and Web Programming

Exceptions...• coded as try/except blocks:

try:

.

.

doSomethingThatCouldThrowException()

if somethingBad:

raise anException()

.

.

except typeOfExceptionToCatch:

handleException()

Monday, January 28, 13

Page 44: Introduction to Python and Web Programming

Classes• Objects must be allocated via className(argList)

• Allocation may include additional arguments used to initialize the object

• Methods (or member functions) may be invoked using dotted notation

• Objects can be deleted with del() or simply by overwriting the reference

• For instance:

from datetime import datedate1 = date(2001,1,1) # allocate instance and supply construction argumentstype(date1) # type datetime.date, user-defined typedate2 = date(2013,1,1)delta = date2 - date1 # notice use of difference operator on user-defined types print delta.total_seconds() # invoke member function total_seconds to compute seconds

Monday, January 28, 13

Page 45: Introduction to Python and Web Programming

Modules & Python Program Structure

• Python files run from top to bottom

• Import statements run their imported files (modules)

• Imported files may import their own set of files

Monday, January 28, 13

Page 46: Introduction to Python and Web Programming

Modules

• Modules allow for decomposition of programs into packages of code & data, to facilitate

• Code reuse

• System namespace partitioning

• Each file is a module, which defines a namespace

• Modules import other modules to access the names they define

Monday, January 28, 13

Page 47: Introduction to Python and Web Programming

Modules...

• import modName - lets a client fetch a module as a whole. Inserts into local namespace as modName.XXX

• from modName import foo - allows clients to fetch particular names from a module into this namespace as foo

• imp.reload modName - provides a way to reload a module w/o restarting python

Monday, January 28, 13

Page 48: Introduction to Python and Web Programming

Python Program Structure

• import statements

• function and class definitions

• module scope variables and code

• Code executes from top to bottom of file (as opposed to main() )

Monday, January 28, 13

Page 49: Introduction to Python and Web Programming

Python program structure...

• Running a.py does nothing except declare the foo function

• When B is run, A’s foo is imported as a.foo

• Running b.py prints “foo”

file:a.py

def foo(): print “foo”

file: b.py

import aa.foo()

Monday, January 28, 13

Page 50: Introduction to Python and Web Programming

File I/O• The File type object is actually built into the language

as a type

• You can only get an instance with the open(fileName,”r+”) method. “r+” = read and write.

• f.read() returns whole file as a string

• f.readlines() returns a list of strings, one for each line in file

• f.write(data) to write data to file

• f.close() to close file when finished

Monday, January 28, 13

Page 51: Introduction to Python and Web Programming

Do

• Create a file named “test”, open it, and write “hello world\n” and then “i love python\n” into it and close it

• Open the file, read its contents and close it

• From outside the Python shell, verify that file named “test” was created and contains the two strings on separate lines

Monday, January 28, 13

Page 52: Introduction to Python and Web Programming

The Python Standard Library

• Included with Python (no need to download 3rd party libs)

• What follows is a small subset

• Selected to help pique your interest

Monday, January 28, 13

Page 53: Introduction to Python and Web Programming

CSV Files

• CSV = Comma Separated Values

• Common format for spreadsheets and databases

• Python includes a csv module to make reading and writing of CSV files easier

• Handles different “dialects” of file formats

Monday, January 28, 13

Page 54: Introduction to Python and Web Programming

csv Module• To write:

f = file(“myData.csv”,”r+”)

myWriter = csv.writer(f)

myWriter.writerow([“www.cnn.com”, ”news”, ”hourly”])

myWriter.writerow([“www.yelp.com”, ”reviews”, ”daily”])

f.close()

• Could use .writerows(arg) if arg is a list of lists

• To read:

f = file(“myData.csv”,”r”)

myReader = csv.reader(f)

for rowAsList in myReader:

print rowAsList

Monday, January 28, 13

Page 55: Introduction to Python and Web Programming

Do

• Create several rows of data with each row having a name string, and address string, and a zip code number as a list of lists.

• Write all of the rows of data to a CSV file using .writerows()

• Open the CSV file outside of Python and verify that all the rows were written

Monday, January 28, 13

Page 56: Introduction to Python and Web Programming

JSON Processing

• JSON encoding is very popular on the web

• json.org has the spec, but looks similar to Python dicts

• Can contain objects (maps) and lists of strings, numbers, booleans, null, or other maps and lists

Monday, January 28, 13

Page 57: Introduction to Python and Web Programming

json Module

• To read/write JSON, use the json module

• For input, parses JSON string into its value

val=json.loads(jsonString)

• For output, converts a value to a JSON encoded string

jsonString=json.dumps(value)

Monday, January 28, 13

Page 58: Introduction to Python and Web Programming

Do{

    "red":"#f00",

    "green":"#0f0",

    "blue":"#00f"

}

• Parse the above string using json.loads() into a dict. Once in the dict, change “green” to “#fff” and then encode back to JSON string using json.dumps() and print the resultant string.

Monday, January 28, 13

Page 59: Introduction to Python and Web Programming

datetime module

• Captures date & time stamps

• Allows manipulations (addition & subtraction of time, comparing time stamps,

• class datetime.now() returns a datetime object with a date&time stamp when statement is run

• class datetime.timedelta contains the difference between two datetime objects

Monday, January 28, 13

Page 60: Introduction to Python and Web Programming

DoThis program should print the time to write a file but has two bugs in it that prevent it from working correctly. Fix the bugs and make it run

so that it prints the proper amount of time. How long did it take?

import datetime

def WriteToFile(fileName, textToWrite):

f = open(fileName,”r”)

f.write(textToWrite)

f.close()

timeBefore = datetime.now()

WriteToFile(“testFile”,”hello world”);

timeAfter = datetime.now()

print timeBefore - timeAfter

Monday, January 28, 13

Page 61: Introduction to Python and Web Programming

random Module

• Used to generate “pseudo” random numbers

• random.randint(a,b) generates a random number between a and b inclusive, uniform distribution

• random.choice(seq) randomly selects one of the elements in the seq

• random.shuffle(seq) shuffles the sequence

Monday, January 28, 13

Page 62: Introduction to Python and Web Programming

import random

def UnshufledDeckOfCards():

deck = []

for num in range (1,11):

for suit in ["hearts", "clubs", "diamonds", "spades"]:

deck.append( str(num) + " of " + suit)

deck = UnshuffledDeckOfCards()

print "Unshuffled deck:"

print deck

random.shuffle(deck)

print "Shuffled deck:"

print deck

The following code is supposed to generate a deck of cards, shuffle them, and then print out the shuffled deck. One line is missing, causing it to fail. What is that one line? Add it and make it run.

Monday, January 28, 13

Page 63: Introduction to Python and Web Programming

socket module

• Provides low level networking interface called sockets

• Normally, one uses the higher level networking abstracts, but this is available

• Supports client and server modes

• socket.socket(socket.AF_INET, socket.SOCK_STREAM)

• socket.connect( (host, port) ) connects to server

• socket.sendall(stuffToSend) sends message to server

• socket.recv(NumBytesToRx) receives up to num bytes

Monday, January 28, 13

Page 64: Introduction to Python and Web Programming

Do

import socket

sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)

sock.connect( ("www.google.com",80) )

sock.sendall("GET / HTTP/1.0\n\r\n\r")

httpResponse = sock.recv(1000)

print httpResponse

Enter the following code and run it and look at the output. If it ran, you just talked to Google’s web server and fetched part of their home page!

Monday, January 28, 13

Page 65: Introduction to Python and Web Programming

telnetlib module

• Supports the telnet protocol

• Allows you to login to a telnet session and send and receive text

• telnetlib.Telnet(host) connects to host via Telnet protocol, returns connection object

• .write(text) sends text to server

• .read_until(text) reads from server until finds “text” and returns all data read

Monday, January 28, 13

Page 66: Introduction to Python and Web Programming

Do

import telnetlib

# Villanova Law Library server

HOST = "153.104.15.249"

tn = telnetlib.Telnet(HOST)

print tn.read_until("DISCONNECT")

Enter this code and run it. Look at the output. If successful, you just connected to the Villanova library’s Telnet server.

Monday, January 28, 13

Page 67: Introduction to Python and Web Programming

X/HTLM Parsing• Two models, stream based (SAX) or document based (DOM)

• A simple stream based parser is available in the module HTMLParser

• You must derive from the HTMLParser class and implement functions to receive callbacks for those parsing events

• handle_starttag - <html>

• handle_endtag - </html>

• handle_data - for data between tags

• Throws a HTMLParseError in the event that gets too confused to parse

Monday, January 28, 13

Page 68: Introduction to Python and Web Programming

from HTMLParser import HTMLParser

# create a subclass and override the handler methods

class MyHTMLParser(HTMLParser):

def handle_starttag(self, tag, attrs):

print "Encountered a start tag:", tag

def handle_endtag(self, tag):

print "Encountered an end tag :", tag

def handle_data(self, data):

print "Encountered some data :", data

# instantiate the parser and fed it some HTML

parser = MyHTMLParser()

parser.feed('<html><head><title>Test</title></head>'

'<body><h1>Parse me!</h1></body></html>')

Enter the following code, run it, and watch the output. Explain how you would modify it to

extract just the text between the tags <h1> and </h1>

Monday, January 28, 13

Page 69: Introduction to Python and Web Programming

Regular Expressions

• RE (regular expressions) module allows you to do pattern matching

• RE is a small language that defines rules for matching patterns in text

• For instance, find a string followed by @ followed by a string followed by .com to find an email addresses in a string

• Optimized for very fast execution

Monday, January 28, 13

Page 70: Introduction to Python and Web Programming

RE Example• The following is a Python script demonstrating a

Python script, fetching an RSS feed, and searching for a RE in the received page

• It is to be run via “python script.py ‘re-expression’ “

• For example, “python script.py Kardashians”

• Fetches the People magazine headlines RSS feed

• Searches for occurrences of the RE in the feed

• If any were found, prints a message, otherwise indicates that none were found

Monday, January 28, 13

Page 71: Introduction to Python and Web Programming

import re # regular expressions

import urllib2 # URL access

import sys # for argv

# Use the first arg as RE search term

whatToFind = sys.argv[1]

# Fetch the headlines from People magazine

response = urllib2.urlopen("http://rss.people.com/web/people/rss/topheadlines/index.xml")

if response.code == 200:

strings = re.findall(whatToFind,response.read())

if len(strings) == 0:

print "Thankfully, another day without news"

else:

print "In the news again"

else:

print "Error fetching the URL”

Monday, January 28, 13

Page 72: Introduction to Python and Web Programming

Web Server

• Python includes a library to implement a simple web server, SimpleHTTPServer

• Serves files from the current directory and below, mapping the dir structure to HTTP requests

Monday, January 28, 13

Page 73: Introduction to Python and Web Programming

Enter the following code into a file and run it. Fire up your browser and enter localhost:8000 as the URL. Your browser should talk to this Python server code, which should will a web page of the directory in which you started the Python interpreter in. Pretty cool!

import SimpleHTTPServer

import SocketServer

PORT = 8000

Handler = SimpleHTTPServer.SimpleHTTPRequestHandler

httpd = SocketServer.TCPServer(("", PORT), Handler)

print "serving at port", PORT

httpd.serve_forever()

Monday, January 28, 13

Page 74: Introduction to Python and Web Programming

Web Service Clients

• Using HTTP to make requests of servers

• Usually requires a 3rd party library not part of the Python standard library, such as SUDS

• Client libraries like SUDS can read a WSDL file and generate what looks like a Python class with member functions which perform all communications with the server, making server communications look as simple as a regular function call

Monday, January 28, 13

Page 75: Introduction to Python and Web Programming

GUIs

• There are several options here:

• Tkinter

• PyQT

• wsPython

Monday, January 28, 13

Page 76: Introduction to Python and Web Programming

Need more?

• Look over the standard library docs at http://docs.python.org/2/library/

• PyPI - Python Package Index, http://pypi.python.org/pypi

• Appx 30,000 packages in PyPI (!)

Monday, January 28, 13

Page 77: Introduction to Python and Web Programming

Python TwitterProject

Monday, January 28, 13

Page 78: Introduction to Python and Web Programming

Python Project

• Using your browser, download the Twitter example and unzip the project on your computer from here:

• https://github.com/pythonsd/twitter-example

Monday, January 28, 13

Page 79: Introduction to Python and Web Programming

Project Problem 1

• Run the command python sample_search.py

• The response.json returns the JSON object as a Python data type. What data type is it? (e.g. , List, String, Dictionary, Tuple)

Monday, January 28, 13

Page 80: Introduction to Python and Web Programming

Project Problem 2

• Create a new output text file and write the response.json to the file

• Open up the output file in another text editor and check the result

Monday, January 28, 13

Page 81: Introduction to Python and Web Programming

Project Problem 3

• Print out the count of how many “tweets” there are in the results

• For example

• Tweet Count: n

Monday, January 28, 13

Page 82: Introduction to Python and Web Programming

Project Problem 4

• Create and print a new list of the tweet ‘text’ from the results of the search

Monday, January 28, 13

Page 83: Introduction to Python and Web Programming

Project Problem 5

• Create and print a new list of the tweet ‘from_user’ from the results of the search

Monday, January 28, 13

Page 84: Introduction to Python and Web Programming

Project Problem 6

• From the results, create and print a dictionary with the ‘from_user’ strings as the keys and the ‘text’ as the values of a Python dictionary.

Monday, January 28, 13

Page 87: Introduction to Python and Web Programming

Upcoming Events• PyLadies

• Join the PyLadies group, encourage women to learn Python

• Monthly Meeting January 24th

• Meet our regular Python developers

• Django Day February 23

• Learn to program a web app!

• November 2-3, BrightScope

Monday, January 28, 13

Page 88: Introduction to Python and Web Programming

Follow Us

• www.meetup.com/pythonsd

• www.meetup.com/sd-pyladies

• www.pythonsd.org

• groups.google.com/group/pythonsd

• @sandiegopython

Monday, January 28, 13