Subtle Accents

Design patterns in Python

Glenn Ramsey

Kiwi Pycon 2011

Who I am

Glenn Ramsey

ME (1993, U of Auckland) mechanical engineering control systems

Freelance developer
(mainly C++, now Python)

Based at Hikutaia

RowPro (digitalrowing.com) since 2001

Interest in software is primarily for modelling and simulation

PhD candidate at U of A (FEA horse's hoof, FORTRAN, Perl) thesis submitted

Hikutaia

Outline

Motivation

General software design / pattern concepts (brief)

Specific examples of Gang of Four patterns in Python

Motivation

16 of 23 [GoF] patterns have a qualitatively simpler implementation in Lisp or Dylan than in C++, for at least some uses of each pattern

Peter Norvig (http://norvig.com/design-patterns/)

Patterns are not needed in Python because design patterns are a sign of a deficiency of a language ... for the purpose that the design pattern addresses.

How is observer implemented in Python?

(equivalent to Boost.Signals, Qt Slot/Signals, .NET events)

Coming from C++ or Java, if you already know the GoF patterns then it would be informative to see how they are implemented in Python.

This talk documents part of my journey from C++ to Ptyhon.

Personal: needed an observer implementation in Python like Boost.Signal, initially couldn't find one.

My engineering background lead me to search for generic design principles in software.

Software vs Engineering design

architecturearchitectureengineeringbuildcodePhysical construction E.g. building a house

Software construction

compile

The code is the design!

Design stage outputSoftware may be cheap to build, but it is incredibly expensive to design J W Reeves, What Is Software Design?, www.developerdotstar.com

Software design

How does one design software, compared
to physical engineering design?

Data + algorithms? - only a part of the solution
Structure and Interpretation of Computer Programs. H Abelson, G Sussman, J Sussman. http://mitpress.mit.edu/sicp/full-text/book/book.html

Software Design Concepts (wikipedia)Abstraction categorize and group conceptsRefinement convert high level to program statementsModularity isolate independent featuresSoftware architecture overall structure of the softwareControl Hierarchy program structureStructural partitioning horizontal vs vertical ?Data structure logical relationship among elementsSoftware procedure an operation within a moduleInformation hiding - information contained within a module is inaccessible to others

Not especially helpful - too abstract!

Object Oriented Design principles

Open Close PrincipleSoftware entities like classes, modules and functions should be open for extension but closed for modifications.

Encapsulation information hiding

Dependency Inversion PrincipleHigh-level modules should not depend on low-level modules. Both should depend on abstractions.

Abstractions should not depend on details. Details should depend on abstractions.

Loose coupling

Interface Segregation PrincipleClients should not be forced to depend upon interfaces that they don't use.

Single Responsibility PrincipleA class should have only one job.

Liskov's Substitution PrincipleDerived types must be completely substitutable for their base types.

Prefer composition over inheritance

http://www.oodesign.com/design-principles.html

Polymorphism

What is a design
pattern?

Wikipedia: In software engineering, a design pattern is a general reusable solution to a commonly occurring problem within a given context in software design. A design pattern is not a finished design that can be transformed directly into code.

Christopher Alexander - Architect

Patterns are discovered not invented

Patterns are not independent from the programming language. Example: subroutines in assembler.

Gamma, Helm, Johnson, Vlissades (1995): Design patterns: elements of reusable object oriented software. Addison Wesley.

Pattern classes

Purpose

CreationalStructuralBehavioural

ScopeClassFactory MethodAdapter (class)InterpreterTemplate Method

ObjectAbstract factoryBuilderPrototypeSingletonAdapter (object)BridgeCompositeDecoratorFacadeFlyweightProxyChain of responsibilityCommandIteratorMediatorMementoObserverStateStrategyVisitor

Gamma, Helm, Johnson, Vlissades (1995): Design patterns: elements of reusable object oriented software. Addison Wesley.

Invisible or simplified in Python due to: first class types first class functions other

Fixed at
compile timeCan change
at runtimeObject
creationCompostion ofclasses or objectsClass and object
interactions Creational patterns: patterns that can be used to create objects.Structural patterns: patterns that can be used to combine objects and classes in order to build structured objects.Behavioral patterns: patterns that can be used to build a computation and to control data flows.

Norvig:16 of 23 patterns are either invisible or simpler, due to:First-class types (6): Abstract-Factory, Flyweight, Factory-Method, State, Proxy, Chain-Of-ResponsibilityFirst-class functions (4): Command, Strategy, Template-Method, VisitorMacros (2): Interpreter, IteratorMethod Combination (2): Mediator, ObserverMultimethods (1): BuilderModules (1): Facade

Why are they invisible/ simplified?

Some patterns are work-arounds for static typing

Python hasFirst class* types

First class* functions

An object* is first-class when it:can be stored in variables and data structures

can be passed as a parameter to a subroutine

can be returned as the result of a subroutine

can be constructed at run-time

has intrinsic identity (independent of any given name)

*The term "object" is used loosely here, not necessarily referring to objects in object-oriented programming.
The simplest scalar data types, such as integer and floating-point numbers, are nearly always first-class.Source:wikipedia

Why are they invisible/simplified? (2)

Python has duck typing

Wikipedia: In computer programming with object-oriented programming languages, duck typing is a style of dynamic typing in which an object's current set of methods and properties determines the valid semantics, rather than its inheritance from a particular class or implementation of a specific interface.

An object only has to have a method with the right name

This means that a base class is not always needed

Therefore a lot of infrastructure code can be avoided

Why are they invisible/simplified? (3)

Override special methodsAutomatic delegationMethods that a class does not know about can be passed on to a another class

When to use a class

Use a class only:if you need to inherit from it

If you need to do something special. E.g.

# Put in const.py...:class _const: class ConstError(TypeError): pass def __setattr__(self,name,value): if self.__dict__.has_key(name): raise self.ConstError, "Can't rebind const(%s)"%name self.__dict__[name]=valueimport syssys.modules[__name__]=_const()

# that's all -- now any client-code canimport const# and bind an attribute ONCE:const.magic = 23# but NOT re-bind it:const.magic = 88 # raises const.ConstError# you may also want to add the obvious __delattr__

Alex Martelli http://code.activestate.com/recipes/65207-constants-in-python/

Iterator built in

Provide a way to access the elements of an aggregate object sequentially without exposing its underlying representation

http://www.dofactory.com/Patterns/PatternIterator.aspx

class Sequence: def __init__(self, size): self.list = [x for x in xrange(size)] self.index = 0 def __iter__(self): return self def next(self): if len(self.list) == self.index: raise StopIteration current = self.list[self.index] self.index += 1 return current

>>> a = Sequence(3)>>> for x in a: print x

012>>>

Command

Encapsulate a request as an object, thereby letting you parameterize clients with different requests, queue or log requests, and support undoable operations.

Known uses: undo/redo.

OO replacement for callbacks.

Specify, queue and execute requests at different times.

http://www.cs.mcgill.ca/~hv/classes/CS400/01.hchen/doc/command/command.html

Command GoF style

Rahul Verma, Chetan Giridhar. Design Patterns in Python. www.testingperspective.com

class Command: """The Command Abstract class""" def __init__(self): pass #Make changes def execute(self): #OVERRIDE raise NotImplementedError

class FlipUpCommand(Command): """The Command class for turning on the light""" def __init__(self, light): self.__light = light def execute(self): self.__light.turnOn()

Command in Python

def greet(who): print "Hello %s" % who

greet_command = lambda: greet("World")# pass the callable around, and invoke it latergreet_command()

class MoveFileCommand(object): def __init__(self, src, dest): self.src = src self.dest = dest self() def __call__(self): os.rename(self.src, self.dest) def undo(self): os.rename(self.dest, self.src)

undo_stack = []undo_stack.append(MoveFileCommand('foo.txt', 'bar.txt'))undo_stack.append(MoveFileCommand('bar.txt', 'baz.txt'))# foo.txt is now renamed to baz.txtundo_stack.pop().undo() # Now it's bar.txtundo_stack.pop().undo() # and back to foo.txt

Simple case:Just use a callableMore complex case:Use a command object butno need for a base classhttp://stackoverflow.com/questions/1494442/general-command-pattern-and-command-dispatch-pattern-in-python (Ants Aasma)

Singleton

Ensure a class has only one instance, and provide a global point of access to it.Excessive consumption may be harmful because:it overloads your liver

makes you seem stupid

it's global

creates very strong coupling with client classes

http://en.csharp-online.net

Singleton GoF style

http://code.activestate.com/recipes/52558-the-singleton-pattern-implemented-with-python/

class Singleton: class __impl: """ Implementation of the singleton interface """ def spam(self): """ Test method, return singleton id """ return id(self) # storage for the instance reference __instance = None

def __init__(self): """ Create singleton instance """ # Check whether we already have an instance if Singleton.__instance is None: # Create and remember instance Singleton.__instance = Singleton.__impl()

# Store instance reference as the only member in the handle self.__dict__['_Singleton__instance'] = Singleton.__instance

def __getattr__(self, attr): """ Delegate access to implementation """ return getattr(self.__instance, attr)

def __setattr__(self, attr, value): return setattr(self.__instance, attr, value)

Singleton in Python

Use a module (Alex Martelli - 99% of cases)Modules are objects too

Allows you to create Fake objects for testing

Just create one instance (99% of the rest)You could assign that to a module variable

If that doesn't work also see the Borg patternShares common state among objects

Strategy

Define a family of algorithms, encapsulateeach one and make them interchangeable.

Known uses: line breaking algorithms

http://java-x.blogspot.com/2006/12/implementing-strategy-pattern-in-java.html

Strategy - statically typed

class Bisection (FindMinima): def algorithm(self,line): Return (5.5,6.6)

class ConjugateGradient (FindMinima): def algorithm(self,line): Return (3.3,4.4)

class MinimaSolver: # context class strategy='' def __init__ (self,strategy): self.strategy=strategy def minima(self,line): return self.strategy.algorithm(line) def changeAlgorithm(self, newAlgorithm): self.strategy = newAlgorithmdef test(): solver=MinimaSolver(ConjugateGradient()) print solver.minima((5.5,5.5)) solver.changeAlgorithm(Bisection()) print solver.minima((5.5,5.5))

From J Gregorio http://assets.en.oreilly.com/1/event/12/_The%20Lack%20of_%20Design%20Patterns%20in%20Python%20Presentation.pdf

Strategy in Python

def bisection(line): Return 5.5, 6.6

def conjugate_gradient(line): Return 3.3, 4.4

def test(): solver = conjugate_gradient print solver((5.5,5.5)) solver = bisection print solver((5.5,5.5))

From J Gregorio http://assets.en.oreilly.com/1/event/12/_The%20Lack%20of_%20Design%20Patterns%20in%20Python%20Presentation.pdf

Invisible because Python has first class functions

Observer

Define a one to many dependency between objects so that when one object changes state, all its dependents are notified and updated automatically

Known uses: model view controller, Qt Signals/Slots, Boost.Signals, most GUI toolkits

http://en.wikipedia.org/wiki/File:Observer.svg

Observer GoF style

class Observer(): def update(self): raise NotImplementedError

class ConcreteObserver(Observer): def __init__(self, subject): self.subject = subject def update(self): data = self.subject.data # do something with data

class Subject(): def __init__(self): self.observers = [] def attach(self, observer): self.observers.append(observer)

def detach(self, observer): self.observers.remove(observer) def notify(self): for o in self.observers: o.update()

class ConcreteSubject(Subject): def __init__(self, data): self.data = data def do_something(self): self.data += 1 self.notify()

Issues Deleted observers

Detach during notify()

Pass parameters to the observer

e.g. http://code.activestate.com/recipes/131499-observer-pattern/

Observer in Python

Simplified - Observer base class is not required use a callableE.g. PyDispatcher (http://pydispatcher.sourceforge.net)Use composition instead of inheritance

from pydispatch import dispatcher

# define the observer functiondef something_was_updated(data, signal, sender): print "data:", data, "signal:", signal, "sender:",sender

class A_Model(): def __init__(self): self.data = 100 # an object to identify the signal self.data_changed= "data"

def do_something(self): self.data *= 1.34 #args are: signal, sender, args dispatcher.send(self.data_changed, self, self.data)

# create an object to be an identifier for a signalsender = A_Model()#args are: receiver, signal, senderdispatcher.connect(something_was_updated, sender.data_changed, sender)sender.do_something()

Decorator

Attach additional responsibilities or functions to an object dynamically. Decorators provide a flexible alternative to subclassing for extending functionality.

Not the same as Python decorators

Objects enclose other objects that share similar interfaces. The decorating object appears to mask or modify or annotate the enclosed object.

http://en.wikipedia.org/wiki/File:Decorator_UML_class_diagram.svg

Decorator GoF style

class Writer(object): def write(self, s): print s class WriterDecorator(object): def __init__(self, wrappee): self.wrappee = wrappee

def write(self, s): self.wrappee.write(s) class UpperWriter(WriterDecorator): def write(self, s): self.wrappee.write(s.upper()) class ShoutWriter(WriterDecorator): def write(self, s): self.wrappee.write('!'.join( \[t for t in s.split(' ') if t]) + '!')

Magnus Therning http://therning.org/magnus/archives/301

w = Writer()w.write('hello')

uw = UpperWriter(w)uw.write('hello')

wd = WriterDecorator(w)wd.write('hello')

sw1 = ShoutWriter(w)sw1.write('hello again')

sw2 = ShoutWriter(uw)sw2.write('hello again')

>>>helloHELLOhellohello!again!HELLO!AGAIN!

Decorator using function decorators

def uppercase(f): def wrapper(*args, **kwargs): orig = f(*args, **kwargs) return orig.upper() return wrapper

def shout(f): def wrapper(*args, **kwargs): orig = f(*args, **kwargs) return '!'.join( \[t for t in orig.split(' ') if t]\) + '!' return wrapper

@shoutdef s_writer(s): return s

print s_writer("hello again") @shout@uppercasedef su_writer(s): return s

print su_writer("hello again")

>>>HELLO AGAINHELLO!AGAIN!

Decorator using delegation

import string

class UpSeq:

def __init__(self, seqobj): self.seqobj = seqobj

def __str__(self): return string.upper(self.seqobj.seq)

def __getattr__(self,attr): return getattr(self.seqobj, attr)

class DNA(): def __init__(self, name, seq): self.seq = seq self.name = name def __getitem__(self, item): return self.seq.__getitem__(item)

def first(self): return self.seq[0]

s=UpSeq(DNA(name='1', seq='atcgctgtc'))

>>>print sATCGCTGTC>>>print s[0:3]atc>>>print s.first()a

Adapted from http://www.pasteur.fr/formation/infobio/python/

UpSeq delegates to DNA

State

Allow an object to alter its behaviour when its internal state changes. The object will appear to change its class

Very common

Known uses: TCP, GUIs

http://en.wikipedia.org/wiki/File:State_Design_Pattern_UML_Class_Diagram.svg

State GoF style

class State(object): """Base state. This is to share functionality"""

def scan(self): """Scan the dial to the next station""" self.pos += 1 if self.pos == len(self.stations): self.pos = 0 print "Scanning Station is", \self.stations[self.pos], self.name

class AmState(State): def __init__(self, radio): self.radio = radio self.stations = ["1250", "1380", "1510"] self.pos = 0 self.name = "AM"

def toggle_amfm(self): print "Switching to FM" self.radio.state = self.radio.fmstate

class FmState(State): def __init__(self, radio): self.radio = radio self.stations = ["81.3", "89.1", "103.9"] self.pos = 0 self.name = "FM"

def toggle_amfm(self): print "Switching to AM" self.radio.state = self.radio.amstate

class Radio(object): """A radio. It has a scan button, and an AM/FM toggle switch.""" def __init__(self): """We have an AM state and an FM state""" self.amstate = AmState(self) self.fmstate = FmState(self) self.state = self.amstate def toggle_amfm(self): self.state.toggle_amfm() def scan(self): self.state.scan()

Note lack of state methodsAbstract stateContextConcrete states# Test radio = Radio()actions = [radio.scan] * 2 + [radio.toggle_amfm] + [radio.scan] * 2actions = actions * 2for action in actions: action()

Jeff ? http://ginstrom.com/scribbles/2007/10/08/design-patterns-python-style/

Scanning... Station is 1380 AMScanning... Station is 1510 AMSwitching to FMScanning... Station is 89.1 FMScanning... Station is 103.9 FMScanning... Station is 81.3 FMScanning... Station is 89.1 FMSwitching to AMScanning... Station is 1250 AMScanning... Station is 1380 AM

State in Python

Switch classes or methods

From: Alex Martelli http://www.aleax.it/goo_pydp.pdf

class RingBuffer(object): def __init__(self): self.d = list() def tolist(self): return list(self.d) def append(self, item): self.d.append(item) if len(self.d) == MAX: self.c = 0 self.__class__ = _FullBuffer

class _FullBuffer(object): def append(self, item): self.d[self.c] = item self.c = (1+self.c) % MAX def tolist(self): return ( self.d[self.c:] + self.d[:self.c] )

Irreversible
state changeInitial state. Use this untilthe buffer gets full.Method change
implementationclass RingBuffer(object): def __init__(self): self.d = list() def append(self, item): self.d.append(item) if len(self.d) == MAX: self.c = 0 self.append = self.append_full def append_full(self, item): self.d.append(item) self.d.pop(0) def tolist(self): return list(self.d)

state change

State in Python (method)

class Sequencer(): def __init__(self): self._action_impl = self.action1 self.count = 1 def action(self): self._action_impl() def next(self): self.count += 1 if self.count > 3: self.count = 1 self._action_impl = \ getattr(self, "action"+str(self.count)) def action1(self): print "1" def action2(self): print "2" def action3(self): print "3"

s = Sequencer() actions = [s.action] + [s.next] actions = actions * 3 for f in actions: f()

Switch methods>>>123>>>

outputUse Bridge so that the binding ofSequencer.action doesn't change

State in Python (class)

>>>First 1Second 2Third 3First 4First 1Second 2Second 3

outputclass Base(): def __init__(self): self.state = 0 def action(self): self.state += 1 print self.__class__.__name__, self.state def change_state(self, next_class): self.__class__ = next_class class Third(Base): def transition(self): self.change_state( First ) class Second(Base): def transition(self): self.change_state( Third )

class First(Base): def transition(self): self.change_state( Second )

state = First()state.action()state.transition()state.action()state.transition()state.action()state.transition()state.action()

state = First()actions = [state.action] + [state.transition]actions = actions * 3

for action in actions: action()

This doesn't work because
state is always First

Bridge

Decouple an abstraction from its implementation so that the two can vary independently

Similar to strategy but isn't simplified in the same way

Strategy is behavioural interchange algorithms

Bridge is structural implementation varies independently from abstraction

C++ pimpl

http://atlas.kennesaw.edu/~dbraun/csis4650/A&D/GoF_Patterns

Factory method

Define an interface for creating an object, but let subclasses decide which class to instantiate. This method lets a class defer instantiation to subclasses

http://en.wikipedia.org/wiki/File:FactoryMethod.svg

Factory Method GOF style

class Person: def __init__(self): self.name = None self.gender = None def getName(self): return self.name def getGender(self): return self.gender class Male(Person): def __init__(self, name): print "Hello Mr." + name class Female(Person): def __init__(self, name): print "Hello Miss." + name class Factory: def getPerson(self, name, gender): if gender == 'M': return Male(name) if gender == 'F': return Female(name)

From: dpip.testingperspective.com

factory = Factory()person = factory.getPerson("Chetan", "M")person = factory.getPerson("Money", "F")

>>>Hello Mr.ChetanHello Miss.Money

Factory method in Python

class Male(object): def __init__(self, name): print "Hello Mr." + name

class Female(object): def __init__(self, name): print "Hello Ms." + name

factory = dict(F=Female, M=Male)

if __name__ == '__main__': person = factory["F"]("Money") person = factory["M"]("Powers")

>>>Hello Ms.MoneyHello Mr.Powers

Adapted from: http://www.rmi.net/~lutz/talk.html

# variable length arg listsdef factory(aClass, *args, **kwargs): return aClass(*args, **kwargs)

class Spam: def __init__(self): print self.__class__.__name__ def doit(self, message): print message

class Person: def __init__(self, name, job): self.name = name self.job = job print self.__class__.__name__, name, job

object1 = factory(Spam)object2 = factory(Person, "Guido", "guru")

>>>SpamPerson Guido guru

Abstract factory

Provide an interface for creating families of related or dependent objects without specifying their concrete classes

http://en.wikipedia.org/wiki/File:Abstract_factory.svg

Abstract Factory GoF style

class PetShop: def __init__(self, animal_factory=None): """pet_factory is our abstract factory. We can set it at will."""

self.pet_factory = animal_factory

def show_pet(self): """Creates and shows a pet using the abstract factory"""

pet = self.pet_factory.get_pet() print "This is a lovely", pet print "It says", pet.speak() print "It eats", self.pet_factory.get_food()

class Dog: def speak(self): return "woof" def __str__(self): return "Dog"

class Cat: def speak(self): return "meow" def __str__(self): return "Cat"

http://ginstrom.com/scribbles/2007/10/08/design-patterns-python-style/

class DogFactory: def get_pet(self): return Dog() def get_food(self): return "dog food"

class CatFactory: def get_pet(self): return Cat() def get_food(self): return "cat food"

# Create the proper familydef get_factory(): return random.choice([DogFactory, CatFactory])()

# Show pets with various factoriesshop = PetShop()for i in range(3): shop.pet_factory = get_factory() shop.show_pet() print "=" * 10

>>>This is a lovely DogIt says woofIt eats dog food==========This is a lovely CatIt says meowIt eats cat food==========This is a lovely DogIt says woofIt eats dog food==========

Abstract factory in Python

Use a module for example similar to the os module

# cat.pyclass Animal(): def __init__(self): print "Cat"

def speak(): return "meow"

class Food: def acquire(self): print "go hunting" def serve(self): print "eat your catch, leave " \ "the entrails on the doorstep"

#dog.pyclass Animal(): def __init__(self): print "Dog" def speak(): return "woof"

class Food: def acquire(self): print "wait for cat to catch" def serve(self): print "eat what the cat left"

import dog as factory_1import cat as factory_2

make_animal(factory_1)identify(factory_1)feed(factory_1)

make_animal(factory_2)identify(factory_2)feed(factory_2)

def make_animal(factory): return factory.Animal()

def identify(factory): print factory.speak() def feed(factory): food = factory.Food() food.acquire() food.serve()

If inheritance
is not needed

Flyweight

Use sharing to support large numbers of fine grained objects efficiently

http://www.lepus.org.uk/ref/companion/Flyweight.xml

Flyweight in Python

http://codesnipers.com/?q=python-flyweights

import weakref

class Card(object): _CardPool = weakref.WeakValueDictionary()

def __new__(cls, value, suit): obj = Card._CardPool.get(value + suit, None) if not obj: obj = object.__new__(cls) Card._CardPool[value + suit] = obj obj.value, obj.suit = value, suit

return obj c1 = Card('9', 'h')c2 = Card('9', 'h')c3 = Card('2', 's')

print c1 == c2, c1 == c3, c2 == c3print id(c1), id(c2), id(c3)

>>>True False False38958096 38958096 38958128

Visible Patterns

Covered by Alex MartelliFacade

Template method

Adapter

OthersMediator

Bridge

Composite

Memento

Summary

Patterns are simplified because Python has:First class objects and types

Duck typing base classes may be optional

Ability to override special methods

References

Alex Martelli, Design Patterns in Python, Google TechTalks, March 14, 2007. (find them on youtube)

Joe Gregorio The (lack of) design patterns in Python, Pycon 2009 (http://bitworking.org/news/428/the-lack-of-design-patterns-in-python-pycon-2009)

Peter Norvig - Design Patterns in Dynamic Programming, Object World, 1996.http://norvig.com/design-patterns/ppframe.htm

Alan Shalloway, James Trott. Design Patterns Explained.

Thank you

http://dl.dropbox.com/u/10287301/Ramsey-KiwiPycon-2011.pdf