new-style classes thomas wouters xs4all [email protected] yhg1s @ #python
TRANSCRIPT
Overview
• Old situation– types, classes– shape of Python
• Type unification– subtyping– descriptors and properties– class- and static-methods
• Metaclasses
Old situation: types
• Python types implemented in C– C structs represent types– C function pointers define functionality– No real class data– C structs represent objects– All instance data in object structs– 'Manual' attribute retrieval– Explicit methods
Old situation: Python classes
• Implemented in terms of Python types– "bolted on top"– 'classes' and 'instances' as distinct types– C function pointers delegated to 'magic'
methods to define behaviour– tp_getattro function searches base classes– Implicit methods from Python functions– Class and instance data in '__dict__' attribute
Old situation: limitations
• Python and C as two distinct worlds
• No real accessors
• No class/static methods
• No immutable classes
• Simplistic inheritance order
• Less control over behaviour
Old situation: consequences
• Interface-based functionality– informal interfaces rather than inheritance
• Containment rather than inheritance– inheritance is not always the answer
• Simple inheritance trees
• Easy to use
Type unification
• Allow (better) mingling of Python and C types (or classes)– bring C and Python closer
• __dict__ and inheritance for C types
• descriptors, properties, class/staticmethods for Python classes
– subclassing C types in C
– subclassing C types in Python
• Correct 'warts' in classic classes• Generalize special cases
Type unification (2)
• Metaclasses: a new type of type• Explicit base class: object
– container of generic functionality
• Old-style classes for compatibility– hidden metaclass
• Translation from C function-pointers to Python __methods__ (slots)
• __dict__ for C types and __slots__ for Python classes
Subclassing
• Subclass C types from Python in the expected manner:
class mylist(list): def __getitem__(self, i): try: return list.__getitem__(self, i) except IndexError: return None
• Resricted multiple inheritance• Not always a good idea!
Subclassing (2)
• __new__, Python's constructor
– called to construct (allocate) the object
– static method, called with class as first argument
– may return an existing value
• __slots__: store data almost like C would
– no __dict__, less memory consumption
Immutable Python typesclass tristate(int):
__slots__ = []
nstates = 3
def __new__(cls, state=0):
state %= cls.nstates
return int.__new__(cls, state)
def __add__(self, o):
return tristate(int.__add__(self, o))
def __sub__(self, other):
return self.__add__(-other)
Subclassing C types in C
• Make sure base type supports subclassing– Py<type>_Check(), Py<type>_CheckExact()– PyMethodDef, PyMemberDef, PyGetSetDef– PyObject_GenericGetAttr as tp_getattro– no type-object hardcoding
• Subclass's PyType object– leave unchanged behaviour up to base type– set tp_base to base class– call base class's tp_init
• Provide compatible object struct
Type checking
• Type-checking is a necessary evil (in C)• PyObject_TypeCheck() for inheritance-aware
PythonC type check• Define Py<type>_Check() in terms of
PyObject_TypeCheck()• Define Py<type>_CheckExact() as type-pointer
comparison• Use Py<type>_CheckExact() for internal
optimizations
PyList_Check*()
#define PyList_Check(op) \
PyObject_TypeCheck(op, \
&PyList_Type)
#define PyList_CheckExact(op)\
((op)->ob_type == \
&PyList_Type)
Py*Def
• Allow subclasses to extend/override parts • PyMemberDef (tp_members) for instance data
– maps C structs to Python attributes– tp_members in type struct
• PyMethodDef (tp_methods) for all methods– wraps C functions in Python objects– specifies argument style and type of method
• PyGetSetDef (tp_getset) for accessors– maps functions to attributes and vice versa
Subclass struct
• Include base class struct in subclass struct typedef struct {
PyListObject list;
PyObject * default;
} defaultlistobject;
• No changes to original memory layout• Multiple inheritance is only possible with
'compatible memory layouts'• C subclasses subclassable in Python
Method Resolution Order
• Old MRO not suited to complex inheritance trees– base classes get queried before some of their
derived classes– base classes get queried multiple times– No convenient way to access base classes
• hardcode base class names
• guess about attributes / methods
New MRO
• Published algorithm: C3– http://www.webcom.com/haahr/dylan/linearizat
ion-oopsla96.html
• Relatively easy to explain– same order as before– eliminates all but the last occurance of the same
class
• Same order for simple inheritance
A
D(B, C)
C(A)B(A)
E(C, B)
F(D, E)
Old-style MRO: F, D, B, A, C, A, E, C, A, B, ANew-style MRO (2.2): F, D, E, B, C, A
super()
• Proxy'object for accessing 'base' classes
• Continues MRO where it left off– requires current class and (derived) instance
• Somewhat inconvenient to use
• Very important for consistency
• Use it anyway
super() use
class BStore(Storage):
def __init__(self, state):
Storage.__init__(self, state)
class BStore(Storage):
def __init__(self, state):
super(BStore, self).__init__(state)
Descriptors
• Generalization of class-getattr magic and C tp_getattr tricks
• Trigger functioncalls when retrieved or stored from an object (getattr/setattr)– __get__()– __set__()– __delete__()
Properties
• Accessors for Python• An application of descriptors• class R(object): def _get_random(self): return random.random() random = property(_get_random)• Also hold docstrings for attributes• 'set' and 'delete' functions don't work with old-
style classes
Caching Property
class cachingprop(object):
__slots__ = ["_name", "_fget"]
def __init__(self, name, fget):
self._name = name
self._fget = fget
def __get__(self, inst, type=None):
if inst is None:
return self
v = self._fget(inst)
inst.__dict__[self._name] = v
return v
Special method types
• classmethods– Passes class as implicit first argument– can be called through class or through
instance– allow for factory functions (or 'alternate
initializers') that create subclasses• dict.fromkeys• tarfile.TarFile.open
Special Method Types (2)
• staticmethods– Passes no special arguments
– Necessary for object.__new__ (or is it?)
– Allows for regular (non-method) Python functions as attributes
Special Method Types (3)
class Buffer(object):
def __init__(self, data):
self.data = data[:]
def fromstring(cls, s):
return cls(s.splitlines())
fromstring = classmethod(fromstring)
def send(self):
self._extern_send(self.data)
_extern_send = staticmethod(sendmodule.send)
Metaclasses
• The class of class
• Usually derives from type
• Relate to classes like classes relate to instances
• Define class behaviour
• Allow for convenient post-processing of classes
Class/instance relation
• Creating the instance passes the contents (arguments) to the class __init__:
class Send(object):
def __init__(self, what, who):
...
Send("my data", him)
Metaclass/class relation
class Meta(type):
def __init__(self, name, bases, attrs):
type.__init__(self, name, bases, attrs)
class Impl(base1, base2):
__metaclass__ = Meta
X = 1
def method(self, it):
return not it
stat = staticmethod(...)
Metaclasses• Behave like classes:
– __new__ called for class creation– __init__ called for class initialization– inheritance
• Mixing metaclasses requires compatibility– derived classes must have same or derived
metaclasses– metametaclasses can automatically derive
metaclasses