on reflection in oo programming languages

Yann-Gaël Guéhéneuc

This work is licensed under a Creative Commons Attribution-NonCommercial-

ShareAlike 3.0 Unported LicenseDépartement de génie informatique et de génie logiciel

On Reflection in OO Programming Languages

[email protected] 1.5.22014/04/14

mailto:[email protected]

2/169

Any questions/comments are welcome [email protected] code available at http://www.ptidej.net/tutorial/javareflection

mailto:[email protected]

http://www.ptidej.net/tutorial/javareflection

3/169

Outline

Rationale Definition Context

– Interconnections Scenarios Theory Conclusion Class Loading

4/169

Outline



5/169

Use of instanceofpublic abstract class Animal {}public class Cat extends Animal {}public class Dog extends Animal {}public class Expression {

public static String speak(final Animal anAnimal) {final String sound;if (anAnimal instanceof Dog) {

sound = "woof";}else if (anAnimal instanceof Cat) {

sound = "miaow";}else {

sound = "";}return sound;

}public static void main(final String[] args) {

final Dog fido = new Dog();System.out.println(Expression.speak(fido));

}}

Must test the type ofthe object at run-time

Breaks informationhiding of Cat and Dog

Requires special case

6/169

Use of instanceof

Reason– Must bypass the compiler and virtual machine’s

choice of the method to call

– Remember that overloading is resolved by the static type of the argument, not its run-time type (cf. Week 2)

7/169

Use of instanceof

Problem

“Don't repeat yourself” (DRY): “Every piece of knowledge must have a single, unambiguous, authoritative representation within a system.”

—Andy Hunt and Dave Thomas(in The Pragmatic Programmer)

8/169

Use of instanceofpublic abstract class Animal {

public abstract String speak();}public class Cat extends Animal {

@Overridepublic String speak() {

return "miaow";}

}public class Dog extends Animal {


return "woof";}

}public class Expression {

public static void main(final String[] args) {final Dog fido = new Dog();System.out.println(fido.speak());

}}

9/169

Use of instanceofpublic class Cat {

public String speak() {return "miaow";

}}public class Dog {

public String speak() {return "woof";

}}public class Expression {

public static void main(final String[] args) throwsIllegalAccessException,IllegalArgumentException,InvocationTargetException {

final Dog fido = new Dog();final Method[] method = Dog.class.getDeclaredMethods();System.out.println(method[0].invoke(fido, new Object[0]));

}}

Discover method ofobject of unrelatedclasses

10/169

Plugins

Plugins are external, user-defined pieces of code loaded, used, and unloaded to benefit from their features only when needed

Problem– Closed-world assumption (at compile-time and

runtime) leads to the need for a mechanism to perform dynamic loading

11/169

Serialisation

Serialisation is the process of translating data structures or object state into a format that can be stored (file, network…)

Problem– Information hiding prevents access to the

internals of the data structures and to the complete object states

12/169

Debugging/Profiling

Debugging/profiling are the processes of monitoring the execution of a program – To understand its behaviour– To measure its space and time complexity

Problem– Accessing the internals of objects and

controlling the program to collect relevant data

13/169

Patching

Patching is the process of changing the behaviour of a class or an object– To modify its behaviour– To add new behaviour

Problem– Modifying the behaviour at run-time without the

need to recompile and restart the program

14/169

Outline



15/169

Definition

Reflection is the ability of a computer program to examine and modify the structure and behaviour of an object at runtime

Problem: inspect objects and change their behaviour at runtimeSolution: reflection

16/169




return "miaow";}



return "woof";}


public static void main(final String[] args) {final Dog fido = new Dog();System.out.println(fido.speak());

}}

17/169



@Overridepublic String miaow() {

return "miaow";}


@Overridepublic String bark() {

return "woof";}


public static void main(final String[] args) {final Dog fido = new Dog();final Method[] method = Dog.class.getDeclaredMethods();System.out.println(method[0].invoke(fido, new Object[0]));

}}

Unrelated classes

Choice at run-time

18/169

Outline

Problems Definition Context


19/169

Context

Reflection– Ability of a computer program to examine and

modify the structure and behaviour of an object at runtime

20/169

Context

Reflection– Theoretical interest

• What is an interpreter?Brian Cantwell Smith “Procedural Reflection in Programming Languages” (Ph.D. thesis, 1982)

– Practical interest• Libraries• Frameworks

21/169

Context




22/169

Context




23/169

Context

Libraries– Collections of data structures and algorithms

(possibly encapsulated in classes) used to develop and run programs

Frameworks– Reusable sets of libraries, tools, and

conventions used to develop and run programs

Problem: connection between clients and libraries/frameworksSolution: linking, forking, subclassing, IPC, and dynamic loading

24/169

Interconnections

Clients–Libraries/Frameworks– Linking– Forking– Subclassing– Inter-process communication– Dynamic loading/invoking

25/169

Interconnections

Linking(Contrast with virtual machines)

– Typically C/C++

– Several object files (.o)– One executable (.exe)

26/169

Interconnections

Forking– Typical in most

languages– Process duplication

• Is a real duplication• Creates a new OS

process

final StringBuffer commandLine = new StringBuffer();

commandLine.append("..\\DOT\\bin\\dotty ");

commandLine.append(aFilePath);

final Process process =

Runtime.getRuntime().exec(commandLine.toString());

final OutputMonitor errorStreamMonitor =

new OutputMonitor(...,process.getErrorStream(),...);

errorStreamMonitor.start();

final OutputMonitor inputStreamMonitor =

new OutputMonitor(...,process.getInputStream(),...);

inputStreamMonitor.start();

try {

process.waitFor();

}

catch (final InterruptedException ie) {

ie.printStackTrace(

Output.getInstance().errorOutput());

}

if (process.exitValue() != 0) {

...

}

27/169

Interconnections

IPC– Typical in most languages– Use remote procedure calls– Use well-defined protocols

• COM• CORBA• XML-RPC

– Web services

28/169

Interconnectionspackage kr.ac.yonsei.cse3009.rpc;

import java.net.URL;import org.apache.xmlrpc.client.XmlRpcClient;import org.apache.xmlrpc.client.XmlRpcClientConfigImpl;

public class Client {public static void main(final String[] args)

throws Exception {

final XmlRpcClientConfigImpl config = new XmlRpcClientConfigImpl();

config.setServerURL(new URL("http://127.0.0.1:8080/xmlrpc"));

final XmlRpcClient client = new XmlRpcClient();client.setConfig(config);

final Object[] params = new Object[] {new Integer(33), new Integer(9) };

final Integer result = (Integer) client.execute("Calculator.add", params);

System.out.println(result);}

}

package kr.ac.yonsei.cse3009.rpc;

import org.apache.xmlrpc.server.PropertyHandlerMapping;import org.apache.xmlrpc.server.XmlRpcServer;import org.apache.xmlrpc.webserver.WebServer;

public class Server {public static void main(final String[] args)

throws Exception {

final WebServer webServer = new WebServer(8080);final XmlRpcServer xmlRpcServer =

webServer.getXmlRpcServer();final PropertyHandlerMapping phm =

new PropertyHandlerMapping();phm.addHandler("Calculator", Calculator.class);xmlRpcServer.setHandlerMapping(phm);

webServer.start();}

}

package kr.ac.yonsei.cse3009.rpc;

public class Calculator {public int add(final int i1, final int i2) {

return i1 + i2;}public int sub(final int i1, final int i2) {

return i1 - i2;}

}

http://127.0.0.1:8080/xmlrpc

29/169

Interconnections

Subclassing– Typical in most object-

oriented languages– Structural, static relation

public class OutputMonitor extends Thread {

...

public OutputMonitor(...) {

this.setName(threadName);

this.setPriority(Thread.MAX_PRIORITY);

...

}

public void run() {

try {

int value = 0;

byte[] bytes;

char lastWrittenChar;

while ((value = this.inputStream.read()) > 0) {

synchronized (System.err) {

synchronized (System.out) {

if (value != 13 && value != 10) {

lastWrittenChar = (char) value;

...

}}

}

}

catch (final IOException ioe) {

...

30/169

Interconnections

Subclassing– Hooks and templates

• Hot spots = hooks• Frozen spots = templates

– Hooks are typically abstract methods– Templates typically use hooks

31/169

Interconnections

Subclassing– Hooks and templates

• JUnit

32/169

Interconnections

Template

Hooks

public abstract class TestCase

extends Assert implements Test {

public void runBare() throws Throwable {

setUp();

try {

runTest();

}

finally {

tearDown();

}

}

protected void setUp() throws Exception {

}

protected void tearDown() throws Exception {

}

...

}

33/169

Interconnections

Subclassing– Heavily used in object-oriented programs– Heavily used in design patterns

(Only Singleton does not explicitly use subclassing)• Abstract Factory• Composite• Decorator• Observer• Visitor

34/169

Interconnections

Dynamic loading– In different programming languages (but not all),

it is the possibility to load, use, and unload a piece of code at runtime

– In Java, it is the possibility to load and unload a class and to choose and invoke its methods(and to access its fields…) at runtime

35/169

Interconnections

Poor man’s profiler– Calls the main() method of a class whose name is

given at runtime, by the user– Displays the time spent in the called method

public final class PoorManProfiler {

public static void main(final String[] args) {

try {

final Class toBeRun = Class.forName(args[0]);

final Method mainMethod =

toBeRun.getMethod("main", new Class[] { String[].class });

final long startTime = System.currentTimeMillis();

mainMethod.invoke(null, new Object[] { new String[0] });

final long endTime = System.currentTimeMillis();

System.out.println();

System.out.println(endTime - startTime);

}

catch (final Exception e) {

e.printStackTrace();

}

}

}

36/169

Return to Reflection

Reflection enables dynamic loading– Reflection dynamic loading– Dynamic loading Reflection

• Think of dynamically-loaded shared libraries in C-based operating systems, such as AmigaOS, Linux, UN*X, Windows…

37/169

Outline



38/169

Scenarios

Original problem

– Different interpretations– Different contexts

Four scenarios Three programming languages

Problem: inspect objects and change their behaviour at runtimeSolution: reflection

39/169

Scenarios

Scenario 1: invoke an arbitrary method on an object (see the problems with instanceof and plugins)

– Given a class C– Given an object o, instance of C– Identify all the methods available on o– Invoke a method using its name foo

40/169

Scenarios

Scenario 2: access the complete (including private) state of an object (see the problem with serialisation)

– Given an object o, instance of C– Save on disk the complete state of o– Restore from disk the object o at a later time

41/169

Scenarios

Scenario 3: count the number of instances of a class created at runtime (see the problem with debugging/profiling)

– Given a class C– Record the number of its instances ever created– Report this number when the program ends

42/169

Scenarios

Scenario 4: patch the method of a class to change its behaviour (see the problem with patching)

– Given an object o, instance of C– Without changing o or C– Change the behaviour of o.foo()

43/169

Scenarios

Java

Smalltalk

C++

44/169

Scenarios

Scenario 1:


public class C {private int i;public C(final int anInt) {

this.i = anInt;}public void foo(final String s) {

System.out.print(s);System.out.println(this.i);

}}

45/169

Scenarios

Scenario 1:


Identify all the methods available on ofoogetClasshashCodeequalstoStringnotifynotifyAllwaitwaitwait

Invoke a method using its name fooThis is foo: 42

46/169

Scenarios

Scenario 1:


final C o = new C(42);

System.out.println("Identify all the methods available on o");final Class<?> classOfO = o.getClass();final Method[] methodsOfC = classOfO.getMethods();for (int i = 0; i < methodsOfC.length; i++) {

final Method method = methodsOfC[i];System.out.print('\t');System.out.println(method.getName());

}

System.out.println("Invoke a method using its name foo");final Method fooMethod = classOfO.getMethod("foo", new Class[] { String.class });fooMethod.invoke(o, new Object[] { "\tThis is foo: " });

47/169

Scenarios

Scenario 1:


Identify all the methods available on ofoogetClasshashCodeequalstoStringnotifynotifyAllwaitwaitwait


48/169

Scenarios

Scenario 2:– Given an object o, instance of C– Save on disk the complete state of o– Restore from disk the object o at a later time

public class C {private int i;public C() {

this(-1);}public C(final int anInt) {



}}

49/169

Scenarios


Save on disk the complete state of oi = 42

Restore from disk the object o at a later timeThis is foo on o2: 43

50/169

Scenarios


final C o1 = new C(42);

System.out.println("Save on disk the complete state of o");final Class<?> classOfO = o1.getClass();final Field[] fieldsOfC = classOfO.getDeclaredFields();for (int i = 0; i < fieldsOfC.length; i++) {

final Field field = fieldsOfC[i];field.setAccessible(true);System.out.print('\t' + field.getName());System.out.println(" = " + field.getInt(o1));

}

System.out.println("Restore from disk the object o at a later time");final C o2 = (C) classOfO.newInstance();final Field fieldiOfC = classOfO.getDeclaredField("i");fieldiOfC.setAccessible(true);fieldiOfC.setInt(o2, 43);o2.foo("\tThis is foo on o2: ");

51/169

Scenarios


Save on disk the complete state of oi = 42


52/169

Scenarios

Scenario 3:– Given a class C– Record the numbers of its instance ever created– Report this number when the program ends

public class C {private static int numberOfInstances;public static int getNumberOfInstances() {

return C.numberOfInstances;}

private int i;public C() {

this(-1);}public C(final int anInt) {

C.numberOfInstances++;this.i = anInt;

}public void foo(final String s) {


}}

53/169

Scenarios

Scenario 3:

kr.ac.yonsei.it.cse3009.reflection.scenario3.C@150bd4dkr.ac.yonsei.it.cse3009.reflection.scenario3.C@1bc4459kr.ac.yonsei.it.cse3009.reflection.scenario3.C@12b66513

– Given a class C– Record the numbers of its instance ever created– Report this number when the program ends

54/169

Scenarios

Scenario 3:

public class Main {public static void main(final String[] args) {

final C o1 = new C(42);final C o2 = new C(1);final C o3 = new C(100);System.out.println(o1 + "\n" + o2 + '\n' + o3);

System.out.println(C.getNumberOfInstances());}

}


55/169

Scenarios

Scenario 3:

kr.ac.yonsei.it.cse3009.reflection.scenario3.C@150bd4dkr.ac.yonsei.it.cse3009.reflection.scenario3.C@1bc4459kr.ac.yonsei.it.cse3009.reflection.scenario3.C@12b66513

To fulfill this scenario, we must modify C


56/169

Scenarios

Class Class is a metaclass– It describes other classes

Class Method is a class– It describes methods

Class Field is a class– It describes fields

57/169

Scenarios

Class Class is a metaclass– It allows to reify classes

Class Method is a class– It allows to reify methods, these language

constructs become first-class entities Class Field is a class

– It allows to reify fields , these language constructs become first-class entities

58/169

Scenarios

Class Class is a metaclass– It allows to reify classes

Class Method is a class– It allows to reify methods, these language

constructs become first-class entities Class Field is a class

– It allows to reify fields, these language constructs become first-class entities

Reification is the process of making available an implicit construct of a language

59/169

Scenarios

We never modified class Class– We only used it to obtain the methods and fields

declared by particular classes, e.g., C

We used static fields and methods to count number of instances

60/169

Scenarios

Scenario 4:

public interface I {public void foo(final String s);

}

public class C implements I {private int i;public C(final int anInt) {



}}


61/169

Scenarios

Scenario 4:

This is o: 42Forwarding method: "foo"

Original arguments: "This is o: "New arguments: "THIS IS O: "

THIS IS O: 42


62/169

Scenarios

Scenario 4:

final I o = new C(42);o.foo("This is o: ");

final InvocationHandler handler = new InvokationHandler(o);final I proxy =

(I) Proxy.newProxyInstance(I.class.getClassLoader(),new Class[] { I.class },handler);

Assert.assertTrue(proxy instanceof I);Assert.assertFalse(proxy instanceof C);Assert.assertTrue(proxy.equals(o));

proxy.foo("This is o: ");


63/169

Scen

ario

s

Sce

nario

4:

public class InvokationHandler implements InvocationHandler {private final I i;public InvokationHandler(final I aConcreteImplementationOfI) {

this.i = aConcreteImplementationOfI;}public Object invoke(

final Object aProxy,final Method aMethod,final Object[] someArgs) throws Throwable {

if (aMethod.getName().equals("foo")) {final String arg = (String) someArgs[0];final String newArg = arg.toUpperCase();

System.out.print("Forwarding method: \"");System.out.print(aMethod.getName());System.out.print("\"\n\tOriginal arguments: \"");System.out.print(arg);System.out.print("\"\n\tNew arguments: \"");System.out.print(newArg);System.out.println('\"');

this.i.foo(newArg);return null;

}else {

return aMethod.invoke(this.i, someArgs);}

}}

–G

iven

an

obje

ct o

, ins

tanc

e of

C–

With

out c

hang

ing o

or C

–C

hang

e th

e be

havi

ouro

f o.foo()

64/169

Scenarios

Scenario 4:

final I o = new C(42);o.foo("This is o: ");

final InvocationHandler handler = new InvokationHandler(o);final I proxy =

(I) Proxy.newProxyInstance(I.class.getClassLoader(),new Class[] { I.class },handler);

Assert.assertTrue(proxy instanceof I);Assert.assertFalse(proxy instanceof C);Assert.assertTrue(proxy.equals(o));

proxy.foo("This is o: ");

Properties of the proxy ensured by the Java VM


65/169

Scenarios

Smalltalk– Everything is an object– Everything is an object, really

static

66/169

Scenarios

Smalltalk– SmalltalkImage is the class that “represent[s]

the current image and runtime environment, including system organization, the virtual machine, object memory, plugins, and source files. [Its] instance variable #globals is a reference to the system dictionary of global variables and class names. [Its] singleton instance is called Smalltalk.”

67/169

Scenarios

Smalltalk– Object is “the root class for almost all of the

other classes in the class hierarchy. […] Class Object provides default behavior common to all normal objects, such as access, copying, comparison, error handling, message sending, and reflection. Also utility messages that all objects should respond to are defined here.”

68/169

Scenarios

Smalltalk– Class “[adds] instance-specific behavior to

various class-describing objects in the system. This typically includes messages for initializing class variables and instance creation messages particular to a class. There is only one instance of a particular Metaclass, namely the class which is being described.”

69/169

Scenarios

Scenario 1:


Object subclass: #CinstanceVariableNames: 'i'classVariableNames: ''poolDictionaries: ''category: 'CSE3009'.

C class compile: 'newWithi: anInt^(self new) i: anInt ; yourself.'.

C compile: 'foo: aTextTranscript show: aText.Transcript show: i.Transcript cr.'.

C compile: 'i: anInti := anInt.'.

70/169

Scenarios

Scenario 1:


Identify all the methods available on oan IdentitySet(#handles: #longPrintString #actionMap#hasContentsInExplorer [...] #foo: [...] #i: [...]#creationStamp)


71/169

Scenarios

Scenario 1:


|o|o := C newWithi: 42.

Transcript show: 'Identify all the methods available on o' ; cr.Transcript show: C allSelectors ; cr.

Transcript show: 'Invoke a method using its name foo' ; cr.(C compiledMethodAt: #foo:) valueWithReceiver: o arguments: #('This is foo: ').

Simple and straightforward

72/169

Scenarios






73/169

Scenarios


Save on disk the complete state of o42


74/169

Scenarios


|o1 o2|o1 := C newWithi: 42.

Transcript show: 'Save on disk the complete state of o' ; cr.Transcript show: (o1 instVarAt: 1) ; cr.

Transcript show: 'Restore from disk the object o at a later time' ; cr.o2 := C new.o2 instVarNamed: 'i' put: 43.o2 foo: 'This is foo on o2: '.

Again, simple and straightforward

75/169

Scenarios

Scenario 2:

– Pharo (a recent implementation of Smalltalk) provides is a general object serialiser that can serialise any object

– Uses reflection to implement these methods!


1@2 serializeToFileNamed: 'hello.fuel'.FLMaterializer materializeFromFileNamed: 'hello.fuel'

Thanks to Marcus Denker for pointing out these helper methods

76/169

Scenarios

Scenario 3:

Object subclass: #CinstanceVariableNames: 'i'classVariableNames: 'NumberOfInstances'poolDictionaries: ''category: 'CSE3009'.





Only difference with Scenarios 1 and 2

77/169

Scenarios

Scenario 3:




Same as Scenarios 1 and 2


78/169

Scenarios

Scenario 3:

C class compile: 'initializeNumberOfInstances := 0.'.

C class compile: 'newNumberOfInstances := NumberOfInstances + 1.^ self basicNew initialize.'.

C class compile: 'numberOfInstances^ NumberOfInstances.'.


“Write access” to the metaclass

79/169

Scenarios

Scenario 3:

|o1 o2 o3|C initialize.o1 := C newWithi: 42.o2 := C newWithi: 1.o3 := C newWithi: 100.

Transcript show: o1 ; show: ' ' ; show: o2 ; show: ' ' ; show: o3 ; cr.Transcript show: C numberOfInstances.


80/169

Scenarios

Scenario 3:

a C a C a C3

To fulfill this scenario, we modifiedC class not C


81/169

Scenarios

Scenario 3:

In Smalltalk, each class has one anonymous meta-class, which can be modified

• Adding new variables and methods which are thus class variables and class methods

• These methods apply on the class, these fields belong to the class (thus unique)


82/169

Scenarios

Scenario 3:

– What about a subclass D of C?– How would its instances be counted?

a C a C a C3


Thanks to Christophe Dony for suggesting this use of class vs. class instance variables

83/169

Scenarios

Scenario 3:– In C++ and Java: static = global

• The class variables is shared by all the instances of its declaring class… and its subclasses!

final D o4 = new D();// Eclipse advises to use C to call getNumberOfInstances().System.out.println(o4);System.out.println(D.getNumberOfInstances());


84/169

Scenarios

Scenario 3:– In Smalltalk: class vs. class instance

• Class variable vs. class instance variable– A class variable is shared by all the instance of the

class and all the instances of its subclasses– A class instance variable is unique to each class,

inherited from the super-class, like instance variables

C class instanceVariableNames:'IndividualClassNumberOfInstances'


85/169

Scenarios


C subclass: #DinstanceVariableNames: ''classVariableNames: ''poolDictionaries: ''category: 'Ptidej'.

D class compile: 'initializeIndividualClassNumberOfInstances := 0.'.

D class compile: 'newNumberOfInstances := NumberOfInstances + 1.IndividualClassNumberOfInstances := IndividualClassNumberOfInstances + 1.^ self basicNew initialize.'.

D class compile: 'numberOfInstances^ NumberOfInstances.'.

D class compile: 'individualClassNumberOfInstances^ IndividualClassNumberOfInstances.'.


86/169

Scenarios


| o4 |D initialize.o4 := D new.Transcript show: o4 ; cr.Transcript show: C numberOfInstances ; cr.Transcript show: D numberOfInstances ; cr.Transcript show: D individualClassNumberOfInstances ; cr.


a D441

87/169

Scenarios

Scenario 3:– In Java: the JVM contains the world

• Possible to connect to a JVM using the Java Debug Interface of the Java Platform Debug Architecture

• Possible to reify the classes existing in the remote JVM and to obtain their instancescom.sun.jdi.ReferenceType.instances(long)

• Special feature of the JVM, “outside” of the normal constructs of the language


http://stackoverflow.com/questions/1947122/is-there-a-simple-way-of-obtaining-all-object-instances-of-a-specific-class-in-j/1947200#1947200

http://stackoverflow.com/questions/1947122/

88/169

Scenarios

Scenario 3:– In Smalltalk: the image contains the world

• Possible to ask it all the instances of a particular class or all the instances currently in memory

Transcript show: (C allInstances size).


Thanks to Marcus Denker for pointing out this solution

89/169

Scenarios

Scenario 4:– Given an object o, instance of C– Without changing o or C– Change the behaviour of o.foo()





90/169

Scenarios


This is o: 42Intercepting message send: foo:

Original arguments: This is o: New arguments: THIS IS O:

THIS IS O: 42

91/169

Scenarios

Scenario 4:

Several choices• ObjectWrappers: http://magaloma.seasidehosting.st/

Kernel#ObjectTracer• MethodWrappers: http://pharo.gforge.inria.fr/

PBE1/PBE1ch15.html• Reflectivity: http://scg.unibe.ch/Research/Reflectivity• …


http://magaloma.seasidehosting.st/

http://pharo.gforge.inria.fr/

http://scg.unibe.ch/Research/Reflectivity

92/169

Scenarios

Scenario 4:








93/169

Scenarios


| o aWrapper |o := C newWithi: 42.o foo: 'This is o: '.

aWrapper := WrapperForC on: o."o become: aWrapper."aWrapper foo: 'This is o: '.aWrapper xxxUnTrace.

94/169


ObjectTracer subclass: #WrapperForCinstanceVariableNames: ''classVariableNames: ''poolDictionaries: ''category: 'CSE3009'.

WrapperForC compile: 'doesNotUnderstand: aMessage"Intercept the selector foo:"

| sel arg newArg |sel := aMessage selector.sel = ''foo:'' ifTrue: [

arg := (aMessage arguments) at: 1.newArg := arg asUppercase.

Transcript show: ''Intercepting message send: '' ; show: sel ; cr.Transcript tab ; show: ''Original arguments: '' ; show: arg ; cr.Transcript tab ; show: ''New arguments: '' ; show: newArg ; cr.

aMessage argument: newArg.aMessage sendTo: (self xxxViewedObject)

]ifFalse: [

"Transcript show: aMessage."]'.

95/169

Scenarios




THIS IS O: 42

| o aWrapper |o := C newWithi: 42.o foo: 'This is o: '.

aWrapper := WrapperForC on: o."o become: aWrapper."aWrapper foo: 'This is o: '.aWrapper xxxUnTrace.

96/169

Scenarios

Scenario 4:

– We created the class WrappedForC that does not understand many messages (as subclass of ProtoObject)

– We wrapped an instance of WrappedForCaround o and use doesNotUnderstand: to proxy messages


97/169

Scenarios

Scenario 4:

– We created the class WrappedForC that does not understand many messages (as subclass of ProtoObject)

– We wrapped an instance of WrappedForCaround o and use doesNotUnderstand: to proxy messages


98/169

Scenarios

Scenario 4:








99/169

Scenarios

Reflectivity– Marcus Denker, RMOD Team,

INRIA Lille, and others

– Extensions to the standard reflection features of Smalltalk (both structural and behavioural)

• Structural reflection is extended by sub-method reflection: method bodies are first-class

• Behavioural reflection is provided by Geppetto, an implementation of Partial Behavioural Reflection

100/169

Scenarios






101/169

Scenarios

Scenario 4:– Sub-method reflection


nodes := (C>>#foo:) sends select: [:each | ((each selector ~= #show:)

or: (each arguments) size ~= 1)ifTrue: [ false ]ifFalse: [ (each arguments at: 1) name = 'aText'. ].

].

102/169

Scenarios

Scenario 4:– Meta-object


gpLink := GPLink metaObject: [:arg1 :node || newArg |newArg := arg1 asUppercase.Transcript show: 'Intercepting message send: '.Transcript show: node ; cr.Transcript tab ; show: 'Original arguments: '.Transcript show: arg1 ; cr.Transcript tab ; show: 'New arguments: '.Transcript show: newArg ; cr.Transcript show: newArg.

].gpLink control: #instead.

103/169

Scenarios

Scenario 4:– Linking


o := C newWithi: 42.o foo: 'This is o: '.

nodes do: [:node |node link: gpLink].

o foo: 'This is o: '.gpLink uninstall.



THIS IS O: 42

104/169

Scenarios

Scenarios 1, 2, 3, and 4

More difficult to implement “as is”• http://stackoverflow.com/questions/359237/why-does-

c-not-have-reflection• http://www.open-std.org/jtc1/sc22/wg21/docs/papers/

2005/n1751.html

http://stackoverflow.com/questions/359237/why-does-

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/

105/169

Scenarios


Third-party solutions• Boost.Mirror: http://kifri.fri.uniza.sk/~chochlik/mirror-

lib/html/index.html#introduction• VisualStudio.NET: http://msdn.microsoft.com/en-

us/library/y0114hz2%28v=vs.110%29.aspx• …

http://kifri.fri.uniza.sk/~chochlik/mirror-

http://msdn.microsoft.com/en-

106/169

Scenarios


Third-party solutions• Boost.Mirror: http://kifri.fri.uniza.sk/~chochlik/mirror-

lib/html/index.html#introduction• VisualStudio.NET: http://msdn.microsoft.com/en-

us/library/y0114hz2%28v=vs.110%29.aspx • …

http://kifri.fri.uniza.sk/~chochlik/mirror-

http://msdn.microsoft.com/en-

107/169

Scenarios

Boost.Mirror– Matúš Chochlík, University of Žilina,

Žilina, Slovakia

“[C]ompile-time and run-time meta-data describing C++ constructs like namespaces, types typedef-ined types, enums, classes with their base classes, member variables, constructors, member functions, etc.”

108/169

Scenarios

Scenario 1:


namespace cse3009 {class C {private:

int i;public:

C(const int _i) : i(_i) {}int get_i(void) const {

return this->i;}void set_i(const int _i) {

this->i = _i;}void foo(const std::string _s) const {

std::cout << _s << this->i << std::endl;}

};}

109/169

Scenarios

Scenario 1:


Identify all the methods available on o(The type of o is cse3009::C)cse3009::C::icse3009::C::foo


110/169

Scenarios

Scenario 1:


int main(void) {cse3009::C * o = new cse3009::C(42);

std::cout << "Identify all the methods available on o" << std::endl;identify_members(o);

std::cout << "Invoke a method using its name foo" << std::endl;invoke_member_function(o);

return 0;}

111/169

Scenarios

Scenario 1:


template<typename O>void identify_members(O * _o) {

using namespace mirror;

typedef MIRRORED_CLASS(O) meta_Class;

std::cout << "\t(The type of o is " << meta_Class::full_name()<< ")" << std::endl;

std::cout << "\t";

mp::for_each_ii< all_member_variables<meta_Class> >(info_printer());std::cout << std::endl << "\t";

mp::for_each_ii< member_functions<meta_Class> >(info_printer());std::cout << std::endl;

}

112/169

Scenarios

Scenario 1:


template<typename O>void invoke_member_function(O * _o) {


typedef MIRRORED_CLASS(cse3009::C) meta_Class;

typedef mp::at_c<overloads<mp::at_c<member_functions<meta_Class>, 0>::type>, 0>::type

meta_foo;

my_invoker_maker::invoker<meta_foo>::type invoker(std::string("\tThis is foo: "));

invoker.call_on(*_o);}

113/169

Scenarios

Boost.Mirror

– Provides essentially the same concepts and features as java.lang.reflect of Java

– Uses templates and (manual) registration to collect metadata that does not exist unless manually created through macro preprocessing

114/169

Scenarios

Scenario 1:


MIRROR_REG_BEGINMIRROR_QREG_GLOBAL_SCOPE_NAMESPACE(std)

MIRROR_REG_CLASS_BEGIN(struct, std, string)MIRROR_REG_CLASS_END

MIRROR_QREG_GLOBAL_SCOPE_NAMESPACE(cse3009)MIRROR_REG_CLASS_BEGIN(class, cse3009, C)

MIRROR_REG_CLASS_MEM_VARS_BEGINMIRROR_REG_CLASS_MEM_VAR_GET_SET(

private, _, int, i, _.get_i(), _.set_i(_i))MIRROR_REG_CLASS_MEM_VARS_ENDMIRROR_REG_MEM_FUNCTIONS_BEGIN

MIRROR_REG_MEM_OVLD_FUNC_BEGIN(foo)MIRROR_REG_MEM_FUNCTION_BEGIN(public, _, void, foo, 1)

MIRROR_REG_MEM_FUNCTION_PARAM(std::string, _s)MIRROR_REG_MEM_FUNCTION_END(1, _)

MIRROR_REG_MEM_OVLD_FUNC_END(foo)MIRROR_REG_MEM_FUNCTIONS_END

MIRROR_REG_CLASS_ENDMIRROR_REG_END

115/169

Scenarios

Scenario 1:


template<class Unused>struct my_manufacturer<std::string, Unused> {

const std::string s;

template<class ConstructionInfo>inline my_manufacturer(const std::string _s,

const ConstructionInfo construction_info) : s(_s) {}

void finish(std::string) const {}

inline std::string operator()(void) const {return s;

}};

typedef mirror::invoker_maker<my_manufacturer, mirror::default_fact_suppliers,my_enumerator, void> my_invoker_maker;

116/169

Scenarios

Boost.Mirror

– Uses complex code but can be easily and advantageously hidden in libraries

– Will possibly become part of Boost and the standard reflection library for C++

• http://kifri.fri.uniza.sk/~chochlik/jtc1_sc22_wg21/ std_cpp_refl-latest.pdf

http://kifri.fri.uniza.sk/~chochlik/jtc1_sc22_wg21/

117/169

Scenarios

Scenario 2:namespace cse3009 {class C {private:

int i;public:

C(const int _i) : i(_i) {}int get_i(void) const {

return this->i;}void set_i(const int _i) {

this->i = _i;}void foo(const std::string _s) const {

std::cout << _s << this->i << std::endl;}

};}


118/169

Scenarios


Save on disk the complete state of o(The type of o is cse3009::C)cse3009::C::i = 42

Restore from disk the object o at a later time43

119/169

Scenarios


int main(void) {cse3009::C * o = new cse3009::C(42);

std::cout << "Save on disk the complete state of o" << std::endl;save_on_disk(o);

std::cout << "Restore from disk the object o at a later time" << std::endl;restore_from_disk<cse3009::C>();

return 0;}

120/169

Scenarios


template<typename O>void save_on_disk(O * _o) {


typedef MIRRORED_CLASS(O)meta_Class;std::cout << "\t(The type of o is " <<

meta_Class::full_name() << ")" << std::endl << "\t";mp::for_each_ii<all_member_variables<meta_Class>>(

info_printer_variables<O>(*_o));std::cout << std::endl;

}

121/169

Scenarios


template<class O>struct info_printer_variables {

O o;

inline info_printer_variables(O _o) : o(_o) {}

template<class IterInfo>void operator()(IterInfo) {

using namespace mirror;std::cout << IterInfo::type::full_name()

<< " = " << IterInfo::type::get(o);if (!IterInfo::is_last::value)

std::cout << ", ";}

};

Get the value of reflected variable

122/169

Scenarios


template<typename O>void restore_from_disk(void) {


typedef typename my_factory_maker::factory<O>::type my_factory_type;my_factory_type my_factory(43);O o(my_factory());std::cout << "\t" << o.get_i() << std::endl;

}

123/169

Scenarios


typedef mirror::factory_maker<my_manufacturer, mirror::default_fact_suppliers,my_enumerator,void> my_factory_maker;

Factory generating instancesof (possibly) different types

124/169

Scenarios


template<class Unused>struct my_manufacturer<void, Unused> {

template<typename Context>my_manufacturer(int _value, Context) { }

void finish(int) { }

template<class ConstructionInfo>inline int add_constructor(

int _value, ConstructionInfo) const {return _value;

}

inline int index(void) {return 0;

}};

To pass on to the constructor

125/169

template<class Product, typename IsEnum>struct my_base_manufacturer {

Product x;

struct constr_param_name_printer {template<class IterInfo>inline void operator()(IterInfo) const {

if (!IterInfo::is_first::value)std::cout << ", ";

std::cout << IterInfo::type::base_name();}

};

struct constr_context_printer {template<class IterInfo>inline void operator()(IterInfo) const {

if (!IterInfo::is_first::value)std::cout << "::";

std::cout << IterInfo::type::base_name();}

};

template<class ConstructionInfo>my_base_manufacturer(int tabs,

ConstructionInfo construction_info,const Product& _x) : x(tabs) { }

void finish(int) {}

inline Product operator()(void) {return x;

}};

template<class Product, class Unused>struct my_enumerator :

my_base_manufacturer<Product, std::true_type> {template<class ConstructionInfo>inline my_enumerator(int tabs,

ConstructionInfo construction_info) :my_base_manufacturer<Product, std::true_type>(

tabs, construction_info, Product()) { }


inline Product operator()(void) {return NULL;

}};

template<class Product, class Unused>struct my_manufacturer {

typedef typename mirror::factory_maker<my_manufacturer,mirror::default_fact_suppliers,my_enumerator,Unused> maker;

typename maker::template factory<Product>::type f;

template<class ConstructionInfo>inline my_manufacturer(

int _value, ConstructionInfo construction_info) :f(construction_info) { }


inline Product operator()(void) {return f();

}};

Less complex than it looks!

126/169

Scenarios

Scenario 3:namespace cse3009 {class C {private:

int i;static int numberOfInstances;

public:C(const int _i) : i(_i) {

numberOfInstances++;}...void foo(const std::string _s) const {

std::cout << _s << this->i << std::endl;}static int getNumberOfInstances() {

return numberOfInstances;}

};}


127/169

Scenarios

Scenario 3:

int cse3009::C::numberOfInstances = 0;

int main(void) {cse3009::C * o1 = new cse3009::C(42);cse3009::C * o2 = new cse3009::C(1);cse3009::C * o3 = new cse3009::C(100);

std::cout << o1 << std::endl << o2 << std::endl << o3 << std::endl;

std::cout << cse3009::C::getNumberOfInstances() << std::endl;

return 0;}


128/169

Scenarios

Scenario 3:

0x800418f80x800419080x800419183


To fulfill this scenario, we must modify C as in Java

129/169

Scenarios

Scenario 4:

Several choices• Bjarne Stroustrup's Template:

http://www.stroustrup.com/wrapper.pdf• Herb Sutter's Functor-based Approach:

http://channel9.msdn.com/Shows/Going+Deep/ C-and-Beyond-2012-Herb-Sutter-Concurrency-and-Parallelism

• …


http://www.stroustrup.com/wrapper.pdf

http://channel9.msdn.com/Shows/Going+Deep/

130/169

Scenarios

Scenario 4:

Several choices• Bjarne Stroustrup's Template:

http://www.stroustrup.com/wrapper.pdf• Herb Sutter's Functor-based Approach:

http://channel9.msdn.com/Shows/Going+Deep/ C-and-Beyond-2012-Herb-Sutter-Concurrency-and-Parallelism

• …


http://www.stroustrup.com/wrapper.pdf

http://channel9.msdn.com/Shows/Going+Deep/

131/169

Scenarios


Bjarne Stroustrup's Template:(Prefix::operator->())This is o: 42

132/169

Scenarios


int main(void) {std::cout << "Bjarne Stroustrup's Template:" << std::endl;cse3009::C o2(42);Prefix<cse3009::C> prefixed_o2(&o2);prefixed_o2->foo("This is o: ");std::cout << std::endl;

return 0;}

133/169

Scenarios


template<class T>class Prefix {

T* p;public:

Prefix(T * _p) : p(_p) { }T* operator->() {

std::cout << "(Prefix::operator->())" << std::endl;return p;

}};

134/169

Scenarios

Scenario 4:

– Is similar to a proxy in Java

– Cannot easily access and modify the arguments of the proxy’ed method call

– Makes it difficult to do something after the original method call


135/169

Scenarios

Java

Smalltalk

C++

136/169

Conclusion

Java

Smalltalk

C++

Differences in semanticsand syntax mostly due tothe presence (or absence)of virtual machines

137/169

Outline



138/169

Theory

Class– A programming construct that encapsulates

some state (fields) and behaviours (methods)– A construct whose instances are objects

Metaclass– A programming construct that encapsulates

some state (fields) and behaviours (methods)– A construct whose instances are classes

Aristote (Ἀριστοτέλης)*-384 †-322

139/169

Theory

An object is an instance of a class– A class generates an object

A class is an instance of a metaclass– A metaclass generates a class

A metaclass is an instance of…(a) A meta-metaclass?(b) Itself?

140/169

Theory

An object is an instance of a class– A class generates an object

A class is an instance of a metaclass– A metaclass generates a class

A metaclass is an instance of…(a) A meta-metaclass?(b) Itself

141/169

Theory

The class Object is the parent of all classes, including metaclasses

The class Class is the generator of all classes, including Object

142/169

TheoryJava

[http://www.ogis-ri.co.jp/otc/hiroba/technical/Squeak4/img/metaHiera9.gif]

Smalltalk

http://www.ogis-ri.co.jp/otc/hiroba/technical/Squeak4/img/metaHiera9.gif

143/169

Theory

The classes– Class– Method– Field– …

reify constructs of the programming languages, they become first-class entities, to allow programs to manipulate these constructs at runtime

144/169

Theory

Java Field C

Method C

Static field C

Static method C

Smalltalk Instance variable C

Instance method C

Class variable C class

Class method C class

Class instance variables– To count the numbers of

instances of subclasses of C(D, E…) independently from one another

145/169

Theory

Earlier, we said that reflection is of– Theoretical interest



146/169

Theory




147/169

Theory




A virtual machine is an interpreter

148/169

Theory

In Java, the metaclass is anonymous and cannot be modified

In Smalltalk, “[t]here is only one instance of a particular Metaclass, namely the class which is being described”

149/169

Theory

Many other variations are possible– LISP– Perl 5– Perl 6– Python– …– MetaclassTalk– NeoClassTalk– …

150/169

Theory

A Meta Object Protocol exposes some or all internal structure of the interpreter to the programmer

The MOP is (often) a set of classes and methods that allow a program to inspect the state of the supporting system and alter its behaviour

151/169

Theory

A MOP can be accessible– Compile-time

• OpenJava– Load-time

• Javassist– Run-time

• MetaJava• AspectJ

152/169

Outline



153/169

Conclusion

Scenario 1: invoke an arbitrary method on an object (see the problems with instanceof and plugins)

Scenario 2: access the complete (including private) state of an object (see the problem with serialisation)

Scenario 3: count the number of instances of a class created at runtime (see the problem with debugging/profiling)

Scenario 4: modify the behaviour at run-time of a method without the need to recompile and restart the program (see the problem with patching)

154/169

Conclusion

Reflection helps addressing these scenarios– Depends on the capabilities of the programming

language and underlying execution environment– Yields more or less elegant code

Use reflection sparingly and purposefully– Complexity– Performance

155/169

Conclusion

Java

Smalltalk

C++

156/169

Conclusion

Java

Smalltalk

C++

Differences in semanticsand syntax mostly due tothe presence (or absence)of virtual machines

157/169

Outline



158/169

Class Loading

Virtual machines– Interpreters– Closed world

Must– Access resources– Load classes

159/169

Class Loading



160/169

Class Loading



161/169

Class Loading

Example– Given an interface ICommand– Load at runtime a class implementing ICommand using its binary name

• A binary name is in the form <pckg1>.….<pckgn>.<ClassName>– net.ptidej.reflection.java.scenario1.C

final Class<ICommand> command =this.classLoader.loadClass(binaryName);

162/169

Class Loading

http://map.sdsu.edu/geog583/images/week8.3.gif

http://map.sdsu.edu/geog583/images/week8.3.gif

163/169

Class Loading

http://www.onjava.com/2005/01/26/graphics/Figure01_MultipleClassLoaders.JPG


164/169

Class Loading

public class ClassLoader extends java.lang.ClassLoader {private final String directory;

public ClassLoader(final java.lang.ClassLoader parent,final String directory) {

super(parent);this.directory = directory;

}

165/169

Class Loadingprotected Class findClass(final String name) {

Class newClass = null;try {

newClass = super.findClass(name);}catch (final ClassNotFoundException cnfe) {

final String osName =this.directory + name.replace('.', '/') + ".class";

try {final FileInputStream fis = new FileInputStream(osName);newClass = this.defineClasses(name, fis);

}catch (final ClassFormatError cfe) {

cfe.printStackTrace(Output.getInstance().errorOutput());}catch (final FileNotFoundException fnfe) {

// fnfe.printStackTrace();}

}

return newClass;}

Use the method defined inthe superclass on this CL

166/169

Class Loadingprivate Class defineClasses(final String name, final InputStream inputStream) {

try {int b;int length = 0;byte[] bytes = new byte[4];while ((b = inputStream.read()) != -1) {

if (length == bytes.length) {final byte[] temp = new byte[length + 4];System.arraycopy(bytes, 0, temp, 0, length);bytes = temp;

}bytes[length] = (byte) b;length++;

}

System.out.println(name);final Class newClass = this.defineClass(name, bytes, 0, length);return newClass;

}catch (final IOException ioe) {

return null;}catch (final NoClassDefFoundError ncdfe) {

ncdfe.printStackTrace(Output.getInstance().errorOutput());return null;

}}

Use the method defined inthe superclass on this CL

167/169

Class Loading



168/169

Class Loadingprotected Class findClass(final String name) {

Class newClass = null;try {

newClass = this.getParent().loadClass(name);}catch (final ClassNotFoundException cnfe) {

final String osName =this.directory + name.replace('.', '/') + ".class";

try {final FileInputStream fis = new FileInputStream(osName);newClass = this.defineClasses(name, fis);

}catch (final ClassFormatError cfe) {

cfe.printStackTrace(Output.getInstance().errorOutput());}catch (final FileNotFoundException fnfe) {

// fnfe.printStackTrace();}

}

return newClass;}

Ask the parent CL if italready knows the class

169/169

Conclusion

Virtual machines must access resources and load classes, in specific cases– Can extend ClassLoader– Must override (typically) method

•Class findClass(String)

– Must be careful with inheritance vs. delegation!•super vs. getParent()

on reflection in oo programming languages

Technology

public class dog

public class expression

public abstract string

override public string

ananimal instanceof

final string sound

final dog fido

ananimal instanceof