s.ducasse marcus denker 1 working with bytecodes: irbuilder and instructionstream
DESCRIPTION
S.Ducasse 3 Reasons for working with Bytecode Generating Bytecode Implementing compilers for other languages Experimentation with new language features Parsing and Interpretation: Analysis Decompilation (for systems without source) Pretty printing Interpretation: Debugger, ProfilerTRANSCRIPT
![Page 1: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/1.jpg)
S.Ducasse 1
QuickTime™ and aTIFF (Uncompressed) decompressorare needed to see this picture.
Marcus [email protected]
Working with Bytecodes: IRBuilder and InstructionStream
![Page 2: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/2.jpg)
S.Ducasse 2
License: CC-Attribution-ShareAlike 2.0http://creativecommons.org/licenses/by-sa/2.0/
![Page 3: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/3.jpg)
S.Ducasse 3
Reasons for working with Bytecode•Generating Bytecode
• Implementing compilers for other languages• Experimentation with new language features
• Parsing and Interpretation:• Analysis• Decompilation (for systems without source)• Pretty printing • Interpretation: Debugger, Profiler
![Page 4: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/4.jpg)
S.Ducasse 4
Overview
• Squeak Bytecodes• Examples: Generating Bytecode •with IRBuilder•Decoding Bytecodes•Bytecode Execution
![Page 5: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/5.jpg)
S.Ducasse 5
The Squeak Virtual Machine•Virtual machine provides a virtual processor•Bytecode: The ‘machinecode’ of the virtual machine
•Smalltalk (like Java): Stack machine• easy to implement interpreters for different processors• most hardware processors are register machines
•Squeak VM: Implemented in Slang • Slang: Subset of Smalltalk. (”C with Smalltalk Syntax”)• Translated to C
![Page 6: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/6.jpg)
S.Ducasse 6
Bytecode in the CompiledMethod•CompiledMethods format:
Header
Literals
Bytecode
Trailer Pointer toSource
Array of all Literal Objects
Number of temps, literals...
(Number>>#asInteger)inspect
![Page 7: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/7.jpg)
S.Ducasse 7
Bytecodes: Single or multibyte•Different forms of bytecodes:•Single bytecodes:
• Example: 120: push self•Groups of similar bytecodes
• 16: push temp 1• 17: push temp 2• up to 31
•Multibyte bytecodes• Problem: 4bit offset may be too small• Solution: Use the following byte as offset• Example: Jumps need to encode large jump
offsets
OffsetType
4bits 4bits
![Page 8: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/8.jpg)
S.Ducasse 8
Example: Number>>asInteger•Smalltalk code:
•Symbolic Bytecode
Number>>asInteger"Answer an Integer nearest the receiver toward zero."
^self truncated
9 <70> self10 <D0> send: truncated11 <7C> returnTop
![Page 9: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/9.jpg)
S.Ducasse 9
Example: Step by Step•9 <70> self•The receiver (self) is pushed on the stack
•10 <D0> send: truncated•Bytecode 208: send litereral selector 1•Get the selector from the first literal•start message lookup in the class of the
object that is top of the stack•result is pushed on the stack
•11 <7C> returnTop•return the object on top of the stack to
the calling method
![Page 10: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/10.jpg)
S.Ducasse 10
Squeak Bytecodes•256 Bytecodes, four groups:
• Stack Bytecodes• Stack manipulation: push / pop / dup
• Send Bytecodes• Invoke Methods
• Return Bytecodes• Return to caller
• Jump Bytecodes• Control flow inside a method
![Page 11: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/11.jpg)
S.Ducasse 11
Stack Bytecodes•push and store, e.g. temps, instVars, literals
• e.g: 16 - 31: push instance variable• Push Constants (False/True/Nil/1/0/2/-1)• Push self, thisContext•duplicate top of stack•pop
![Page 12: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/12.jpg)
S.Ducasse 12
Sends and Returns•Sends: receiver is on top of stack
• Normal send• Super Sends• Hard-coded sends for efficiency, e.g. +, -
•Returns•Return top of stack to the sender •Return from a block•Special bytecodes for return self, nil,
true, false (for efficiency)
![Page 13: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/13.jpg)
S.Ducasse 13
Jump Bytecodes•Control Flow inside one method•Used to implement control-flow efficiently•Example:
9 <76> pushConstant: 110 <77> pushConstant: 211 <B2> send: <12 <99> jumpFalse: 1513 <20> pushConstant: 'true'14 <90> jumpTo: 1615 <73> pushConstant: nil16 <7C> returnTop
^ 1<2 ifTrue: ['true']
![Page 14: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/14.jpg)
S.Ducasse 14
What you should have learned...• ... dealing with bytecodes directly is
possible, but very boring.•We want reusable abstractions that hide the
details (e.g. the different send bytecodes)
•We would like to have frameworks for • Generating bytecode easily • Parsing bytecode • Evaluating bytecode
![Page 15: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/15.jpg)
S.Ducasse 15
Generating Bytecodes
• IRBuilder• Part of the new compiler (on SqueakMap)
• Like an Assembler for Squeak
![Page 16: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/16.jpg)
S.Ducasse 16
IRBuilder: Simple Example•Number>>asInteger
iRMethod := IRBuilder newrargs: #(self); "receiver and args"pushTemp: #self;send: #truncated;returnTop;ir.
aCompiledMethod := iRMethod compiledMethod.
aCompiledMethod valueWithReceiver:3.5 arguments: #()
![Page 17: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/17.jpg)
S.Ducasse 17
IRBuilder: Stack Manipulation•popTop - remove the top of stack•pushDup - push top of stack on the stack•pushLiteral: •pushReceiver - push self•pushThisContext
![Page 18: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/18.jpg)
S.Ducasse 18
IRBuilder: Symbolic Jumps• Jump targets are resolved: •Example: false ifTrue: [’true’] ifFalse:
[’false’]iRMethod := IRBuilder new
rargs: #(self); "receiver and args"pushLiteral: false;jumpAheadTo: #false if: false;pushLiteral: 'true'; "ifTrue: ['true']"jumpAheadTo: #end;jumpAheadTarget: #false;pushLiteral: 'false'; "ifFalse: ['false']"jumpAheadTarget: #end;returnTop;ir.
![Page 19: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/19.jpg)
S.Ducasse 19
IRBuiler: Instance Variables•Access by offset•Read: getField:
• receiver on top of stack•Write: setField:
• receiver and value on stack•Example: set the first instance variable to 2iRMethod := IRBuilder new
rargs: #(self); "receiver and args declarations"pushLiteral: 2;pushTemp: #self;setField: 1;pushTemp: #self;returnTop;ir.
aCompiledMethod := iRMethod compiledMethod.aCompiledMethod valueWithReceiver: 1@2 arguments: #()
![Page 20: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/20.jpg)
S.Ducasse 20
IRBuilder: Temporary Variables•Accessed by name•Define with addTemp: / addTemps:•Read with pushTemp:•Write with storeTemp:•Examle: set variables a and b, return value
of aiRMethod := IRBuilder newrargs: #(self); "receiver and args"addTemps: #(a b);pushLiteral: 1;storeTemp: #a;pushLiteral: 2;storeTemp: #b;pushTemp: #a;returnTop;ir.
![Page 21: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/21.jpg)
S.Ducasse 21
IRBuilder: Sends•normal send
• super send
• The second parameter specifies the class were the lookup starts.
....builder send: #selector toSuperOf: aClass;
builder pushLiteral: ‘hello’builder send: #size;
![Page 22: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/22.jpg)
S.Ducasse 22
IRBuilder: Lessons learned• IRBuilder: Easy bytecode generation
•Not part of Squeak yet•Will be added to Squeak 3.9 or 4.0
•Next: Decoding bytecode
![Page 23: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/23.jpg)
S.Ducasse 23
Parsing and Interpretation• First step: Parse bytecode
• enough for easy analysis, pretty printing, decompilation
•Second step: Interpretation• needed for simulation, complex analyis (e.g.,
profiling)
•Squeak provides frameworks for both:• InstructionStream/InstructionClient (parsing)• ContextPart (Interpretation)
![Page 24: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/24.jpg)
S.Ducasse 24
The InstructionStream Hierarchy
InstructionStreamContextPartBlockContextMethodContextDecompilerInstructionPrinterInstVarRefLocatorBytecodeDecompiler
![Page 25: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/25.jpg)
S.Ducasse 25
InstructionStream• Parses the byte-encoded instructions•State:
• pc: programm counter• sender: the method (bad name!)
Object subclass: #InstructionStreaminstanceVariableNames: 'sender pc'classVariableNames: 'SpecialConstants'poolDictionaries: ''category: 'Kernel-Methods'
![Page 26: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/26.jpg)
S.Ducasse 26
Usage•Generate an instance:
•Now we can step through the bytecode with
• calls methods on a client object for the type of bytecode, e.g. • pushReceiver• pushConstant: value• pushReceiverVariable: offset
instrStream := IntructionStream on: aMethod
instrStream interpretNextInstructionFor: client
![Page 27: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/27.jpg)
S.Ducasse 27
InstructionClient•Abstract superclass•Defines empty methods for all methods
that InstructionStream calls on a client• For convienience:
• Client don’t need to inherit from this
Object subclass: #InstructionClientinstanceVariableNames: ''classVariableNames: ''poolDictionaries: ''category: 'Kernel-Methods'
![Page 28: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/28.jpg)
S.Ducasse 28
Example: A testInstructionClientTest>>testInstructions
"just interpret all of methods of Object"| methods client scanner|
methods := Object methodDict values. client := InstructionClient new.
methods do: [:method |scanner := (InstructionStream on: method).[scanner pc <= method endPC] whileTrue: [
self shouldnt: [scanner interpretNextInstructionFor: client] raise: Error.
].].
![Page 29: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/29.jpg)
S.Ducasse 29
Example: Printing Bytecodes• InstructionPrinter: Print the bytecodes as
human readable text•Example: print the bytecode of
Number>>asInteger:String streamContents: [:str |
(InstructionPrinter on: Number>>#asInteger) printInstructionsOn: str
]
result:
'9 <70> self10 <D0> send: truncated11 <7C> returnTop'
![Page 30: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/30.jpg)
S.Ducasse 30
InstructionPrinter•Class Definition:
InstructionClient subclass: #InstructionPrinterinstanceVariableNames: 'method scanner
stream oldPC indent'classVariableNames: ''poolDictionaries: ''category: 'Kernel-Methods'
![Page 31: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/31.jpg)
S.Ducasse 31
InstructionPrinter•Main Loop:
InstructionPrinter>>printInstructionsOn: aStream "Append to the stream, aStream, a description of each
bytecode in the instruction stream."
| end | stream := aStream. scanner := InstructionStream on: method. end := method endPC. oldPC := scanner pc. [scanner pc <= end]
whileTrue: [scanner interpretNextInstructionFor: self]
![Page 32: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/32.jpg)
S.Ducasse 32
InstructionPrinter
•Overwrites methods form InstructionClient to print the bytecodes as text
•e.g. the method for pushReceiver:InstructionPrinter>>pushReceiver"Print the Push Active Context's Receiver on Top Of Stack bytecode."
self print: 'self'
![Page 33: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/33.jpg)
S.Ducasse 33
Example: InstVarRefLocatorInstructionClient subclass: #InstVarRefLocator
instanceVariableNames: 'bingo'....
interpretNextInstructionUsing: aScanner bingo := false.
aScanner interpretNextInstructionFor: self.^bingo
popIntoReceiverVariable: offset bingo := true
pushReceiverVariable: offsetbingo := true
storeIntoReceiverVariable: offset bingo := true
![Page 34: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/34.jpg)
S.Ducasse 34
InstVarRefLocator•CompiledMethod>>#hasInstVarRef
hasInstVarRef"Answer whether the receiver references an instance variable."
| scanner end printer |
scanner := InstructionStream on: self.printer := InstVarRefLocator new.end := self endPC.
[scanner pc <= end] whileTrue: [(printer interpretNextInstructionUsing: scanner) ifTrue: [^true].
].^false
![Page 35: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/35.jpg)
S.Ducasse 35
InstVarRefLocator•Example for a simple bytecode analyzer•Usage:
aMethod hasInstVarRef
(TestCase>>#debug) hasInstVarReftrue
(Integer>>#+) hasInstVarReffalse
![Page 36: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/36.jpg)
S.Ducasse 36
Example: Decompiling to IR•BytecodeDecompiler
•Usage:
InstructionStream subclass: #BytecodeDecompilerinstanceVariableNames: 'irBuilder blockFlag'classVariableNames: ''poolDictionaries: ''category: 'Compiler-Bytecodes'
BytecodeDecompiler new decompile: (Number>>#asInteger)
![Page 37: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/37.jpg)
S.Ducasse 37
Decompiling•uses IRBuilder for building IR
• e.g. code for the bytecode pushReceiver:
BytecodeDecompiler>>pushReceiver
irBuilder pushReceiver
![Page 38: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/38.jpg)
S.Ducasse 38
ContextPart: Execution Semantics •Sometimes we need more than parsing
• “stepping” in the debugger• system simulation for profiling
InstructionStream subclass: #ContextPartinstanceVariableNames: 'stackp'classVariableNames: 'PrimitiveFailToken QuickStep'poolDictionaries: ''category: 'Kernel-Methods'
![Page 39: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/39.jpg)
S.Ducasse 39
Simulation• Provides a complete Bytecode interpreter
•Run a block with the simulator: (ContextPart runSimulated: [3 factorial])
![Page 40: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/40.jpg)
S.Ducasse 40
Profiling: MesageTally•Usage:
•Other example:
MessageTally tallySends: [3 factorial]
This simulation took 0.0 seconds.**Tree**1 SmallInteger(Integer)>>factorial 1 SmallInteger(Integer)>>factorial 1 SmallInteger(Integer)>>factorial 1 SmallInteger(Integer)>>factorial
MessageTally tallySends: [’3’ + 1]**Leaves**4 SmallInteger(Integer)>>factorial 3 SmallInteger(Integer)>>factorial 1 BlockContext>>DoIt
![Page 41: S.Ducasse Marcus Denker 1 Working with Bytecodes: IRBuilder and InstructionStream](https://reader035.vdocuments.mx/reader035/viewer/2022062317/5a4d1b1e7f8b9ab059994432/html5/thumbnails/41.jpg)
S.Ducasse 41