thnad's revenge
DESCRIPTION
At a previous JRubyConf, we talked about Thnad, a fictional programming language. Thnad served as a vehicle to explore the joy of building a compiler using JRuby, BiteScript, Parslet, and other tools. Now, Thnad is back with a second runtime: Rubinius. Come see the Rubinius environment through JRuby eyes. Together, we'll see how to grapple with multiple instruction sets and juggle contexts without going cross-eyed.TRANSCRIPT
![Page 1: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/1.jpg)
Welcome to “Thnad’s Revenge,” a programming language implementation tale in three acts.
![Page 2: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/2.jpg)
Not to be confused with...
![Page 3: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/3.jpg)
http://en.wikipedia.org/wiki/Yars'_Revenge
...Yars’ Revenge, the awesome Atari video game from the ’80s.
![Page 4: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/4.jpg)
Cucumber Recipes
Ian Deeswith Aslak Hellesøy
and Matt Wynne
pragprog/titles/JRUBYdiscount code: JRubyIanDees
Before we get to the talk, let me make a couple of quick announcements. First, we’re updating the JRuby book this summer with a JRuby 1.7-ready PDF. To celebrate that, we’re offering a discount code on the book during the conference. Second, I’m working on a new book with the Cucumber folks, which has some JRuby/JVM stuff in it—if you’d like to be a tech reviewer, please find me after this talk.
![Page 5: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/5.jpg)
I. Meet Thnad
II. Enter the Frenemy
III. Thnad’s Revenge
(with apologies to Ira Glass) Act I, Meet Thnad, in which we encounter Thnad, a programming language built with JRuby and designed not for programmer happiness, but for implementer happiness. Act II, Enter the Frenemy, in which we meet a new Ruby runtime. Act III, Thnad's Revenge, in which we port Thnad to run on the Rubinius runtime and encounter some surprises along the way.
![Page 6: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/6.jpg)
I. Meet Thnad
Thnad is a programming language I created last summer as an excuse to learn some fun JRuby tools and see what it's like to write a compiler.
![Page 7: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/7.jpg)
The name comes from a letter invented by Dr. Seuss in his book, “On Beyond Zebra.” Since most of the real letters are already taken by programming languages, a fictional one seems appropriate.
![Page 8: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/8.jpg)
A Fictional ProgrammingLanguage
Optimized for Implementer Happiness
Just as Ruby is optimized for programmer happiness, Thnad is optimized for implementer happiness. It was designed to be implemented with a minimum of time and effort, and a maximum amount of fun.
![Page 9: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/9.jpg)
function factorial(n) { if (eq(n, 1)) { 1 } else { times(n, factorial(minus(n, 1))) }}
print(factorial(4))
Here’s a sample Thnad program demonstrating all the major features. Thnad has integers, functions, conditionals, and... not much else. These minimal features were easy to add, thanks to the great tools available in the JRuby ecosystem (and other ecosystems, as we’ll see).
![Page 10: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/10.jpg)
Thnad Features
1. Names and Numbers
2. Function Calls
3. Conditionals
4. Function Definitions
In the next few minutes, we’re going to trace through each of these four language features, from parsing the source all the way to generating the final binary. We won’t show every single grammar rule, but we will hit the high points.
![Page 11: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/11.jpg)
As Tom mentioned in his talk, there are a number of phases a piece of source code goes through during compilation.
![Page 12: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/12.jpg)
Stages of Parsing
tokenize
parse
transform
emit
These break down into four main stages in a typical language: finding the tokens or parts of speech of the text, parsing the tokens into an in-memory tree, transforming the tree, and generating the bytecode. We’re going to look at each of Thnad’s major features in the context of these stages.
![Page 13: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/13.jpg)
1. Names and Numbers
First, let’s look at the easiest language feature: numbers and function parameters.
![Page 14: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/14.jpg)
'42' :number
"42"
root
{:number => '42'}
Our parser needs to transform this input text into some kind of Ruby data structure.
![Page 15: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/15.jpg)
Parsletkschiess.github.com/parslet
I used a library called Parslet for that. Parslet handles the first two stages of compilation (tokenizing and parsing) using a Parsing Expression Grammar, or PEG. PEGs are like regular expressions attached to blocks of code. They sound like a hack, but there’s solid compiler theory behind them.
![Page 16: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/16.jpg)
'42' :number
"42"
root
rule(:number) { match('[0-9]').repeat(1).as(:number) >> space? }
{:number => '42'}
The rule at the bottom of the page is Parslet’s notation for matching one or more numbers followed by a optional space.
![Page 17: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/17.jpg)
rule(:number => simple(:value)) { Number.new(value.to_i) }
Thnad::Number.new(42){:number => '42'}
Thnad::Number
:value
42
root
:number
"42"
root
Now for the third stage, transformation. We could generate the bytecode straight from the original tree, using a bunch of hard-to-test case statements. But it would be nicer to have a specific Ruby class for each Thnad language feature. The rule at the bottom of this slide tells Parslet to transform a Hash with a key called :number into an instance of a Number class we provide.
![Page 18: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/18.jpg)
BiteScriptgithub/headius/bitescript
The final stage, outputting bytecode, is handled by the BiteScript library, which is basically a domain-specific language for emitting JVM opcodes.
![Page 19: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/19.jpg)
main do ldc 42 ldc 1 invokestatic :Example, :baz, [int, int, int] returnvoidend
Here's an example, just to get an idea of the flavor. To call a method, you just push the arguments onto the stack and then call a specific opcode, in this case invokestatic. The VM you're writing for is aware of classes, interfaces, and so on—you don't have to implement method lookup like you would with plain machine code.
![Page 20: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/20.jpg)
“JVM Bytecode for Dummies”Charles Nutter, Øredev 2010
slideshare/CharlesNutter/redev-2010-jvm-bytecode-for-dummies
When I first saw the BiteScript, I thought it was something you'd only need if you were doing deep JVM hacking. But when I read the slides from Charlie's presentation at Øredev, it clicked. This library takes me way back to my college days, when we'd write assembler programs for a really simple instruction set like MIPS. BiteScript evokes that same kind of feeling. I'd always thought the JVM would have a huge, crufty instruction set—but it's actually quite manageable to keep the most important parts of it in your head.
![Page 21: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/21.jpg)
class Number < Struct.new :value def eval(context, builder) builder.ldc value endend
We can generate the bytecode any way we want. One simple way is to give each of our classes an eval() method that takes a BiteScript generator and calls various methods on it to generate JVM instructions.
![Page 22: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/22.jpg)
class Name < Struct.new :name def eval(context, builder) param_names = context[:params] || [] position = param_names.index(name) raise "Unknown parameter #{name}" unless position
builder.iload position endend
Dealing with passed-in parameters is nearly as easy as dealing with raw integers; we just look up the parameter name by position, and then push the nth parameter onto the stack.
![Page 23: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/23.jpg)
2. Function Calls
The next major feature is function calls. Once we have those, we will be able to run a trivial Thnad program.
![Page 24: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/24.jpg)
'baz(42, foo)'
{:funcall => {:name => 'baz', :args => [ {:arg => {:number => '42'}}]}} {:arg => {:name => 'foo'}}]}}
:funcall
:name :args
"baz" :arg :arg
:number
"42"
:name
"foo"
root
We’re going to move a little faster here, to leave time for Rubinius. Here, we want to transform this source code into this Ruby data structure representing a function call.
![Page 25: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/25.jpg)
Thnad::Funcall.new 'foo', [Thnad::Number.new(42)]
Thnad::Funcall
:name :args
"foo" Thnad::Number
:value
42
root
Now, we want to transform generic Ruby data structures into purpose-built ones that we can attach bytecode-emitting behavior to.
![Page 26: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/26.jpg)
class Funcall < Struct.new :name, :args def eval(context, builder) args.each { |a| a.eval(context, builder) } types = [builder.int] * (args.length + 1)
builder.invokestatic \ builder.class_builder, name, types endend
The bytecode for a function call is really simple in BiteScript. All functions in Thnad are static methods on a single class.
![Page 27: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/27.jpg)
3. Conditionals
The first two features we’ve defined are enough to write simple programs like print(42). The next two features will let us add conditionals and custom functions.
![Page 28: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/28.jpg)
'if (0) { 42 } else { 667 }'
{:cond => {:number => '0'}, :if_true => {:body => {:number => '42'}}, :if_false => {:body => {:number => '667'}}}
:cond
:number
"0"
:if_true
:body
:number
"42"
:if_false
:body
:number
"667"
root
A conditional consists of the “if” keyword, followed by a body of code inside braces, then the “else” keyword, followed by another body of code in braces.
![Page 29: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/29.jpg)
Thnad::Conditional
:cond :if_true :if_false
Thnad::Number
:value
0
Thnad::Number
:value
42
Thnad::Number
:value
667
root
Thnad::Conditional.new \ Thnad::Number.new(0), Thnad::Number.new(42), Thnad::Number.new(667)
Here’s the transformed tree representing a set of custom Ruby classes.
![Page 30: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/30.jpg)
class Conditional < Struct.new :cond, :if_true, :if_false def eval(context, builder) cond.eval context, builder builder.ifeq :else if_true.eval context, builder builder.goto :endif builder.label :else if_false.eval context, builder builder.label :endif endend
The bytecode emitter for conditionals has a new twist. The Conditional struct points to three other Thnad nodes. It needs to eval() them at the right time to emit their bytecode in between all the zero checks and gotos.
![Page 31: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/31.jpg)
4. Function Definitions
On to the final piece of Thnad: defining new functions.
![Page 32: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/32.jpg)
'function foo(x) { 5 }'
{:func => {:name => 'foo'}, :params => {:param => {:name => 'x'}}, :body => {:number => '5'}}
:func
:name
"foo"
:params
:param
:name
"x"
:body
:number
"5"
root
A function definition looks a lot like a function call, but with a body attached to it.
![Page 33: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/33.jpg)
Thnad::Function.new \ 'foo', [Thnad::Name.new('x')], Thnad::Number.new(5)
Thnad::Function
:name :params :body
"foo" Thnad::Name
:name
"x"
Thnad::Number
:value
5
root
Here’s the transformation we want to perform for this language feature.
![Page 34: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/34.jpg)
class Function < Struct.new :name, :params, :body def eval(context, builder) param_names = [params].flatten.map(&:name) context[:params] = param_names types = [builder.int] * (param_names.count + 1)
builder.public_static_method(self.name, [], *types) do |method|
self.body.eval(context, method) method.ireturn end endend
Since all Thnad parameters and return types are integers, emitting a function definition is really easy. We count the parameters so that we can give the JVM a correct signature. Then, we just pass a block to the public_static_method helper, a feature of BiteScript that will inspire the Rubinius work later on.
![Page 35: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/35.jpg)
Compiler
We’ve seen how to generate individual chunks of bytecode; how do they all get stitched together into a .class file?
![Page 36: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/36.jpg)
builder = BiteScript::FileBuilder.build(@filename) do public_class classname, object do |klass|
# ...
klass.public_static_method 'main', [], void, string[] do |method|
context = Hash.new exprs.each do |e| e.eval(context, method) end
method.returnvoid end endend
Here’s the core of class generation. We output a standard Java main() function...
![Page 37: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/37.jpg)
builder = BiteScript::FileBuilder.build(@filename) do public_class classname, object do |klass|
# ...
klass.public_static_method 'main', [], void, string[] do |method|
context = Hash.new exprs.each do |e| e.eval(context, method) end
method.returnvoid end endend
...inside which we eval() our Thnad expressions (not counting function definitions) one by one.
![Page 38: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/38.jpg)
Built-insplus, minus, times, eq, print
Thnad ships with a few basic arithmetic operations, plus a print() function. Let’s look at one of those now.
![Page 39: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/39.jpg)
public_static_method 'minus', [], int, int, int do iload 0 iload 1 isub ireturnend
Here’s the definition of minus(). It just pushes its two arguments onto the stack and then subtracts them. The rest of the built-ins are nearly identical to this one, so we won’t show them here.
![Page 40: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/40.jpg)
II. Enter the Frenemy
So that's a whirlwind tour of Thnad. Last year, I was telling someone about this project—it was either Shane Becker or Brian Ford, I think—and he said,...
![Page 41: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/41.jpg)
Rubinius
...“Hey, you should port this to Rubinius!” I thought, “Hmm, why not? Sounds fun.” Let’s take a look at this other runtime that has sprung up as a rival for Thnad’s affections.
![Page 42: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/42.jpg)
• As much as performance allows
• Initially 100%, now around half (?)
• Core in C++ / LLVM
• Tons in Ruby: primitives, parser, bytecode
Ruby in Ruby
The goal of Rubinius is to implement Ruby in Ruby as much as performance allows. Quite a lot of functionality you’d think would need to be in C is actually in Ruby.
![Page 43: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/43.jpg)
RubySpec, FFIBrought to you by Rubinius
(Thank you!)
We have Rubinius to thank for the executable Ruby specification that all Rubies are now judged against, and for the excellent foreign-function interface that lets you call C code in a way that’s compatible with at least four Rubies.
![Page 44: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/44.jpg)
Looking Inside Your Code
Rubinius also has tons of mechanisms for looking inside your code, which was very helpful when I needed to learn what bytecode I’d need to output to accomplish a particular task in Thnad.
![Page 45: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/45.jpg)
class Example def add(a, b) a + b endend
For example, with this class,...
![Page 46: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/46.jpg)
AST$ rbx compile -S example.rb[:script, [:class, :Example, nil, [:scope, [:block, [:defn, :add, [:args, :a, :b], [:scope, [:block, [:call, [:lvar, :a], :+, [:arglist, [:lvar, :b]]]]]]]]]]
...you can get a Lisp-like representation of the syntax tree,...
![Page 47: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/47.jpg)
Bytecode$ rbx compile -B example.rb...================= :add =================Arguments: 2 required, 2 totalLocals: 2: a, bStack size: 4Lines to IP: 2: -1..-1, 3: 0..6
0000: push_local 0 # a0002: push_local 1 # b0004: meta_send_op_plus :+0006: ret ----------------------------------------
...or a dump of the actual bytecode for the Rubinius VM.
![Page 48: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/48.jpg)
“Ruby Platform Throwdown”Moderated by Dr Nic, 2011
vimeo/26773441
For more on the similarities and differences between Rubinius and JRuby, see the throwdown video moderated by Dr Nic.
![Page 49: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/49.jpg)
III: Thnad’s Revenge
Now that we’ve gotten to know Rubinius a little...
![Page 50: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/50.jpg)
Let’s port Thnad to Rubinius!
...let’s see what it would take to port Thnad to it.
![Page 51: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/51.jpg)
Our Guide Through the Wilderness@brixen
photo: JSConf US
Brian Ford was a huge help during this effort, answering tons of my “How do I...?” questions in an awesome Socratic way (“Let’s take a look at the Generator class source code....”)
![Page 52: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/52.jpg)
Same parser
Same AST transformation
Different bytecode
(But similar bytecode ideas)
Because the Thnad syntax is unchanged, we can reuse the parser and syntax transformation. All we need to change is the bytecode output. And even that’s not drastically different.
![Page 53: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/53.jpg)
Thnad’s Four Features,Revisited
Let’s go back through Thnad’s four features in the context of Rubinius.
![Page 54: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/54.jpg)
1. Names and Numbers
First, function parameters and integers.
![Page 55: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/55.jpg)
# Numbers:ldc 42
# Names:iload 0
# Numbers:push 42
# Names:push_local 0
JVM RBX
See how similar the JVM and Rubinius bytecode is for these basic features?
![Page 56: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/56.jpg)
class Number < Struct.new :value def eval(context, builder) builder.push value endend
All we had to change was the name of the opcode both for numbers...
![Page 57: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/57.jpg)
class Name < Struct.new :name def eval(context, builder) param_names = context[:params] || [] position = param_names.index(name) raise "Unknown parameter #{name}" unless position
builder.push_local position endend
...and for parameter names.
![Page 58: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/58.jpg)
2. Function Calls
Function calls were similarly easy.
![Page 59: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/59.jpg)
ldc 42ldc 1invokestatic #2; //Method //add:(II)I
push_const :Examplepush 42push 1send_stack #<CM>, 2
JVM RBX
In Rubinius, there are no truly static methods. We are calling the method on a Ruby object—namely, an entire Ruby class. So we have to push the name of that class onto the stack first. The other big difference is that in Rubinius, we don’t just push the method name onto the stack—we push a reference to the compiled code itself. Fortunately, there’s a helper method to make this look more Bitescript-like.
![Page 60: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/60.jpg)
class Funcall < Struct.new :name, :args def eval(context, builder) builder.push_const :Thnad args.each { |a| a.eval(context, builder) } builder.allow_private builder.send name.to_sym, args.length endend
Here’s how that difference affects the bytecode. Notice the allow_private() call? I’m not sure exactly why we need this. It may be an “onion in the varnish,” a reference to a story by Primo Levi in _The Periodic Table_.
![Page 61: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/61.jpg)
flickr/black-and-white-prints/1366095561flickr/ianfuller/76775606In the story, the workers at a varnish factory wondered why the recipe called for an onion. They couldn’t work out chemically why it would be needed, but it had always been one of the ingredients. It turned out that it was just a crude old-school thermometer: when the onion sizzled, the varnish was ready.
![Page 62: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/62.jpg)
3. Conditionals
On to conditionals.
![Page 63: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/63.jpg)
0: iconst_01: ifeq 94: bipush 426: goto 129: sipush 66712: ...
37: push 038: push 039: send :==41: goto_if_false 4743: push 4245: goto 4947: push 66749: ...
JVM RBX
Here, the JVM directly supports an “if equal to zero” opcode, whereas in Rubinius we have to explicitly compare the item on the stack with zero.
![Page 64: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/64.jpg)
class Conditional < Struct.new :cond, :if_true, :if_false def eval(context, builder) else_label = builder.new_label endif_label = builder.new_label
cond.eval context, builder builder.push 0 builder.send :==, 1
builder.goto_if_true else_label
if_true.eval context, builder builder.goto endif_label
else_label.set! if_false.eval context, builder endif_label.set! endendLabels are also a little different in Rubinius, too; here’s what the bytecode for conditionals looks like now.
![Page 65: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/65.jpg)
4. Function Definitions
The trickiest part to implement was function calls.
![Page 66: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/66.jpg)
public int add(int, int); iload_1 iload_2 iadd ireturn
push_rubinius push :addpush #<CM>push_scope push_self push_const :Thnadsend :attach_method, 4
JVM RBX
Remember that in Ruby, there’s no compile-time representation of a class. So rather than emitting a class definition, we emit code that creates a class at runtime.
![Page 67: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/67.jpg)
class Function < Struct.new :name, :params, :body def eval(context, builder) param_names = [params].flatten.map(&:name) context[:params] = param_names
# create a new Rubinius::Generator builder.begin_method name.to_sym, params.count self.body.eval(context, builder.current_method) builder.current_method.ret builder.end_method endend
The code to define a method in Rubinius requires spinning up a completely separate bytecode generator. I stuck all this hairy logic in a set of helpers to make it more BiteScript-like.
![Page 68: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/68.jpg)
class Rubinius::Generator def end_method # ...
cm = @inner.package Rubinius::CompiledMethod
push_rubinius push_literal inner.name push_literal cm push_scope push_const :Thnad send :attach_method, 4 pop endend
Here’s the most interesting part of those helpers. After the function definition is compiled, we push it onto the stack and tell Rubinius to attach it to our class.
![Page 69: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/69.jpg)
Compiler
How does the compiled code make its way into a .rbc file?
![Page 70: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/70.jpg)
g = Rubinius::Generator.new
# ...
context = Hash.newexprs.each do |e| e.eval(context, g)end
# ...
As with JRuby, we create a bytecode generation object, then evaluate all the Thnad statements into it.
![Page 71: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/71.jpg)
main = g.package Rubinius::CompiledMethod
Rubinius::CompiledFile.dump \ main, @outname, Rubinius::Signature, 18
Finally, we tell Rubinius to marshal the compiled code to a .rbc file.
![Page 72: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/72.jpg)
Runner (new!)
That means we now need a small script to unmarshal that compiled code and run it. This is new; on the Java runtime, we already have a runner: the java binary.
![Page 73: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/73.jpg)
#!/usr/bin/env rbx -rubygems
(puts("Usage: #{} BINARY"); exit) if ARGV.empty?
loader = Rubinius::CodeLoader.new(ARGV.first)method = loader.load_compiled_file( ARGV.first, Rubinius::Signature, 18)result = Rubinius.run_script(method)
Here’s the entirety of the code to load and run a compiled Rubinius file.
![Page 74: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/74.jpg)
Built-ins
As we’ve just seen, defining a function in Rubinius takes a lot of steps, even with helper functions to abstract away some of the hairiness.
![Page 75: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/75.jpg)
g.begin_method :minus, 2g.current_method.push_local 0g.current_method.push_local 1g.current_method.send :-, 1g.current_method.retg.end_method
For example, here’s the built-in minus() function. I wanted to avoid writing a bunch of these.
![Page 76: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/76.jpg)
function plus(a, b) { minus(a, minus(0, b))}
I realized that you could write plus() in Thnad instead, defining it in terms of minus.
![Page 77: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/77.jpg)
function times(a, b) { if (eq(b, 0)) { 0 } else { plus(a, times(a, minus(b, 1))) }}
If you don’t care about bounds checking, you can also do times()...
![Page 78: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/78.jpg)
function eq(a, b) { if (minus(a, b)) { 0 } else { 1 }}
...and if()!
![Page 79: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/79.jpg)
stdthnadlib?!?We have a standard library!
That means we have a standard library! Doing the Rubinius implementation helped me improve the JRuby version. I was able to go back and rip out most of the built-in functions from that implementation.
![Page 80: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/80.jpg)
Thnad Onlinegithub/undees/thnad/tree/mastergithub/undees/thnad/tree/rbx
Here’s where you can download and play with either implementation.
![Page 81: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/81.jpg)
This has been a fantastic conference. Thank you to our hosts...
![Page 82: Thnad's Revenge](https://reader034.vdocuments.mx/reader034/viewer/2022052215/55516eadb4c9057e458b485b/html5/thumbnails/82.jpg)
Special Thanks
Kaspar Schiess for Parslet
Charles Nutter for BiteScript
Ryan Davis and Aja Hammerly for Graph
Brian Ford for guidance
Our tireless conference organizers!
...and to the makers of JRuby, Rubinius, Parslet, BiteScript, and everything else that made this project possible. Cheers!