Download - Hacking with ruby2ruby
![Page 1: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/1.jpg)
Hacking with ruby2rubyMarc Chung
OpenRainblog.marcchung.comblog.openrain.com@heisenthought
![Page 2: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/2.jpg)
Hacking with ruby2rubyMarc Chung
OpenRainblog.marcchung.comblog.openrain.com@heisenthought
![Page 3: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/3.jpg)
![Page 4: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/4.jpg)
ruby2ruby
Ruby2RubyWalks the Ruby AST
To emit Ruby
![Page 5: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/5.jpg)
sudo gem install ruby2ruby
![Page 6: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/6.jpg)
Limitations
• Only Ruby 1.8
• Not 1.9, yet
• Built into Rubinius
![Page 7: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/7.jpg)
function rhino_rocks() { return “Rhino, rocks!”;}
![Page 8: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/8.jpg)
js> rhino_rocks();
Rhino, rocks!
![Page 9: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/9.jpg)
js> rhino_rocks.toString();
function rhino_rocks() { return “Rhino, rocks!”;}
![Page 10: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/10.jpg)
js> eval(rhino_rocks.toString()).call();
Rhino, rocks!
![Page 11: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/11.jpg)
ruby_rocks = proc { return “Ruby, rocks”;}
![Page 12: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/12.jpg)
irb> ruby_rocks.call;
Ruby, rocks!
![Page 13: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/13.jpg)
irb> require “ruby2ruby”irb> require “parse_tree_extensions”irb> ruby_rocks.to_ruby
proc { return \“Ruby, rocks\” }
![Page 14: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/14.jpg)
irb> eval(ruby_rocks.to_ruby).call
Ruby, rocks!
![Page 15: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/15.jpg)
![Page 16: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/16.jpg)
define_methodclass Voice define_method(:shout) do |word| word.upcase! endend
puts Ruby2Ruby.translate(Voice)
![Page 17: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/17.jpg)
class Voice < Object def yell(word) word.upcase! endend
![Page 18: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/18.jpg)
aliasclass AliasClass def old_busted "42" end
alias_method :new_hotness, :old_bustedend
puts Ruby2Ruby.translate(AliasClass)
![Page 19: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/19.jpg)
class AliasClass < Object def new_hotness "42" end def old_busted "42" endend
![Page 20: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/20.jpg)
#module_functionmodule HelperModule def help puts "Help me" end module_function : helpend
puts Ruby2Ruby.translate(HelperModule)
![Page 21: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/21.jpg)
module HelperModule def help puts("Help me") end def self. help puts("Help me") endend
![Page 22: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/22.jpg)
ActiveRecord Models
class Doctor < ActiveRecord::Base has_many :patientsend
![Page 23: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/23.jpg)
See, Ruby, Run!
• Drmap
• git clone git://github.com/mchung/drmap.git
• Optionally, install Journeta
![Page 24: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/24.jpg)
Under the hood
• Abstract syntax trees
• S-expressions
• ParseTree
![Page 25: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/25.jpg)
class Book def title “Ruby programming” endend
![Page 26: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/26.jpg)
case NODE_BREAK:case NODE_NEXT: if (node->nd_stts) add_to_parse_tree(self, current, node->nd_stts, locals);
break;
case NODE_YIELD: if (node->nd_stts) add_to_parse_tree(self, current, node->nd_stts, locals);
if (node->nd_stts && (nd_type(node->nd_stts) == NODE_ARRAY || nd_type(node->nd_stts) == NODE_ZARRAY) && !node->nd_state) rb_ary_push(current, Qtrue);
break;
![Page 27: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/27.jpg)
[:class, :Book, [:const, :Object], [:defn, :title, [:scope, [:block, [:args], [:str, "Ruby programming"]]]]]
![Page 28: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/28.jpg)
class Book < Object def title “Ruby programming” endend
![Page 29: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/29.jpg)
/opt/local/usr/bin
• parse_tree_show
• r2r_show
![Page 30: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/30.jpg)
class Book def title “Ruby programming” endend
![Page 31: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/31.jpg)
$ parse_tree_show book.rb
![Page 32: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/32.jpg)
s(:class, :Book, nil, s(:scope, s(:defn, :title, s(:args), s(:scope, s(:block, s(:str, "Ruby programming"))))))
![Page 33: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/33.jpg)
$ r2r_show book.rb
![Page 34: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/34.jpg)
class Book def title() “Ruby programming” endend
![Page 35: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/35.jpg)
Operator Precedence with Guy Decoux: a b c, d
[[:fcall, :a, [:array, [:fcall, :b, [:array, [:vcall, :c], [:vcall, :d]]]]]]
ParseTree.new.parse_tree_for_string(“a b c, d”)
![Page 36: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/36.jpg)
ruby2java
![Page 37: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/37.jpg)
class JavaClass def self.main puts “Java, rocks!” endend
![Page 38: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/38.jpg)
[:class, :JavaClass, [:const, :Object], [:defs, [:self], :main, [:scope, [:block, [:args], [:fcall, :puts, [:array, [:str, "Hello, Java"]]]]]]]
![Page 39: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/39.jpg)
class Ruby2Java < SexpProcessor def process_class(exp) “public class #{exp.shift} #{next_token(exp, true)}” end
def process_const(exp) “extends #{exp.shift}” end....end
![Page 40: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/40.jpg)
public class JavaClass { public static void main(String argv[]) { return “Hello, Java”; }}
![Page 41: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/41.jpg)
![Page 42: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/42.jpg)
Distributing Ruby
• drmap - multi-machine worker queue
• gisting - map/reduce framework
![Page 43: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/43.jpg)
drmapirb> fn = proc {|x| x**10}
irb> Marshal.dump(fn)TypeError: no marshal_dump is defined for class Proc
irb> YAML::dump(fn)=> "--- !ruby/object:Proc {}\n\n"
irb> YAML::dump(fn.to_ruby)=> "--- proc { |x| (x ** 10) }\n"
irb> Marshal.dump(fn.to_ruby)=> "\004\b\"\eproc { |x| (x ** 10) }"
![Page 44: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/44.jpg)
Enumerable#drmap
![Page 45: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/45.jpg)
module Enumerable
def drmap(&block) pool = Drmap::BeanstalkPool.new
jid = rand(100) each_with_index do |element, idx| pool.put_job(jid, block.to_ruby, element) end
results = [] while results.size < length results << pool.next_result(jid) end results end
end
![Page 46: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/46.jpg)
def process loop do begin job_payload = next_job fn = job_payload[:proc] data = job_payload[:data] result = eval(fn).call(data) save(job_payload[:jid], result) success! rescue => e fail!(e) end endend
![Page 47: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/47.jpg)
drmap
• Trivial
• Distributed
• Powerful
![Page 48: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/48.jpg)
Demo time!
![Page 49: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/49.jpg)
Gisting
• MapReduce in Ruby
• http://github.com/mchung/gisting
![Page 50: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/50.jpg)
map / reduce
• Functional programming
• Iterating over collections
• (5..10).map {|x| x+1} #=> [6, 7, 8, 9, 10, 11]
• (5..10).reduce(:+) #=> 45
![Page 51: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/51.jpg)
Google MapReduce
• Modeled after map() and reduce()
• Programming model
• Programs are trivially parallelizable
![Page 52: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/52.jpg)
Demo time!
![Page 53: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/53.jpg)
Gisting::Spec
![Page 54: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/54.jpg)
input.file_pattern = "file.txt" input.map do |map_input| words = map_input.strip.split("\t") Emit(words[1], "1") end
![Page 55: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/55.jpg)
output.reduce do |reduce_input| count = 0 reduce_input.each do |value| count += value.to_i end Emit(count)end
![Page 56: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/56.jpg)
inputs = args spec = Gisting::Spec.new inputs.each do |file_input| input = spec.add_input input.file_pattern = file_input input.map do |map_input| # 2722 mailbox 2006-05-23 00:08:39 words = map_input.strip.split("\t") Emit(words[1], "1") end end output = spec.output output.filebase = "/Users/mchung/Public/datasets/output" output.num_tasks = 2 output.reduce do |reduce_input| count = 0 reduce_input.each do |value| count += value.to_i end Emit(count) end
result = MapReduce(spec) pp result
![Page 57: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/57.jpg)
Gisting
• MapReduce programming model in Ruby
• Uses ruby2ruby to serialize procs
• Uses EventMachine for Map/ReduceServer
![Page 58: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/58.jpg)
• -: 94978
• mailbox: 20872
• google: 6758
• ebay: 2832
• yahoo: 2674
• yahoo.com: 2198
• myspace.com: 1916
• google.com: 1882
![Page 59: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/59.jpg)
self.give(:thanks)OpenRain
openrain.com
My Git Repogithub.com/mchung
Marc Chungblog.marcchung.com
Ruby::AZrubyaz.org
![Page 60: Hacking with ruby2ruby](https://reader035.vdocuments.mx/reader035/viewer/2022081413/54675395af795997368b54cd/html5/thumbnails/60.jpg)
Resources
Code Generation: The Safety Scissors Of Metaprogrammingmwrc2008.confreaks.com/03bowkett.html
Forbidden Fruit: A Taste of Ruby's Parse Treegoruco2008.confreaks.com/03_wanstrath.html
Ruby2Rubyseattlerb.rubyforge.org/ruby2ruby/
ParseTreeparsetree.rubyforge.org