design summit - rails 4 migration - aaron patterson
DESCRIPTION
ManageIQ currently runs on Ruby on Rails 3. Aaron "tenderlove" Patterson presents his effort to migrate to RoR 4, which entails some changes in the code to take advantage of the latest advances in RoR. For more on ManageIQ, see http://manageiq.org/TRANSCRIPT
( °͡ ʖ͜ °͡) .oO(Hi)
Aaron Patterson
Ruby Core Rails Core
Enterprise Software
Twitter: tenderlove
GitHub: tenderlove
Instagram: tenderlove
Yo: tenderlove
Tender Parents
OMG
INTERNET
POINTS
Revert Commits Count Too!
More mistakes == more points!!!!
Short Stack Engineer
Gorbachev Puff Puff Thunderhorse
SEA-TAC Airport YouTube
I am new!
LostPassport
Very 😴
I am new!
Upgrading to Rails 4.x
Upgrading to Edge Rails
Rails Release Timeline
4.2 is Very Soon™
Next is 5.0
Upgrading Rails
Why?
Performance
Memory Reduction
Cost
How?
Push upstream
Move to gems
Change implementation
Issues Today
Rails Fork
Rails Monkey Patches
Monkey Patches
Rails Fork
Active Record Object Allocation Time
ms = Benchmark.ms do records = rows.map { |model| # … }.uniq end
logger.debug(' %s Inst Including Associations (%.1fms - %drows)' % [join_base.active_record.name || 'SQL', ms, records.length])
We can’t upstream this :-(
ActiveSupport Notifications
ActiveSupport::Notifications.instrument( "some.key", payload ) do records = rows.map { |model| # ... }.uniq end
ActiveSupport::Notifications.subscribe('some.key' ) do |value| # same logger statement end
Truncate table
# Executes the truncate statement. def truncate(table_name, name = nil) execute("TRUNCATE TABLE #{quote_table_name(table_name)}", name) end
Pushed up stream
Added a monkey patch to backport
64 bit primary keys
In PG: "integer" == 32bit
In PG: "bigint" == 64bit
Default is 32bit
Why?
¯\_(ツ)_/¯
Schema.rb
create_table "accounts" do |t| t.string "name" t.integer "acctid" end
No PK
declared
Means32bit
We can upstream this
Determine backwards compat scheme
Default PK sequences
+ return if options[:id] == false + return unless self.respond_to?(:set_pk_sequence!) + + value = ActiveRecord::Base.rails_sequence_start + set_pk_sequence!(table_name, value) unless value == 0
We can’t upstream this
Extract and "super"
Add "extension" points.
Auto-reconnect
Supposedly fixed?
+ def self.did_retry_db_connection(connection,count) + logger.info "CONNECTION RETRY: #{connection.class.name} retry ##{count}." if logger + end
Push Up Better Exceptions
Connection Extensions
Database Statistics
Create a new gem: activerecord-pg-stats
Virtual Columns
Declaration
virtual_column :name, :type => :string
2 Responsibilities
Declare a "column type" for reporting
Declare associations used by the method
Separate these concerns
Extract reporting
Push up relation declaration.
RJS
Land the Angular patch!!!!
jquery-rjs
prototype-rails
Monkey Patches
First Glance
Churn
"How often is stuff changing?"
git log --all -M -C --name-only --format='format:' "$@" | sort | grep -v '^$' | uniq -c | sort -n | awk 'BEGIN {print "count\tfile"} {print $1 "\t" $2}'
445 vmdb/app/models/miq_server.rb 498 vmdb/app/models/miq_provision_workflow.rb 508 vmdb/app/models/miq_provision.rb 590 vmdb/app/models/miq_report.rb 720 vmdb/app/controllers/report_controller.rb 777 vmdb/app/models/vm.rb 802 vmdb/app/models/host.rb 825 vmdb/app/helpers/application_helper.rb 1186 vmdb/app/controllers/application.rb
Cyclomatic Complexity
Quantify Code Complexity
flog
1965.9: vmdb/db/migrate/20100302205959_collapsed_initial_migration.rb:7 1131.5: vmdb/app/controllers/ops_controller/settings/common.rb:621 1024.9: vmdb/app/controllers/application_controller/filter.rb:137 951.9: vmdb/app/controllers/application_controller/performance.rb:220 862.9: vmdb/spec/models/ems_refresh/refreshers/vc_refresher_spec.rb:27 842.8: vmdb/app/controllers/ops_controller/settings/common.rb:881 824.5: vmdb/spec/models/ems_refresh/refreshers/rhevm_refresher_3_1_spec.rb:230 791.5: vmdb/app/helpers/application_helper.rb:648
application_helper $$$$$$
Static Analysis
Find unused methods
Ripper
require 'ripper'
class MyParser < Ripper def on_def name, _, _ # do something end end
parser = MyParser.new "some code" parser.parse
class MyParser < Ripper attr_reader :methods, :calls
def initialize code, file, line super @methods = [] @calls = [] end
def on_def name, *rest @methods << name end alias :on_command :on_def
def on_defs target, dot, name, *args @methods << name end
def on_call target, dot, name @calls << name end
def on_fcall name @calls << name end alias :on_symbol :on_fcall alias :on_vcall :on_fcall end
Find.find(ARGV[0]) do |file| next if File.directory? file ext = file[/(?<=\.)\w+$/] next unless ext case ext when 'rb' parser = MyParser.new(File.read(file), file, 1) parser.parse when 'erb' parser = MyParser.new ERB.new(File.read(file)).src, file, 1 parser.parse end end
Dynamic Dispatch
superclass.send("virtual_#{m}")
Static Analysis + Ruby = 😰
Probably also why Brakeman isn’t perfect
This Summit is GREAT!
Communication
Browser
WebServer
Memcached
Every 500msBrowser
WebServer
Memcached
Every 500msBrowser
WebServer
Memcached
Cache Size
Code Complexity
Bugs
Server Load
Every 500msBrowser
WebServer
Memcached
Always Question Assumptions
Ask Why
Business Requirements aren’t set in stone.
Thanks Everyone!
Questions?