POLOWORKING WITH REAL WORLD DATA IN

DEVELOPMENT@nettofarah

1 / 39

nettofarahgithub.com/nettofarah

@nettofarahhttp://bit.ly/polo-rb

2 / 39

Real World Production Data inDevelopment

3 / 39

Because Managing Data is hardEspecially across Different environments

4 / 39

Data in Rails.env.devis...

Ugly, Incomplete, Weird and Biased

5 / 39

Some Consequences of InconsistentData

6 / 39

!"!", ## misalignment, formvalidation, special characters $$

7 / 39

The Worst Consequence

8 / 39

Rails.env.devwill look %

9 / 39

But how do people solve theseproblems?

10 / 39

We can test it in PRODUCTION

11 / 39

Rails.env.staging ?

12 / 39

Custom db/migrationsfor data

13 / 39

rake db:seed

14 / 39

CSVs and Spreadsheets

15 / 39

16 / 39

Sample Database Snapshots

17 / 39

Your good ol'

ActiveRecord::Associations

to

.sql

18 / 39

POLO lets you turn this...

19 / 39

class Chef < ActiveRecord::Base has_many :recipes has_many :ingredients, through: :recipesend

class Recipe < ActiveRecord::Base has_many :recipes_ingredients has_many :ingredients, through: :recipes_ingredientsend

class Ingredient < ActiveRecord::Baseend

class RecipesIngredient < ActiveRecord::Base belongs_to :recipe belongs_to :ingredientend

20 / 39

Into this...

21 / 39

inserts = Polo.explore(Chef, 1)

# Chef -> ActiveRecord::Base object# 1 -> Database ID. (Chef with ID 1)

INSERT INTO `chefs` (`id`, `name`) VALUES (1, 'Netto')

22 / 39

It works with associations too

23 / 39

inserts = Polo.explore(Chef, 1, :recipes)

# :recipes -># ActiveRecord::Associations::HasManyAssociation## a Chef has many Recipes

INSERT INTO `chefs` (`id`, `name`) VALUES (1, 'Netto')

INSERT INTO `recipes` (`id`, `title`, `num_steps`, `chef_id`) VALUES (1, 'Turkey Sandwich', NULL, 1)

INSERT INTO `recipes` (`id`, `title`, `num_steps`, `chef_id`) VALUES (2, 'Cheese Burger', NULL, 1)

24 / 39

It also works with nestedassociations

25 / 39

inserts = Polo.explore(Chef, 1, { :recipes => :ingredients })# { :recipes => :ingredients } -># load every recipe and ingredientes

...INSERT INTO `recipes` (`id`, `title`, `num_steps`, `chef_id`) VALUES (1, 'Turkey Sandwich', NULL, 1)

INSERT INTO `recipes` (`id`, `title`, `num_steps`, `chef_id`) VALUES (2, 'Cheese Burger', NULL, 1)

INSERT INTO `recipes_ingredients` (`id`, `recipe_id`, `ingredient_id`) VALUES (1, 1, 1)

INSERT INTO `recipes_ingredients` (`id`, `recipe_id`, `ingredient_id`) VALUES (2, 1, 2)...

INSERT INTO `ingredients` (`id`, `name`, `quantity`) VALUES (1, 'Turkey', 'a lot')

INSERT INTO `ingredients` (`id`, `name`, `quantity`) VALUES (2, 'Cheese', '1 slice')...

26 / 39

&&Show me the magic!

27 / 39

Collect all the objectslib/polo/collector.rb#L16

28 / 39

asn = ActiveSupport::Notifications# Extracted so this can fit in a slide

asn.subscribed(collector, 'sql.active_record') do# Set up ActiveRecord::Preloader base_finder = @base_class. includes(@dependency_tree) .where(id: @id)

# Store SELECTS in some global storage collect_sql(@base_class, base_finder.to_sql)end

29 / 39

''Transform them to SQLlib/polo/sql_translator.rb#51

30 / 39

attributes = record.attributes

keys = attributes.keys.map do |key| "`#{key}`"end

values = attributes.map do |key, value| column = record.column_for_attribute(key) attribute = cast_attribute(record, column, value) connection.quote(attribute)end

joined_keys = keys.join(', ')joined_values = values.join(', ')

table_name = record.class.table_name

"INSERT INTO `#{table_name}`" +" (#{joined_keys}) VALUES (#{joined_values})"

31 / 39

PROFIT! ((

32 / 39

I need your help! ))

33 / 39

on Githubhttp://github.com/IFTTT/polo

34 / 39

Pull Requests **

35 / 39

PostgreSQL, SQLite and Oraclesupport

(advanced features only, basic usage stillworks)

36 / 39

ActiveRecord < 3.2 support

37 / 39

Check out some other really coolOpen Source projects

http://ifttt.github.io

38 / 39

Questions?

39 / 39