testing survival guide
DESCRIPTION
In this talk that I gave at RailsWayCon I talk about practices that help to maintain readable, fast and simple test. I also show some examples where hard tests point to design issues. In the last part i introduced some tool that may help to maintain a good test suite.TRANSCRIPT
TESTING SURVIVAL GUIDEThilo Utke
A high level view on how to keep sane while doing tdd
WHY DO WE TEST?
why we take this extra step
Write Extra Code
Interrupt the Flow
Add Complexity to our Codebase
More Maintenance
TESTING CONS
the negative effects are either blends or can be minimized
Think Double
Avoid Errors
Narrow Down Bugs
Prevent Regression
Improve Design
TESTING PROS
This is what I get out of TDD
Failing Scenarios:
cucumber features/currency_conversion.feature:129 # Scenario: Using the web-service supplied exchange rate for XAU
cucumber features/statistics.feature:161 # Scenario: view statistics for completed withdrawals
cucumber features/statistics.feature:189 # Scenario: view statistics for completed deposits
cucumber features/transaction_confirmation.feature:6 # Scenario: Fund transactions must be confirmed before the funding is added to the balance
In situations like this I ask myself the following questions
BUG IN CODE?
BUG IN TEST?
EXTERNAL DEPENDENCY?
A NEW CUCUMBER VERSION?
A BUG IN RUBY BIG DECIMAL!
forget to use rvm ruby
TESTS GIVE US CONFIDENCE
ALLOW US TO MOVE FORWARD
IMHO Lack of confidence reason why software in Enterprise tend to become outdatedNot to philosophical
DOING TDD WITH RUBY IS EASY
class Test{ //JMock & JUnit public void testWithCategoryNameGetNameOfPostCategory(){ final Category category = context.mock(Category.class) //MockObjects final Post post = context.mock(Post.class) oneOf (post).name; will(returnValue('TestPost')); //Stub Methods oneOf (post).category; will(returnValue(category)) context.checking(new Expectations() {{ //Mock oneOf (category).name; will(returnValue('TestCat')) }} PostNamesWithCategoryLoader loader = new PostNamesWithCategoryLoader context.assertIsSatisfied(); }}
might be improved a little (anotations?)
# mocha
def test_show_gets_name_of_post_category category = mock('category', :name => 'test') # mock, name must be called post = stub('post', :category => category) #stub Post.stubs(:find).returns(post) #partial stub for find get :showend
Ruby the better Language for Testing DSL
DOING TDD RIGHT IS HARD
How often do you swear at your breaking test? How often does you feel that tests break your flow?
Simple
Fast
Maintainable
Durable
Side-effect Free
Repeatable
Thats what you want.
GUIDELINES NOT RULES
1. Part I assume the basics like using setups, cleaning upmostly unittests
1. SIMPLICITY
Break down the domain problem in no brainers
THINK DOUBLE
What I want
Required Info
How get these Info
Presentation
Controller
DomainModel
For customer information I want a listing of payment providers and their logos. Their logo should be linked when I provide a link. If their is no logo, show the link instead.
STORY
How many started to think of it as one problem to solve?Split it up in separate problems.
LOGO URL
X X
X
X
REDUCE TO NO-BRAINER
Do this on complex problems and you will implement them easier, sometimes this will leave you wondering.
it "should show the description for a payment provider" do payme = PaymentProvider.new(description: 'Pay me') payment_provider_listing(payme).should include(payme.description)end
BE EXPLICIT
Not a good Idea.
it "should show the description for a payment provider" do payme = PaymentProvider.new(description: 'Pay me') payment_provider_listing(payme).should include('Pay me')end
BE EXPLICIT
Be explicit.
payment_provider_listing(payme).should == '<a href="pay.me">PayMe</a>, Get Paid'
VS
payment_provider_listing(payme).should include('<a href="pay.me">PayMe</a>')
1 THING AT A TIME
That also relates to another problem
2. DURABILITY
payment_provider_listing(payme).should == '<a href="pay.me">PayMe</a>, Get Paid'
VS
payment_provider_listing(payme).should include('<a href="pay.me">PayMe</a>')
1 THING AT A TIME
payment_provider_listing(payme).should match(/<a.*href="pay\.me".*>PayMe<\/a>/)
RELAX SPECIFICATION
Improves durability but not so easy to read anymore. Some things contradict.
it "should initialize the correct gateway with the order" do order = Order.make(:ecurrency => 'GoldPay') GoldPayGateway.should_receive(:new).with(order, anything) GatewayFactory.build(order)end
RELAX SPECIFICATION
Next Topic
MAINTAINABILITY
it "should return the last created owner as the current owner" do gateway = Gateway.create!(api_key: 'XYZ', url: 'api.pay.me') provider = PaymentProvider.create!(url: 'pay.me', name: 'Payme', gateway: gateway) owner_1 = Owner.create!(first_name: 'Phil', name: 'Adams', provider: provider) owner_2 = Owner.create!(first_name: 'Maria', name: 'Williams', provider: provider) provider.current_owner.should == owner_2 end
This is not good!
it "should return the last created owner as the current owner" do @provider_with_two_owners.current_owner.should == @owner_2 end
This neitherYou ask why?
2. CONTEXT
Context is importantWhat are you dealing with
it "should return the last created owner as the current owner" do gateway = Gateway.create!(api_key: 'XYZ', url: 'api.pay.me') provider = PaymentProvider.create!(url: 'pay.me', name: 'Payme', gateway: gateway) owner_1 = Owner.create!(first_name: 'Phil', last_name: 'Adams', provider: provider) owner_2 = Owner.create!(first_name: 'Maria', last_name: 'Williams', provider: provider) provider.current_owner.should == owner_2end
TOO NOISY
To much
it "should return the last created owner as the current owner" do @provider_with_two_owners.current_owner.should == @owner_2 end
NO CONTEXT
To little
it "should return the last created owner as the current owner" do provider = Provider.make owner_1 = Owner.make provider: provider owner_2 = Owner.make provider: provider provider.current_owner.should == owner_2 end
MAINTAIN CONTEXT
right amout
describe 'transaction' do before(:each) do @payer = User.make end describe "percentage payout bonus set" do before(:each) do PayoutBonus.create!(amount: 20, unit: :percentage) end end describe "fixed payout bonus set" do before(:each) do PayoutBonus.create!(amount: 10, unit: :usd) end endend
SPLIT SETUP TO DRY CONTEXT
another contradiction because this also increases creates complexity by adding new places
BETTER UNDERSTANDABLE THAN DRY
others have to work with your code
3. SPEED
Most important for unit tests as you run them over and over again
TEST IN ISOLATION
it "should be false when order has a single product from a single partner" do partner = Partner.make product = Product.make new_partner_name: partner.name order = Order.make_unsaved partner: partner, contact: Contact.make, address: Address.make order.items.build product: product, price: 100, scale_basis: 1, quantity: 1 order.save! order.reload order.should_not have_multiple_product_partnersend
Most speed is gained if only that code executes that is necessary for that test
TEST IN ISOLATION
it "should be false when order has a single product from a single partner" do product = Product.make new_partner_name: "Pear" order = Order.make_unsaved order.items << OrderItem.make_unsaved :product = product order.should_not have_multiple_product_partnersend
Most speed is gained if only that code executes that is necessary for that test
ISOLATION THROUGH MOCKING
Instead of real dependencies inject mocksdifferent techniques
FAKES
• Mimic the behavior of the real object but don’t share all characteristics
good example are in memory data storage vs. persistence.
FAKE USAGE
class FakeActivityLogger def log(object) @changes[object.id] ||= [] @changes[object.id] << object.changes end def changes_for(object) @changes[object.id] endend
it "should call loggers on changes" do logger = FakeActivityLogger.new @logger_config.register(logger, User) user = User.make(name: 'Paul') user.update_attribute(:name, 'Paula') logger.changes_for(user).should = [:name, 'Paul', 'Paula']end
STUBS
• Pretend to be some object but without any logic
STUB USAGE
it "should change the given attribute" do logger = stub('stub_logger', log: true) @logger_config.register(logger, User) user = User.make(name: 'Paul') user.update_attribute(:name, 'Paula') user.name.should == 'Paula'end
MOCKS
• Pretend to be some object, also no logic but monitor if interaction with them is specified
MOCK USAGE
it "should call loggers on changes" do logger = mock('mock_logger') @logger_config.register(logger, User) user = User.make(name: 'Paul') logger.expects(:log).with(user).once user.update_attribute(:name, 'Paula')end
MOCKS AND FAKES CAN HIDE INTEGRATION BUGS
Integration or Acceptancetests to the rescueexcessive use of mocking my counteract fast testing if more integration test is required
WRONG USAGE OF MOCKS HURT DURABILITY
MOCK BEHAVIOR UNDERT TEST AND STUB THE REST
Rule of thump
LISTEN TO YOUR TESTS “If something hurts you probably doing it wrong”
examples taken from real code I was involved.
TO MANY DEPENDENCIES
it "should be false when order has a single product from a single partner" do partner = Partner.make product = Product.make new_partner_name: partner.name order = Order.make_unsaved partner: partner, contact: Contact.make, address: Address.make order.items.build product: product, price: 100, scale_basis: 1, quantity: 1 order.save! order.reload order.should_not have_multiple_product_partnersend
Bad Design
TO MANY DEPENDENCIES
• split up
• add layer
• decouple logic
MANY MOCKS / SETUP FOR INTERNALS
before(:each) do @converter = mock_model CurrencyConverter, convert: 4, exchange_fee: 4, convertible?: true CurrencyConverter.stub!(:new).and_return(@converter) @modified_converter = mock_model ModifiedCurrencyConverter convert: 200 ModifiedCurrencyConverter.stub!(:new).and_return(@modified_converter) @user = mock_model(User, cleared_balance: 1000, add_balance_transaction: true, request_balance_transaction: true) @currency_conversion = CurrencyConversion.new source_amount: 100, source_currency: 'USD', destination_currency: 'EUR', user: @user end it "should request source amount plus fee from users source currency balance" do @user.should_receive(:request_balance_transaction).with(@currency_conversion, 104, 'USD') @currency_conversion.save! end
5 mocks/partial mocks
INTERNAL DEPENDENCIES
• Inject Dependencies
• If from callbacks, use a observer or think about Presenter/Service
STUB CHAINS/METHOD CHAINS
it "should be false when order has a single product from a single partner" do order = Order.make_unsaved item = OrderItem.make_unsaved :name => 'iPet' order.stub_chain(:items, :delivered, :from_partner, :last => item) last_delivered_item_for_partner_label(partner, order).should include('iPed')end
artificialexposes to many internals
EXPLAINING COMMENTS ON EXPECTATIONS
it "should sum all credits for the partner" do credit1 = @partner.credits.make(:order => @order, :payment => 100) credit1.items.make :price => 100, :quantity => 1 credit2 = @partner.credits.make(:order => @order, :payment => 100) credit2.items.make :price => 150, :quantity => 1 @partner.credits.sum_for_month(Date.today.month, Date.today.year).should == 297.5 # including 19% taxend
TOOLS
help to write faster/better testsbeside your test/mock framework of choice
MORE INFRASTRUCTURE
more to maintain
BENEFIT > COST ?
SPORK
+Reduce startup time for testing frameworks (RSpec, Cucumber, Test-Unit)
- Reloading breaks for code loaded in environment/initializers
Great timesaver in unittest for bigger projects with lot of gems and plugins
BUNDLER
+full dependency resolution at once
+version lockdown
- beta
fixed version are great, no surprises with unexpected updates.
FACTORIES(MACHINIST, FACTORY GIRL)
+Greatly remove noise in tests
+Dry Setups
+Keep Context
- DB Overhead
HYDRA/PARALLEL SPEC
+distribute tests on multiple cores or even machines
- extra setup
- concurrency/load order issues
So far no serious project running with them
CAPYBARA
+Allow to run cucumber features agains different backends
+full stack testing with culerity or selenium where required
- not one feature on many backends
setup is a super easy with cucumber
WEBMOCK/SHAM_RACK
+Allow to fake or mock http apis
- Don’t tell you when the real api changes ;)
ENVIRONMENTS
+Allow to isolate tools completely
- Extra startup time
cucumber is doing it, you can do this too.
INFO
• Name: Thilo Utke
• Company: Upstream-Agile GmbH
• Web: http://upstre.am
• Twitter : @freaklikeme