testing grails: experiencies from the field

Testing Grails

Source Code: github.com/zanthrash/loopback

Experiences from the Field

Colin Harrington :: SpringOne2gx 2011

Friday, October 28, 2011

Testing the CRAP out of your Grails App


whoamiColin Harrington

@ColinHarrington

[email protected]@objectpartners.com


mailto:[email protected]

mailto:[email protected]

“Testing is the engineering rigor of software development”

- Neal Ford


History


Pre Grails 1.0Testing Sucked


Grails 1.0


2.0Changes &

Enhancements


• Testing Basics

• Stubs, Spies, & Mocks

• Unit Testing

• Integration Testing

• Functional Testing

• What’s changing in Grails 2.0

• Grails Testing Ecosystem

• Experiences

Agenda


1st Class Citizen

• create-* scripts

• baked in the framework

• JUnit

• grails.test.*


Class Hierarchy


Running Tests All tests grails test-app

Failed test grails test-app -rerun

Specific test class grails test-app PresentationController

Specific test case grails test-app PresentationController.testFoo

Only unit test grails test-app unit:

Only integration test grails test-app integration:

Only Controller test grails test-app *Controller

Only test in a package grails test-app com.foo.*

Show output grails test-app -echoOut -echoErr


Phases & Types

Phases

unit

integration

functional

other

Types

unit

integration

functional

spock

custom

etc.

:http://ldaley.com/post/615966534/custom-grails-test


http://ldaley.com/post/615966534/custom-grails-test

http://ldaley.com/post/615966534/custom-grails-test

Unit Testing• No I/O

• Prove the basic correctness of the system

• State Testing

• Collaboration Testing

‣ Test Behavior NOT Implementation


void testAddSucceedsWhenSpeakerIsFound() {controller.params.title = "Test Presentation"def user = new User(id:1)

controller.springSecurityService = [currentUser: user] controller.accessCodeService = [createFrom: {title, event -> "abcd1234"}] mockDomain Presentation

controller.add()

controller.with{ assert redirectArgs.size() == 2 assert redirectArgs.controller == "speaker" assert redirectArgs.action == "index" assert flash.message == "'Test Presentation' added with access code: abcd1234" }}


Setup and Teardown• Always call super.setUp() and super.tearDown()

• super.setUp() creates:

• mock the applicationContext

• container for loaded codecs

• container for saved meta classes

• container for errorMap (for validation errors)

• Registers Sets, Lists, Maps, Errors so can be converted to XML and JSON


Setup and Teardown

• super.tearDown():

• removes the mocked meta-class from the GroovySystem.metaClassRegistry

• sets the original meta-class back in the GroovySystem.metaClassRegistry

• resets the mocked domain class id’s


Test Doubles

•Stubs

•Spies

•Mocks


Stubs

• A stand in for the real implementation with the simplest possible implementation

controller.accessCodeService = [ convertFrom: {title, event -> '1234abc'} ]


Spies• Stand in to capture an observation point

• Useful when:

• there is no output to inspect

• only concerned that an associated method or closure is called


Test Spy: Map with closure

void testControllerWithServiceCall() { def fooCalled = false controller.myService = [foo:{ a -> fooCalled = true}]

controller.methodThatCallsFooOnService()

assertTrue(fooCalled)}


Test Spy: mockFor(Class, loose=true)

void testControllerWithServiceCall() { def fooCalled = false

def mockService = mockFor(MyService, true) mockService.demand.foo() { a -> fooCalled = true} controller.myService = mockService.createMock()

controller.methodThatCallsFooOnService()

assertTrue fooCalled

}


Mocks

• mockFor()

• Used to contain the surprises & control:

• input params

• output

• method call count

• method call order


mockFor

• strict vs. loose

• createMock()

• verify()

• AVOID doing MetaClass programming yourself!


Strict Mocks: mockFor(Class)

void testControllerWithServiceCall() {

def mockService = mockFor(MyService) mockService.demand.bar(2..2) { a -> return a + 1} controller.myService = mockService.createMock()

controller.methodThatCallsBarOnService()

assert controller.redirectArgs.action == ‘foo’

mockService.verify()

}


Testing Domain Classes

• State Testing

• Constraint Testing

• mockForConstraintsTest()

• Domain Expectations Plugin


mockForConstraintsTest()

• Use in a GrailsUnitTestCase

• Think as double entry book keeping

• Signatures:

• mockForConstraintsTest( Domain )

• mockForConstraintsTest( Domain, [new Domain()]

✴ list param is used to test ‘unique’ constraint


mockForConstraintsTests(Class)

void testEventNameShouldNotValidateIfNull() { def event = new Event()

assertFalse event.validate() assertTrue event.hasErrors()

assert event.errors['name'] == 'nullable'}


Domain Expectations Plugin

• Designed to TDD your domain constraints

• Uses a GORM-like dynamic finder syntax

• expect[FieldName][Suffix]

• eg: user.expectAddressIsNotNullable

• Or a natural language like syntax

• eg: user.“address is not nullable”


Domain Expectations Example

void testEventNameConstraints() { Expectations.applyTo Event

def event = new Event()

event.expectNameIsNotNullable() event.expectNameIsNotBlank() event.expectNameHasMaxSize(40) event.expectNameHasAValidator()

event.expectNameIsValid(“jojo”)event.expectNameIsNotValid(“Bob”, ‘validator’)

}


Testing Controllers

• ControllerUnitTestCase (1.3.x)

• Mocking Domain Classes

• Mocking Services

• Mocking Grails Config


ContollerUnitTestCase

• Call super.setUp()

• Inspects the name of your test and instantiates an instance of your Controller Class and assigns it to the “controller” field.

• mocks out all the stuff in the controller class and adds helper methods for testing


What’s Mocked Out?Controller Methods

Helper MethodsHelper Methods

redirect getRedirectArgs getResponse

forward getForwardArgs getSession

chain getChainArgs getParams

render getRenderArgs getFlash

withFormat getTemplate getChainModel

withForm getModelAndView getActionName

log setModelAndView getControllerName

getRequest getServletContextFriday, October 28, 2011

Mocking Domain Objects

• mockDomain(Person)

• mockDomain(Person, [new Person()])

‣ used to simulate existing saved records


What you get

findAll load validate delete

findAllWhere getAll getErrors discard

findAllBy* ident hasErrors refresh

findBy* exists setErrors attach

get count clearErrors addTo*

read list save removeFrom*

create

= don’t do anything in testingFriday, October 28, 2011

• String based HQL queries

• Composite Identifiers

• Dirty checking methods

• Direct interaction with Hibernate

What you don’t get


Controller Unit Test Example

void testAddSucceedsWhenSpeakerIsFound() { controller.params.title = "Test Presentation" def user = new User(id:1) controller.springSecurityService = [currentUser: user] controller.accessCodeService = [createFrom: {title, event -> "abcd1234"}] mockDomain Presentation

controller.add()

controller.with{ assert redirectArgs.size() == 2 assert redirectArgs.controller == "speaker" assert redirectArgs.action == "index" assert flash.message == "'Test Presentation' added with access code: abcd1234" }}


Mocking Grails Config

mockConfig(''' braintree { merchantKey = "abcd1234" }''')

With String

With actual Config.groovy file *

mockConfig( new File('grails-app/conf/Config.groovy').text)

* violates No I/O ruleFriday, October 28, 2011

Testing URL Mappings

• GrailsUrlMappingsTestCase

• UrlMappingsUnitTestMixin (2.0+)

• Forward, Reverse, View and Response Code mappings


UrlMappings.groovyclass UrlMappings {! static mappings = {! ! "/$controller/$action?/$id?"{! ! ! constraints {! ! ! ! // apply constraints here! ! ! }! ! }! ! "/login"(controller: 'login', action:'auth')

! ! "/comment/$action/$accessCode"(controller: 'comment')

! ! "/"(controller:'speaker', action:"index")! ! "500"(view:'/error')! ! "404"(view:'/error404')! }}


Testing URLMappings

void testLoginPage() { assertUrlMapping(“/login”, controller:‘login’, action: ‘auth’)}

void testErrorPages() { assertUrlMapping(500, view: ‘error’) assertUrlMapping(404, view: ‘error404’)}

void testSiteRoot() { assertUrlMapping(‘/’, controller: ‘speaker’, action: ‘index’)}


Testing URLMappingsvoid testCommentMappings() {! assertUrlMapping("/comment/code/12345", controller: 'comment', action: 'code') {! ! accessCode = "12345"! }! assertUrlMapping("/comment/post/12345", controller: 'comment', action: 'post') {! ! accessCode = "12345"! }}

void testDefaultControllerActionIdMapping() {! assertUrlMapping("/event/show/3210", controller: 'event', action: 'show') {! ! id = 3210!!! }! assertUrlMapping("/presentation/show/125", controller: 'presentation', action: 'show') {! ! id = 125! }}


Testable Mappings

• Forward

• Reverse

• Status Code

• Views


static mappings = {

"/product/$id"(controller: "product") { action = [GET: "show", PUT: "update", DELETE: "delete", POST: "save"] }!}

No HTTP Methods support until 2.0


Testing Taglibs

• unit or integration

• TagLibUnitTestCase

• GroovyPagesTestCase


Unit Testing TagLibsclass LoopbackTagLibTests extends TagLibUnitTestCase {! protected void setUp() {! ! super.setUp()! }

! void testFixedWidthIpDefaultsTag() {! ! tagLib.fixedWidthIp(null, null)! ! assertEquals "................", tagLib.out.toString()! }

! void testFixedWidthIpParameters() {! ! tagLib.fixedWidthIp(ip: '127.0.0.1', width: '20', pad: '=', null)! ! assertEquals "127.0.0.1===========", tagLib.out.toString()

! ! shouldFail {! ! ! tagLib.fixedWidthIp(ip: '127.0.0.1', width: 'not-a-number', pad: '=', null)! ! }! }}


Integration Testing TagLibs

import grails.test.*

class LoopbackTagLibGSPTests extends GroovyPagesTestCase {

! void testFixedWidthIp() {! ! def template = '<g:fixedWidthIp ip="127.0.0.1" width="21" pad="-"/>'

! ! assertOutputEquals('127.0.0.1------------', template)! }

}


Testing GSPs

• Integration Tests

• GroovyPagesTestCase

• Views

• Templates

• assertOutputEquals(expected, template, model)


Integration Testing

• Domain Objects. Grails vs Rails

• Services

• Plugin utilization

• Rendering of Views

slower than unit testsFriday, October 28, 2011

Testing Services

• Autowired Beans

• Transactionality

• Full Database interactions

• Fixtures + Build Test Data


Functional Testing

• Testing from the outside

Fully functional app

Real database

Real HTTP traffic

Real integrations

Geb

Canoo Webtest

Selenium RC

EasyB

Fitness

BDD


Geb + Spockimport geb.spock.GebSpec

class MySpec extends GebSpec {

def "test something"() { when: go "/help"

then: $("h1").text() == "Help" }

}


Spock

• Highly expressive specification language

• Given ➡ When ➡ Then

• Drop in replacement for Testing Plugin for both Unit & Integration test

• Simple but detailed assertions

• Parameterized testing with wiki like syntax


Testing with Spock: Parallel Hierarchy

Testing Plugin Spock Plugin

GrailsUnitTestCase UnitSpec

ControllerUnitTestCase ControllerSpec

TagLibUnitTestCase TagLibSpec

GroovyTestCase IntegrationSpec

GroovyPagesTestCase GroovyPagesSpec


Spock Parameterized Test Example: class ParameterizedSpec extends UnitSpec{

@Unroll("#a + #b = #c") def "summation of 2 numbers"() { expect: a + b == c

where: a |b |c 1 |1 |2 2 |1 |3 3 |2 |6 }}

Condition not satisfied:a + b == c| | | | |3 5 2 | 6 false


Spock ControllerSpec Example: class PresentationControllerSpec extends ControllerSpec{

@Shared def user = new User(id:1)

@Unroll("add presentation for user: #testUser with title: #title") def "call to add a new presentation for logged in user"() { given:"collaborators are mocked" mockDomain Presentation mockDomain Speaker, [new Speaker(user:user)] controller.springSecurityService = [currentUser: testUser ] controller.accessCodeService = [createFrom:{title, event -> "1234abcd"}] and:"prarams are set" controller.params.event = "Event Name" controller.params.date = new Date() controller.params.title = title when:"call the action" controller.add()

then:"should be redirected to '/speaker/index'" controller.redirectArgs.controller == 'speaker' controller.redirectArgs.action == 'index'

and: "should have correct flash message" controller.flash.message == expectedFlashMessage

where: title | testUser |expectedFlashMessage 'test' | user |"'test' added with access code: 1234abcd" null | user |"Could not add null to your list at this time" 'test' | null |"Could not find a Speaker to add test to your list at this time" }}


2.0 Changes

• Testing Hierarchy is dead

• Replaced with annotated mix-ins

• In-memory GORM implementation

• Unit Tests finally use GrailsMockHttpServletReponse


2.0 Testing Annotations & Mixins

@TestFor(ClassUnderTest)

@Mock(Class) OR @Mock([Class1, Class2])

@TestMixin(Mixin)

- DomainClassUnitTestMixin- ControllerUnitTestMixin- FiltersUnitTestMixin- GroovyPageUnitTestMixin- ServiceUnitTestMixin- UrlMappingsUnitTestMixin


2.0 Domain Constraints Example

@TestFor(Event)class EventTests {

@Before void setup() { mockForConstraintsTests Event }

@Test void eventNameShouldNotValidateIfNull() { def event = new Event()

assertFalse event.validate() assertTrue event.hasErrors() assert event.errors['name'] == 'nullable' }}


2.0 Controller Example@TestFor(PresentationController)@Mock([User, Speaker, Presentation])class PresentationControllerTests {

User user

@Before void setUp() { controller.params.title = "Test Presentation" user = new User(id:1) new Speaker(user:user).save(validate: false) }

@Test void testAddSucceedsWhenSpeakerIsFound() { controller.springSecurityService = [currentUser: user] controller.accessCodeService = [createFrom: {title, event -> "abcd1234"}]

controller.add()

assert response.redirectedUrl == '/speaker/index' assert flash.message == "'Test Presentation' added with access code: abcd1234" }}


Ingredients for Success

• Jenkins

• Git / VCS

• Servers


Tools


Plugins

• Spock (http://code.google.com/p/spock/)

• Spock Plugin (http://www.grails.org/plugin/spock)

• Geb (http://geb.codehaus.org/latest/index.html)

• Domain Expectation Plugin (http://www.grails.org/plugin/domain-expectations)

• Build Test Data Plugin (http://www.grails.org/plugin/build-test-data)

• Remote Control Plugin (http://www.grails.org/plugin/remote-control)

• Fixtures Plugin (http://www.grails.org/plugin/fixtures)

• Greenmail Plugin (http://www.grails.org/plugin/greenmail)


http://code.google.com/p/spock/

http://code.google.com/p/spock/

http://www.grails.org/plugin/spock

http://www.grails.org/plugin/spock

http://geb.codehaus.org/latest/index.html

http://geb.codehaus.org/latest/index.html

http://www.grails.org/plugin/domain-expectations

http://www.grails.org/plugin/domain-expectations

http://www.grails.org/plugin/build-test-data

http://www.grails.org/plugin/build-test-data

http://www.grails.org/plugin/greenmail

http://www.grails.org/plugin/greenmail

Perspective


Spectrum

• Run-before-checkin

• Continuous Testing (on each checkin)

• Scheduled Testing

• Load Testing?

• Manual Testing

• Regression Testing


The Future is now

• Git + Gerrit + Jenkins

http://alblue.bandlem.com/2011/02/gerrit-git-review-with-jenkins-ci.html

http://www.infoq.com/articles/Gerrit-jenkins-hudson






Considerations:• Passing Tests != stable != quality // Think!

• Test your Migrations { structure, data }

• Finite Resources

• Developers fight with ‘Done’

• Lost in the weeds.

• lack of skill with the tools, (its OK)

• Professional Yak shaving


Code Coverage• Focus on test quality NOT test coverage

void testWhyCodeCoverageNumbersAreCrap() { def service = new AccessCodeService() service.metaClass.methods.*name.sort().unique().each { try{ service.invokeMethod(it, ['blah', 'blah']) } catch (Exception ex ){ //eat it } } }


Now what?

• Download the Testing Cheat Sheet

• http://bit.ly/GRAILSTESTING

• Look into Geb and Spock

• Take the small steps forward!

Source Code: github.com/zanthrash/loopbackFriday, October 28, 2011

http://bit.ly/GRAILSTESTING

http://bit.ly/GRAILSTESTING

Questions ?

Thanks


Thank you


testing grails: experiencies from the field

Technology