Nicholas C. ZakasYahoo!, Inc.

YUIConfNovember 9, 2010

Who's this guy?

@captain_picard

@slicknet

Who's this guy?

Principal Front End Engineer

Contributor,Creator of YUI Test

Author Lead Author Contributor Lead Author

YUI TestHow'd we get here?

Automated Integration Testingwith YUI Test (Talk)October 2009

YUI Test DebutJuly 2007 (YUI 2.3.0)

Async TestingTestReporterXML and JSON Test FormatsDecember 2007 (YUI 2.4.0)

Mock ObjectsFriendly Test NamesSeptember 2009 (YUI 3.0.0)

Test-Driven Developmentwith YUI Test (Talk)October 2008

YUI Test DebutJuly 2007 (YUI 2.3.0)

Getting started with YUI Test (YUIBlog)December 2008

Writing effective tests(YUIBlog)January 2009

Automation HooksJUnit XML and TAP FormatsApril 2010 (YUI 3.1.0)

ZOMG!!!Today

Yeti ReleasedAugust 2010

Today

YUI TestStandalone Library

YUI TestSelenium Driver

YUI TestCoverage

Today

YUI TestStandalone Library

YUI TestSelenium Driver

YUI TestCoverage

YUI Test Standalone Library has no YUI dependencies

You can load it completely on its own – outside of YUI 2.x and YUI 3.x

Why?To address a series of ongoing

problems

ProblemInconsistencies across versions

YUI Test for YUI 2.xYAHOO.tool.TestCaseYAHOO.tool.TestSuiteYAHOO.tool.TestRunnerYAHOO.tool.TestReporterYAHOO.tool.TestManagerYAHOO.tool.TestFormatYAHOO.util.AssertYAHOO.util.ArrayAssertYAHOO.util.DateAssertYAHOO.util.ObjectAssert

YUI Test for YUI 3.xY.Test.CaseY.Test.SuiteY.Test.RunnerY.Test.ReporterY.Test.FormatY.AssertY.ArrayAssertY.DateAssertY.ObjectAssertY.Mock

Hmmmm...Y.Test.CaseY.Test.SuiteY.Test.RunnerY.Test.Reporter

Y.Test.FormatY.AssertY.ArrayAssertY.DateAssertY.ObjectAssertY.Mock

YAHOO.tool.TestCaseYAHOO.tool.TestSuiteYAHOO.tool.TestRunnerYAHOO.tool.TestReporterYAHOO.tool.TestManagerYAHOO.tool.TestFormatYAHOO.util.AssertYAHOO.util.ArrayAssertYAHOO.util.DateAssertYAHOO.util.ObjectAssert

YUI Test Standalone 1.0.0YUITest.TestCaseYUITest.TestSuiteYUITest.TestRunnerYUITest.ReporterYUITest.PageManagerYUITest.TestFormatYUITest.AssertYUITest.ArrayAssertYUITest.DateAssertYUITest.ObjectAssertYUITest.Mock

ProblemI'd like to use YUI Test, but not YUI

ProblemEveryone else is doing it

What's it look like?

<script type="text/javascript" src="yuitest.js"></script><script type="text/javascript">var testCase = new YUITest.TestCase({ name: "Same ol' test case",

testSomething: function(){ YUITest.Assert.isTrue(true); },

"This should work": function(){ YUITest.Assert.isTrue(works); }});

YUITest.TestRunner.add(testCase);YUITest.TestRunner.run();</script>

FamiliarSyntax!

var testSuite = new YUITest.TestSuite("name");testSuite.add(testCase1);testSuite.add(testCase2);

YUITest.TestRunner.add(testSuite);YUITest.TestRunner.run();

FamiliarSyntax!

Not just a port!API cleanup & bug fixes

YUITest.ObjectAssertareEqual()hasKey()hasKeys()ownsKey()ownsKeys()ownsNoKeys()inheritsKey()inheritsKeys()ownsOrInheritsKey()ownsOrInheritsKeys()

Friendlier Test Namesvar testCase = new YUITest.TestCase({ name: "My Test Case",

"This should be a test": function(){

//this is a test in YUI Test for YUI 3.x

}, "As is this one": function(){

//this is not a test in YUI Test for YUI 3.x //but is in YUI Test Standalone 1.0.0

}});

Counting Asserts

var testCase = new YUITest.TestCase({ name: "My Test Case",

"This should be a test": function(){

//this test passes in YUI Test for YUI 3.x //but fails in YUI Test Standalone 1.0.0

}});

Beyond _should

var testCase = new YUITest.TestCase({ name: "My Test Case",

_should: { //old "Another test": "Error message" },

"Another test": function(){ //new someFunction(); }});

Beyond _should

var testCase = new YUITest.TestCase({ name: "My Test Case",

"Another test": function(){ //new YUITest.Assert.throwsError("Error message", someFunction); }});

What about the YUI 2.x and 3.x versions?

YUI Test Standalone becomes the core implementation of both

Today

YUI TestStandalone Library

YUI TestSelenium Driver

YUI TestCoverage

Easy automation for continuous integration

HudsonContinuous build/testing system

Used to periodically build from source and validate by running tests

http://hudson-ci.org/

?JUnit

Selenium

TAPSelNG

Screen scrape

Hack

Hudson

?JUnit

TAPSelNG

Screen scrape

Hack

Hudson

Selenium

SeleniumWeb application testing system

Primarily used by QA organizations for functional testing

http://seleniumhq.org/

http://seleniumhq.org/projects/remote-control/

Selenium Remote Control

Hudson SeleniumRC

?

Screen scraping to gather results = ick

Hudson SeleniumDriver

SeleniumRC

Selenium DriverThree primary goals

Completely handle communication between Hudson and Selenium RC

Yes, Uhura once wore yellow – focus, people

#1

#2

Extract as much data as possible from testsReturn the results in a format that Hudson can consume and report on

#3

Don't force already-existing tests to changeSeriously, it's just wrong

Test Pages

var testCase = new YUITest.TestCase({ name: "My Test Case",

//...});

YUITest.TestRunner.add(testCase);YUITest.TestRunner.run();

Test pages must automatically beginrunning tests without additional interaction

Command LineUsage: java -jar yuitest-selenium-driver-x.y.z.jar [options] [test files]

Global Options -h, --help Displays this information. --browsers <browsers> Run tests in these browsers (comma-delimited). --conf <file> Load options from <file>. --coveragedir <dir> Output coverage files to <dir>. --erroronfail Indicates that a test failure should cause an error to be reported to the console. --host <host> Use the Selenium host <host>. --port <port> Use <port> port on the Selenium host. --resultsdir <dir> Output test result files to <dir>. --silent Don't output test results to the console. --tests <file> Loads test info from <file>. -v, --verbose Display informational messages and warnings.

Examplesjava -jar yuitest-selenium-driver.jar http://www.example.com/tests/test_ui

java -jar yuitest-selenium-driver.jar --host selenium.example.com --port 9080 http://www.example.com/tests/test_ui

java -jar yuitest-selenium-driver.jar --browsers *firefox,*iexplore http://www.example.com/tests/test_ui http://www.example.com/tests/test_util

java -jar yuitest-selenium-driver.jar --tests tests.xml

Tests File

<?xml version="1.0"?><yuitest> <tests base="http://www.example.com/tests/" timeout="10000"> <url>test_core</url> <url timeout="30000">test_util</url> <url>test_ui</url> </tests></yuitest>

Easiest way to specify multiple tests toexecute on the same host machine

Demo

(Sorry Slideshare viewers)

Today

YUI TestStandalone Library

YUI TestSelenium Driver

YUI TestCoverage

What is code coverage?

I dunna know whathappened, all my tests

passed!

What was it you weretesting, Mr. Scott?

...

Code coverage tells you what code executed during testing

Code can have multiple paths

function doSomething(){ if (condition){ doSomethingElse(); } else { doAThirdThing(); }}

The goal of unit testing is to exercise each pathThe goal of code coverage is to identify if you did

doSomething()

condition

doSomethingElse() doAThirdThing()

doSomething()

condition

doSomethingElse() doAThirdThing()

doSomething()

condition

doAThirdThing()doSomethingElse()

Step OneInstrument JavaScript

Coverage covered-file.jsfile.js

Command LineUsage: java -jar yuitest-coverage-x.y.z.jar [options] [file|dir]

Global Options -h, --help Displays this information. --charset <charset> Read the input file using <charset>. -d, --dir Input and output (-o) are both directories. -v, --verbose Display informational messages and warnings. -o <file|dir> Place the output into <file|dir>. Defaults to stdout.

Step TwoRun tests with instrumented

JavaScript

Step ThreeGenerate reports

Extract Coverage Data Yourself

//object resultsresults = YUITest.TestRunner.getCoverage();

//standard JSON results formatresults = YUITest.TestRunner.getCoverage( YUITest.CoverageFormat.JSON);//alternate formatresults = YUITest.TestRunner.getCoverage( YUITest.CoverageFormat.XebugJSON);

Coverage Report

Reportresults.json

Command LineUsage: java -jar yuitest-coverage-report-x.y.z.jar [options] [file]

Global Options -h, --help Displays this information. --format <format> Output reports in <format>. Defaults to HTML. -v, --verbose Display informational messages and warnings. -o <file|dir> Place the output into <file|dir>.

CoverageSelenium Driver

Selenium Driver automatically pulls in coverage data

Complete Testing Solution

YUI TestStandalone Library

YUI TestSelenium Driver

YUI TestCoverage

JavaScript continuous integration testing system

Compatibility• YUI Test Standalone Library 1.0.0• YUI Test for YUI 3.1.1+• YUI Test for YUI 2.8.1+

Contribute• GitHub:

http://github.com/yui/yuitest/

• YUILibrary:http://yuilibrary.com/projects/yuitest/

• YUIBlog Announcement:http://www.yuiblog.com/blog/2010/11/09/introducing-the-new-yui-test

Questions?

Etcetera• My blog:

www.nczonline.net

• Twitter:@slicknet

• These Slides:http://slideshare.net/nzakas/

• Rate Me:http://spkr8.com/t/4999

Creative Commons Images Used• http://www.flickr.com/photos/anks/3311228772/

• http://www.flickr.com/photos/eschipul/4716821041/