ui testing

UI Testing, handling complex UIs at scale

Josh Black :: FEDucation :: March 2nd, 2016

.FED@IBM

Discussion Time A look at the landscape of testing options and techniques for developers.

.FED@IBM

Wikipedia An investigation conducted to provide information about the quality of a product or service under test.

.FED@IBM

TestingHelps developers tell whether the product is working or not.

.FED@IBM

OverviewThe Basics What are the various kinds of tests? How do I know when to use each one?

Writing a Test How can we structure our tests in a way that's clean, readable, and effective?

Real World What kind of tools can I leverage to start testing in my project today?

0Part 0 Setting the Stage

What is a test?

.FED@IBM

Testing When a developer changes a piece of code in a project, how confident am I that sh*t won't break?

.FED@IBM

Testing a Dropdown

class Dropdown { /* ... */ }

const d = new Dropdown({ items: ['Home', 'Cognitive', 'Watson', 'Services'], defaultText: 'Please select an item', target: document.querySelector('.dropdown') });

.FED@IBM

HTML

<ul class="dropdown"> <li class="dropdown__item"> <p id="current">Please select an item</p> <span class="dropdown__arrow"></span> <ul class="dropdown-menu"> <li class="dropdown-menu__item"><a href="#">Home</a></li> <li class="dropdown-menu__item"><a href="#cognitive">Cognitive</a></li> <li class="dropdown-menu__item"><a href="#watson">Watson</a></li> <li class="dropdown-menu__item"><a href="#services">Services</a></li> </ul> </li> </ul>

.FED@IBM

JavaScript

const d = new Dropdown({ items: ['Home', 'Cognitive', 'Watson', 'Services'], defaultText: 'Please select an item', target: document.querySelector('.dropdown') });

dropdown.arrow.addEventListener('click', () => { dropdown.classList.toggle('is-expanded'); });

.FED@IBM

.FED@IBM

.FED@IBM

Testing Allows us to make a trade-off between reliability and speed.

.FED@IBM

Testing a Dropdown

class Dropdown { /* ... */ }

const d = new Dropdown({ items: ['Home', 'Cognitive', 'Watson', 'Services'], defaultText: 'Please select an item', target: document.querySelector('.dropdown') });

.FED@IBM

Testing a Dropdown

describe('Dropdown', () => { it('should ...', () => { // ... }); });

.FED@IBM

Testing a Dropdown

describe('Dropdown', () => { it('should expand the list of options when clicked', () => { // ... }); });

.FED@IBM

Testing a Dropdown

describe('Dropdown', () => { it('should expand the list of options when clicked', () => { // ... });

it('should collapse the list of options when an item is selected', () => { // ... }); });

.FED@IBM

Test-Driven Development

Test Behavior-Driven Development

Add some initial, failing tests

Add Tests

.FED@IBM

Write an initial attempt at making your test pass

Write code

.FED@IBM

Run tests, re-working until they go green

Run Tests

.FED@IBM

Now that you have passing tests, go back and refine

Refactor

.FED@IBM

Unit Tests

Allows us to test small pieces of functionality in isolation.

Integration Tests

End-to-End Tests

Understand and test how a user flows through the system

Test the integration of small pieces of functionality with each other

Testing Basic Types

.FED@IBM

Unit Tests

const addIntegers = (a, b) => a + b;

describe('add', () => { it('should add two integers together', () => { const a = 1; const b = 2;

expect(add(a, b)).toBe(a + b); // true || false }); });

.FED@IBM

Unit Tests

const addIntegers = (a, b) => a + b;

addIntegers(1.5, 'a'); // 1.5a

.FED@IBM

Unit Tests

const addIntegers = (a, b) => { const { isInteger } = Number;

invariant( isInteger(a) && isInteger(b), 'Expected integer arguments, instead got: [%s, %s]', a, b );

return a + b; }

.FED@IBM

Unit Tests

const addIntegers = (a, b) => /* ... */

describe('add', () => { it('should warn if an argument is not an integer', () => { const a = 1; const b = 'a';

expect(add(a, b)).toThrow(); }); });

.FED@IBM

Integration Tests

Scenario

.FED@IBM

Visualization

API Response Validation

Duration

Count Visualization

Scenario

.FED@IBM

Validation

Duration

Count

Scenario

.FED@IBM

Validation

Duration

Count

Testing a Validator

describe('Validator', () => { it('should validate a response from our API', () => { // ... }); });

.FED@IBM

Testing getDuration

describe('getDuration', () => { it('should get a duration value from an API response', () => { // ... }); });

.FED@IBM

Integration Test

import Validator from '../Validator'; import getDuration from '../getDuration';

describe('getResponseDuration', () => { it('should get a duration value from an API response', () => { const response = {}; const validated = Validator(response);

expect(validated.passed).toBe(true); expect(validated.value).toBe(response.result.data);

// ... }); });

.FED@IBM

Integration Test

describe('getResponseDuration', () => { it('should get a duration value from an API response', () => { const response = {}; const validated = Validator(response);

expect(validated.passed).toBe(true); expect(validated.value).toBe(response.result.data);

const durationCount = getDuration(validated.value); expect(durationCount).toBe(300); }); });

.FED@IBM

End-to-End Tests

E2E Focus on real user scenarios, catching bugs before they reach the user.

.FED@IBM

E2E Scenario

.FED@IBM

Hits EnterLogin Form Clicks Form Enter Credentials

Testing Pyramid

.FED@IBM

E2E

Integration

Unit

Testing Pyramid

.FED@IBM

E2E

Integration

Unit

Unit Testing

Allows us to create a fast, reliable feedback loop that isolates errors.

Requirements: Knowledge in areas such as dependency management, mocking, and hermetic testing.

Testing Pyramid

.FED@IBM

E2E

Integration

Unit

Integration Tests

Allows us to verify the behavior of small groups of units

Testing Pyramid

.FED@IBM

E2E

Integration

Unit

End-to-End Tests

Covers a small set of our product features to help catch bugs that may reach customers

Take a Look Just Say No More to End-to-End Tests Covers the practicality of E2E tests, highlighting the advantages of favoring unit/integration tests over E2E.

Move Fast and Don't Break Things Discusses the integration and role of testing in the context of Continuous Integration and Delivery

Link

Link

1Part 1 Writing a test

.FED@IBM

How to actually get started

Setup Test runners, assertions, and spies, oh my!

.FED@IBM

mocha

chai

karma

expect

expect.js

jasmine

sinon

qunitselenium

web driver

ava

jsdom

.FED@IBM

Test Runner

Allows us to write chunks of code to describe what should happen. Handles executing tests for us and returning the result.

Assertion Library

Spies

Sometimes we need to check and verify that what we think will be called is actually called

Enables us to state what we believe to be true, false, and a variety of other states.

Testing Checklist

.FED@IBM

Test Runner

Allows us to write chunks of code to describe what should happen. Handles executing tests for us and returning the result.

Assertion Library

Spies

Sometimes we need to check and verify that what we think will be called is actually called

Enables us to state what we believe to be true, false, and a variety of other states.

Testing Checklist

.FED@IBM

DOM Mocking / Driver

Allows us to verify that what we expect to happen in our product is actually happening

Test Runner

describe('Module', () => { it('should ...', () => { // ... }); });

.FED@IBM

Test Runner

.FED@IBM

Take a Look Mocha Classic JavaScript test framework for JavaScript

Jasmine DOM-less simple JavaScript Testing Framework

https://www.npmjs.com/package/mocha

https://www.npmjs.com/package/jasmine

Karma Spectacular Test Runner for JavaScript https://www.npmjs.com/package/karma

Ava Futuristic Test Runner for JavaScript https://www.npmjs.com/package/ava

Assertions

Assertions Specify what you expect the outcome of a scenario to be.

.FED@IBM

Assertions

describe('Module', () => { it('should ...', () => { const result = Module();

expect(result).toBe(2); }); });

.FED@IBM

Assertions

describe('Module', () => { it('should ...', () => { const result = Module();

expect(result).toEqual({ title: 'FEDucation', description: 'Coolest FED gathering in the world' }); }); });

.FED@IBM

Assertions

describe('Module', () => { it('should ...', () => { const result = Module();

expect(result).toEqual(['some', 'collection']); }); });

.FED@IBM

Assertions

describe('Module', () => { it('should ...', () => { const result = Module();

expect(result).toThrow(); }); });

.FED@IBM

Assertions

describe('Module', () => { it('should ...', () => { const result = Module();

expect(result).to.be.instanceOf(Error); }); });

.FED@IBM

Assertions

it('supports loading multiple keys in one call', async () => { var identityLoader = new DataLoader(keys => Promise.resolve(keys));

var promiseAll = identityLoader.loadMany([ 1, 2 ]); expect(promiseAll).to.be.instanceof(Promise);

var values = await promiseAll; expect(values).to.deep.equal([ 1, 2 ]);

var promiseEmpty = identityLoader.loadMany([]); expect(promiseEmpty).to.be.instanceof(Promise);

var empty = await promiseEmpty; expect(empty).to.deep.equal([]); });

.FED@IBM

Take a Look Expect Write better assertions

Chai BDD/TDD assertion library for node.js and the browser

https://www.npmjs.com/package/expect

https://www.npmjs.com/package/chai

Should Test framework agnostic BDD-style assertions https://www.npmjs.com/package/should

Assert Native assertion library for Node.js

Spies

Spies Stubs out any function and tracks calls to it and all arguments

.FED@IBM

Assertions

const sum = (a, b) => a + b;

describe('A spy', () => { it('should track that the spy was called', () => { spyOn(sum); sum(1, 2); expect(sum).toHaveBeenCalled(); expect(sum).toHaveBeenCalledWith(1, 2); }); });

.FED@IBM

Assertions

describe('Module', () => { it('should track that the spy was called', () => { spyOn(console, 'error'); const result = Module('invalid argument'); expect(console.error).toHaveBeenCalled(); expect(console.error) .toHaveBeenCalledWith('invalid argument'); }); });

.FED@IBM

Isolation

Achieves similar results to mocks, stubs, dummies, fakes, etc for accomplishing isolation

Consistent Interface

Partial Mocking

You as the developer choose what portions of an interface you want to mock

Spies follow the same interface as the functions that they're mocking

SpiesHighlights

.FED@IBM

Take a Look Jasmine DOM-less simple JavaScript Testing Framework

Expect Write better assertions

Sinon JavaScript test spies, stubs and mockshttps://www.npmjs.com/package/sinon

DOM Mocking

End-to-End Tests

Prohibitively Expensive Loading up the browser, and performing a multitude of tests doesn't fit our ideal feedback loop for testing.

Flakiness Race conditions, or other processes that may yield inconsistent results for a test due to using the DOM.

Tooling In the past, we'd have to use complex tool chains like Selenium with Web Driver just to get E2E that don't actually help us, or the user.

Take a Look jsdom A JavaScript implementation of the DOM and HTMl standards

Karma Brings a productive testing environment to developers

https://www.npmjs.com/package/jsdom

https://www.npmjs.com/package/karma

Selenium Web Driver Driving a browser natively as a user would http://docs.seleniumhq.org/projects/webdriver/

DOM Mocking

import jsdom from 'mocha-jsdom'; import { expect } from 'chai';

describe('mocha tests', function () { jsdom()

it('has document', function () { var div = document.createElement('div') expect(div.nodeName).eql('DIV') }); });

.FED@IBM

DOM Mocking

it('should work for DOM events', function () { var div = document.createElement('div'); div.addEventListener('click', () => console.log('FEDs rule!'));

spyOn(console, 'log');

div.click();

expect(console.log).toHaveBeenCalled(); expect(console.log).toHaveBeenCalledWith('FEDs rule!'); });

.FED@IBM

DOM Mocking

import jsdom from 'mocha-jsdom'; import { expect } from 'chai';

const markup = '<html>...</html>';

describe('mocha tests', function () { jsdom(markup)

it('has document', function () { var app = document.querySelector('#app') expect(app.nodeName).eql('DIV') }); });

.FED@IBM

Recap The Basics Overview of Unit, Integration, and End-to-End tests.

The Tools Identify the necessary components that developers need for a robust testing framework.

The Practice How do we go about structuring our tests, and begin writing tests using the techniques/testing frameworks that we've identified0

Google Focus on the user and all else will follow

.FED@IBM

Testing Enables us to move fast and not break things

.FED@IBM

@joshblackfr github.com/joshblack

github.ibm.com/joshblack

Thanks! Any Questions?