agile and the automated testing of accessibility
TRANSCRIPT
![Page 1: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/1.jpg)
Agile and the automated testing of accessibility
![Page 2: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/2.jpg)
© 2015 - All Rights Reserved 2
![Page 3: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/3.jpg)
© 2015 - All Rights Reserved 3
![Page 4: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/4.jpg)
Agile Tools
5© 2015 - All Rights Reserved
![Page 5: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/5.jpg)
Shift Left Testing
Catching a defect in the design and requirements phase can reduce cost to fix by up
to 100:1
Software Defect Reduction Top 10 List, Barry Boehm and Victor R. Basili, January 2001
https://www.cs.umd.edu/projects/SoftEng/ESEG/papers/82.78.pdf
© 2015 - All Rights Reserved 6
![Page 6: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/6.jpg)
Rise of Continuous
• Lag between committing code and getting feedback allows damage to magnify– Costs rise– Velocity decreases
• Continuous Integration• Continuous Deployment
http://www.startuplessonslearned.com/2009/06/why-continuous-deployment.html
© 2015 - All Rights Reserved 7
![Page 7: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/7.jpg)
Automation
© 2015 - All Rights Reserved 8
![Page 8: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/8.jpg)
© 2015 - All Rights Reserved 9
Accessibility Testing’s Image
![Page 9: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/9.jpg)
Aim of Software Testing
• Reduce the risk of the existence of a serious problem on a supported configuration to an acceptable level– Define “serious problem”– Define “supported configuration”– Define “acceptable level”
© 2015 - All Rights Reserved 10
![Page 10: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/10.jpg)
© 2015 - All Rights Reserved 11
![Page 11: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/11.jpg)
Android Fragmentation
12© 2015 - All Rights Reserved
![Page 12: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/12.jpg)
Android Fragmentation
We have seen 24,093 distinct devices download our app in the past few months. In our report
last year we saw 18,796. In 2013 we saw 11,868
OpenSignal 2015
© 2015 - All Rights Reserved 13
![Page 13: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/13.jpg)
Combinations
4 operating systems, 2 supported versions5 Assistive technologies4 Browsers, 2 supported versions3 Responsive breakpoints
216 combinations excluding device combinations
© 2015 - All Rights Reserved 14
![Page 14: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/14.jpg)
© 2015 - All Rights Reserved 15
![Page 15: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/15.jpg)
Sparse Testing Matrix
Browser/AT
Chrome KB IE KB
Firefox KB Safari KB Dragon IE NVDA FF Jaws IE VO Safari TB FF Orca FF
Operating
Sytem
Windows
OS X
iOS
Android
Linux
© 2015 - All Rights Reserved 16
![Page 16: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/16.jpg)
© 2015 - All Rights Reserved 17
![Page 17: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/17.jpg)
Accessibility “Acceptance Automation”
18© 2015 - All Rights Reserved
![Page 18: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/18.jpg)
Acceptance Automation
• Comes from the “requirements”– Designers mark up their wireframes and comps– Stories must contain specific accessibility
acceptance criteria• Can go to any level of detail– Test the specific alt attributes on images– Test the DOM order specified in the wireframe– Etc.
© 2015 - All Rights Reserved 19
![Page 19: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/19.jpg)
Open Source Rules Engine
© 2015 - All Rights Reserved 20
![Page 20: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/20.jpg)
aXe Manifesto
• Must have a zero false positive rate• Must be lightweight and fast• Must work in all modern browsers on all
platforms• Must be tested automatically
© 2015 - All Rights Reserved 21
![Page 21: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/21.jpg)
aXe-Core
• Small ~100KB• Fast
– No network connection, no new browser start• Intelligent
– Supports cross domain iframes– Turns document level rules on/off automatically
• Unlimited– No throttling, no server limits
• Reliable– No double-execution or missing CSS problems– No network connectivity problems– No server capacity problems
• Secure– No server access across the wire– No sending data to a server
• Mozilla Public License 2.0– Available via GitHub, npm and Bower
© 2015 - All Rights Reserved 22
![Page 22: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/22.jpg)
aXe Chrome extension
© 2015 - All Rights Reserved 23
![Page 23: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/23.jpg)
aXe Firefox extension
© 2015 - All Rights Reserved 24
![Page 24: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/24.jpg)
aXe extension
• Developers– Use when coding
• Testers– Use during functional testing
• Accessibility Experts– Use for validation
© 2015 - All Rights Reserved 25
![Page 25: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/25.jpg)
aXe API
• Two major calls– axe.getRules() – returns a list of the rules loaded– axe.a11yCheck() – analyze a page or portion of a page
• Getting started– Install package
• bower install –-save-dev axe-core
– Include library• <script src=“(path_to)/axe.js”></script>
– Call function• a11yCheck(target, options, callback);
© 2015 - All Rights Reserved 26
![Page 27: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/27.jpg)
Integrations
• Java Selenium• JavaScript Selenium (axe-webdriverjs)• Grunt (grunt-axe-webdriver)• Cucumber and R-Spec (axe-matchers)• Firefox Dev Tools• Chrome Dev Tools• Ember-aXe (ember-axe)
more…
© 2015 - All Rights Reserved 28
![Page 28: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/28.jpg)
Summary
• Manage your matrix– Optimize for resources and risk
• Leverage automation– Generic automation– Specific automation– Optimize for up-front vs. ongoing effort
© 2015 - All Rights Reserved 29
![Page 29: Agile and the automated testing of accessibility](https://reader036.vdocuments.mx/reader036/viewer/2022062304/56649f325503460f94c4de8d/html5/thumbnails/29.jpg)
Questions
• https://addons.mozilla.org/firefox/addon/axe-devtools/
• Chrome: search Google for “chrome axe extension”
© 2015 - All Rights Reserved 30