testing automaton
TRANSCRIPT
![Page 1: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/1.jpg)
Testing AutomatonLuis Majano @lmajano
![Page 2: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/2.jpg)
Who am I
• Luis Majano
• Computer Engineer
• Imported from El Salvador
• CEO of Ortus Solutions
• Creator of all things Box
![Page 3: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/3.jpg)
Why people don’t testWhy we need to test
How to testHow to automate
![Page 4: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/4.jpg)
Where to start?
Write Tests
![Page 5: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/5.jpg)
WHY PEOPLE DON’T TEST
COMFORT
![Page 6: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/6.jpg)
WHY PEOPLE DON’T TEST
New MethodologyNew Learned Behavior
It is a leap….
![Page 7: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/7.jpg)
BIGGEST LIE IN SOFTWARE DEV
Don’t worry, we will create the
tests and refactor it later!
![Page 8: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/8.jpg)
![Page 9: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/9.jpg)
Bugs cost money
Cost To Fix Time detected Requirements Design Building Testing Post-Release
Time Introduced
Requirements 1x 3x 5-10x 10x 10-100x
Design --- 1x 10x 15x 25-100x
Building -- -- 1x 10x 10-25x
^ Kaner, Cem; James Bach, Bret Pettichord (2001). Lessons Learned in Software Testing: A Context-Driven Approach. Wiley. p. 4. ISBN 0-471-08112-4.
![Page 10: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/10.jpg)
• Just do it!
• You will get dirty
• It can hurt (a little)
• Learned behavior
NO MORE EXCUSES
IT WILL ACCELERATE YOUR DEVELOPMENT
![Page 11: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/11.jpg)
TDD/BDD=ProcessContinuousIntegrationContinuousDelivery
ContinuousImprovement
AUTOMATION
![Page 12: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/12.jpg)
![Page 13: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/13.jpg)
TestBox : What you get
API Docs
Core
Test Browser
Test Harness
Test Runner
Samples
install testbox —saveDev
install testbox@be —saveDev
![Page 14: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/14.jpg)
Test Harness
Automated test results!
xUnit/BDD Test Bundles
Harness bootstrap
HTML Runner
ANT Runner
![Page 15: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/15.jpg)
Test Bundle CFCs• Inherits from testbox.system.BaseSpec
•TesBox will then execute all tests/specs/suites within 1 or more bundles
![Page 16: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/16.jpg)
RUN TESTS
• Execute bundles via the URL• http://localhost/tests/bundle.cfc?method=runRemote
• SOAP Runner• HTTP/REST Runner• ANT Runner• NodeJS Runner• CommandBox Runner• Custom Runner
![Page 17: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/17.jpg)
TESTBOX REPORTERS
•ANTJunit : Variant of JUnit for Jenkins•Codexwiki : Mediawiki syntax•Console : Server console•Doc : Semantic HTML•Dot : Awesome Dots•JSON•JUnit•Raw : CFML Struct•Simple : A basic HTML reporter•Text : Back to the 80’s, Text•MinText : Great for CommandBox Runners•XML •Tap : A test anything protocol reporter•Min : Minimalistic Report
![Page 18: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/18.jpg)
MANUAL TESTING
BundleCFC Runner
![Page 19: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/19.jpg)
CommandBox Runnerpackage set testbox.runner = "http://127.0.0.1:57538/tests/runner.cfm"
testbox run
1
2
![Page 20: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/20.jpg)
{ "runner": "http://127.0.0.1:57538/tests/runner.cfm", "directory": "/tests/specs", "recurse": true}
NodeJS Test Runnernpm install -g testbox-runner
create .testbox-runnerrc
1
2
Run it : testbox-runner2
![Page 21: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/21.jpg)
ANT TESTING
![Page 22: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/22.jpg)
DEVELOPER AUTOMATION
1.Grunt Watchers + LiveReload2.Grunt Watchers + testbox-runner3.Grunt Watchers + commanbox4.ColdBox Elixir
![Page 23: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/23.jpg)
CONTINOUS INTEGRATION
![Page 24: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/24.jpg)
CI SERVERS
![Page 25: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/25.jpg)
Jenkins + CommandBoxwget http://mirrors.jenkins-ci.org/war-stable/latest/jenkins.war
box server start warpath=jenkins.war
1
2
![Page 26: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/26.jpg)
Travis CI - Sign Uphttps://travis-ci.org/
Sign Up
1
2
![Page 27: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/27.jpg)
Accounts
Activate For Repositories
1
2
Create .travis.yml3
Commit to repository4
Travis CI - Getting Started
![Page 28: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/28.jpg)
TRAVIS + ANTlanguage: javabranches: only: - developmentsudo: requireddist: trustybefore_install:- sudo apt-key adv --keyserver keys.gnupg.net --recv 6DA70622- sudo echo "deb http://downloads.ortussolutions.com/debs/noarch /" | sudo tee -a /etc/apt/sources.list.d/commandbox.listinstall:- sudo apt-get update && sudo apt-get --assume-yes install commandbox- box install- box server startbefore_script:# Startup the app- curl http://localhost:49616/script: - ant -f workbench/build.xml
![Page 29: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/29.jpg)
TRAVIS + COMMANDBOXlanguage: javasudo: requireddist: trustybefore_install:- sudo apt-key adv --keyserver keys.gnupg.net --recv 6DA70622- sudo echo "deb http://downloads.ortussolutions.com/debs/noarch /" | sudo tee -a /etc/apt/sources.list.d/commandbox.listinstall:- sudo apt-get update && sudo apt-get --assume-yes install commandbox- cd tests- box install- box server startbefore_script:# Startup the app- curl http://localhost:49615/script: > testResults="echo $(box testbox run)"; echo "$testResults"; if grep -i "\[Failures: [1-9][0-9]\?[0-9]\?\]\|\[Errors: [1-9][0-9]\?[0-9]\?\]\|<t[^>]*>\|<b[^>]*>" <<< $testResults; then exit 1; fi
![Page 30: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/30.jpg)
MULTIPLE VERSIONS
MULTIPLE ENGINES
![Page 31: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/31.jpg)
TRAVIS + MULTIPLE ENGINESlanguage: javasudo: requiredjdk:- oraclejdk8env: matrix: - [email protected] - ENGINE=lucee@5 - ENGINE=adobe@2016 - ENGINE=adobe@11 - ENGINE=adobe@10before_install:- sudo apt-key adv --keyserver keys.gnupg.net --recv 6DA70622- sudo echo "deb http://downloads.ortussolutions.com/debs/noarch /" | sudo tee -a /etc/apt/sources.list.d/commandbox.listinstall:- sudo apt-get update && sudo apt-get --assume-yes install commandbox- box installbefore_script:- box server start cfengine=$ENGINE port=8500script: > testResults="$(box testbox run runner='http://127.0.0.1:8500/tests/runner.cfm' )"; echo "$testResults"; if grep -i "\[(Failures|Errors): [1-9][0-9]\?[0-9]\?\]\|\[Errors: [1-9][0-9]\?[0-9]\?\]\|<t[^>]*>\|<b[^>]*>" <<< $testResults; then exit 1; finotifications: email: false
![Page 32: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/32.jpg)
Resources
• TestBox : ortussolutions.com/products/testbox
• CommandBox: ortussolutions.com/products/commandbox
• Slack: boxteam.herokuapp.com
• CFML Slack: #box-products
• travis-ci.org
• jenkins.io
![Page 33: Testing automaton](https://reader031.vdocuments.mx/reader031/viewer/2022021919/5877572c1a28ab84388b7935/html5/thumbnails/33.jpg)
Thank you!