test automation and innovation with open source tools
DESCRIPTION
slides of the presentation about test automation with open source tools at Credit Suisse in ZürichTRANSCRIPT
Testautomation & Innovation with Open Source Tools
Michael Palotas Head of Quality Engineering Europe eBay International
Zürich, May 2012
Founded in 1995 Based in San Jose, California 27770 Employees worldwide
eBay Inc. Facts
104 million active buyers and sellers worldwide
68 Billion USD transaction volume 2011
2000 USD transaction volume every second
Platform Facts
9 petabytes of data In our Hadoop and Teradata clusters
2 billion page views each day
75 billion database calls each day
Data Facts
70,000,000 Downloads of the eBay apps
5 billion USD transacted 2011 on mobile devices
1 million items are listed with using mobile apps every week
Mobile Facts
• Introduction • Testautomation why? • Automation history at eBay • Why open source? • Testautomation who? • Testautomation how? • Outlook
Topics today
To get early and instant Feedback
regression tests only
safety net to save time & which is reinvested into manual testing
Test Automation – why?
NOT to replace manual testing
End-to-End Automation History at eBay
http://www.argyllhoteliona.co.uk/home_baked.php http://www.seleniumhq.org
Home baked solution Selenium IDE
Selenium RC Selenium 2 Selenium GRID
(Forget that) It’s free
Faster innovation cycles
Independence
Engagement / opportunities to grow
Hiring easier
It’s free
Why open source?
Support for tool drops
Your competitor uses the same tool
Need a different mindset
You need to solve your problems yourself
Potential risks
Testautomation – WHO?
http://gigidavidge.com/
Testautomation – HOW?
High Level Test Architecture
Biz Domain Layer
Browser
Flow Objects
Tests
WebDriver
Page Objects
Biz Domain Layer
• Modularization and reuse of components: maintainable & extendable e2e tests
• Crucial to have in depth knowledge of the domain
• Enable data driven testing by generic test cases
eCommerce shopping site like eBay
Checkout Page Success Page Shopping Cart
Page View Item
Page
Pages and Flows
View Orderlist
View Category
Update
Delete View Product
Search Item
View Category Review
Select Payment Provider
Order Summary
Instant Feedback - Selenium Grid
Sequential Execution Execution time: 7000 minutes (~116 hours = ~5 days)
Test 1
Test 2
Test …
Test 4500
Execution Time
Test 3
Footnote about the scale of the tests @eBay Europe: - 4500 end-to-end regression test cases - 2-3 minutes execution time for each test
Parallel Execution Execution time: 60 minutes by using 150 Virtual Machines / Browsers using the WebDriver GRID
Test Test Test
Execution Time
Test
Test Test Test Test
Test Test Test Test
Par
alle
l Exe
cutio
n
Par
alle
l Exe
cutio
n
Summary & Outlook
• Automation is changing
• Open source • Skillset
• Mobile … stay tuned
• Try it yourself
Thank You! Questions or Comments?