tools for better testing for open source project · open source project dong ma hewlett packard...
TRANSCRIPT
TOOLS FOR BETTER TESTING FOR OPEN SOURCE PROJECT
DONG MA
HEWLETT PACKARD ENTERPRISE
Open Source Summit Japan 2017
Friday 2 June 2017
DONG MA
• Hewlett Packard Enterprise software engineer, focused on
Free and Open Source Software development
• OpenStack Team software engineer focused on Jenkins,
OpenStackCI and QA
• FOSSology project core contributor
TOPICS
• What’s QA and why are they important for open source
projects
• What’s the OpenStack QA?
• Tools for Better Testing in OpenStack
• How to leverage the OpenStack way
WHAT’S QA?
• What’s Quality Assurance(QA)?
• Why QA important for Open Source project?
WHAT’S THE OPENSTACK QA?
Develop, maintain, and initiate tools and plans to ensure the
upstream stability and quality of OpenStack, and its
release readiness at any point during the release cycle.
OPENSTACK CI WORKFLOW
OPENSTACK INFRASTRUCTURE
OPENSTACK GATE TESTING
• Python Unit Tests
• Style Rule Checkers
• Devstack
• Tempest
CURRENT OPENSTACK QA PROJECTS
• devstack-plugin-ceph
• Karma-subunit-reporter
• os-performance-tools
• tempest
• tempest-lib
• devstack-tools
• patrole
• bashate
• devstack-vagrant
• tempest-plugin-cookiecutter
• devstack-plugin-cookiecutter
• eslint-config-openstack
• devstack
• openstack-health
• hacking
• stackviz
• grenade
• coverage2sql
TOOLS IN OPENSTACK
• Devstack
• Tempest
• Subunit2sql
• OpenStack Health
• Elastic Recheck
• Stackviz
DEVSTACK
DevStack is a series of extensible scripts used to quickly
bring up a complete OpenStack environment based on the
latest versions of everything from git master. It is used
interactively as a development environment and as the
basis for much of the OpenStack project’s functional testing.
TEMPEST
• The OpenStack Integration Test Suite
• Tempest has batteries of tests for OpenStack API
validation, Scenarios, and other specific tests useful in
validating an OpenStack deployment.
OPENSTACK HEALTH
SUBUNIT2SQL
• Tool for storing test results data in a SQL database
• Provides a DB schema and a python API for interacting
with the database
ELASTIC RECHECK
• Designed to answer the question “Have you seen this recently?”
• Leverages elastic search to identify failures with known
fingerprints
• Contains a repository of elastic-search queries with known
failures
• Has 2 parts:
• A bot which watches changes and reports identified failures to gerrit
and IRC
• A dashboard which shows failure categorization
ELASTIC RECHECK
STACKVIZ
Visualization tool of individual CI build results
DEMO
• OpenStack QA Status
• http://status.openstack.org/
HOW TO LEVERAGE OPENSTACK WAY
• Understand the importance of QA in Open Source Project
• Leverage the CI and QA process in OpenStack
• Tools we can use
QUESTIONS
• Contact
• Email: [email protected]
• HPE Email: [email protected]
• Freenode: larainema
• Twitter: @wintermadong
• Resource
• https://wiki.openstack.org/wiki/QA
THANK YOU!