Quality Assurance in SakaiPast, Present and Future

Pete Peterson, Sakai FoundationMegan May, Indiana University

Agenda

• What is QA?• Past • Present• Future• Discussion/Questions?

July 2009 210th Sakai Conference - Boston, MA, U.S.A.

What is QA/QC?• Quality Assurance (QA) is the process of verifying or determining

whether products or services meet or exceed customer expectations. QA considers all aspects of design, development, production, and service. When implemented correctly, QA provides a continuous and consistent improvement of processes within a product lifecycle.

• Quality Control (QC), a subset of QA, is employed to ensure a certain level of quality in a product or service. The basic goal of QC is to ensure that the products, services, or processes provided meet specific requirements.

• Often, Quality Control is confused with Quality Assurance. Though the two are very similar, there are some basic differences. QC is concerned with the product, while QA is process–oriented. Simply put, QA ensures that a product or service is developed and implemented in the right way; while QC evaluates whether or not the end result is satisfactory.

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 3

What is QA/QC?

There are three essential questions that need to be answered and tested long before a product is placed into production, and QA/QC activities focus on answering these questions: a.Does it do what we expect it to do? b.Does it do what the end-user expects it to do? c.Does it do “a” and “b” correctly and consistently?

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 4

Key Points of QA/QC

• Quality is not free: There are costs associated with implementing these additional steps into your projects and development processes—time, money, resources, etc. These costs, however, are small in comparison to the benefits of having increased end-user satisfaction, reduced support requirements, and functionally accepted applications and systems.

• Repeatable: The QA/QC process should be repeatable and flexible, so it can be easily applied to other projects as they arise.

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 5

Key Points of QA/QC

• Comprehensive: QA/QC should be included in the project from the beginning to the end. The goal of this participation is to examine the project from all angles. This not only allows the QA/QC group to create better tests, but also to provide feedback and a global perspective to the developer community.

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 6

Key Points of QA/QC

• Ongoing: QA/QC doesn’t stop once a project is complete; in fact this is where much of the QA is applied. Examining pitfalls and past mistakes can improve design, development and testing, as can documenting procedural successes and areas that need improvement.

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 7

Key Points of QA/QC

• Autonomous: It is essential for the QA/QC group to have a high level of autonomy. Often the QA/QC process is performed in-house by the developers or project managers themselves. Although well-intentioned, this approach is almost always ill-advised and precarious. It is difficult to remove oneself, step back and see things from the diverse perspectives needed for an effective QA/QC analysis.

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 8

Past

“Those who don't know the [past] are destined to repeat it.” Edmund Burke

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 9

Development

Testing & Software Release

Requirement Gathering and

Design

Sakai’s Testing & Release Management Process

Formulation of QA Network

Amsterdam BostonCambridge Cape TownColumbia IndianarSmart Charles Sturt Georgia

TechRecent additions of Nagoya, Marist & UMich

Past QA Workgroup Participation

2.4.0

93 People

37 Institutions

11 Countries

2.2.1

23 People

10 Institutions

4 Countries

2.3.0

44 People

17 Institutions

5 Countries

2.5.0

56 People

17 Institutions

4 Countries

Redefining processes

• Defined milestones •

http://confluence.sakaiproject.org/x/VwAiAQ

• No new features in maintenance releases• Better oversight of merges into

maintenance release branches

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 13

Other Accomplishments

• Expansion of test script library• Raised awareness of the importance of unit

and performance testing throughout community

• Availability of Static Code review• Integration of Accessibility WG activities in

general QA cycle• Inclusion of 3rd party licensing review in QA

process

July 2009 1410th Sakai Conference - Boston, MA, U.S.A.

Sakai QA - Present

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 15

Sakai QA - Present

• Finalizing Sakai 2.6 release• Finalizing Sakai 2.5 release• So everything is done right?

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 16

Sakai QA - Present

• Sakai 2.6.1 and 2.5.6 coming soon to a QA server near you

• Testing for maintenance branch merges• Updates of test plan

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 17

Sakai QA - Present

The Links Slide• http://www.sakaiproject.org/portal• http://jira.sakaiproject.org/• http://confluence.sakaiproject.org/• QA

http://confluence.sakaiproject.org/display/QA/Home

• Release Managementhttp://confluence.sakaiproject.org/display/REL/Release+Management+Working+Group

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 18

Resources

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 19

K2

Sakai 2.7

K1

Sakai 2.6

Sakai 3

Sakai 2.5

Future

• Automation• Inclusion of test cases in every jira• Continuous testing

July 2009 10th Sakai Conference - Boston, MA, U.S.A. 20