What does Quality Assurance Mean to Sakai?

Megan May

Sakai Quality Assurance Director

Role of QA Director

• Coordination! – Help bring parties interested in the same testing

efforts together– Foster initiatives that support quality throughout

the community– Assist in bridging the gap between test and

design & development teams– Ensure release activities are transparent

WG Membership

2.4.0

93 People

37 Institutions

11 Countries

2.2.0

82 People

28 Institutions

6 Countries

2.2.1

23 People

10 Institutions

4 Countries

2.3.0

44 People

17 Institutions

5 Countries

QA Participants for 2.4 Mark Lowe

Marc Zaldivar,Jeff Navid,Kevin Chan,Bob Conroy, Kathleen E Moore,Soo Kim,Daniel Perry,Harriet Truscott,Steven Githens,Tim Archer,Melissa ZhuoSeth Theriault,Kin Wang,Andrew Poland,Chris Maurer,Derek Ramsey,James Moyer,John Dunn,John Gosney,Kate Ellis,Kevin Brokamp,Kevin Deckard,Lorie Shuck,Lynn Ward, Mark Lowe,Sandy Zimmerman, Steve Smail,Michael Lockett,Mark Notess,Tony Camilli,Sam Ottenhoff,Earle Nietzel,Sravanthi Tipirneni,Venkatesh Subramanian,Vidhya Rajan,Jeff Ziegler,Steve Lonn,Angela Rabuck,Anand Mehta,Corey McGarrahan, Ernie Souhrada,Hannah Reeves,Jan Smith,Kara Stiles,Jesse Schibilia,Robert Long,Huong Nguyen,Laura Burchard,Jim Pease,Salwa Khan,UC Berkeley Training & Support Team,Lisa Wilson,Thomas Amsler,Jim Williamson,Nuno Fernandes, John Leasia,Ivy Tech Students, Gaurav Bhatnagar,Gonzalo Silverio,Jeff Ziegler,Jean-François Lévêque,David Horwitz,Greg Doyle,Stephen Marquard,David Haines,Linda Place,Pramod Prabhu,Sean DeMonner,Sheeba Gandhi,Anastasia Cheetham,Peter Thiessen,Alan Berg,Bas Toeter,Dawn Eckert,Ann Paterson,Cheryl Wogahn,Adam Price,Clay Fenlason,Dave Ross,Deborah Bracken,Diego Luis Del Blanco Orobitg,Jack HC LEE,Karen Miles,Onur Sirin,Ratana Lim,Ronald Chik,Shoji Kajita,Vincent Siveton, Steve Swinsburg

QA Network

Amsterdam Boston

Cambridge Cape Town

Columbia Indiana

rSmart

Where does QA fit in? What kinds of activities occur?

Development

Testing & Software Release

Requirement Gathering and

Design

Focus on Functional Testing

Partners with accessibility WG

Integration Week/Integration Meetings

“Code Freeze”

Selective Merging

Stealth Provisional Tools

Documentation Finalized Release!

Weekly Tags

Communication

Information Gathering

CollabHow to be in the know!https://collab.sakaiproject.org/portal

Jirahttp://bugs.sakaiproject.org/jira/secure/Dashboard.jspa

Confluencehttp://bugs.sakaiproject.org/confluence/display/QA/Home

Year in Review• 6 Releases

– 2.2.0 (5/12-7/19)

– 2.2.1 (7/15-8/15)

– 2.2.2 (9/18)

– 2.3.0 (9/15-11/3)

– 2.3.1 & 2.2.3– 2.4.0 (3/17-5/21)

Summary - JIRA Activity

2.3.0 – 85% Verification

2.2.0 – 85% Verification

2.2.1 – 85% Verification

2.3.1 & 2.2.3 – NA/Security Release

2.2.2– NA/Security Release

(1,345 JIRA Tickets)

(611 JIRA Tickets)

(504 JIRA Tickets)

(119 JIRA Tickets)

2.4.0 – 82% Verification

Strides Forward• Deployment testing on wider variety of platforms• Data retention & conversion script testing• Integration of Accessibility WG activities in general QA

cycle• Push out testing earlier• Automated testing tools

• Static code review• Selenium usage• Site Profiles

Reflection on Release process

Proposed Kernel + Bundle Release Timing

NovRelease

Kernel Kernel QAQA

Kernel Kernel DevDev

UIFreeze

APIFreeze

Bundle DevelopmentBundle Development PatchPatchWeekWeek

CodeFreeze

Bundle QABundle QA

MayRelease

Contrib X DevelopmentContrib X Development QAQA

Contrib Y DevelopmentContrib Y Development QAQA

Contrib Z DevelopmentContrib Z Development QAQA

Perceived Quality LevelDevelopment

Management

QA

Support

Overall Perceived level of Quality

Quality, like beauty, is clearly in the eyes of the beholder.

Perceived Quality PollUser Perceptions

What we can do!

• Infiltration of entire development process

• Formulate Working Groups

• Refine processes & grow WG

Development

Testing & Software Release

Requirement Gathering and Design

Areas where QA is needed

Pledge Resources

• Why?– What happens in the community has a direct

local impact

• Easy point of entry into Sakai

Questions

Other Sessions of Interest• Expanding Community: Student

Involvement in QA Testing (today, immediately following)

• Toward Further Automation of the Quality Assurance cycle (Thursday, 9:35-10:05 in Matterhorn 2)

• Sakai Accessibility Update (Thursday, 2:05-2:35 in Zurich 1)