Csci 565Spring 2009H.Reza

Event-based systemsGUI TestingConcepts, strategies, etcThe Currency Conversion ProgramTo show unit testing for GUITo show integration testing for GUITo show system testing for GUI

What is GUI?Universal client alternative to traditional forms/commands based user interfaces that accepts as input user-generated and/or system-generated events from a set of finite events and produces predicable graphical outputs

GUIs are organized in hierarchical fashion and Composed of objects:Objects has a set of fixed properties having discrete valuesMenuprovide almost continuous availability of the various features of the systems;Buttons /keyboard shortcuts enable the user to navigate and access the various functions of their application. User interacts and manipulate with objects using events (or sequence of events)Windows allow multiple windows to be displayed at the same time Windows provides functionality using fields in which text or numeric data can be entered Windows operate in two mode:Modal (restrict the user's foucs and interactions to one window)Modalless ( does not restrict monopolozes the GUI interactions)

The GUI events can be classified asRestricted-focus events: open modal windowsE.g. Set LanguageUnrestricted-focus events open modeless windowsE.g., Replace in MS WorldTermination Events : close modal windowsE.g., OK and CancelSystem- interaction events: interact with underlying software to perform some actionE.g., Copy event used for coping objects to the clipboard

GUI testing issuesThe event-driven nature of GUIspresents the first serious testing difficulty because users can click on any pixel on the screen and there so many of them Unsolicited eventsE.g., when the printer is out of papersThe infinite sequence of events (or permutations of events)GUI can be represented by Object oriented paradigmHidden synchronization and dependencies (objects in the same windows and different windows)E.g., if a check box is set to true, a text box intended to accept a numeric value elsewhere in the window may be made inactive or invisible. Many events in/Many events outTestability (difficulty to specify, to execute and to analyze test cases)event-based system +hidden interactionsCoverage criteria??? (memon et al uses events, and events sequences)Suffers from some of problems related to COTs (or precompiled components)

There are serious legal issues related to both design and testing of GUILegal issuesPrivacy/security E.g., access to the privileged/stored dataSafety/ reliabilityE.g., GUI for Safety critical systems can affect life-or-death decisions (functionality and ease of use)AccessibilityEqual access for disabled users

Focus on errors to reduce the scope of tests Separation of concerns (divide and conquer)Use Test design techniques when it is applicable (e.g., blackbox)

GUI errors includeData validationIncorrect field defaultsMishandling of server process failuresMandatory fields, not mandatoryWrong fields retrieved by queriesIncorrect search criteriaField orderMultiple database rows returned, single row expectedCurrency of data on screensWindow object/DB field correspondenceCorrect window modality?Window system commands not available/dont workControl state alignment with state of data in window?Focus on objects needing it?Menu options align with state of data or application mode?Action of menu commands aligns with state of data in windowSynchronization of window object contentState of controls aligns with state of data in window?

Testing Strategies includeFunctionalityUnit testing (Buttons)Integrations Testing (synchronizations)System Testing (functional behaviors according to requirements or functional testing)Non-Functionality(Quality) Usability testingSecurity testing

Model-Based Approach to test GUI requires:Identify components and EventsSelect a hierarchal formal model (e.g., EFSM) to represent GUI elements (components)Intra vs. inter componentsUse model to generate threats or event sequences of various lengths corresponding to visible & observable system outputsTotal number of events sequences increases when the length increaseIt might impractical to test for ALL possible event sequencesbe selective?

Unit and Integration testing minimally neededMain focus is on System Testing Model?Event-driven Petri nets (EDPNs)Testing threads

Event drivenUnit button level testingNot much integration testingSystem level testing: ModelsUML Behavioral modelsStateChartsFinite State Machines

Starts testing (functional and structural) of user-supplied code for buttonsComputeValid and invalid data (e.g., non-numerical, etc)ClearQuitTwo approachesUsing Drivers (good)Using GUI (system level for input/output events)

Not too much of Integration testingThree possible options:Focus on all logic in one place and use the value of the option buttons as conditions in IF testsUse OO approach using methods invocation to exchange the rate valueUse visual BasicEvent procedures for each option button uses constant value for specific exchange rate(e.g. Private Sub OptEU_click () exchange =1.35 End sub)

Heavily usage of System testingNeed event-driven modelEvent-Driven PN (EDPN) to specify and to generate scenarios/threadsTables 19.1 and Table 19.2

The process to generate threads (positive/negative scenarios) includeMap the elements of CCP to EDPNMap input/out events to input /output events of EDPNMap Data to EDPN data placesMap each Action to the transition in EDPNCreate EDPN for each ASF (tasks)Compose the EDPNsUse ASF graphGenerate a subset (or suitable?) of threads

How many threads are needed?Coverage criteria?Reasonable coverage includesTest ALL port inputsTest All port outputsTest All ASF (tasks or requirements)Difficult to test each ASFTest ThreatsThere are too manyNeed to be selective

Definition: EDPN is tripartite-directed graph (P, D, S, In, Out)P: a set of port eventsD: a set of data placesS: a set of transitionsIn: a set of ordered Paris from (P D) SOut: a set of ordered pairs from S (P D)P, D, S correspond to Data, Input/output Events, Action,

Represent the snap shot of the simulating EDPN

Tuple(p4, d2, p12, p17,p21)descriptionm1(1,0,0,0,0)S4 enabled; s4 firedm2(0,1,1,1,1)Nothing enabledm3..mn

Port Input EventsP1: Enter US Dollar AmountP2: Click on BrazilP3: Click on CanadaP4: Click on European UnionP5: Click on JapanP6: Click on Compute ButtonP7: Click on Clear ButtonP8: Click on Quit Button

Output EventsP9: Display U.S. dollar amountP10: Display Brazilian RealsP11: Display Canadian dollarsP12 :Display European Community EurosP13: Display Japanese YenP14: Display ellipsisP15: Indicate BrazilP16: Indicate CanadaP17: Indicate European CommunityP18: Indicate Japan

Output EventP19: Reset selected countryP20: Reset BrazilP21: Reset CanadaP22: Reset European CommunityP23: Reset JapanP24: Reset equivalent currency amountP25: End application

Atomic System Functions (or actions) maps to transitionS1: Store US Dollar amountS2: Sense Click on BrazilS3: Sense click on CanadaS4: Sense click on EUS5: Sense click on JapanS6: Sense click on clear buttonS7: sense click on the Clear buttonS8: Sense click on Quit button

D1: US dollar amount enteredD2: country selected

The composition is not definedPlace fusions?Looks like itTransition fusions?

Consider a set of threads T = {T1, T2, T3, T4}Scenario 1: 1) enter US amount, 2) select EU, 3) press compute button, 4) press quitT1 = {s1, s4, s6, s8}Scenerio2: 1) User enters US$, 2) Click on Brazil, 3) Clicks on Compute, Clicks on Clear button)T2 = {s1, s2, s6, s7}Scenario 3:T3 = {s3, s1, s6, s7}Scenario 4:T4 = {s5, s1, s7, s8}In general the set of T covers (minim set of usages for CCP)Every action (or S)Every port input (every input data and input event)Every port output (every output data and output event)

Next-level user behaviorCommon usage scenarioConverts amount to all 4 currencies, clears and quitsT5 = [s1, s2, s6, s3, s6, s4, s6, s5, s6, s7, s8]

Unexpected user behavior (negative scenario) Neg-Sencario1: User changes his/her mind about currencyT6 = [s1, s2, s3, s4, s5, s6, s7, s8]unexpected behavior (negative scenario)Neg-Scenario2:User toggles between 2 countries, then quitsT7 = [s1, s2, s3, s2, s3, s2, s3, s8]

Unit testing & integration testing not importantSystem testing most complicated test.ASFs and EDPNs can help identify and validate test casesValid usages (positive scenarios)Invalid usages (negative scenarios)

How to use MBT to test GUI?How to measure the benefit and cost of using MBT in GUI?Usability testing/evaluation: requires specified requirements or formal models of GUIExpert reviewsHeuristic evaluationGuidelines reviewConsistency inspectionCognitive walkthroughFormal usability inspectionUsability Testing and LaboratoriesSurveys InstrumentsAcceptance testsEvaluation during Active Use

Software Testing by P. JorgensenModel-Based Testing by Utting

It is difficult to specify testsbecause much of the underlying functionality in a GUI application is undocumented. Because of the event-driven nature of GUIs, a considerable amount of programming effort is expended on dealing with hidden interactions that come to light during the informal programmer testing so tend to go undocumented.It is difficult to prepare testsbecause the number tests required to exercise paths through the application which a user might follow has escalated dramatically. If we consider using menus, function keys and mouse movements to exercise system features, the number of tests increased further.It is difficult to execute tests.Using a manual pointing device is virtually unrepeatable and certainly error prone. Creating tests which stimulate hidden interactions, set or amend visible (or invisible) GUI objects is troublesome. Separating tests of application code from the GUI elements of the operating system is tricky.It is difficult to analyze testsbecause there is constant change on the screen and behind the screen. Windows on which results are displayed may appear and all other visible windows may be refreshed simultaneously making visual inspection difficult. Expected results may not be directly displayed but on hidden windows. Attributes of objects to be verified may be invisible or difficult to detect by eye. Windows that display invalid results may be hidden by other windows or on windows that are minimized.

**We can create a very general use-cases that include both use and misuse cases. Using Use cases, we then can create ASF graph using the order between actions. Once the ASF graph is created, we then generate scenarios. Scenarios or threads.*An infinite number of threads can be generate. It is impossible to test all of them.

*