formative evaluation
DESCRIPTION
Formative Evaluation. cs3724: HCI. ANALYZE. analysis of stakeholders, field studies. claims about current practice. Problem scenarios. Scenario-Based Design. DESIGN. Activity scenarios. metaphors, information technology, HCI theory, guidelines. iterative analysis of - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/1.jpg)
Formative Evaluation
cs3724: HCI
![Page 2: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/2.jpg)
Problem scenarios
summativeevaluation
Information scenarios
claims about current practice
analysis ofstakeholders,field studies
Usability specifications
Activityscenarios
Interaction scenarios
iterativeanalysis ofusability claims andre-design
metaphors,informationtechnology,HCI theory,guidelines
formativeevaluation
DESIGN
ANALYZE
PROTOTYPE & EVALUATE
Scenario-B
ased Design
![Page 3: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/3.jpg)
Usability Engineering
Reqs Analysis
Evaluate Design
Prototype
![Page 4: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/4.jpg)
Usability Engineering
Reqs Analysis
Evaluate
many iterations
Design
Develop
![Page 5: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/5.jpg)
Usability Engineering
Formative evaluation
Summative evaluation
![Page 6: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/6.jpg)
User Interface Metrics
• Ease of learning• learning time, …
• Ease of use• perf time, error rates…
• User satisfaction• surveys…
Not “user friendly”
![Page 7: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/7.jpg)
Claims Analysis
• + advantages
• - disadvantages
• E.g. file management:
+ +
![Page 8: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/8.jpg)
Claims Analysis
• + advantages
• - disadvantages
• E.g. file management:
+ fast learning+ spatial learning aid memory- switching between devices- long to locate a known file…
+ flexible, power+ fast for experts - Learning commands - aesthetics lacking?...
![Page 9: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/9.jpg)
Usability Evaluation
• Analytic Methods:• Heuristic Evaluation
• Models: GOMS, Fitts Law
• Empirical Methods:• Usability Testing
– Wizard of Oz
• Controlled Experiment
![Page 10: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/10.jpg)
Nielsen’s 10 Heuristics1. Visible status, feedback2. User control, undo, exits3. Familiar, speak user’s language4. Consistent, standards5. Recognition over recall6. Efficient, expert shortcuts7. Aesthetic, minimalist8. Prevent errors9. Error recovery10. Help, task-based
![Page 11: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/11.jpg)
Nielsen’s 10 Heuristics1. Visible status, feedback wysiwyg
2. User control, undo, exits wizards
3. Familiar, speak user’s language Acrobat error msg
4. Consistent, standards Word, PPT
5. Recognition over recall web nav, phone menu
6. Efficient, expert shortcuts Word bold
7. Aesthetic, minimalist phone book
8. Prevent errors HomeFinder
9. Error recovery undo, back
10. Help, task-based IIS help doc
![Page 12: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/12.jpg)
Speak the User’s Language
![Page 13: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/13.jpg)
Help documentation
Context help, help doc, UI
![Page 14: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/14.jpg)
Applying Heuristics
• Lets look at a website…
![Page 15: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/15.jpg)
Early Usability Testing
• Formative: helps guide design
• Early in design process• when architecture is finalized, then its too late!
• A few users (3-5)
• Identify usability problems, incidents
• Qualitative feedback from users
• Quantitative usability measures
![Page 16: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/16.jpg)
Usability Test Procedure• Wizard of Oz: act out the system• Benchmark task set• No cheating! Don’t teach them how to use the UI• Repeat:
• Give user a task• Ask user to “think aloud”• Observe, note mistakes and problems• Avoid interfering, hint only if completely stuck
• Interview • Verbal feedback• Questionnaire
• 30 min - 1 hour / user
![Page 17: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/17.jpg)
Analyze Results• Initial reaction:
• “stupid user!”, “that’s developer X’s fault!”, “this sucks”
• Mature reaction:• “how can we redesign UI to solve that usability problem?”• the user is always ‘right’
• Identify usability problems• Learning issues: e.g. can’t figure out or didn’t notice feature• Performance issues: e.g. arduous, tiring to solve tasks• Subjective issues: e.g. annoying, ugly
• Problem severity: critical vs. minor• Redesign
![Page 18: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/18.jpg)
Pizza Ordering System
• Scenario:
• Volunteers!
• Design ideas
• Wizard of Oz usability test
• Redesign
![Page 19: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/19.jpg)
![Page 20: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/20.jpg)
In action: Pizza Ordering System
• Scenario: VT students, studying/busy/tired, poor, wants discounts, starving, credit card / hokie card/ no cash, 2 in together, half-n-half, split cost
• Volunteers!
• Design ideas
• Wizard of Oz usability test
• Redesign
![Page 21: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/21.jpg)
Pizza R Us
peperoni
mushrooms
pineapple
WholeHalf-n-half
PRICE: $10.00
Drag your toppings:
checkout
![Page 22: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/22.jpg)
• Size of pizza:– large– Small
• First half:
• Toppings:– Peperoni– Mushrooms– Pineapple
• Price: 0.00
• 2nd half:
• Toppings:– Peperoni– Mushrooms– pineapple
![Page 23: Formative Evaluation](https://reader035.vdocuments.mx/reader035/viewer/2022062719/5681328d550346895d9923c7/html5/thumbnails/23.jpg)
• % pza –ppm –large -…