avaliação da usabilidade e da acessibilidade do ambiente virtual de aprendizagem teleduc: levando...
DESCRIPTION
IHC+CLIHC 2011 - Evaluation CompetitionTRANSCRIPT
Avaliação da Usabilidade e da Acessibilidade do Ambiente Virtual de
Aprendizagem TelEduc: levando o “aprender” a todos
André C. da Silva Ariel Vargas
Caroline C. Letizio Ricardo E. Caceffo
Prof. Dr. Heloísa V. da RochaProfessor
Institute of Computing - NIEDUNICAMP
Evaluation Competition October, 28th 2011
Agenda
• Ideas• Materials• Methods• Examples• Results• Conclusion
IdeasVirtual Learning Environments (VLE) characteristics
Funcionalities
Security
Performance
Scalability
Availability
UsabilityAccessibility
Reliability
Maintainability... Is the VLE for everyone?
Ideas
• TelEduc Environmento First Brazilian VLE open sourceo More than 4,000 installationso Analyzed version: 4.2o It has been used in our university
http://www.teleduc.org.br/
Materials
• Communication among all participants → Mail/Correio and Discussion Forums/Fórum de Discussão
• Administration of several contents → Support
Materials/Material de Apoio
• Development, delivery and monitoring of activities → Portfolio/Portfólio
Methods
Usability
Log Interaction Analysis
Inspection
User Tests
Nielsen
Questionnaire
Guidelines
Heuristic
WebHint
Methods - WebHint
Users
• Students with some VLE experience: 17
• One user with tool to change font size
Methods
Accessibility
WCAG (W3C)
Inspection
User Tests
IBM
Tanaka (2009)
Guidelines
Heuristic
Hera
Methods – Tanaka(2009)
• Similarly Nielsen usability evaluation
• 5 Heuristics:
6)Support for different types of input and output7)Contents for all kind of users8)Avoid independence of use9)Respect the user preferences10)Efficiency of alternative interaction
Examples
There is not showed a confirmation message
Examples of problems found
Usability: Add comment to a portfolio item
- Nielsen Heuristics: Visibility and status of system
- Log Interaction Analysis: Half of the users had executed the task distinct than expected
- Questionnaire: Half of the user agreed partially
"The information showed by the system after I performed some action request are enough to make me understand the
results of my action"
It is not possible to check a checkbox using only the keyboard
Accessibility: It is not possible use the system only with keyboard input.
- Accessibility Heuristcs: Support for different types of input and output
- Guidelines: Checkpoint 6.4 - For scripts and applets, ensure that event handlers are input device-independent
Examples of problems found
Results
Tool Errors x Severity Total
0 1 2 3 4
Mail/Correio 0 3 1 1 0 5
Portfolio/Portfólio 0 4 5 12 1 22
Discussion Forums/Fórum de Discussão
0 2 3 2 0 7
Suport Material/ Material de Apoio
1 7 2 4 0 14
General/Geral 0 3 2 2 2 9
Total 1 19 13 21 3 58
• Usability Heuristic Evaluation
Results
• Questionnaire
• WebHint Method:
o 5 usability's problems found
o A few number of users
Results
• Accessibility Heuristic Evaluation
ResultsTool Errors x Severity Total
0 1 2 3 4
Mail/Correio 0 0 0 0 13 13
Portfolio/Portfólio 0 0 0 0 9 9
Discussion Forums/Fórum de Discussão
0 0 0 2 12 14
Suport Material/ Material de Apoio
0 0 0 0 16 16
General/Geral 0 0 2 5 11 18
Total 0 0 2 7 61 70
• Guidelines Evaluations
ResultsPriority Mail/
CorreioForum Portfolio Support
Material / Material de Apoio
Total
1 2 1 1 2 6
2 8 8 8 8 32
3 1 3 3 3 10
Total 11 12 12 13 48
Conclusions
• Methods are complementary: • The usability results show that the TelEduc has good
usability, also pointing specific problems that can be solved to improve VLE's usability.
• The accessibility results show critical accessibility problems in TelEduc – Is NOT for everyone!!
o Accessibility 2 inspection methods
o Usability 1 inspection method 2 user tests methods
Thanks
• Instituto de Computação - Unicamp• Capes• CNPq• FAPESP
More information about this workhttp://www.students.ic.unicamp.br/~ra059664/ihc2011/
Extra: Accessibility and Usability
• In some situations were found problems that affect both usability and accessibility of TelEduc
• For example:• User starts the upload of some content in his/her
portfolio• User opens another tab and continues browsing
TelEduc, for example to read mail• When the user comes back to the portfolio tab, s/he
is not able to know if the upload was successfully or not
Extra: Accessibility and Usability
• It happens because the success or not success message is showed for only a few seconds (usability issue) and can't be “read” by the screen reader (accessibility issue)