jessica 690tcc 4.22.14(final)
TRANSCRIPT
![Page 1: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/1.jpg)
E V A L U A T I N G
U S E R S ’EX P ER I E NC
Ethe
![Page 2: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/2.jpg)
JESSICA E. LEAUANAE
University of Hawaii, 2014 Candidate M.Ed., Educational Technology
![Page 3: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/3.jpg)
BACKGROUND
2011-2014: Instructional Designer
2003: BA English, BYU Provo
2011: Accepted to Master’s Educational Technology Program, University of Hawaii
This presentation is in partial fulfillment of my Master’s Project for ETEC 690
![Page 4: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/4.jpg)
![Page 5: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/5.jpg)
TRUE ?
A. If you teach someone how drinking is bad for them, they won’t become an alcoholic.
B. If someone goes to a management training class, they’ll be good managers.
C. If someone takes a really good photography class, they’ll be a good photographer.
D. None of the above.
![Page 6: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/6.jpg)
PROJECT PURPOSE:
Evaluate the User Experience of an e-learning environment and implement
improvements using iterative rapid prototyping.
![Page 7: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/7.jpg)
REAL LIFE CASE SCENARIO
June 2013Creation of Online Training Site Using Rapid Prototyping
August 2013250+ new hires begin utilizing site
October 2013Need for Site Improvements and Redesign
Journey Begins
![Page 8: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/8.jpg)
SYSTEMATIC APPROACH
How can we approach improvements systematically
under tight budget
and even tighter timeframes?
WHAT DO THE EXPERTS SAY?
![Page 9: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/9.jpg)
INSTRUCTIONAL DESIGN
IDEAL• ADDIE Method of Design• Formative and Summative
Assessments• Large sampling• Can be a long and lengthy
process• Can be costly
REALITY• Alternative methods of ID-
Rapid Prototyping• Targeted, real-time
evaluations• Small sampling• Fast iterative revisions
based on end user• Less costly• Must provide Proof of
Concept before investments are made
![Page 10: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/10.jpg)
• Discount Usability Testing• User centered design-user is the
central focus • Suitable for in-house, e-learning
modules produced by content experts.
• Reduce costs, faster results, increased customer satisfaction, yields a more successful product
*Designing an Affordable Usability Test for e-Learning Modules
STEVE KRUG:DON’T MAKE ME THINKROCKET SURGERY MADE EASY
![Page 11: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/11.jpg)
DONALD KIRKPATRICK:EVALUATING TRAINING PROGRAMS
• Simplistic approach based on user reactions
• Dominant model evaluating training programs for last 30 years
• Strategic alignment- the degree to which training programs conforms to and advances the goals of the organization
• Systematic matching of goals and outcomes
*Evaluating Distance Delivery and E-Learning. Is Kirkpatrick’s Model Relevant?
![Page 12: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/12.jpg)
IN TODAY’S DIGITAL WORLD
![Page 13: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/13.jpg)
THE USER IS KING
![Page 14: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/14.jpg)
{UX}
![Page 15: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/15.jpg)
{UX}Usability Testing
1.
![Page 16: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/16.jpg)
{UX}Usability Testing Learnability Testing
(Learner Experience)
1.
2.
![Page 17: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/17.jpg)
ETEC 690 MASTER’S PROJECT…
• Usability testing and learner experience evaluations
• Collect, gather and analyze data• Redesign based on user feedback• Rapid succession of build, test, re-design,
deploy
![Page 18: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/18.jpg)
![Page 19: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/19.jpg)
TEST
What: Test different versions of the site
![Page 20: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/20.jpg)
TEST
• What: Test different versions of the site
• Who: Current Virtual Instructors,
Lead status
![Page 21: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/21.jpg)
PARTICIPANT DEMOGRAPHIC
48 Participants Overall• Virtual Instructors• 90% Female, 10% Male• Ages 35-44• 69% Masters Degree• 52% Taught Online• 94% Taken an online course• 92 % Passed Tech Savvy Test
![Page 22: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/22.jpg)
TEST
• What: Test different versions of the site
• Who: Current Virtual Instructors,
Lead status
• Variables to test:Kirkpatrick’s
Learnability + Krug’s Usability
![Page 23: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/23.jpg)
KIRKPATRICK’S TRAINING EVALUATION
• 4 different evaluations measuring learners’ experience: Learner reaction Technology dimension Course dimension Design dimension
![Page 24: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/24.jpg)
• 48 participants total• 138 evaluations completed• 3 different iterations of site tested• embedded online within the site
![Page 25: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/25.jpg)
KRUG’S USABILITY TESTING FRAMEWORK
1 Usability Test Testing 3 different site versions Small test group: 8 participants divided across
3 cycles 18 Tasks to complete: Think aloud Testing is remote via Blackboard Collaborate 45-60 Minutes long Session is recorded Low fidelity prototype
![Page 26: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/26.jpg)
USABILITY TESTING DATA COLLECTION
![Page 27: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/27.jpg)
TEST REVISE RE-DESIGN
{X 3 versions}DA
TA AN
ALYS
IS
USAB
ILITY
&
LEAR
NABIL
ITY
RAPID
PR
OTOTY
PING
![Page 28: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/28.jpg)
![Page 29: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/29.jpg)
Learning experiences are like journeys…
![Page 30: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/30.jpg)
Learning experiences are like journeys…
![Page 31: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/31.jpg)
HOW WAS THE JOURNEY?
![Page 32: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/32.jpg)
JOURNEY
TRAINING SITE
1
![Page 33: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/33.jpg)
TIME: Built in 2 months
MONEY: Lack of resource, Prototype built in Google site, Google Aps for education
PROCESS: Convert analog content to digital. More than a transcription of paper-based content
PEOPLE: MeOne man band- content developer, graphic artist, video editor, voiceover, tech support etc.,
QUALITY ASSURANCE:None, no feedback
![Page 34: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/34.jpg)
SITE VERSION 1: WHERE DO I BEGIN ?
![Page 35: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/35.jpg)
Or do I read the Mission?
Ohhh…surveys! I like surveys, should I jump
over and start completing them?
Do I start with the Training Modules?
ITERATIVE 1: 11.07.13
![Page 36: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/36.jpg)
LEARNER EXPERIENCE SUMMARYLEARNER REACTION:58% Navigation confusing
TECHNOLOGY DIMENSION:41% Login not intuitive
COURSE DIMENSION:61% Good scaffolding of concepts47% Scope and sequence appropriate
DESIGN DIMENSION:74% Enjoyed multi-modalites63% Video and audio syncing not aligned39% Different video delivery is distraction to learning.
![Page 37: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/37.jpg)
JOURNEY
TRAINING SITE
2
![Page 38: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/38.jpg)
SITE VERSION 2: CLUTTER IS DISTRACTING
![Page 39: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/39.jpg)
![Page 40: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/40.jpg)
LEARNER EXPERIENCE SUMMARYLEARNER REACTION:49% Navigation confusing
TECHNOLOGY DIMENSION:33% Login not intuitive48% Technology allowed self-pacing
COURSE DIMENSION:72% Good scaffolding of concepts56% Scope and sequence appropriate
DESIGN DIMENSION:74% Enjoyed multi-modalites70% Video and audio syncing not aligned44% Different video delivery is distraction to learning.
![Page 41: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/41.jpg)
JOURNEY
TRAINING SITE
3
![Page 42: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/42.jpg)
SITE ITERATION 3: CHOICES ALTER OUTCOMES
![Page 43: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/43.jpg)
![Page 44: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/44.jpg)
LEARNER EXPERIENCE SUMMARYLEARNER REACTION:27% Navigation confusing
TECHNOLOGY DIMENSION:39% Login not intuitive57% Fast and responsive
COURSE DIMENSION:88% Good scaffolding of concepts62% Scope and sequence appropriate
DESIGN DIMENSION:62% Enjoyed multi-modalites59% Video and audio syncing not aligned41% Different video delivery is distraction to learning.
![Page 45: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/45.jpg)
JOURNEY
TRAINING SITE
4
![Page 46: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/46.jpg)
SITE ITERATION 4: THE USER IS KING
![Page 47: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/47.jpg)
4 versions later…I FINALLY GOT IT
RIGHT!
![Page 48: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/48.jpg)
USABILITY TEST FINDINGS
![Page 49: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/49.jpg)
TAKEAWAYS
• Improvements based on User• Fast deploy• CMS/LMS issues not resolved• Journey, rough start, smoother finish• Alternative design approach using systematic
evaluation methods
![Page 50: Jessica 690TCC 4.22.14(Final)](https://reader035.vdocuments.mx/reader035/viewer/2022062902/577cc9e21a28aba711a4dc9f/html5/thumbnails/50.jpg)
Mahalo!