making checklists social - stanford university€¦ · 05/04/2012 · making checklists social...
TRANSCRIPT
![Page 1: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/1.jpg)
Making Checklists Social
Scott Klemmer Jesse CirimeleNicolas Kokkalis
1
POMI research group
Thursday, April 5, 2012
![Page 2: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/2.jpg)
How many of you...?
Milkman, K et. al. in PNAS 2011
Thursday, April 5, 2012
![Page 3: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/3.jpg)
Checklists Everywhere
Pronovost et al, 2006
Thursday, April 5, 2012
![Page 4: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/4.jpg)
Checklists Everywhere
Pronovost et al, 2006
Ten-day Trial: infection rates from 11% to 0.
Michigan hospital study:infection down 66%saved $175 millionsaved 1500 lives
Thursday, April 5, 2012
![Page 5: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/5.jpg)
Checklists are Mobile
N Seyff et. al. in Software IEEE 2006
Thursday, April 5, 2012
![Page 6: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/6.jpg)
Checklists can be Socially Created or Used
Our Work
Thursday, April 5, 2012
![Page 7: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/7.jpg)
Creating Checklists Socially:Providing Action Plans helps people
complete Tasks
Nicolas Kokkalis, Johannes Huebner, Steve Diamond, Dominic Becker, Michael Chang, Thomas Koehn,
Florian Schulze, Moontae Lee, Scott Klemmer
Thursday, April 5, 2012
![Page 8: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/8.jpg)
Can personal task lists benefit from social creation?
Thursday, April 5, 2012
![Page 9: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/9.jpg)
Complete more and easier
No Action Plans
Provided Action Plans
Self-created Action Plans
Less
Ef
fort
Mor
e Ef
fort
More CompletedLess Completed
Thursday, April 5, 2012
![Page 10: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/10.jpg)
Can the crowd provide actionability to people’s tasks?
Thursday, April 5, 2012
![Page 11: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/11.jpg)
To enable our studies we created an online community around tasks: TaskGenies
People can submit their tasks, others can decompose them
Access is provided via WWW for desktop, App for mobile, and e-mail
Taskgenies Platform
Thursday, April 5, 2012
![Page 12: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/12.jpg)
TaskGenies User Interfaces
Desktop: Submit Task & Create Plans Mobile: Submit Tasks
Thursday, April 5, 2012
![Page 13: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/13.jpg)
Experiment: Method & Measures
280 people submitted 20 tasks they planed to complete
Control: no interventionCrowd: provided w/ crowd-created plansSelf: asked to create own plans
1 week later: measured # of tasks completed
Thursday, April 5, 2012
![Page 14: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/14.jpg)
F(2,230)=6.10, p<.05
Crowd-created plans helped people complete more tasks
Thursday, April 5, 2012
![Page 15: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/15.jpg)
Crowd Plans help diverse Tasks
High Level Tasks Lingering Tasks
!"#
$!"#
%!"#
&!"#
'!"#
(!"#
)!"#
*+,--#.#/0--#102301## 45647-080-##
Control Crowd
Task
s C
ompl
eted
(χ2=16.00, p<0.0001) (χ2=3.68, p=0.052) (χ2=17.72, p<0.0001) (χ2=6.24, p<0.05)
!"#
$!"#
%!"#
&!"#
'!"#
(!"#
)!"#
*+*,-.*/01.*/#23454## -.*/01.*/#23454##
Thursday, April 5, 2012
![Page 16: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/16.jpg)
69% of task had at least one analog
Thursday, April 5, 2012
![Page 17: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/17.jpg)
Scaling Up by reusing Action Plans
Using a corpus of 6000 action plans95% of tasks used an existing plan5% of tasks used crowd
NLP condition better than Control
Thursday, April 5, 2012
![Page 18: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/18.jpg)
Email: de facto task list of people
Thursday, April 5, 2012
![Page 19: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/19.jpg)
GmailValet.com
Thursday, April 5, 2012
![Page 20: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/20.jpg)
Restricting Access: Only 1 label, Only the last 100 messages
Thursday, April 5, 2012
![Page 21: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/21.jpg)
The Valet View
Thursday, April 5, 2012
![Page 22: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/22.jpg)
Trust but verify: Record all actions
Thursday, April 5, 2012
![Page 23: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/23.jpg)
Take aways
Action plans facilitate effective work, but we have them too rarely. Making checklists social helps us get things done.
Thursday, April 5, 2012
![Page 24: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/24.jpg)
Using Checklists Socially:Supporting crisis response teams
with checklists and multiple displays
Jesse Cirimele, Leslie Wu, Kyle Barrett, Katherine Chen, Kyle Harrison, Larry Chu, Stu Card, Scott
Klemmer
Thursday, April 5, 2012
![Page 25: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/25.jpg)
Surgery
Thursday, April 5, 2012
![Page 26: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/26.jpg)
Surgery
• Pervasive and essential (234 million surgical procedures per year)
Thursday, April 5, 2012
![Page 27: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/27.jpg)
Surgery
• Pervasive and essential (234 million surgical procedures per year)
• Complex178 tasks per day in ICU
Thursday, April 5, 2012
![Page 28: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/28.jpg)
Surgery
• Pervasive and essential (234 million surgical procedures per year)
• Complex178 tasks per day in ICU
• High mortality1/300 die in surgery
Thursday, April 5, 2012
![Page 29: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/29.jpg)
Types of errors
Thursday, April 5, 2012
![Page 30: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/30.jpg)
Types of errors
• Social
• team coordination problems
• communication issues (miss something)
Thursday, April 5, 2012
![Page 31: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/31.jpg)
Types of errors
• Social
• team coordination problems
• communication issues (miss something)
• Individual
• lose time (late)
• get treatment algorithms wrong
Thursday, April 5, 2012
![Page 32: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/32.jpg)
Braincan be slow, error prone
Thursday, April 5, 2012
![Page 33: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/33.jpg)
Braincan be slow, error prone
Cognitive AidFast and Accurate
Thursday, April 5, 2012
![Page 34: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/34.jpg)
Braincan be slow, error prone
Cognitive AidFast and Accurate
... butThursday, April 5, 2012
![Page 35: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/35.jpg)
Cognitive AidStatic, Visible to One
Interactive AidDynamic, Visible to Group
Thursday, April 5, 2012
![Page 36: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/36.jpg)
insights
Thursday, April 5, 2012
![Page 37: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/37.jpg)
insights
• Always ready
Thursday, April 5, 2012
![Page 38: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/38.jpg)
insights
• Always ready
• Nurse as primary controller
Thursday, April 5, 2012
![Page 39: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/39.jpg)
insights
• Always ready
• Nurse as primary controller
• Big display facilitates group coordination
Thursday, April 5, 2012
![Page 40: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/40.jpg)
Thursday, April 5, 2012
![Page 41: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/41.jpg)
Cog Aid => Resource Management
• cognitive aids are a resource to help the doctor during a crisis
• but so are
• other people in the room
• information about the patient
• people on-call in case of emergency
• communication
Thursday, April 5, 2012
![Page 42: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/42.jpg)
OR layoutThursday, April 5, 2012
![Page 43: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/43.jpg)
Tablet Controller
Large Screen Display
Thursday, April 5, 2012
![Page 44: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/44.jpg)
Scenario walkthrough
Thursday, April 5, 2012
![Page 45: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/45.jpg)
The Wizard of Oz and the OR
Thursday, April 5, 2012
![Page 46: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/46.jpg)
Thursday, April 5, 2012
![Page 47: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/47.jpg)
• Checklists (cognitive aids, task lists)
• can play an important role in many situations
• can help us more when we carry them with us
• can be more effective when support socially
Conclusion
Thursday, April 5, 2012
![Page 48: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/48.jpg)
Thank you!
Questions?
37Thursday, April 5, 2012
![Page 49: Making Checklists Social - Stanford University€¦ · 05/04/2012 · Making Checklists Social Scott Klemmer Jesse Cirimele Nicolas Kokkalis 1 POMI research group Thursday, April](https://reader033.vdocuments.mx/reader033/viewer/2022050607/5faebd8e8f8a69755e66faee/html5/thumbnails/49.jpg)
http://hci.stanford.eduhttp://aim.stanford.edu
Jesse Cirimele, Leslie Wu, Kyle Barrett, Katherine Chen, Kyle Harrison, Larry Chu, Stu Card, Scott Klemmer
Thursday, April 5, 2012