using neuroscience theories and tools to inform information system research
TRANSCRIPT
PIERRE-MAJORIQUE LÉGER, Ph.D.Co-director, Tech3LabDepartment of Information Technologies, HEC Montréal
November 2015
Using neuroscience theories & tools to inform system research
© Copyright Tech3Lab 2015
Prof. Pierre-Majorique Léger, Ph.D.
Full ProfessorDirector ERPsim LabCo-director Tech3Lab [email protected]
GRAD stUDIEs
POst-DOctORAtE
LAbs VIsItInG PROfEssOR
Lab Serious games to learn enterprise systems and business analytics
https://www.researchgate.net/profile/Pierre-Majorique_Leger
EASE OF USE?
© Copyright Tech3Lab 2015© Copyright Tech3Lab 2015
EASE OF USE?
EASE OF USE?
EASE OF USE?
© Copyright Tech3Lab 2015
UXManaging Business User Experience
BUSINeSS INteLLIGeNCe
MArketING AND eLeCtrONIC COMMerCe
INFOrMAtION teChNOLOGIeS
INNOVAtION 2013
Management comittee
SYLVAIN SÉNÉCAL, Ph.D.Co-directorFull Professor
FrANÇOIS COUrteMANChe, Ph.D.Research professional
MArC FreDette, Ph.D.ResearcherAssociate Professor
PIerre-MAJOrIQUe LÉGer, Ph.D.Co-directorFull Professorn
eLISe LABONtÉ-LeMOYNe, Ph.D.Post-Doctoral researcher
© Copyright Tech3Lab 2015
Financing Agencies
© Copyright Tech3Lab 2015
Tech3Lab 2015 : The most important UX Lab in North America
© Copyright Tech3Lab 2015
sEctIOn 1 Emotion and Attention in HcIsEctIOn 2
Mobility and tactile interfaces in HcI sEctIOn 3
Learning and adaption in HcI
sEctIOn 1 Emotion and Attention in HcI
© Copyright Tech3Lab 2015
What is eye-tracking?
© Copyright Tech3Lab 2015
tobii Eyetracker X60
© Copyright Tech3Lab 2015
© Copyright Tech3Lab 2015
tobii Eyetracker X60
© Copyright Tech3Lab 2015
© Copyright Tech3Lab 2015
Source: http://eyetracking.me/?page_id=9
What is Eye Tracking
© Copyright Tech3Lab 2015
Processing order: Gaze path
© Copyright Tech3Lab 2015
Visual attention: Heatmap
© Copyright Tech3Lab 2015
!!
!
83(4'1'9+!120:0-!&'()*+!,"#$!'//)01*21+0!83(4'1'9+!120:0!"+207<!
Figure 1. Illustration of pupillary responses in cognitive taskexecution, (A) Mental multiplication task and pupil size change, (B) Difficulty effect on pupil dilation evoked by mental multi-plication of two numbers displayed together for eight seconds (adopted from Klingner's Dissertation (Figure 5.5) [1]).
(B)
(A)
Source: Hossein & Yeasin, 2014
© Copyright Tech3Lab 2015
© Copyright Tech3Lab 2015
Emotional attention: HeatmaptEcHnOLOGy DEVELOPED At tech3LabCourtemanche, F., et al., Method and Product forVisualizing the Emotions of a User, in Provisional patent application 2015. p. p. 14.
sEctIOn 2 Mobility and tactile interfaces in HcI
The effect of touching on advertising effectiveness
• Needfortouch+• tact iledevice=
•upto2XmorechaNcestorecogNizebraNds
Senecal, S., Leger, P. M., Fredette, M., Courtemanche, F., Cameron, A. F., Mirhoseini, S., & Riedl, R. (2013). Touch Screen as Input Device: Does it Influence Memory Retrieval?.
The touch effectvirtually touching the product has significant effect on brand and product recognition
The impact of treadmill desk on cognition
Seated condition
Walking condition
Treamillhabituation
Reading taskwhile walking
Reading taskwhile seated
Adjustmentof equipment
5 min. 40 min. 10 min. 10 min. 10 min.EEG
Recall taskPost-
experimentalsurvey
▪ Proportion of EEG frequency band power for electrodes with significant differences between groups
Labonté-LeMoyne, É., Santhanam, R., Léger, P. M., Courtemanche, F., Fredette, M., & Sénécal, S. (2015). The delayed effect of treadmill desk usage on recall and attention. Computers in Human Behavior, 46, 1-5.
Avec la permission de ICI Radio-Canada
Texting while walking:an expensive switch cost
Courtemanche, F., Léger, P.-M., Cameron, A.-F., Faubert, J., Labonté-LeMoyne, É., Sénécal, S., Fredette, M. et Bellavance, F. (2014). «Texting While Walking: Measuring the Impact on Pedestrian Visual Attention», Gmunden Retreat on NeuroIS 2014, Gmunden, Autriche, 5-7
sEctIOn 3 Learning & adaption in HcI
Measuringbrain activity
© Copyright Tech3Lab 2015
1011
© Copyright Tech3Lab 2015
© Copyright Tech3Lab 2015
© Copyright Tech3Lab 2015
ÜííéWLLìéäç~ÇKïáâáãÉÇá~KçêÖLïáâáéÉÇá~LÉåLÉLÉQLqÉíêáë|alp|NV=ÜííéWLLìéäç~ÇKïáâáãÉÇá~KçêÖLïáâáéÉÇá~LÉåLÉLÉQLqÉíêáë|alp|NVUSKéåÖ=USKéåÖ
© Copyright Tech3Lab 2015
Learning in the brain
Source: Hill and Schneider, 2006
Controlled processing
Automatic processing
Learning in the brain
Source: Hill and Schneider, 2006
Rapid acquisition Easy alteration of memory
Slow and high effort Deteriorates under stress
Slow acquisition Difficult to alter
Fast and low effort Robust under stress
Reduction of control network activation
which scaffold learning
http://upload.wikimedia.org/wikipedia/en/e/e4/Tetris_DOS_19 http://upload.wikimedia.org/wikipedia/en/e/e4/Tetris_DOS_1986.png 86.png
Source: Haier, 1992
Amateur Expert
ALPHA relaxed/reflecting
alert/working
drowsy/ideating
sleep/dreaming
deep dreamless
BETA
THETA
DELTA
© Copyright Tech3Lab 2015
Astronaut/aquanaut Nicole P. Stott and Ross Hein, a professional aquanaut from the University of North Carolina at Wilmington (UNCW), assist University of Cincinnati physician Tim Broderick apply a net with electrodes for an electroencephalogram (EEG) study
© Copyright Tech3Lab 2015
ALPHA relaxed/reflecting
alert/working
drowsy/ideating
sleep/dreaming
deep dreamless
BETA
THETA
DELTA
Pope, Alan T., Edward H. Bogart, and Debbie S. Bartolome. “Biocybernetic system evaluates indices of operator engagement in automated task.” Biological psychology 40.1 (1995): 187-195.
© Copyright Tech3Lab 2015
ERPsim on SAP HANA
now available on sAP HAnA
Automated administrative tasks
Simulated market
Simulated passing of time
Business analytics
Business decisions
VIRtUAL systEM PARtIcIPAntsERP systEM
1 12 23
floW=d iff icultyaNd compet eNcy+
psychophys iological stat eoft helearNer
Predicting the flow state
16%
18%
© Copyright Tech3Lab 2015
Léger, Pierre-Majorique, et al. “Neurophysiological correlates of cognitive absorption in an enactive training context.” Computers in Human Behavior 34 (2014): 273-283.
© Copyright Tech3Lab 2015
Attentional heatmap in during software interaction
© Copyright Tech3Lab 2015
Mobile Eye-tracking AssEssInG AttEntIOn AnD bEHAVIOUR In sOftwARE AnD MEcHAnIcAL DEVELOPMEnt
© Copyright Tech3Lab 2015
Understanding problem solvingAssEssInG sELEctIVE AttEntIOn In A MULtIPLAn EnVIROnMEnt
© Copyright Tech3Lab 2015
Monitoring learner’s engagement in real-time
© Copyright Tech3Lab 2015
HyPOtHèsEGénérer des émotions compatibles avec l’engagement dès la prise en main du jeu
MEsUREs Activité électrodermale, mouvement oculaire et rythme cardiaque
L’EffEt DEs tyPEs DE tUtORIEL DE JEU sUR L’IntEntIOn D’UtILIsAtIOn D’APPLIcAtIOns MObILEs
Chercheur : Pierre-Majorique Léger, Ph.D. (HEC Montréal) Partenaire industriel : Hibernum Créations inc Financement : CRSNG EGP 484281 – 15
cARtE DE cHALEUR OcULOMÉtRIQUE
PARcOURs OcULAIRE
© Copyright Tech3Lab 2015
EffEt DE LA RÉDUctIOn DE L’ÉMOtIOn nÉGAtIVE LIÉE AU DEsIGn sUR L’EffIcAcItÉ DEs APPLIcA-tIOns ÉDUcAtIVEs Et DE fORMAtIOn
Chercheur : Pierre-Majorique Léger, Ph.D. (HEC Montréal) Partenaire industriel : HMH Tribal Nova Financement : CRSNG EGP 485446 - 15
cARtE DE cHALEUR OcULOMÉtRIQUE
PARcOURs OcULAIRE
HyPOtHèsEMaintenir l’apprenant dans un état compatible avec la volonté de persévérer
MEsUREs Mesure de la charge mentale avec le diamètre de la pupille et l’analyse automatique des émotions faciales
floWofplayer(a)=d iff iculty+compet eNcy+
player(a)+player(b)
Predicting flow in multiplayer context
66%
Bastarache, Léger, Courtemanche, Sénécal and Fredette (2015) Measuring Flow Using Psychophysiological Data in a Multiplayer Gaming Context, Information Systems and Neuroscience, Lecture Notes in Information Systems and Organisation.
conclusion
© Copyright Tech3Lab 2015
Upcoming book at Springer: R. RIEDL, P.-M. LÉGER
fundamentals of neuroisInformation Systems and the Brain
Series: Studies in Neuroscience, Psychology and Behavioral Economics
▪ Provides the fundamentals of NeuroIS
▪ Guides the reader to theories and tools which are relevant for this field
▪ Contains tutorial-like illustrations which enhance understanding
▪ Written by experts in the field
© Copyright Tech3Lab 2015
sEctIOn 1 Emotion and Attention in HcIsEctIOn 2
Mobility and tactile interfaces in HcI sEctIOn 3
Learning and adaption in HcI
Thank you!