razvan petrusel presentation caise 2013

Download Razvan petrusel   presentation caise 2013

Post on 27-Jun-2015

138 views

Category:

Technology

2 download

Embed Size (px)

TRANSCRIPT

  • 1. Eye-tracking the Factors of Process Model Comprehension Tasks Rzvan Petruel, Jan MendlingBabe-Bolyai University Cluj-Napoca, Romania razvan.petrusel@econ.ubbcluj.ro Wirtschaftsuniversitt Wien, Austria jan.mendling@wu.ac.at

2. Overview Address the gap of research on the factors that influence the comprehension tasks. 2 Motivation and Relevant Region Definition The Experiment Conclusions 3. Motivation 3 Q1 - Can Z and AA be executed in the same case? Q2 - After O has been executed, and the default path is taken at the next gateway, then Z must always be executed? 4. Notion of Relevant Region 4 o Notion of Dominator o Relevant Region for activities a and b includes all elements on a path from the dominator node of a and b 5. Eye-tracking example 5 6. Eye-tracking example 6 7. Eye-tracking example 7 8. Eye-tracking example 8 9. Hypothesis H1: The Relevant Region elements are fixated a longer time than other model elements by the subjects that provided the correct answer to the comprehension question; H2: More elements of the Relevant Region are fixated than other model elements by the subjects that provided the correct answer; H3: The higher the percentage of time spent fixating the Relevant Region elements, the more likely is a correct answer; H4: The higher the share of Relevant Region elements a person fixates (scan-path recall and/or f- measure), the more likely is a correct answer. 9 10. Experiment Conducted using 26 experts from academia and industry in Vienna, Eindhoven and Cluj- Napoca; Eye-tracking of subject behavior while answering 6 comprehension questions based on 2 structured and 4 unstructured models; Measured: Model elements inspected; Time spent fixating each model element. 10 11. Measurements 11 ATT ADT ALX ALY ARI APW APH AQU AFX TotalTime DeltaTimeX_Gaze Y_Gaze Region PupilWidthPupilHeightQuality Fixation 0.0000 0.0000 0.1869 0.5495 3 0.0759 0.0576 1 0.0167 0.0167 166.523 0.1829 0.5426 3 0.0746 0.0557 1 0.0167 0.0334 167.122 0.1796 0.5335 3 0.0752 0.0572 1 0.0167 0.0500 166.523 0.1759 0.5257 3 0.0758 0.0582 1 0.0167 0.0619 B:ROI[03] for 0.183466 sec 0.0668 167.968 0.1728 0.5149 3 0.0762 0.0568 1 0.0168 0.0835 166.659 0.1708 0.5061 3 0.0696 0.0607 1 0.0167 0.0953 B:Fixation 10263 ( 0.194, 0.501 ) for 0.116743 sec 0.1001 166.579 0.1683 0.4958 3 0.0735 0.0572 1 0.0167 0.1168 166.703 0.1680 0.4936 3 0.0735 0.0559 1 0.0167 0.1335 166.535 0.1694 0.4894 3 0.0698 0.0592 1 0.0167 0.1501 166.661 0.1697 0.4896 3 0.0764 0.0573 1 0.0167 0.1501 A:Drift 7313 ( 0.183, 0.539 ) --> ( 0.170, 0.490 ) = 0.05134 distance for 0.216858 sec 0.1668 167.059 0.1684 0.4895 3 0.0778 0.0572 1 0.0334 12. Results Model_question no 19_0 19_6 29_5 30_3 39_6 50_1 Correct (no.) 18 18 17 6 16 14 Incorrect (no.) 5 6 2 10 0 9 Correct (%) 78.26% 66.67% 89.47% 37.5% 100% 60.87% Subject 1 2 3 15 25 Question code 10_0 10_0 10_0 10_0 10_0 Outcome 1 1 0 1 0 TIR 47% 86% 12% 65% 59% Count ME Actually visited 16 5 17 6 12 Count RR Elem 3 3 2 3 3 Total RR Elem 3 3 3 3 3 SPP 0.19 0.60 0.12 0.50 0.25 SPR 1.00 1.00 0.67 1.00 1.00 SPF 0.32 0.75 0.20 0.67 0.40 SPF2 0.54 0.88 0.34 0.83 0,63 Variable F F2 SPP SPR TIR Sample size 146 146 146 146 146 Arithmetic mean 0.5742 0.5742 0.4745 0.6591 0.5729 Standard deviation 0.2214 0.2214 0.2547 0.2687 0.2839 13. Results H1 and H2 13 Variable F F2 SPP SPR TIR ANOVA F-ratio 28,247 29.650 17.290 21.446 17.964 ANOVA Significance P