An Empirical Study of Spreadsheet Authors’ Mental Models in Explaining and Debugging Tasks

Bennett Kankuzi, Jorma Sajaniemi

School of Computing, Joensuu CampusUniversity of Eastern Finland, Finland

Outline

•Introduction•Methodology•Results•Discussion•Conclusion

16.9.2013Kankuzi, Sajaniemi VL/HCC 2013 2

Introduction

•Many spreadsheets have non-trivial errors (Panko, 1998; Powell et. al, 2009)

•The spreadsheet comprehension and debugging problem is still very much there

•It is important to understand spreadsheet authors’ mental models when doing different spreadsheet process activities

– can help to understand why the spreadsheet process is so error-prone – can help to develop the right tools and techniques for spreadsheet

activities

16.9.2013Kankuzi, Sajaniemi VL/HCC 2013 3

Introduction (cont’d)

•Research Question: – what are the mental models of spreadsheet authors when they are

working with spreadsheets?

•Mental model - “a mental image of the world around us that we carry in our heads depicting only selected concepts and relationships that represent real systems” (Doyle & Ford, 1998)

•What are those aspects of the spreadsheet that a spreadsheet user finds appropriate for the task he or she has at hand?

16.9.2013Kankuzi, Sajaniemi VL/HCC 2013 4

Methodology

•Study organized into two activities– first activity dwelt on an explaining task while second activity

involved a debugging task

•In the explaining task: – each participant requested to explain their own chosen

spreadsheet to the researcher•Debugging task:

– each participant given an erroneous copy of their spreadsheet and asked to locate the seeded errors and to fix them

16.9.2013Kankuzi, Sajaniemi VL/HCC 2013 5

Methodology (cont’d)

•Adapted Good’s program summary analysis in analysis of transcript content

•Each object reference coded on a yes/no scale with respect to three object types: “real-world”, “domain-specific”, and “spreadsheet-specific”

16.9.2013Kankuzi, Sajaniemi VL/HCC 2013 6

Results

16.9.2013Kankuzi, Sajaniemi VL/HCC 2013 7

Discussion

•Nature of mental models vary depending on the task at hand

•When explaining a spreadsheet, the participants talk (i.e., think) mainly in terms of domain and real-world concepts

•When locating errors, study participants think mainly in terms of domain and spreadsheet concepts

16.9.2013Kankuzi, Sajaniemi VL/HCC 2013 8

Discussion (cont’d)

16.9.2013Kankuzi, Sajaniemi VL/HCC 2013 9

Discussion (cont’d)

•Nature of mental models vary depending on the task at hand

•When explaining a spreadsheet, the participants talk (i.e., think) mainly in terms of domain and real-world concepts

•When locating errors, study participants think mainly in terms of domain and spreadsheet concepts

16.9.2013Kankuzi, Sajaniemi VL/HCC 2013 10

Discussion (cont’d)

•When fixing errors, study participants think mainly in terms of spreadsheet concepts although domain concepts are also more pronounced in a secondary role

16.9.2013Kankuzi, Sajaniemi VL/HCC 2013 11

Discussion (cont’d)

Implications for Tool Development:•Results of study indicate that when explaining a

spreadsheet, the real-world and domain models are prominent

– a tool intended to aid in comprehension of a spreadsheet should make prominent real-world and domain concepts and map those concepts easily to spreadsheet-specific details

– spreadsheet comprehension tools should not just focus much on amplifying spreadsheet specific details such as cell references because doing so will cause a mismatch in how spreadsheet authors think

16.9.2013Kankuzi, Sajaniemi VL/HCC 2013 12

Discussion (cont’d)

Implications for Tool Development:•To locate and fix an error (debugging), one mainly

switches back and forth between the domain model and spreadsheet model

– a good debugging tool must not only display possible spreadsheet errors in spreadsheet terms (e.g., cell references) but also in domain terms in order to help authors easily discern errors since there will be a more pronounced mapping between the problem domain and the spreadsheet

16.9.2013Kankuzi, Sajaniemi VL/HCC 2013 13

Conclusion

•Reported on an empirical study carried out to explore the nature of mental models of spreadsheet authors when explaining and debugging own spreadsheets

•Found that the nature of mental models vary depending on the task at hand

•Findings provide insights on the need for developing spreadsheet authoring and debugging tools that correspond to spreadsheet authors’ mental models given the task at hand

16.9.2013Kankuzi, Sajaniemi VL/HCC 2013 14

Thank you for your attention!

www.uef.fi