electronic paper -prototyping tools · characteristics of prototyping approaches. this thesis has...
TRANSCRIPT
SUPPORTING ANNOTATIONS IN
ELECTRONIC PAPER-PROTOTYPING TOOLS
Amir M Naghsh
A thesis submitted in Partial fulfilment of the requirements for the Degree of Doctor of
Philosophy at Sheffield Hallam University
August 2007
Amir M Naghsh i
To my grand parents who inspired me through my life, especially to Shahzadeh Ayati
who always had faith in me.
Amir M Naghsh ii
Communication
The most obvious factor that distinguishes humans from other animals is the variety of
ways in which we communicate. Human Interactions are a rich blend of language, facial
and body movement, and symbols; we also assess the context of any message to obtain the
maximum amount of information. Our style of communication is one of the clearest
markers of identity.
Lord Robert Winston, Human, page 301
Amir M Naghsh iii
Amir M Naghsh iv
ABSTRACT
It is generally accepted that user involvement is essential for the successful design of an
interactive system. Enabling users to envisage or make sense of design proposals (whether those
proposals originate with ‘professional designers’ or from the users themselves) is an essential
element of all participatory approaches to design. Users can only make informed choices when the
proposals being discussed are meaningful to them. In this process, stakeholders exchange a large
number of documents and models, and may spend a great deal of time meeting to reach a common
ground. This becomes challenging in design projects where stakeholders are fully dispersed (both
geographically and in regards of competence).
Previous research indicates that electronic paper-prototyping can be used to rapidly create simple
prototypes of interactive systems, such as websites. Recent studies point to various ways in which
the design process depends on communicative activities around collaboratively produced
prototypes. Little is known about how to provide and maintain a variety of communication
channels around electronic paper-prototypes to enable end-users and other stakeholders to
contribute to design dialogues.
This thesis examines a range of tools for early prototyping of interactive systems, each of which
provides some facilities that may be useful in enabling electronic paper prototyping, but each of
which also suffers limitations in terms of their ability to support communication activities. The
thesis presents theoretical and empirical analysis to explore the feasibility and requirements for
creating an electronic paper prototyping tool that can facilitate computer supported collaboration
in the early stages of designing interactive systems.
Collaboration can be facilitated by applications for sharing and annotation. This thesis reports
research conducted on the use of annotation in asynchronous collaboration systems. Furthermore,
this thesis presents design, development and evaluation of Gabbeh, an electronic paper
prototyping tool, designed to facilitate communication and collaboration through annotating the
prototype at the early stages of design. The core innovation in Gabbeh is that it allows different
stakeholders to add arbitrary annotations in the form of comments either when the prototype is
being designed or when it is being executed.
This thesis reports the results of observational and longitudinal studies of using Gabbeh and
explores how different stakeholders may use annotations in an electronic paper prototyping tool.
Finally, this thesis suggests the important areas for further investigations.
Amir M Naghsh i
ACKNOWLEDGMENTS
I wish to express my sincere thanks to all those who have encouraged and backed me
up while writing this thesis.
First, I would like to express my sincere gratitude to my director of studies, Dr. Andy
Dearden, for his continuous support of me at Sheffield Hallam University. I would
like to thank Andy for giving me the opportunity to begin this work and for then
giving me the skills, continues guidance and experience to finish it. Without his
knowledge and help, this thesis would never have begun and without his support and
understanding it certainly would never have got been finished.
I would also like to thank my advisors, Dr. Mehmet B. Ozcan and Prof. Jawed Siddiqi
for their great comments. I would especially like to single out Mehmet, who gave me
more feedback and time out of his busy schedule than anyone next to Andy.
I am also grateful to all those who took part in my studies or answered my questions,
my anonymous thanks to them all.
Friends are something without which I could not have made it through this time in my
life. I would like to thank Haleh, Rashmee, Mohammad, Pav, Georgios and Sara who
have provided pleasant distraction over the years.
I would like to offer my special and sincere thanks to my parents for all years of
support and forgiveness for not being there when they needed me the most. I would
like to impress my gratitude to my sister Ati, my aunt Elaheh, and my uncle Mostafa
who gave me support, encouragement and space to finish this thesis. Last but not least
I would like to thank my aunt, Dr. Malihe Shahidan, who was always there to meet
and talk about my ideas and mark up my papers and chapters.
To all the significant others whom I couldn't mention here, please accept my
gratitude. I might fade in your memory but you will always grow in mine with every
step I make.
Amir M Naghsh ii
CERTIFICATION
I certify that this thesis, and the research to which it refers, are the product of my own
work and that they do not incorporate without acknowledgment, any materials
previously submitted for a degree or diploma in any university or institute. I also
certify that this thesis do not contain any material previously published or written by
other people except where due acknowledgment is made in the text.
I would like to clarify that some of the listed publications have been conducted in
collaboration with Dr. Andy Dearden, Dr. Mehmet B. Ozcan and Prof. Jawed Siddiqi.
The paper presented at the Fifteen Annual Meeting of the Psychology of
Programming Interest Group (PPIG) (Dearden, Siddiqi and Naghsh, 2002) has
considered a limited number of Cognitive Dimensions to explore different
characteristics of prototyping approaches. This thesis has extended this work by
applying the full Cognitive Dimensions framework to compare the electronic paper
prototyping tools. The enhanced analysis has been presented in Chapter 4 of this
thesis.
Early design of Gabbeh has been reported at the annual conference of British Human
Computer Interaction group (HCI’04) (Naghsh and Ozcan, 2004) and at the
Participatory Design Conference (PDC’04) (Dearden, Naghsh and Ozcan, 2004).
Gabbeh has been enhanced further and the first version and second version of Gabbeh
are presented in Chapters 6 and 8 of this thesis.
The publication at the workshop of Design, Specification and Verification of
Interactive Systems (DSVIS’05) (Naghsh, Dearden and Ozcan, 2005) has reported the
design of the first version as well as observation studies of Gabbeh. These are
presented in detail in Chapters 6 and 7 of this thesis.
Amir M Naghsh iii
TABLE OF CONTENTS
ABSTRACT IV
ACKNOWLEDGMENTS I
CERTIFICATION II
TABLE OF CONTENTS III
LIST OF FIGURES VII
LIST OF TABLES IX
LIST OF PUBLICATION X
CHAPTER 1. INTRODUCTION 1
1.1. USER PARTICIPATION AND PROTOTYPING 1 1.2. COMMUNICATION IN DESIGN 3 1.3. THE THESIS 4 1.4. OUTLINE 5
CHAPTER 2. REVIEW OF PROTOTYPING 7
2.1. INTRODUCTION 7 2.2. WHAT TO PROTOTYPE? 7 2.2.1. HOW THE PROTOTYPE IS USED? 8 2.2.2. THE FORM OF PROTOTYPES 9 2.3. LOW-FIDELITY APPROACHES 11 2.3.1. STORYBOARDS 11 2.3.2. PAPER PROTOTYPING 12 2.3.3. PICTIVE 13 2.3.4. RAPID SOFTWARE PROTOTYPING 14 2.4. HIGH-FIDELITY APPROACHES 14 2.4.1. RAPID APPLICATION DEVELOPMENT 14 2.4.2. EXTREME PROGRAMMING (XP) 15 2.4.3. EXECUTABLE FORMAL SPECIFICATION 16 2.5. MEDIUM-FIDELITY APPROACHES 18 2.6. ELECTRONIC PAPER PROTOTYPING TOOLS 19 2.6.1. PATCHWORK 19 2.6.2. SILK AND DENIM 20 2.6.3. INDESIGN 24
iv
2.6.4. FREEFORM 26 2.7. SUMMARY 26
CHAPTER 3. CASE-STUDY: COMPARING EXISTING ELECTRONIC PAPER
PROTOTYPING TOOLS 28
3.1. INTRODUCTION 28 3.2. PAPER PROTOTYPING 30 3.3. INDESIGN 31 3.4. DENIM 33 3.5. FREEFORM 35 3.6. DISCUSSION 37
CHAPTER 4. COGNITIVE DIMENSION FRAMEWORK 40
4.1. INTRODUCTION 40 4.2. RELATING THE COGNITIVE DIMENSIONS TO PROTOTYPING 41 4.2.1. VISCOSITY 41 4.2.2. ABSTRACTION 42 4.2.3. CLOSENESS OF MAPPING 43 4.2.4. CONSISTENCY 43 4.2.5. DIFFUSENESS 44 4.2.6. ERROR-PRONENESS 44 4.2.7. HARD MENTAL OPERATIONS 45 4.2.8. HIDDEN DEPENDENCIES 45 4.2.9. PREMATURE COMMITMENT 45 4.2.10. PROGRESSIVE EVALUATION 46 4.2.11. ROLE-EXPRESSIVENESS 46 4.2.12. SECONDARY NOTATION 47 4.2.13. PROVISIONALITY 48 4.2.14. VISIBILITY AND JUXTAPOSABILITY 48 4.3. A REVIEW OF TOOLS USING COGNITIVE DIMENSIONS FRAMEWORK 48 4.3.1. PAPER PROTOTYPING 49 4.3.2. DENIM 51 4.3.3. INDESIGN 54 4.3.4. FREEFORM 55 4.4. DISCUSSION 57 4.5. CONCLUSION 60
CHAPTER 5. ANNOTATION 62
5.1. INTRODUCTION 62 5.2. ANNOTATION 62 5.3. EXAMPLES OF ANNOTATION TOOLS IN USE 64 5.4. FORMS AND FUNCTIONS OF ANNOTATION 64 5.4.1. PERSONAL ANNOTATIONS 65 5.4.2. PUBLIC ANNOTATIONS 69 5.5. CHARACTERISTICS FOR ANNOTATION TOOLS 71 5.5.1. OPERATIONAL CHARACTERISTICS 71 5.5.2. USER INTERFACE CHARACTERISTICS 72 5.6. CONCLUSION 73
v
CHAPTER 6. GABBEH 75
6.1. INTRODUCTION 75 6.2. DESIGNING AN ANNOTATION TOOL 76 6.2.1. CONSISTENCY TO CURRENT DESIGN AND EASY TO USE 76 6.2.2. FLUIDITY OF FORM 76 6.2.3. IN SITU ANNOTATION 77 6.2.4. INFORMAL USE 77 6.2.5. RETRIEVING AND FILTERING 78 6.2.6. AVAILABILITY 78 6.3. DENIM 79 6.3.1. THE DENIM USER INTERFACE 79 6.3.2. ARCHITECTURE OF DENIM 82 6.4. GABBEH 87 6.4.1. THE GABBEH USER INTERFACE 87 6.4.2. IMPLEMENTING GABBEH 96 6.5. CONCLUSION 105
CHAPTER 7. OBSERVATIONAL STUDIES 106
7.1. INTRODUCTION 106 7.2. OBJECTIVES 106 7.3. LAYOUT 107 7.4. PILOT STUDY 111 7.5. REVISED STUDIES 112 7.6. RESULTS 113 7.6.1. WERE COMMENTS USED? 113 7.6.2. WERE DISCUSSIONS SUPPORTED? 116 7.6.3. HOW WERE COMMENTS USED? 118 7.7. DISCUSSION 123 7.7.1. MANAGING LARGE NUMBERS OF COMMENTS 123 7.7.2. IDENTIFYING AND MANAGING DISCUSSIONS 124 7.7.3. SUPPORTING PROGRESS AWARENESS AND TRACKING 124 7.8. CONCLUSION 124
CHAPTER 8. SUPPORTING DISCUSSIONS IN GABBEH 126
8.1. INTRODUCTION 126 8.2. ENHANCING THE DESIGN OF THE COMMENTING TOOL 127 8.2.1. PROGRESS AWARENESS 127 8.2.2. SUPPORTING DISCUSSIONS 127 8.2.3. MANAGING LARGE NUMBERS OF COMMENTS 129 8.3. GABBEH VERSION 2 130 8.3.1. PROGRESS AWARENESS 130 8.3.2. SUPPORTING DISCUSSIONS 133 8.3.3. MANAGING LARGE NUMBERS OF COMMENTS 137 8.4. CONCLUSION 138
CHAPTER 9. LONGITUDINAL STUDY 139
9.1. INTRODUCTION 139
vi
9.2. OBJECTIVES 139 9.3. LAYOUT 140 9.4. RESULTS 141 9.5. HOW WERE DISCUSSIONS USED? 145 9.5.1. PROGRESS AWARENESS 145 9.5.2. DISCUSSING DESIGN ISSUES 149 9.5.3. DIRECTING ATTENTION 151 9.5.4. RECORDING DESIGN ISSUES 152 9.6. REVIEWING AND REFLECTING THE COMMENT CLASSIFICATION 153 9.7. DISCUSSION 156 9.7.1. PROGRESS AWARENESS 156 9.7.2. RICHNESS OF COMMUNICATION CHANNEL 157 9.7.3. EMPOWERING USERS 157 9.8. SUMMARY 158
CHAPTER 10. CONCLUSION 159
10.1. SUMMARY 159 10.2. CONTRIBUTIONS 159 10.2.1. GABBEH 159 10.2.2. USE OF ANNOTATIONS 159 10.2.2.1. A CLASSIFICATION OF ANNOTATIONS 159 10.2.2.2. USE OF DISCUSSIONS 160 10.2.3. COGNITIVE DIMENSIONS 160 10.2.4. RECOMMENDATIONS FOR SUPPORTING ANNOTATION IN ELECTRONIC PAPER
PROTOTYPING TOOLS 161 10.3. FUTURE WORK 163 10.3.1. ENHANCEMENT TO THE CURRENT VERSION OF GABBEH 163 10.3.2. DESIGN RATIONALE 165 10.3.3. SUPPORTING ANNOTATION IN SITUATED SETTINGS 165 10.3.4. DISTRIBUTED PARTICIPATORY DESIGN 166
REFERENCES 167
APPENDIX A 181
APPENDIX B 185
RESEARCH GROUP WEBSITE TASK BRIEF 185 VISIT SHEFFIELD WEBSITE TASK BRIEF 186
APPENDIX C 191
A T-SHIRT WEBSITE 191 READY T-SHIRTS 191 DESIGN TOOL 192 RESULTS 193 FINAL DESIGN 201
APPENDIX D 204
vii
LIST OF FIGURES
Figure 2-1 Horizontal and Vertical dimensions of prototyping. (Nielsen, 1993, p. 94) ______________8 Figure 2-2 PICTIVE setting (Muller, 1991) ______________________________________________13 Figure 2-3 ZAL Interfaces____________________________________________________________17 Figure 2-4 Make a dialog box appear when the button is pressed. (Landay, 1996a) _______________20 Figure 2-5 DENIM, shown here in “Storyboard View” _____________________________________22 Figure 2-6 Denim - run mode, a simple browser providing back, forward and refresh button. The user
can interact with the model by clicking on the hyperlinks____________________________________23 Figure 2-7 Damask user interface (Lin, 2003) ____________________________________________24 Figure 2-8 Active area of a sketch within InDesign ________________________________________25 Figure 2-9 Showing links between screens within InDesign. Scanned sketches are shown in left part of
screen ___________________________________________________________________________25 Figure 2-10 Design view (Left), Storyboard view (Right) ____________________________________26 Figure 3-1 A screen state used in the paper prototype with addition of an interface element in the form
of a message dialogue box____________________________________________________________30 Figure 3-2 Navigational view in InDesign _______________________________________________32 Figure 3-3 Simulating interaction ______________________________________________________34 Figure 3-4 Page view in FreeForm_____________________________________________________36 Figure 3-5 Storyboard view in FreeForm ________________________________________________36 Figure 6-1 DENIM window___________________________________________________________79 Figure 6-2 DENIM design mode at storyboard view _______________________________________80 Figure 6-3 DENIM design mode at page view ____________________________________________81 Figure 6-4 DENIM in run mode _______________________________________________________81 Figure 6-5 DENIM relies on Satin _____________________________________________________82 Figure 6-6 UML diagram of DENIM scenegraph__________________________________________83 Figure 6-7 Tree data structure of DENIM scenegraph ______________________________________84 Figure 6-8 A visual representation of DENIM scenegraph. DenimSheet boundary is highlighted with a
red line, DenimLabel with a black line, DenimSketch with a green line and TimedStroke with a blue line.
Also TypedText and ScribbledText are highlighted within the sketch ___________________________85 Figure 6-9 Diagram of how strokes are handled as gesture and ink ___________________________86 Figure 6-10 Login box: allowing stakeholders to select different versions of Gabbeh _____________87 Figure 6-11 Gabbeh toolbox, Commenting pencil for creating and editing comments (fifth from left), pad
for filtering the comments (sixth from left) _______________________________________________88 Figure 6-12 Comments can be created by writing words or phrases directly on the design sheet at any
zoom levels _______________________________________________________________________88 Figure 6-13 the comment dialogue box allowing the designer to select the colour as well as entering the
typed text to create a comment ________________________________________________________89 Figure 6-14 Using the drawing or commenting pencil to associate a comment with one or many design
objects ___________________________________________________________________________90 Figure 6-15 Representing a comment shared between components from two pages in the design_____90 Figure 6-16 Design sheet crowded with comments_________________________________________92 Figure 6-17 Using the commenting pencil for managing the visibility of comments _______________92 Figure 6-18 A coloured node represents the comment once it is set to invisible___________________93 Figure 6-19 Managing comments in Gabbeh _____________________________________________93 Figure 6-20 The run mode interface for Gabbeh. File button is to open a design file, Run button to start
the execution of the design model and save button to save the added comments to the design model __94 Figure 6-21 Limited functionality browser with commenting features __________________________94 Figure 6-22 Viewing comments in the 'run mode' __________________________________________95 Figure 6-23 Adding comments when Gabbeh is executed____________________________________95
viii
Figure 6-24 Dependency Graph for DenimPanel __________________________________________97 Figure 6-25 DenimComponent extending DenimIntrinsicComponent _________________________100 Figure 6-26 Introducing new tools in Gabbeh ___________________________________________101 Figure 7-1 Illustration of the layout of the evaluation exercises in two different location. The only
communication channel was through the email and prototype while there was no face-to-face meeting
________________________________________________________________________________108 Figure 7-2 This is a picture taken in the pilot study demonstrating the designer-users working together
using a graphic tablet and laptop connected to each other. Both designer-users were contributing to the
design by using the tablet, keyboard and the mouse. Halfway through the design, two designer-users
swapped places ___________________________________________________________________109 Figure 7-3 Picture taken from the first design exercise (group1). The picture is demonstrating one
designer (left) in charge of the tablet while the other designer (right) is more distanced and works with
the computer. The video record showed both designer-users discussed the problem but only one (left)
was creating the design. While the other one (right) was mainly giving feedback and making sure that
the agreed design was developed and in some occasions give assistance by typing on the keyboard _110 Figure 7-4 The comments were used in all exercises. This figure is demonstrating the use of comments
as Gabbeh was used to develop a prototype (Group 2, Session 2) ____________________________115 Figure 7-5 This thread of two was identified in the design developed by the group 2. The designer-users
have changed part of the design in response to users comment. Here the design-users are explaining the
new design which receives the user’s approval___________________________________________117 Figure 7-6 This thread was identified in the design from the pilot study. The designer-users explain the
design while requesting the user’s approval _____________________________________________117 Figure 7-7 Examples of clarifying comments (group 2, session 2). The blue comment is clarifying the
design. Designer-users have made comment to explain what would happen once the user clicked on one
of the items in the publication lists. It has allowed designer-users to avoid designing a new page to
simulate the dynamic behaviour and its functional output in the form of a new page. _____________119 Figure 7-8 An example of verifying comments as part of a thread ____________________________120 Figure 7-9 The end-user requesting a new page to be added - An example of altering the design____121 Figure 7-10 Red comment shows an example for understanding comments helping the end-user to
understand the provisional part of design_______________________________________________122 Figure 8-1 A discussion example from observational studies ________________________________128 Figure 8-2 Short comments were used to approve the design________________________________129 Figure 8-3 The dialogue box for allowing the users to provide a name ________________________131 Figure 8-4 The new associated point to a comments is highlighted on the page _________________132 Figure 8-5 Comments displayed in the run-view when no comment is selected.__________________133 Figure 8-6 Displaying a comment when it is selected ______________________________________134 Figure 8-7 Editing an existing comment ________________________________________________134 Figure 8-8 comment inserted as a reply to an existing comment _____________________________135 Figure 8-9 Closed view of a discussion_________________________________________________136 Figure 8-10 Expanded view of a discussion _____________________________________________136 Figure 8-11 A comment with flag _____________________________________________________137 Figure 8-12 filtering toolbar _________________________________________________________138 Figure 9-1 Page "design", sent to user in iteration one ____________________________________147 Figure 9-2 Page "design", sent to user in iteration 9, evolving through 8 iterations ______________147 Figure 9-3 A discussion that is representing comments from user and designer giving instruction to each
other ___________________________________________________________________________148 Figure 9-4 Showing a discussion, there has been a two iterations gap between second and third
comment. ________________________________________________________________________149 Figure 9-5 An example of debating a design issue ________________________________________150 Figure 9-6 Another example of debating a design issue ____________________________________150 Figure 9-7 User’s feedback__________________________________________________________151 Figure 9-8 Clarifying design through discussion _________________________________________152 Figure 9-9 A discussion, recording a design issue ________________________________________153 Figure 9-10 A discussion on technical issues ____________________________________________153 Figure 9-11 Confirmation by designer _________________________________________________155 Figure 9-12 Confirmation by Designer _________________________________________________156 Figure 9-13 Confirmation by user_____________________________________________________156
ix
LIST OF TABLES
Table 4-1 Summary of 14 cognitive dimensions in relation to electronic paper-prototyping tools_____60 Table 5-1Mapping annotations form into functions ________________________________________69 Table 7-1 This table presents the number of comments created for each session as well as presenting the
overall comments made while using Gabbeh in comparison to DENIM________________________114 Table 7-2 List of identified discussions for each session divided in two main groups of Designer-initiated
or User-initiated discussions_________________________________________________________117 Table 7-3 Total number of comment-phrases associated to each category for the three evaluation studies
________________________________________________________________________________123 Table 7-4 Number of comment-phrases associated to each category for the pilot study ___________123 Table 9-1 Results for the comments associated to the whole design ___________________________142 Table 9-2 Number of discussions associated with each page of the design _____________________143 Table 9-3 Discussions grouped based on the number of sessions_____________________________144 Table 9-4 Discussions grouped based on number of the comments involved ____________________144 Table 9-5 Number of comment phrases associated to each category __________________________154
x
LIST OF PUBLICATION
Naghsh, A. M., Dearden, A. and Ozcan, M. B. (2006). Investigating annotation in electronic paper-prototypes. In: Gilroy, Stephen W. and Harrison, Michael D. (eds.). DSV-IS'05 Interactive Systems, Design, Specification, and Verification, 12th
International Workshop. Lecture Notes in Computer Science, Springer, pp. 90-101.
Naghsh, A. M. and Dearden, A. (2005). Supporting collaboration in electronic paper prototyping. In: The Workshop on Cognition and Collaboration Analyzing Distributed
Community Practices for Design at CHI 2005, Portland, Oregon, USA.
Naghsh, A. M. and Ozcan, M. B. (2004). GABBEH - A tool for computer supported collaboration in electronic paper prototyping. In: A., Dearden and L., Watts (eds.). HCI'04 the Annual Conference of British HCI Group: Design for Life, Leeds, UK, Springer, pp. 77-80.
Naghsh, A. M. (2004). An electronic prototyping tool to facilitate annotations in the early sages of design. In: Doctoral Consortium at Participatory Design Conference
2004, Toronto, Canada.
Naghsh, A. M. (2004). Towards computer supported collaboration in electronic paper prototyping. In: Doctoral Consortium at CSCW'04 the ACM Conference on Computer
Supported Cooperative Work, Chicago, USA.
Naghsh, A. M. and Dearden, A. (2004). A demonstration of GABBEH: A tool to support collaboration in electronic paper prototyping. In: CSCW'04 the ACM
Conference on Computer Supported Cooperative Work, Chicago, USA.
Dearden, A., Naghsh, A. M. and Ozcan M. B. (2004). Support for participation in electronic paper prototyping. In: The Proceedings of the Participatory Design
Conference, CA, USA, CPSR Palo Alto, pp. 105 – 108.
Dearden, A., Siddiqi, J. I. and Naghsh A. M. (2003). Using cognitive dimensions to compare prototyping techniques. In: The First Joint Conference of EASE & PPIG
(15th Annual Meeting of the Psychology of Programming Interest Group), Keel, UK.
xi
Amir M Naghsh 1
Chapter 1. INTRODUCTION
1.1. USER PARTICIPATION AND PROTOTYPING
The conventional wisdom within the information systems community suggests
that user participation is central to the successful development of information
systems [Ives and Olson 1984].
Involving users directly in the software development process has found its clearest
expression in the participatory design (PD) approach. PD emerged in the Scandinavian
countries that were concerned with bringing democracy into the work place (Floyd et
al., 1989). The focus of PD has since shifted to a belief that direct user participation in
the development process is a key factor in a successful design (Ehn, 1993). Such
participation must proceed iteratively from the early stages of design (Rosson et al.,
1988) by involving different stakeholders (e.g. designers, users and developers) who
are collaboratively engaged in producing the design proposals (Bodker et al., 1993).
Therefore, it is an essential element of all participatory approaches to enable users to
envisage or make sense of design proposals (whether those proposals originate with
‘professional designers’ or from the users themselves). Users can only make informed
choices when the proposals being discussed are meaningful to them.
Prototyping is one popular method of helping users (and designers) to understand
possible alternatives. Prototyping is suitable for gaining experience of new applications
by implementing parts of software requirements to learn more about actual
requirements or about alternative designs that can be used in the refinement of
requirements.
“A prototype is a practical implementation of a system built expressly to learn
more about a problem or solution to a problem” (Davis, 1992).
The traditional approaches to software development such as the waterfall model pay
little attention to user involvement in the design process. These approaches generate
prototypes that are mainly intended to help designers and software engineers in
providing a description of future systems. In contrast, Bodker and Gronbæk (1991), in
explaining their cooperative approach describe prototyping as a useful process for
Amir M Naghsh 2
uncovering unspecified and unstated aspects of users’ work and encouraging them to
contribute more in improving the prototype through the design process. They found
prototyping useful in simulating the dynamic behaviour of a future system.
“Users need to be actively involved in prototyping - passive participation in
demonstrations and unplanned evaluations of prototypes is insufficient to get
benefits from prototyping.” (Gronbæk, 1991, p. 31)
Bodker and Gronbæk (1991) explain that cooperative prototyping has its roots in
exploratory approaches, as a cooperative activity between users and designers, rather
than an activity of designers utilising users’ stated requirements.
The cooperative prototyping approach aims to establish a design process
where both users and designers are participating actively and creatively,
drawing on their different qualifications. To facilitate such a process, the
designers must somehow let the users experience a fluent work-like situation
with a future computer application; that is, users’ current skills must be
brought into contact with new technological possibilities. (Bodker and
Gronbæk 1991, p. 200)
It is not essential for users and designers to understand each other as long as they can
relate to the prototype and materials used in the design process (Ehn, 1988). Therefore,
a key issue in enabling users to be more involved and creative in the design activities is
to use tools and materials which offer family resemblance to the tools and materials
that users work with in their ordinary work places (Ehn & Kyng, 1991). The use of
pencil and paper is an established participatory approach for designing interactive
systems and has been suggested to encourage user participation in the design process
(Ehn & Kyng, 1991; Preece et al., 2002).
Whilst paper-prototyping has many advantages in promoting user participation, it also
has some limitations. In particular:
• The lack of an explicit representation of the navigational structure could make
it difficult for users to understand and revise the dynamic behaviour of paper-
prototypes (O’Neill et al., 1999);
• It is difficult to review a paper-prototype when users and designers are
dispersed and are not able to arrange a face-to-face meeting;
Amir M Naghsh 3
• Paper-prototypes may be difficult to relate to other representations being used
within design, such as detailed specifications of behaviour and functionality;
• The lack of support for design memory could make it difficult for users to
search through sketches in the future to find out why a particular design
decision was made (Landay, 1996a); and
• Paper-prototypes are hard to modify. By increasing the number of sketches the
designer must frequently redraw the common features of the design for all the
sketches (Landay, 1996a).
As pen-based interaction devices have become more widely available, some systems
have applied pen-based interaction to explore ways in which some of the benefits of
paper prototyping could be realised in interactive systems design. Examples include
SILK, DENIM and DAMASK (Landay, 1996b; Lin et al., 2000; Newman et al. 2003;
Lin, 2003) and FreeForm (Plimmer & Apperley, 2003). These systems might be
described as supporting a form of ‘electronic-paper prototyping’. Other work has also
explored ways in which the benefits of paper-prototyping might be realised in software
prototyping environments, without relying on pen-based interaction (van de Kant et al.,
1998; Nixon, 2001). These approaches overcome some of the limitations of paper-
prototyping. In particular, these systems can make the dynamic behaviour of the
proposed system easier for users to perceive and can permit the prototype to be
distributed electronically.
However, the designs of these existing tools are primarily oriented towards the needs
of software developers directly involved in creating designs, rather than considering
how other stakeholders can provide inputs to design.
1.2. COMMUNICATION IN DESIGN
Earlier studies in design work (Bellotti and Bly, 1996; Button and Sharrock, 1996;
Henderson, 1992; Pycock and Bowers, 1996; Wojahn et al., 1998) address various
ways in which communicative activities support design processes. Button and Sharrock
(1996) noted that the design process depends on communications, and on
transformation process involving design artefacts (e.g. prototypes). Perry and
Sanderson (1998) described the interactions between designers and other stakeholders
as key dimensions in the design process. The communication dimension and the role
and transformation of artefacts in design work intersect, in that the artefacts are subject
to discussion, negotiation, and alteration and help to mediate and organise the
Amir M Naghsh 4
communication. Bäumer et al. (1996) in a series of case studies found that one of the
important goals in building prototypes was to make the intended solution easily
communicable and allow different stakeholders to contribute to it.
Näslund (1997) discussed the different groups of stakeholders who are involved within
a software development project. He explained that the number of potential stakeholder
groups who may be involved during the design process is extensive. These groups may
be external or internal to the organisation in which the design group is located.
Members of all these stakeholder groups work together to review and discuss artefacts
to improve them to reach common ground on the basis that one artefact can and should
be developed.
While a key dimension in the design process is the interactions among stakeholders, it
has become a major difficulty and challenging when stakeholders are dispersed due to
the distributed nature of projects (Heeks et al., 2001; Audy et al., 2004; Coar, 2004).
Morris et al. (1999) explained that people use the technology to communicate
asynchronously and overcome the time and space constraints in distributed settings.
Wojahn et al. (1998) noted the importance of providing affordable ways to
communicate about collaboratively produced artefacts and described making
annotations as a common communication practice which could be easily shared, and
passing such shared annotations among members of a team is a common practice even
among those who work in the same office room or building. Bottoni et al (2004)
explained that people used annotation when using paper documents to make design
decisions, communicate comments with other team members or highlight important
design issues.
Recent studies (Cadiz et al., 2000; Brush et al., 2001; Weng and Gennari, 2004)
investigated how annotations can facilitate asynchronous communication. Cadiz et al.
(2000) explained that annotations that are shared deliberately support communication
and collaboration between members of a group.
1.3. THE THESIS
While participatory design approaches are primarily focused on facilitating projects
with stakeholders who are co-located during design activities, this thesis aims to
investigate how the use of annotation in electronic paper-prototyping tools could
facilitate asynchronous communication to encourage user involvement in early design,
particularly when stakeholders are fully distributed. Overall this thesis has the
following aims:
Amir M Naghsh 5
• To review existing literature and research on tools and methods that is used in
designing and prototyping interactive systems at early stages of design. In
particular, to compare existing electronic paper-prototyping tools to identify
their weaknesses and strengths.
• To identify a set of features and requirements that need to be considered when
providing annotation features to support asynchronous communication in
electronic paper-prototyping tools.
• To design and develop such an electronic paper-prototyping tool to demonstrate
the feasibility of supporting communication through annotations.
• Finally to evaluate the implemented tool and to investigate the uses made of
annotations in designing interactive systems.
1.4. OUTLINE
Chapter 2 gives an introduction to prototyping in designing interactive systems and
reviews the existing prototyping methods and tools. The chapter considers the
traditional concept of fidelity to review methods and tools belonging to three
categories of low-fidelity, high-fidelity and medium-fidelity.
Chapter 3 investigates the use of electronic paper prototyping tools by conducting a
pragmatic investigation. The chapter argues that the traditional concept of fidelity is
not sufficiently fine-grained to analyse important similarities and differences between
the available set of prototyping methods and tools.
Chapter 4 uses the Cognitive Dimension to evaluate the electronic paper prototyping
approaches. Based on the findings presented in the chapter, it highlights the importance
of providing “secondary notation” in electronic paper-prototyping tools. The chapter
shows that existing electronic paper-prototyping tools do not support secondary
notation and are therefore limited in their ability in supporting communication among
different stakeholders.
Chapter 5 reviews various studies investigating the use of annotation in text books to
digital documents. The chapter presents the findings of previous studies on
investigating the use of annotation in supporting asynchronous collaboration around
digital documents and reports on the form and functions of annotation, as well as the
characteristics for such annotation tools.
Amir M Naghsh 6
Chapter 6 discusses the design and implementation of an electronic paper-prototyping
tool with an annotation facility. The chapter reports on a list of requirements as a
possible design for such an electronic paper-prototyping tool based on the reviewed
literature in chapter 5 and the outcome of a number of design sessions. Iterates further,
the chapter introduces Gabbeh and reports the user interface of Gabbeh and its
implementation.
Chapter 7 reports on the evaluation of the comment facility introduced in Gabbeh. The
chapter explains the objectives and layout used in the observational studies. Further,
the chapter presents the findings and identifies a possible classification of uses of
annotations during the design process. Finally, this chapter reports the analysis of the
findings and explains areas of concern with the first version of Gabbeh.
Chapter 8 discusses the design of the commenting tool and required enhancements to
the design to support communication in form of discussions. The chapter reports the
further implementation of Gabbeh and presents the improved user interface and
features introduced in the second version of Gabbeh.
Chapter 9 reports on a longitudinal case-study conducted to understand how Gabbeh
supports communication through annotation in a distributed setting. The chapter
presents the outcome of the study and analysis. Finally the chapter uses the data to
validate the classification of uses of annotation and extends it further.
Chapter 10 concludes the thesis, discusses the impact of the research and summarises
the contributions and lessons learned from previous chapters. Finally it assesses the
research and points towards the areas for future work.
Amir M Naghsh 7
Chapter 2. REVIEW OF PROTOTYPING
2.1. INTRODUCTION
This chapter reviews the prototyping literature concerning the use of prototypes in the
development process. The chapter further reviews prototyping tools and approaches
and discusses how such tools and approaches improve user participation.
2.2. WHAT TO PROTOTYPE?
The entire idea behind prototyping is to save on the time and cost to develop
something that can be tested with real users. These savings can only be
achieved by somehow reducing the prototype compared with the full system:
either cutting down on number of features in the prototype or reducing the level
of functionality of the features such that they seem to work but do not actually
do anything. (Nielsen, 1993, p. 94).
To achieve some of these savings Nielsen (1993) introduced two dimensions of
vertical and horizontal prototyping.
Nielsen (1993) explained vertical prototyping as a narrow version of the finished
system which only has a reduced number of features but includes in-depth
functionality for the few selected features. Therefore, vertical prototyping can only
evaluate a limited part of the finished system while allowing users to perform real tasks
using selected features probably with some real data.
Nielsen explained horizontal prototyping as a surface layer that includes the entire user
interface of the finished system but with reduced or no functionality. A horizontal
prototype without underlying functionality allows users to feel the entire interface
while not letting them perform any real tasks. The main advantage of horizontal
prototypes is in allowing stakeholders to concentrate on the bigger picture by assessing
the interface as a whole. This type of prototype can be implemented quickly with the
use of less formal tools in comparison to vertical prototyping (see Figure 2-1).
Amir M Naghsh 8
Figure 2-1 Horizontal and Vertical dimensions of prototyping. (Nielsen, 1993, p. 94)
In addition to the dimensions already explained, prototyping approaches can be
categorised from two points of view; one is based on the prototyping process and the
other is based on the fidelity of the prototype.
This chapter presents a review of traditional ways of categorising prototyping
approaches from these two points of view. Furthermore, the chapter reviews a number
of prototyping methods and tools that encourage user involvement and participation in
an iterative design process.
2.2.1. HOW THE PROTOTYPE IS USED?
Davis (1992) has described a framework for categorising the prototyping processes
based on the development processes. Davis explained that there are three broad
categories of prototyping processes in the literature of software engineering:
a) Throwaway prototyping: A throwaway prototype is built quicker than
evolutionary and operational prototyping and implements poorly understood
requirements. The throwaway prototype is not used as part of the finished
system and it is used in an experimental manner to learn about “which
requirements are real and which are not” to be considered in the development
of the final system.
b) Evolutionary prototyping: This is built to a higher quality standard. The
prototype evolves to become the final system. It implements only confirmed
Amir M Naghsh 9
requirements and is used to explore those requirements that exist but have not
been captured. It is also used experimentally.
c) Operational prototyping: An evolutionary prototype is constructed and made
into a baseline by developing requirements that are well understood. The copy
of this baseline would be sent to other user sites. A trained prototyper would be
sent to the user site to monitor users when interacting with the software system
and record any problems or requirements which would free the user from
having to record them. Changes to the top of baseline software would be made
by the prototyper in a throwaway manner. The user then uses the new system
composed of the baseline system and throwaway prototype. Useless changes
would be discarded and any other changes would be reported to the
development team and incorporated into the next version of the baseline.
2.2.2. THE FORM OF PROTOTYPES
Prototypes traditionally have been categorised based on the level of fidelity of the
prototype (Walker et al., 2002; Leone et al., 2000; Rudd et al., 1996; Wong, 1992). The
determining factor in the fidelity of a prototype is the degree to which that prototype
presents application and the interaction of the system from the user's point of view
(Tullis, 1990).
Rudd et al (1996) described low-fidelity prototypes with limited or no functionality.
They are constructed quickly and allow designers and users to focus on high-level
overviews of a final system by providing a representation of the general look and
perhaps the feel of the interface, rather than the visual details and styles (Landay and
Myers, 2001).
Low-fidelity prototypes are created to increase users' participation and users' feedback
at the early stages of design by communicating, educating and informing ideas. Nielsen
(1992) suggested that the unfinished look of low-fidelity prototypes encourages users
to suggest more changes to the design. Virzi (1989) claimed low-fidelity prototypes are
an effective approach for exploring the design space. Nielsen (1993) explained that
low-fidelity prototypes can be used in horizontal prototyping and therefore make it
quicker to generate the entire user interface.
High-fidelity prototypes represent the core functionality of the final system (Rudd et
al., 1996). They are interactive and users can operate and even perform some real tasks
with the prototype. Leone et al. (2000) described how the look of high-fidelity
prototypes can be made in a way that users could find them similar to the final systems
Amir M Naghsh 10
which are typically created in later stages of the design process. Their creation requires
more time and effort compared to low-fidelity prototypes. Nielsen (1993) suggested
that high-fidelity prototypes could be used in vertical prototyping which gives a closer
look and feel to the finished system.
Studies in comparing the low and high fidelity prototypes to evaluate the usability of
the design have showed that low-fidelity prototypes are as effective as high-fidelity
prototypes for evaluation by users (Virzi et al., 1996; Walker et al. 2002).
Other research has compared the low and high-fidelity prototypes to identify their
advantages to determine at what stage in the development process they could be used.
Retting (1994) argued that designers often use low-fidelity prototypes early in the
design process. Many researchers (Preece et al., 2002; Muller, 1992; Ehn and Kyng,
1991; Ehn, 1989) have suggested the use of low-fidelity prototypes for creating an
early impression of the possible design of an interactive system.
“Informal observations and discussions with designers show that they like to
sketch out ideas. Sketching allows them to be more creative and exploratory
than if they were using a computer. Sketching also made it easier to rapidly
prototype and iterate on user interfaces” (Hong et al, 2002).
Rauch et al. (1997) also discussed the advantages of low and high-fidelity prototypes
and argued that low-fidelity is more appropriate for the early stages of the design
process for exploring the design while high-fidelity is more appropriate for the later
stages of the design process.
The finished look of high-fidelity prototypes has some disadvantages for the early
stages of the design process such as discouraging users from suggesting changes
(Nilsen, 1992) or causing distraction by shifting the users’ focus to the low level details
such as colour or font size (Hong et al., 2001; Wong 2002). However, despite the
disadvantages that high-fidelity prototypes may cause at early stages of design, a study
conducted by Newman and Landay (2000) interviewing eleven professional designers
found out that designers may move to high-fidelity prototypes early in design since
clients may find low-fidelity prototypes unprofessional.
Many researchers (Engelberg and Seffah, 2002; Leone et al., 2000) focussed on the
gap between low and high fidelity prototypes and suggested a medium-fidelity
prototype.
Amir M Naghsh 11
“In our experience, medium-fidelity prototyping allows for a greater flexibility
in employing a prototype of higher fidelity [compared to the low-fidelity] and
navigation capability earlier in the design process. […] this type of prototype
can be built faster, shown to users sooner and iterated by developers
throughout the design process (Leone et al., 2000, p. 234)”.
Some of the examples of tools and approaches supporting medium-fidelity prototyping
include Microsoft PowerPoint, HTML documents and HyperCard.
To bridge the gap between high and low-fidelity prototypes, many studies (Liu and
Khooshabeh, 2003; Hong et al., 2001; Landay and Myers, 1995; Uceta et al., 1998)
suggested the importance of keeping the informal appearance of the low-fidelity
prototypes while increasing the fidelity and their automation. They studied the use of
sketches and paper-prototypes as examples of low-fidelity prototypes and argued that
increasing the fidelity can improve the simulation and interaction of the paper-
prototype while the informal representation can encourage more user participation.
Some of the works in this area (Landay, 1996b; Damn et al., 2000; Lin et al., 2000;
Wilson et al., 1997) have explored ways in which some of the benefits of paper-
prototyping could be realised in software environments. These recent tools can be
referred to as "electronic paper-prototyping" that can be used in horizontal prototyping
and offer more functionality and better navigation to improve the evaluation of the
design by users. Tools supporting electronic pape- prototyping approaches generate
prototypes that can be categorised as a form of medium-fidelity prototyping. Such
prototypes can be used as an alternative to high and low-fidelity prototyping
approaches.
Below, this chapter reviews a number of tools and methods divided into three
categories of low, high and medium-fidelity. In addition, the chapter focuses on the
electronic paper-prototyping tools as a separate section to review them in more detail.
The chapter discusses each of the approaches from the point of view of improving user
involvement and the development process that it may support.
2.3. LOW-FIDELITY APPROACHES
2.3.1. STORYBOARDS
Andriole (1989) imitated film production by introducing Storyboarding as a
prototyping approach for user requirements validation. Curtis and Vertelney (1990)
provided a detailed introduction to storyboard prototyping explaining that storyboards
Amir M Naghsh 12
consist of a sequence of paper sketches illustrating a scenario. Each of these sketches
can represent a screen of a future system or a scene of a user interacting with a device.
A sequence of sketches could illustrate a function that a future system may perform or
could show how a user interacts with a device when performing a task.
Paper storyboards are portable and can be exchanged with other stakeholders.
Storyboards do not need a member of the design team to simulate the dynamic
behaviour of the design. Storyboards can be used as part of the throwaway prototyping
processes at the early stages of design.
2.3.2. PAPER PROTOTYPING
Another approach that makes use of paper sketches is paper-prototyping. Unlike the
storyboards that illustrate only one possible scenario in the form of a linear sequence of
sketched screens, paper-prototypes include a possible user interface that illustrates
many of the functions that a future system may perform.
Using paper-prototyping gives the opportunity to designers and users (who are not
necessarily programmers) to get involved in generating a prototype and developing the
design of an interactive system. Interface elements such as menus, windows, dialogues
and icons can be sketched on paper or created in advance using cards, pens and office
materials (Muller, 1992). When the paper-prototype has been prepared a member of
the design team sits before a user and ‘plays the computer’ by moving interface
elements around in response to the user’s actions. The user makes selections and
activates interface elements by using their finger as a mouse and writing ‘typed’ input.
A further person can assist the session by providing task instructions and encouraging
the user to express their thoughts and impressions. Other observers may make notes or
a video record may be created.
The individual playing the role of the computer must be fully aware of the
functionality of the intended system in order to simulate the computer. However, due
to the use of paper and a human operator, this form of prototype cannot be used
reliably to simulate real system response times.
Rettig (1994) noted that breaking the user-developer barriers by encouraging
communication and collaboration between the designers and users in the early stages
of the development process is one of the most important advantages of the paper-
prototyping method. Consequently usability problems could be detected at a very early
stage in the design process.
Amir M Naghsh 13
This method is effective when the basic control types of a future interface are
known but user feedback on a suitable layout is required. The method does not
(necessary) require users to draw the interface although they can supplement
the design with additional elements or annotations to add contents (Rettig
1994, p. 23).
Paper-prototypes are quick to build and refine and are used in throwaway and
operational development processes. Paper-prototypes provide a valuable and cost-
effective means of evaluating and iterating design options before a team gets
committed to one implementation. However, because of their simplicity, they are not
suitable for evaluation of fine design detail (Retting, 1994).
2.3.3. PICTIVE
PICTIVE (Plastic Interface for Collaborative Technology Initiatives through Video
Exploration) was created at Bellcore by Muller [1991] combining low technology
office objects with higher technology video recording.
PICTIVE is a sophisticated method that provides the users with a set of paper,
cardboard or plastic interface elements which they can lay out on a flat surface in what
they feel is an appropriate way while designs are discussed and developed as a group
or individually.
PICTIVE, by using ordinary office materials, offers a more equal opportunity to both
users and developers to get involved in designing interfaces of an interactive system
rather than the evaluation of a designed interface.
Figure 2-2 PICTIVE setting (Muller, 1991)
Amir M Naghsh 14
PICTIVE uses video as a recording technique to reassure participants that their ideas
and views in sessions are going to be used. This provides an informal and complete
record of design.
The success of the exercise relies on the presence of a co-ordinator in the meeting. The
main role of this person is to ensure that the group stay focused upon the design
problem and ensuring that every member of the group is given the opportunity to stand
up and present his or her own ideas.
2.3.4. RAPID SOFTWARE PROTOTYPING
Use of high-level programming languages and software tools was suggested (Gronbæk,
1989, Wilson and Rosenberg, 1988) to implement prototypes that represent the entire
interface of a new interactive system (horizontal prototyping). These prototypes can be
considered as low-fidelity since they require little programming effort in creating them
and they support limited or no functionality to process data. These prototypes can be
constructed quickly at the early stages of design. Since software tools and frameworks
for implementing the final system are used to create rapid prototypes, they enable the
prototypes to be reused in the production of the intended system or evolve to become
the final system. Therefore, it is possible to apply either evolutionary or throwaway
process to them.
These prototypes have the feel and look of the finished system, they support iterative
design by obtaining users’ feedback and using them in the refinement of the prototype.
However, Gronbæk (1989)’s research, studying nine projects, he noted that these
prototypes are mainly intended to help designers and developers work.
The users did not adequately evaluate the horizontal prototypes, because of the
limited functionality, lack of motivation, and bad conditions. Consequently the
horizontal prototypes were reduced to be just substitutes for parts of the
traditional written requirements specification (Gronbæk, 1989, p. 9).
2.4. HIGH-FIDELITY APPROACHES
2.4.1. RAPID APPLICATION DEVELOPMENT
Techniques that focus on creating and modifying executable software prototypes are
widely used in commercial software development in order to obtain early feedback
from users (Rapid Application Development) (Martin, 1991; Agarwal, 2000).
Amir M Naghsh 15
Rapid Application Development (RAD) can be used as a high-fidelity prototype when
a few selected functions are implemented in detailed as part of the vertical prototyping
which supports processing real data.
The use of vertical prototypes in two of the projects seemed to stimulate quite
useful user response, which could be utilised in making stepwise development
and test of a new computer system with ongoing end-user involvement
(Gronbæk, 1989, p. 9).
Such prototypes require more effort, time and cost in creating them. These prototypes
are often used in evolutionary prototyping where they can evolve from being a
narrowed vertical portion of a system to become the entire future system.
Gronbæk (1989) explained that although such high-fidelity prototypes seem to support
user-involvement, they are selective and to implement such prototype for the entire
system it requires a large amount of resources. In addition, as Nielsen (1992) explained
the finished look of such prototypes and the cost and time involve to incorporate
changes into the prototype could discourage end users from making suggestions.
2.4.2. EXTREME PROGRAMMING (XP)
XP is another approach in software development that like RAD generates high-fidelity
prototypes that evolve and become the final system in an evolutionary process. In
contrast to RAD, which involves long development cycles (Gronbæk, 1989), XP
applies shorter development cycles where an executable version of the system is
delivered by the end of each cycle (Beck and Andres, 2004). XP aims to reduce the
cost of change by adopting changing requirements in between its shorter iteration
cycles.
XP highlights the importance of user involvement and allows a potential user of the
system to be included as part of the development team on the site. The on-site user is
representing an entire community who will use the system. XP assign a series of tasks
to the on-site user, including providing feedback by the end of each cycle, driving the
product by setting the priorities and answering the development queries on the site.
The user's feedback is used to generate the requirements which are called stories in XP.
Although the possibility of integrating more user-centred approaches with XP has been
suggested (Sharp and Robinson, 2004), current practice in XP does not permit direct
participation of real end-users in designing the system; instead it suggests the use of
users’ representatives. In addition, high level skills required for XP processes also do
not allow the on-site user to be involved along with the programmers participating
Amir M Naghsh 16
actively and creatively in developing the system as Bodker and Gronbæk (1991)
suggested in their cooperative approach.
2.4.3. EXECUTABLE FORMAL SPECIFICATION
Using formal specifications is another way to generate ideas about how an interactive
system can be designed, and it helps to evaluate the quality of a solution at an early
stage.
“Formal Methods are mathematically based techniques for describing system
properties. Such formal methods provide frameworks within which people can
specify, develop and verify system in a semantic rather than ad hoc manner
(Wing, 1990, p. 8).”
Formal specifications describe the core functionality of the final system. Executable
formal specifications are interactive and can be used for exploration and evaluation of
the system. They are not as quick and easy to create as paper-prototypes but some of
them can be made in such a way that users can find them similar to the final system
(Palanque and Bastide, 1997).
It is been argued by a number of researchers (Fuchs, 1992; Plat et al., 1992; Palanque
and Bastide, 1997) that specifications in formal languages such as Z, could be used as
part of the prototyping process for safety critical systems or where the behaviour of a
system is hard to describe except mathematically. Such a mathematical model can be
used in explaining the detailed aspects about a design's behavioural properties. Users'
feedback and involvement is essential in validation of these specifications.
Fuchs (1992) explains that executable specifications represent the conceptual and
behavioural model of the system that is going to be implemented which can allow
users and developers to interact with it.
Although researchers have suggested allowing users and developers to interact with
executable formal specifications in order to validate requirements (Cooper et al. 2002;
Bastide et al., 2003; Parry and Ozcan, 2000; Ozcan et al., 1998; Siddiqi et al., 1997;
Fuchs, 1992), there is not much of evidence showing real end-users interacting with
executable systems.
Formal specifications could be used in different stages of the development process, but
they would not form any part of the finished deliverable system. Therefore, formal
methods are usually used as part of a throwaway process. The following section
presents examples of formal executable specifications.
Amir M Naghsh 17
2.4.3.1. EXAMPLES
ZAL (Z Animation in Lisp) is a Lisp based environment, which constructs an
executable version of a Z specification (Roast and Siddiqi, 1997; Siddiqi et al., 1997).
TranZit (Morrey et al., 1998) is an editor for Z that makes it possible for developers
and the designer team to move from informal to formal by using an initial specification
written in English from user needs and construct a Z specification (Siddiqi et al.,
1999). TranZit contains a transformer engine that converts the Z specification into Lisp
syntax that will be used by ZAL. Once the converted version of Z has been
transformed into ZAL, it is possible to assign values to the ZAL representation of the
requirements in the form of text input and output to validate the Z model (Figure 2-3).
ZAL representation is mainly based on mathematical notation and it is limited in
providing user interfaces to encourage user involvement and support evaluation.
Figure 2-3 ZAL Interfaces
Further research has suggested the use of visualisation in displaying the mathematical
notation in the form of familiar graphical representation (Cooper et al., 2002; Ozcan et
al., 1998). Visualisation in Z (VIZ) (Parry and Ozcan, 2000) is an extension to ZAL
which provides alternative representations in the form of familiar graphics that offer a
better understanding for the end users. First, objects and their actions in a ZAL
representation are identified, and then graphics from a component library are used to
generate static and dynamic visual representation of identified objects. Then VIZ
attaches the generated graphical models to the underlying ZAL representation and
allows user evaluation.
Amir M Naghsh 18
Palanque and Bastide (1997) used Petri nets a graphical and mathematical tool and its
diagrammatic representations to model a system specification by emphasising the
dynamic behaviour of the system and the tasks of users interacting with the system.
They explained that they have used an object structured dialect of Petri nets which is
called interactive cooperative objects or ICO formalism (Bastide and Palanque, 1995).
Object-Oriented features of ICO formalism used in modelling the structure and static
aspects of the interactive system, while the Petri nets is used to explain the dynamic
behaviour of the system.
The use of formal description based on Petri nets has been refined further into a tool
called PetShop (Bastide et al., 2003). The main use of this tool is for interactive critical
systems such as air traffic control. PetShop offers an iterative development process and
uses an interface builder to visualise the underlying formal ICO model for evaluating
the prototype by allowing users to interact with it.
Although such tools as VIZ and PetShop enable developers to validate the formal
specification by visualising them early in the design and allowing end users to interact
with them, the complexity of the underlying formal model does not allow the end user
to participate directly in generating the prototype.
2.5. MEDIUM-FIDELITY APPROACHES
Leone et al. (2000) suggested HTML as a method for medium fidelity prototyping to
quickly generate user interfaces and demonstrate to the end users. They explained how
using HTML addressed some of the limitations of low and high-fidelity prototyping.
Unlike high-fidelity prototypes it was quick, and cheap to construct and easier to
maintain. It would support an iterative process and capture user requirement in early
design. Unlike low-fidelity paper-prototypes they can support close navigation to the
final system and allow users to interact with them. They reported that they received
more constructive feedback from medium-fidelity prototypes in comparison to high
and low fidelity prototypes.
Engelberg and Seffah (2002) used PowerPoint as a tool to support medium-fidelity
prototyping. Similar to HTML, PowerPoint was used to prototype the interactive and
navigational aspects of interfaces in early design to capture users feedback. Although
using PowerPoint offered benefits such as being easy to learn, quick in constructing the
prototype and supporting hyperlinks, they found it weak for designing and sketching
Amir M Naghsh 19
dynamic menus, designing a page that is larger than a PowerPoint predefined page
size, and also difficult in modifying larger models.
Another tool that can be used in medium-fidelity prototyping is HyperCard. This was
an application program from Apple Computers and it has been out of production since
2004 (and is not supported by Apple anymore). HyperCard uses a stack of cards, each
of which can contain text, picture and other interface elements available from the tool
box. It offers functionality by supporting scripts that can be attached to the buttons.
The simplest script is similar to the hyperlink which can link two different cards.
2.6. ELECTRONIC PAPER PROTOTYPING TOOLS
Landay (1996a) argues that an interactive user-interface design tool that supports
electronic sketching (an electronic pape- prototyping tool) would give designers more
freedom to change sketches and more flexibility in creating and evaluating a design
prototype. This flexibility is particularly important in the early stages of design itself,
when designers need the freedom to sketch rough design ideas quickly, the ability to
test designs by letting users interact with sketches, and the flexibility to fill in the
design details as they make choices.
‘Electronic paper-prototyping’ tools try to realise the benefits of paper-prototyping in
encouraging user participation by allowing designers and users to participate directly
and creatively when designing prototypes (Bodker and Gronbæk, 1991).
This section introduces a number of electronic paper-prototyping tools that can be used
in medium fidelity prototyping.
2.6.1. PATCHWORK
Patchwork, designed by van Kant et al. (1998) is a prototyping tool, where designers
express their ideas in the form of storyboards. Patchwork consists of patches that are
bitmaps and can be in any shape. These patches are easily modifiable and rough-
looking building blocks. Many of the patches are digital representations of low
technology office materials. It is also possible to use digital photography to import
sketches from paper, whiteboard or mock-ups into patches.
Patchwork supports storyboards. The designer can use the workspace to create a
prototype. Any time during the design process, the designer can switch from design
mode to run mode to evaluate the design. In the run mode, it is possible to interact with
the prototype by clicking on the patches that are linked to other screens. Each screen in
the storyboard consists of a background and a number of patches. In addition,
Amir M Naghsh 20
patchwork allows the designer to sketch on the background of the storyboard or the
patches using coloured pens. It is explained that a limited set of colours have been
provided to prevent the designers from focusing on design details too early.
Patchwork is intended as basic tool to encourage user participation in building
throwaway prototypes early in design.
2.6.2. SILK AND DENIM
SILK stands for Sketching Interfaces Like Crazy (Landay and Myers, 2001). SILK is
written in common Lisp and runs on top of Garnet system1. Garnet is a user interface
development environment for common Lisp. Designers can run SILK on a graphic
tablet such as a TabletPC or a Wocam Cintiq. It allows designers to quickly sketch out
GUI interfaces where designers are not required to specify details that are not known
or are not important at the early stages of design (e.g. Size, Colour, Shape, etc).
SILK then retains the sketch-like look of the components. It provides a storyboard
view where designers can view more than one sketched interface and identify
navigational links between them. These sketches can then be executed on the basis of
the state transitions, allowing a user to experience the proposed navigational behaviour
of a design.
Further, SILK can transform the sketches into operational interfaces by recognising the
gestures as visual basic components (e.g. list, button, etc). SILK can be applied in
throwaway and evolutionary prototyping.
Figure 2-4 Make a dialog box appear when the button is pressed. (Landay, 1996a)
1 See http://www.cs.cmu.edu/~garnet/
Amir M Naghsh 21
DENIM2 (Newman et al., 2003) is an outgrowth of SILK which has been implemented
in Java environment. Denim is a sketching tool to assist website designers to build
throwaway prototypes at the early stages of design.
Similar to SILK, DENIM also runs on a graphic tablet such as a TabletPC or a Wacom
Cintiq. However, Denim does not support recognition of drawn gestures. Instead it
supports more views that allow designers to use them to sketch out the overall structure
of a site (site map view); sketch the contents of the pages as a set of ‘scribbles’ (page
view); define hyperlinks from scribbles in one page to another page (storyboard view);
and then execute the resulting hypertext in a reduced functionality browser.
Figure 2-5 presents a screenshot from DENIM. The slider bar to the left of the screen
allows the site to be viewed at different levels of detail – varying from a site overview
that simply identifies the pages included, through a navigation view where the overall
navigation can be examined; down to a detailed view where fine details of individual
pages can be manipulated. At the bottom of the screen is a toolbar that includes a
pencil - for sketching new pages, new content within pages, or adding links; an eraser;
a hand – for moving the working surface to bring different areas into view; and a stamp
for adding typed text fields. To add links, the user selects the pencil and draws a line
from a scribble in one page to the destination page.
Denim also supports abstractions and allows the designer to create templates for the
design. Modifying the template can result in making global changes to all the pages
that have used that template. The template can also be used to create components such
as a menu navigation bar that can be used within the pages.
2 Denim version 1 is considered for the thesis that has been available since 2002. However, this section includes references to Denim version 2 that became available in late 2006.
Amir M Naghsh 22
Figure 2-5 DENIM, shown here in “Storyboard View”
DENIM also supports storyboards and allows designers and users interact with design
through a run mode (see Figure 2-6). It displays sketched pages in a limited
functionality browser and enables the users and designers to navigate the designed site
by clicking on active areas.
Amir M Naghsh 23
Figure 2-6 Denim - run mode, a simple browser providing
back, forward and refresh button. The user can interact with the model by clicking on the hyperlinks
DAMASK (Lin, 2003) is an extension of DENIM tool that is developed further to
support the early stage design for cross device user interfaces such as mobile phones,
PDAs and PCs (see Figure 2-7). DAMASK includes a series of design patterns from
the book: The Design of Sites (van Duyne et al., 2002). Each pattern has a different
solution for each device.
Amir M Naghsh 24
Figure 2-7 Damask user interface (Lin, 2003)
Damask allows the designer to split the design view into multiple screens to view the
design for different devices at the same time. To create a cross-device interface,
designers have to first design the interface for one of the devices and then specify a
design pattern to be used. The selected pattern will generate the user interface for other
devices. Therefore, the designer doesn’t need to sketch out the interface for each
device. Damask also allows designers to define custom patterns which can be shared
by other team members.
2.6.3. INDESIGN
InDesign (Nixon, 2001) uses storyboards to support navigational representations of
paper-prototypes. It displays scanned paper sketches in individual screens and lets
users and designers navigate through these screens by clicking on the active area of
each one. InDesign connects one screen to another by using a navigational link
between two screens and adding a label on a displayed sketch within the screen to
show the actual place of the navigational link.
Amir M Naghsh 25
Figure 2-8 Active area of a sketch within InDesign
Figure 2-9 Showing links between screens within InDesign. Scanned sketches are shown in left part of screen
Amir M Naghsh 26
2.6.4. FREEFORM
FreeForm (Plimmer & Apperley, 2003) is a tool for designing Visual Basic forms by
interacting with an electronic whiteboard. Users draw their designs using specialised
whiteboard pens, and then by clicking on the converter button the marks are recognised
and replaced by user interface widgets. Forms can be associated with navigation links
which can be positioned within the storyboard view. Evaluation is then possible in
FreeForm in a specialised mode, in which navigation links are indicated by specially
coloured target areas which provide the dynamic behaviour of the prototype.
FreeForm, similar to SILK, allows the evolution of the design by recognising gestures
that correspond to particular Visual Basic interaction components such as tables, drop-
down menus, text fields etc. A button is provided to convert the finished design into a
set of Visual Basic forms.
Figure 2-10 Design view (Left), Storyboard view (Right)
2.7. SUMMARY
Methods and tools which have been explained in this chapter are designed to support
iterative design. Iterative design involves creating an artefact, evaluating and reviewing
it, and then repeating the process several times.
High-fidelity prototyping approaches such as executable software prototypes,
executable formal specifications and XP are intended for use by software engineers.
Although some of the high-fidelity prototyping approaches (e.g. XP) support iterative
design through shorter cycles, it can still be argued that they have longer cycles to
evaluate and iterate design options than low-fidelity tools and methods discussed.
Medium fidelity prototypes have been designed to overcome some of the
disadvantages of low-fidelity prototypes. These approaches are intended for use by UI
designers and support quick prototyping and iteration. Some of the examples of
Amir M Naghsh 27
medium fidelity prototyping tools and approaches are Microsoft PowerPoint, Hyper-
card and HTML documents.
Active user participation in designing interactive systems has been identified as one of
the primary keys in developing useful and useable software products (O’Neill, 2001).
Although the medium fidelity prototypes have been designed to overcome some of the
disadvantages of low-fidelity prototypes, none of the tools mentioned earlier realise the
benefits of low-fidelity approaches in encouraging direct user involvements.
Therefore it is essential for users to feed into the design from the early stages of the
design process, because that is when fundamentally different ideas can and should be
generated and observed. Low-fidelity prototyping methods such as paper-prototypes
give the opportunity to members of different stakeholder groups, mainly users, to get
involved in generating a prototype rather than just the evaluation of a developed
prototype.
However, paper-prototypes have disadvantages as explained in Chapter 1, such as: the
difficulty in reviewing them when stakeholder teams are distributed; lack of explicit
representation of the navigational structure; lack of support for design memory and
difficulty in modifying them.
The electronic paper-prototyping tools discussed in this chapter allow designers to
sketch user-interface elements or rough layout for web pages and identify navigational
links between states or pages by drawing arrows. These sketches can then be executed
on the basis of the state transitions, allowing a user to experience the proposed
navigational behaviour of the design. These approaches try to realise the benefits of
paper-prototyping in encouraging user participation.
However, none of the above studies report on comparing the mentioned electronic
paper-prototyping tools. Therefore to be able to compare these tools, the next chapter
investigates the use of electronic paper-prototyping tools as part of a pragmatic
investigation.
Amir M Naghsh 28
Chapter 3. CASE-STUDY: COMPARING
EXISTING ELECTRONIC PAPER
PROTOTYPING TOOLS
3.1. INTRODUCTION
The type and properties of prototyping tools and methods can have a significant effect
on the design process and the effectiveness of the activities involved (Ehn & Kyng,
1991). Therefore, it is important to be able to compare the properties of existing
electronic paper-prototyping tools in order to select an appropriate tool for this design
investigation.
As explained in chapter 2 none of the reviewed literature reports on comparing the
existing electronic paper-prototyping tools. Therefore to be able to compare these
tools, it is essential to conduct a new investigation.
The new investigation was aimed to learn how these tools were used in practice, which
indicated the need for using a qualitative approach. However, conducting a qualitative
study such as collaborative or observational study with a number of recruited
participants would generate a massive volume of information which would be very
time consuming to process. It was also proved to be a difficult task to recruit
participants who would have the time to spend to learn each of the tools, as well as the
time to perform the design task using each of the tools. The total time required for
recruiting and training the participants, conducting the study and processing the results
was outside this research time frame and resources.
Therefore, to conduct a new investigation that would offer some degree of evaluation
of these electronic paper prototyping tools, it was decided that the author of the thesis
should conduct a case study. This would also allow the author to learn how these tools
were used in practice. In addition to the author, the supervisory team simulated the end
users by providing feedback on the implemented prototypes.
Amir M Naghsh 29
Hence, a case study was conducted aiming to investigate the use of electronic paper
prototyping tools in order to learn more about the similarities and differences of the
properties that they offer.
Different tools were used in the study to explore a simple design task. DENIM,
FreeForm and InDesign were considered for the investigation. In addition, paper
prototyping was also included in order to be able to compare with the tools that try to
realise paper prototyping benefits. SILK and Patchwork were excluded from this
investigation. SILK never was released to the public, due to the resources Garnet and
common Lisp require to be able to run SILK. Patchwork was excluded since there was
no access available to it. DAMASK was also excluded because of its high level of
similarity to DENIM.
The design task was a simple student marks program. The student marks program had
to be designed in such a way as to keep a record of each university course and each
enrolled student. The main purpose of the system was to allow users to assign students
to a course module and allocate a mark to each student. In addition, it had to allow
users to add, edit and remove records of students and course modules.
Other requirements included enabling users to use the system to generate the following
reports:
• A list of units taken by each individual student;
• A list of students assigned to a given course module;
• The results for each individual student indicating all modules taken and assigned marks;
• The results for all students associated with one course module, also indicating whether each student has passed or failed the module.
To conduct the investigation, a paper-prototype of the student marks system was
created first. The paper-prototype was used as the starting point in the form of a
visualised requirement to generate the following prototypes of the student marks
program by applying different methods and tools:
• an executable prototype using DENIM;
• an executable prototype using InDesign;
• an executable prototype using FreeForm.
Amir M Naghsh 30
3.2. PAPER PROTOTYPING
To create the paper prototype, A4 sized paper was used to sketch out the layout of each
individual screen state. Special icons were sketched on paper, cut out and put onto the
sketched interfaces to model the computer screen which then made it possible to
change the arrangement of the interface elements quickly and ‘on the fly.’ To interact
with each individual modelled screen pencil and post-it notes were used.
Figure 3-1 A screen state used in the paper prototype with addition of an interface element in the form of a message
dialogue box
Displaying sequences of screens did not really help to demonstrate the representation
of the navigational structure, but going through different sequences of the interfaces
made it possible to test multiple versions of the system’s interaction design.
The paper prototype was quickly constructed and simple skills were used in the
creation of the prototype. Using the paper prototype with other researchers, giving
feedback and making dialogues about the design seemed to be an easy task when
evaluating it.
The sketches have to be organised in a way to allow the designer to find them as
quickly as possible in order to respond to the user when interacting with the paper
prototype. Although this was not a large design task in terms of sketched screens,
organising the sketches and interface parts was a concern. Since each sketched screen
of the paper prototype allows the user to interact with it in many different ways, it
Amir M Naghsh 31
became more challenging to organise the screens as their number was increasing. It
became difficult for the designer to remember what other screens are related to each
individual screen, and interacting with which part of a screen requires the designer to
display another screen.
The other main difficulty was remembering the history of a presented series of screens
which make up an interaction sequence. An interaction sequence starts from a sketched
screen and allows the user to perform a task through interacting with a number of
screens and reaching the final screen. Remembering which screens were displayed
before reaching the current sketch is difficult when there are too many sketched
screens in a paper prototype.
Keeping track of all sketched screens that needed modifications to reflect a change in
the design proved to be a difficult task. For example changing the location of the logo
requires all the sketched screens to be updated, but adding a new item to a sub-menu
requires the designer to find all the screens with that sub-menu to alter the design.
After completion, the paper prototype as a visualised requirement was used to create
prototypes using InDesign, DENIM and FreeForm.
3.3. INDESIGN
To create a prototype using InDesign, sketched interfaces on paper were scanned and
put into the image library of InDesign.
To create an interactive sequence of sketched screens, the relation between screens was
recognised and links between them were created (see Figure 3-2). Being able to
execute the prototype gave the opportunity to have a better understanding of how the
dynamic behaviour of the final system might appear. However, use of hyper-links
between screens was more of a way to change the state from one screen to another to
follow a storyboard or illustration of a scenario. In particular, it was not possible to
interact with each individual screen layout and interfaces in any form of input.
Amir M Naghsh 32
Figure 3-2 Navigational view in InDesign
Since InDesign does not support pen-based technology, it was not possible to alter the
design and layout of a screen once it was scanned and transferred into the image
library. It required the effort of finding the relevant sketched interface on the paper,
making the changes on the paper and scanning the interface again to transfer it into
InDesign repeatedly. Once the scanned sketch was available in the image library the
screen holding the older version of the scanned sketched had to be retrieved in order to
assign the updated scanned sketch to it. Keeping track and finding relevant screens
holding specific sketches became difficult as the size and number of screens was
increasing during the process.
In addition, the naming style of the screens which was combination of “screen” and a
number indicating the order of the screen’s creation was not aiding the understanding
of the screen’s content. Since it was not possible to change the screens’ name to
something more meaningful, retrieving screens of interest became more difficult as the
number of screens was increasing. To find an interesting screen the content of many
screens which seem to have similar layouts at the storyboard level had to be verified by
changing the view to the page which had more details.
Alternatively, a graphic editor to alter the scanned screens could be used; however it
still would require the screen holding the older version of the interface to be found
before assigning the altered version of the sketched interface.
Amir M Naghsh 33
More importantly, since the links which connected different screens were not attached
to the underlying sketches, once an altered sketched interface was assigned to an old
screen, all the links from that screen needed to be updated.
3.4. DENIM
To create a prototype using DENIM, screens of paper prototype were used as a starting
point and were re-sketched in the DENIM environment using a graphic tablet. In
DENIM the designer had to name each screen.
The most used zoom levels were sitemap, storyboard and page levels. The sitemap
level gave a view of the prototype as connected labels. The labels were used to indicate
the screen names. This representation was used to review connected pages and identify
other relations between screens.
The storyboard zoom level was also often used for viewing several pages
simultaneously and observing the navigational relationship between the pages. Tools
such as pen were used to create identified hyperlinks between screens by linking a
phrase or shape from one screen to another. The page level was mainly used while
sketching the screen contents or when further modifications to a screen were required.
The other two major levels at the extreme ends of the zoom bar were rarely used.
Supporting pen-based interaction in DENIM made it much easier to modify and alter
the layout of the interfaces in comparison to InDesign. In addition to the pen-based
interaction, a text-field simulating an input box and typed-text were used when
designing the content of the pages in DENIM.
One of the advantages of the paper prototype was allowing interaction with the layout
of screens by moving around sketched interface elements (see Figure 3-1), which was
not supported in DENIM environment. Therefore, it was required to duplicate the
whole sketch in order to simulate the interaction of some part of a sketched layout (see
Figure 3-3). For instance, to illustrate the procedure of adding a student name to a
module, the same page was duplicated with a minor change from one page to another
page.
Amir M Naghsh 34
Figure 3-3 Simulating interaction
Duplicating an interface screen was not a straightforward task and many states of the
same screen increased the design’s size and consequently the time and effort needed
for further modifications. In particular, multiple duplicates of the same page with
similar names caused confusion that required zooming further into the page using the
storyboard level to locate the correct page by viewing its content.
Another issue with DENIM was the difficulty with the positioning of the sketches
when changing the zoom level. For example, when the zoom level was changed to
zoom in, the pages were positioned so far apart that it became difficult to find other
pages and it was necessary to pan or zoom back out. The other way round, when
changing the zoom level to zoom out, the pages would become too close to each other,
covering one another, which required the designer to select and move some of the
pages around to be able to have them all visible.
Being able to move the sketches around at the storyboard zoom level, made it possible
to arrange a number of pages of interest, to group them based on their content, or to
review and compare them together. However, unlike the paper-prototype the pages at
the storyboard zoom level did not provide detailed content of the sketches and required
further zooming in to view such detailed contents. The zoom levels such as page view
were more useful in providing an appropriate level of details, while they were
restricted with a maximum number of one or two pages that they could display at one
Amir M Naghsh 35
time. On the other hand, the site map zoom level had the benefit of displaying many
pages next to each other, but the content of the sketches was not provided at this level.
Selecting the sketched pages and then moving them around the DENIM sheet, proved
to not be as easy a task as it was in paper-prototypes. It was possible to select an
individual page at any of the zoom levels, however the facility to select a group of
pages was only provided at the sitemap zoom level, where the content of the sketches
was not displayed. This has the restriction of making a selection based on the
understanding of the page label or making the extra effort to zoom in and zoom out to
learn about the page content.
DENIM’s simple browser with reduced functionality was used to execute the prototype
and interact with it at any time during the design process. DENIM supported more
inputs in the form of left and right mouse clicks. It also allowed a potential user to type
into the typed-text object in the run mode. These simulated a closer look and feel to the
dynamic behaviour of the final system in comparison to InDesign.
3.5. FREEFORM
To create a prototype in FreeForm, sketched screens of paper prototype were also used
as a starting point to re-sketch them in FreeForm by using a graphic tablet. FreeForm
provided two representations of the sketched pages: page view and storyboard view.
The layout and details of each of the screens were sketched at the page view (see
Figure 3-4). To identify navigational links between screens the storyboard view was
used, which displayed a number of pages at a time (see Figure 3-5).
Unlike DENIM and paper-prototype, the arrangement of screens at the storyboard level
was restricted and it was very difficult to move the pages around to find a number of
pages of interest in order to group them together. The sketched pages in storyboard
view were displayed in the order of their creation in the form of a grid with three
columns and many rows. Therefore, at all times only a limited number of screens
depending on the size of the monitor could be displayed. The scroll bar had to be used
to access other sketches.
Two empty slots to contain sketched pages were provided at the storyboard view in
FreeForm, one at the beginning and other one at the end of the list of displayed pages.
The two empty slots were provided as temporary locations to be used when rearranging
the order of displayed pages at the storyboard level. To arrange a number of pages
together in FreeForm, the designer could only move one page at the time, making an
Amir M Naghsh 36
empty slot for another page to replace it. It became a hideous task to rearrange pages
when the prototype grew in its size.
Another difficulty with FreeForm is the way that sketched screens are named. Each
newly created page is named “sketch” with a number indicating the order of which that
page was created. Since it was not possible to change the page names to indicate a
more sensible meaning, it became difficult to keep track of what a page (for example
called “sketch 5”) was referring to. This became more problematic as the number of
screens in the prototypes grew.
Figure 3-4 Page view in FreeForm
Figure 3-5 Storyboard view in FreeForm
Amir M Naghsh 37
FreeForm supports a specialised mode to execute the prototype, in which navigation
links were indicated by specially coloured target areas to allow users to interact with a
executed version of the prototype. Similar to DENIM and InDesign the run mode in
FreeForm was provided to support a closer look and feel to the dynamic behaviour of
the finished system. In addition, when the prototype was executed it was possible to
add free ink marks onto an overlay of each page to simulate the user filling in the
boxes. However, the marks were held on a single overlay, and a button was provided
to clear the overlay or it was cleared once FreeForm exited the execution mode without
allowing the content of the overlay to be recorded as part of the design.
Unlike DENIM and InDesign, FreeForm provides a function to map and transform the
scribbles in sketched pages to Visual Basic components. The transformation of
sketches to components seemed to be taking more time than creating them from scratch
in Visual Basic. It was time consuming particularly because the content of each
individual sketched page had to be mapped before being transformed to a Visual Basic
form. Once the page’s content is mapped, a label is added next to each of the scribbles
that have been mapped indicating what that scribble was mapped to. By clicking on
each label, a drop-down menu is displayed giving the option to choose other
possibilities in case FreeForm has guessed the wrong component. Therefore, the user
has to go through each individual page, to verify the content that has been mapped
piece by piece. It is possible to skip this verification step and modify the forms in
Visual Basic; however it will be a more difficult task since access to the sketched
pages is not provided any more at that point.
In addition, FreeForm did not transform identified links between sketched pages. Once
the transformation was complete, the designer was left with a number of Visual Basic’s
forms that were required to be reviewed again to identify their relation to each other
and how they have to be linked.
3.6. DISCUSSION
Although all the discussed electronic paper prototyping tools can be viewed as
supporting medium fidelity prototyping, the case study showed these tools have many
differences as well as similarities.
Unlike FreeForm and DENIM, InDesign does not support pen based technology.
FreeForm allows the sketched pages to be transformed into Visual Basic forms while
DENIM and InDesign do not support this recognition. On other hand, it is much easier
Amir M Naghsh 38
to move the sketched pages around to arrange related pages together in DENIM and
InDesign.
The question then arises of how to categorise the similarities and differences among
these tools which belong to medium fidelity prototyping. The first question that arises
is how to use the concept of fidelity to categorise these tools? Does FreeForm have a
higher fidelity in contrast to other two tools because it is allowing user inputs? Or does
DENIM have a higher fidelity because it allows the use of text-fields which are very
similar to what would be used in the finished system?
The concept of fidelity is mainly focused on the degree to which a prototype presents
the final application and its interaction from the user’s point of view. Therefore, even
if it was possible to make a distinction in terms of the fidelity that different electronic
paper prototyping are offering, a second question arises - is the traditional concept of
fidelity sufficiently fine-grained to highlight the similarities and differences between
the electronic paper prototyping tools?
FreeForm might be considered to have a higher level of fidelity in comparison to
DENIM, since it can produce Visual Basic forms. However, how could it be explained
through the concept of fidelity that FreeForm requires more effort in comparison to
DENIM to allow its user to link sketches in the storyboard view? Or describe the effort
needed to transform the sketched pages into Visual Basic forms, when it can cause a
high level of errors in the recognition process and requires a lot of time to verify them?
Can this concept explain that it is much more difficult to select and move the pages in
FreeForm in comparison to DENIM and InDesign? Or for example, how could the
concept of fidelity explain that it is more difficult to modify the sketches in InDesign in
comparison to other two tools?
Although it might be possible to compare the prototypes created by using electronic
paper-prototyping tools and argue that they offer similar but not the same level of
fidelity, it is still not possible to compare the important similarities and differences that
these tools offer. In a single dimensional concept such as fidelity, everything is
compared against one individual characteristic which is the degree to which a
prototype presents a final application. More importantly, such a concept focus on what
the tool generates and it does not consider the activity, for instance design, and the way
the tool has been used in performing such an activity.
The combination of the tool and the prototype can be considered as a complex
information artefact. As explained earlier single dimension concepts such as fidelity
Amir M Naghsh 39
are not sufficient to compare these complex information artefacts and it is crucial to
apply frameworks with multiple dimensions. Such a framework would require
dimensions that are directly concern with the activities (such as designing interactive
systems) that different electronic paper prototyping tools are used to pursue them, as
well as dimensions that are concern with the generated prototype and the tool itself.
Heuristic evaluation is a framework that has multiple dimensions. However, since this
concept does not consider the activities that are involved when generating a prototype
and the way the tool has been used in performing such activities, it is not sufficient for
comparing complex information artefacts. This concept focuses more on the usability
and efficiency of what the tool generates. For example it is not possible to use this
concept to explain that FreeForm requires more effort in comparison to DENIM to link
the sketches in the storyboard view. Heuristic evaluation is an informal usability
inspection technique in which experts, guided by a set of usability principles knows as
“heuristic”, evaluate whether user-interface elements, such as dialog boxes, menus,
navigation structure, etc., conform to the principles (Preece et al., 2002, p. 408).
Cognitive Dimensions unlike other concepts such as heuristic evaluations or fidelity is
a multi dimensional framework that is designed to be used with none expert users for
evaluating complex information artefacts. Green (1994) explains the cognitive
dimensions as a discussion tool that raises the level of discourse among different
participants (e.g. end users, designers, programmers) involved in a design activity by
providing terms that can be used in discussions to try to understand the relation
between the activity and desirable characteristics of devices, which together make up a
short check. The next chapter applies cognitive dimension framework as an alternative
to the concept of fidelity.
Amir M Naghsh 40
Chapter 4. COGNITIVE DIMENSION
FRAMEWORK
4.1. INTRODUCTION
As explained in Chapter 3, the concept of fidelity is not sufficiently fine-grained to
compare the similarities and differences among different electronic paper-prototyping
tools. This chapter applies the cognitive dimension framework that has previously been
used to evaluate software systems, programming languages, and software design
notations (Roast, 1998; Roast et al., 2000; Clark, 2001; Kutar et al., 2002; Khazaei and
Triffitt, 2002) as an alternative way to explore the characteristics of different electronic
paper-prototyping approaches.
Green & Blackwell (1998) offer ‘Cognitive Dimensions’ as a useful conceptual
framework for assessing notational systems. This approach considers a range of
different vocabulary that can be used in describing the usability of a notational system.
Cognitive dimensions are a tool to aid non-HCI specialists in evaluating
usability of information-based artefacts (summative evaluation). Since they are
addressed to non-specialists, they consciously aim for broad-brush treatment
rather than lengthy, detailed analysis. Their checklist approach helps to ensure
that serious problems are not overlooked (Green and Blackwell, 1998, P. 5).
To apply the framework, first the information structure of artefacts must be built. The
Cognitive Dimensions uses the concepts of notation, environment and medium to
structure the information artefacts and define a notational system.
The notation comprises the perceived marks or symbols that are combined to
build an information structure. The environment contains the operations or
tools for manipulating those marks. The notation is imposed upon a medium,
which may be persistent, like paper, or evanescent, like sound (Green and
Blackwell, 1998, P. 8).
Amir M Naghsh 41
Tools and methods used in the development of prototypes for interactive systems can
be viewed as ‘notational systems’, where tools and their facilities can be viewed as the
environment, material and marks that are used in generating the prototype can be
viewed as the notation. Finally, the means such as paper, graphic tablet, PC and
whiteboard that are used to interact with the prototype can be considered as the
medium.
The next section gives a brief review of the full set of dimensions supported with
examples and followed by an analytical discussion to describe their relevance for
analysing electronic paper-prototyping environments.
4.2. RELATING THE COGNITIVE DIMENSIONS TO PROTOTYPING
4.2.1. VISCOSITY
Resistance to change: the cost of making small changes.
Repetition viscosity: a single goal-related operation on the information
structure (one change 'in the head') requires an undue number of individual
actions.
Knock-on viscosity: one change 'in the head' entails further actions to restore
consistency (Green and Blackwell, 1998, p. 12)
Viscosity means how much work is required to affect a small change. A good example
for repetition viscosity is the time and cost required to change the area code of a city in
a paper phone book. Updating the area code is a single change that requires an undue
number of actions. All the numbers in each page of a phone book have to be checked
and changed manually if they have the specific area code.
A timetable with a high interdependency is the example used by Green and Blackwell
(1998) for knock-on viscosity. Changing a lecture location from classroom A to
classroom B requires the lecture in classroom B to move to classroom C and lecture in
classroom C needs to be moved and so on. A single change to the timetable causes
reorganisation of everything else.
Viscosity has interactions with other dimensions, in particular abstraction, secondary
notation, hidden dependency and premature commitment (Green and Blackwell, 1998).
A simple example of how viscosity can be applied to the prototyping environment is
having to make a general change manually when the prototyping environment has no
Amir M Naghsh 42
tools to make global updates. An example could be when the user wants to add a new
option to the menu bar. In a paper-prototype, the designer has to modify the menu on
each individual sketched interface while in a software prototype the designer only
modifies the template for menu bar and it will update all the screens which contains the
related menu bar.
4.2.2. ABSTRACTION
An abstraction is a class of entities, or a grouping of elements to be treated as
one entity, either to lower the viscosity or to make the notation more like the
user’s conceptual structure (Green and Blackwell, 1998, p. 24).
Abstraction changes the underlying notation by reducing the viscosity of a notation.
An example is the use of styles in the word processor. For example, changing the font
size for “heading 1” style will change all the titles that use this heading style. A user
can add a new style as well as learning an existing one and applying it to the whole
document.
The Abstractions barrier is determined by minimum number of abstractions which
must be mastered by users before using the system, as well as their readiness, or desire
to accept new abstractions. Abstraction hungry systems can only be used when a large
number of abstractions are defined and set up prior to any work. Abstraction tolerant
systems allow users to define abstractions but do not require user-defined abstraction
to make the system work. Abstraction hating systems do not permit user-defined
abstractions.
Green and Blackwell (1998) explain abstractions as investments for the future. They
explain by changing the notation now it will be easier to use it in future, or makes
what’s built in the notation more re-usable or easier to modify since abstractions can
reduce the viscosity. Green and Petre (1996) suggest abstractions as the classic solution
to viscosity problems. Introducing more abstractions means more components can be
treated as a group thus reducing the repetition viscosity.
However, Blackwell et al (2001), Green and Petre (1996) along with Green and
Blackwell (1998) explain disadvantages of abstractions as being costly and requiring a
long time to set up. In addition, systems that enforce abstraction are potentially
difficult to learn and have a higher level of hard mental operation. Difficulty in
learning how abstractions are set up and are used could permanently exclude some of
the stakeholders. Therefore, systems with an abstraction hunger can be inappropriate
for exploratory tasks since only skilled designers can use them.
Amir M Naghsh 43
4.2.3. CLOSENESS OF MAPPING
Closeness of representation to domain (Green and Blackwell, 1998, p. 39)
Blackwell et al (2001) explain this dimension as how closely the notation is related to
the result it is representing.
Green and Blackwell (1998) provide two ‘thumbnail illustrations’ to explain in this
dimension. A close mapping: the visual programming language LabVIEW, designed
for use by electronics engineers, is closely modelled on an actual circuit diagram,
minimising the number of concepts that need be learnt. A distant mapping: in early
versions of Microsoft Word, to count the characters in a file, it had to be saved to disc
– whereupon it would calculate the size of the file and number of its characters.
Closeness of mapping determines the visual and behavioural similarities between the
symbols in a notation and the result it is describing. This work interprets this
dimension as the mapping of visual and behaviour similarities between the prototype
and the finally delivered software.
It is important that a relatively close mapping is available to some degree between the
interfaces in a prototype and the final interfaces in the finished system. The closeness
of mapping in behaviour respect is also suitable which can help users in interacting
with the prototype and have a better understanding of how the final system will
perform.
Although closeness of mapping is encouraged, too mush of closeness of mapping
between the prototype’s visual and the final system is not recommended by researchers
(Hong et al., 2000; Nielson, 1993) in the early stages to avoid users from focussing
attentions at low-level details of design such as fonts colour or the background picture
in a website.
4.2.4. CONSISTENCY
Similar semantics are expressed in similar syntactic forms [within the notation]
(Green and Blackwell, 1998, p. 39).
Green (1983) argued that a consistent programming language was easier to learn.
Blackwell et al. (2001) indicated that users could learn the structure of an information
artefact through recognising and learning patterns in notation and preventing such
patterns from being visible to users could compromise the usability.
Amir M Naghsh 44
A good example for consistency is mobile phones’ menus. In a consistent version, the
same keys are used to move up and down between different levels of menus. For
instance, in mobile phones the top right key on the keypad is always used to go one
level up in the menus.
To relate consistency to the prototyping environment, the similarities between the
marks that generate the prototype (notation) and the similarities between the facilities
that are supported by the prototyping tools (medium) can be considered.
4.2.5. DIFFUSENESS
Verbosity of language (Green and Blackwell, 1998, p. 39).
Some notations can use long-winded description in the form of many symbols, series
of commands or a lot of space to achieve the results that can be achieved more
compactly using other notations. Notations with high diffuseness can limit users who
are carrying out tasks which require working memory, such as exploratory design
(Green and Blackwell, 1998). Such notations force users to use their working memory
for remembering the long-winded descriptions. For example, the abstraction hungry
systems can have a high diffuseness level by forcing users setting up many levels of
abstraction when using the notation.
In the prototyping environment, the number of actions and marks that are required
when generating part of the prototype can be considered as the diffuseness.
4.2.6. ERROR-PRONENESS
Notation invites mistakes (Green and Blackwell, 1998, p.40).
Green and Petre (1996) have used the conventional way of categorising errors in two
distinguished category as ‘mistakes’ and ‘slips’. They explain a slip as doing
something that the user ‘didn’t mean’ to do. It happens when user knew what to do but
- due to the notation’s poor ability to discriminate, inadequate syntax-checking, bad
dialogue design or memory overload - did it wrong. While they explain a mistake as an
error that could happen in the design of parts which are deeply difficult.
The determining factor for error-proneness dimension for the prototyping approaches
is how much the notational system forces the stakeholders in making errors. For
instance, it is very unlikely in paper-prototyping that when a designer has a clear idea
of the layout of an interface, the designer makes errors due to the difficulties in
creating marks to design the interface and sketches something that was not intended.
Amir M Naghsh 45
4.2.7. HARD MENTAL OPERATIONS
High demand on cognitive resources (Green and Blackwell, 1998, p. 40).
A notation can make things difficult to be worked out in the head. Hard mental
operations is how much effort is required to understand a notation. The hard mental
operation of a notation can be considered in terms of the effort that is required in
performing three activities of writing, reading and executing the notation.
A simple example is the level of thinking required by the user to read, write and
execute a prototype created in a formal environment such as Z in comparison to a
paper-prototype. The prototype created in Z has a higher hard mental operation since
the user needs to make more effort and thinking to understand it in comparison to the
paper-prototype.
4.2.8. HIDDEN DEPENDENCIES
A hidden dependency is a relationship between two components such that one
of them is dependent on the other, but that the dependency is not fully visible. In
particular, the one-way pointer, where A points to B but B does not contain a
back-pointer to A (Green and Blackwell, 1998, p. 17).
A good example of hidden dependencies is hyperlink between HTML pages. The
HTML link is a one-way link and it is not visible at the end point which is the page that
is being linked to. Hidden dependencies slow down the process of finding information.
In the example of HTML pages, since the hyper-links are hidden from the other end, to
determine what pages are pointing to a specific page is costly and requires long
extensive search processes. Therefore, it is important to reduce the hidden dependency
or tolerate it to reduce the effort required for information findings.
When creating prototypes many parts of the design are dependent on one another. For
example, there are hidden dependencies between the pages that share similar layout in
paper-prototype. The designer has to keep record of pages that are similar, so that
when the layout of one page has changed, the same adjustment is incorporated into the
other similar pages.
4.2.9. PREMATURE COMMITMENT
Constraints on the order of doing things force the user to make a decision
before the proper information is available (Green and Blackwell, 1998, p. 21).
Green and Petre (1996) explain that premature commitment could happen when (a) the
notation contains many internal constraints or dependencies, (b) there are order
Amir M Naghsh 46
constraints in terms of what can be created first and forces the user to make a decision
before all the information is available, and (c) enforcing the user to look ahead in an
inappropriate order.
A simple example is route planning and not being aware of road traffic. For example, a
driver would not know anything of road works and possible delays until he/she reaches
that specific location on his/her way.
Premature commitment in the prototyping environment can be considered as the degree
of commitment that users and designers are forced to make to initial choices that have
been decided early in design while not all the information was available. For example,
it would be a premature commitment to decide on number of screens for illustrating an
interaction in a paper-prototype prior to having a clear understanding of the interaction
and the functionalities that it includes.
4.2.10. PROGRESSIVE EVALUATION
Work-to-date can be checked at any time (Green and Blackwell, 1998, p. 40).
Blackwell et al (2001) explain that the notational system can facilitate evaluation by
letting users to check on work-to-date frequently. Green and Blackwell (1998) indicate
the importance of such frequent evaluation even for experienced users in difficult
times, such as working under pressure or constant interruptions.
For example, spreadsheets re-compute repeatedly at each opportunity, therefore users
can develop a solution gradually while checking it at the same time.
It is essential for designers to be able to run the prototype with users at every given
opportunity and obtain users’ feedback to evaluate the design. For instance, it is
possible to review the paper-prototype with users at any time during the design
process.
4.2.11. ROLE-EXPRESSIVENESS
The purpose of a component (or an action or a symbol) is readily inferred
(Green and Blackwell, 1998).
Role-expressiveness means how easy it is to explain ‘what is this bit for?’ when using a
notation. In other words, it means how easy it is to understand the use for a part of the
notation or how it relates to other parts when using the notation. For example, a GUI
builder which offers role-expressiveness should allow the programmer to understand
what each component does when looking at the icons in the toolbox.
Amir M Naghsh 47
In the prototype environment, role-expressiveness can be considered as the level of
understanding that the user or designer could have in using the marks that create the
prototype or in using the facilities provided by the tool to perform tasks.
4.2.12. SECONDARY NOTATION
Extra information carried by other means than the official syntax.
Redundant recoding gives a separate and easier channel for information that is
already present in the official syntax. Escape from formalism allows extra
information to be added, not present in the official syntax. (Green and
Blackwell, 1998, p. 29).
Systems supporting secondary notation let users to record things that have not been
considered by the designer within the notation. The user can use the secondary notation
freely as informal additions that do not have any official meaning according to the
formal notation. Such ability to add arbitrary additional material is an essential aspect
of human communication that can be used as a channel between user and designer to
express extra information.
A simple example of redundant recoding is when designing a sitemap of a website;
usually the pages that belong to same section are grouped together. Another example
for secondary notation is the use of coloured post-it notes for categorising a textbook,
or comments in programme codes for future use.
Closely related to secondary notation is the possibility of an escape from the
formalism altogether. No programming language captures everything that a
programmer has to say about a program. If programs are to be used as a
means of communication, some other mechanism must be provided with which
to escape from the bounds of formalism (Green and Petre, 1996, p. 32).
Another example is offering secondary notation to reduce the understanding time of a
notation. Crosby et al. (2002) suggested that providing comments (escape from
formalism) and the use of a good naming style (redundant recoding) could reduce the
understanding time. Further a Blinman and Cockburn (2005) study showed that in
software development systems which were supported by a good naming style and
interfaces, documentations were understood better in comparison to the systems
without such supports.
In the prototyping environment, secondary notation can be considered as the forms of
additional information that are added to the prototype that were not included in the first
Amir M Naghsh 48
place. An example is the comments that users and designers scribble on the sketched
pages in a paper-prototype while reviewing the prototype.
4.2.13. PROVISIONALITY
Degree of commitment to actions or marks.
Pencils are used by architects, typographers and other designers to make faint
blurry marks, meaning ‘something more or less like this goes more or less
here’, as well as precise hard marks. (Green and Blackwell, 1998, p. 41)
Provisionality can reduce the premature commitment. Notations that support undefined
or vague marks are said to offer high provisionality by making provisional actions and
offering multiple design options before any commitment to the design. A high degree
of provisionality is appropriate when the prototype is not intended as a representation
of any finished system and it would be used to simulate and explore new ideas early in
the design.
4.2.14. VISIBILITY AND JUXTAPOSABILITY
Juxtaposability: ability to place any two components side by side.
Visibility: ability to view components easily (Green and Blackwell, 1998, p.
34).
For example paper-prototype has high level of Juxtaposability since it is possible to
select any number of sketched pages and rearrange them to view any two pages at the
same time. However, a paper-prototype can have a low visibility since seeking a page
of interest could require the user or designer to search through a large pile of sketched
pages.
Another example of Juxtaposability is the split option provided by Microsoft Word
that allows the user view two different part of same document at the same time. The
word documents that use the table of contents have a higher level of visibility since it is
easier to find a page of interest.
4.3. A REVIEW OF TOOLS USING COGNITIVE DIMENSIONS FRAMEWORK
Having introduced the cognitive dimensions framework, it is now possible to examine
a range of established prototyping techniques and tools including paper-prototyping
and different electronic paper-prototyping tools reviewed in chapter 2. Initial analysis
was reported at Psychology of Programming Interest Group 2003 (Dearden et al, 2003)
Amir M Naghsh 49
that considered a limited number of dimensions to investigate the feasibility of
applying cognitive dimensions in exploring the different characteristics of prototyping
approaches. Furthermore, the author of the thesis used the results of the case study
presented in chapter 3 and applied the full cognitive dimensions framework to review
prototyping notation by exploring all dimensions: viscosity, abstraction, closeness of
mapping, consistency, diffuseness, hard mental operations, hidden dependencies,
premature commitment, progressive evaluation, provisionality, role-expressiveness,
secondary notation, visibility and juxtaposability.
4.3.1. PAPER PROTOTYPING
In terms of the cognitive dimensions discussed above, paper-prototyping relies on
providing a low viscosity representation to encourage more user involvement at the
early stages of design. The low viscosity of paper-prototyping is provided by the use of
simple tools such as pencil and paper which users can adapt easily and modify the
design to express their desires. In particular, paper-prototyping is offering a low knock-
on viscosity by enabling users to easily modify any part of the design at one time.
Maintaining low viscosity lower the costs associated with repeated iterations in the
early stages of the development process. Prototyping assisting such a low viscosity
(such as paper-prototyping) are more suitable in encouraging users to participate with
supplying feedback in the early stages. However, as Lin et al. (2000) points out one of
the drawbacks of the paper-prototype is the time and effort required to make changes
to a design that need to be reflected across multiple sketches. In other words, a paper-
prototype may suffer from a high repetition viscosity.
Paper-prototyping does not enable the user to modify the notation to define new
facilities or terms within the notation. In other words paper-prototyping is an
abstraction hating notation. Having a low abstraction prevents paper-prototyping to
overcome its high repetition viscosity problems by allowing many components to be
treated as one entity for the ease of later modifications and incrementation. However,
as Green and Blackwell (1998) explain thinking in abstraction is hard and systems
supporting them are difficult to learn. Therefore, the low abstraction of paper-
prototyping makes it easier for users to become involved in the design process by not
requiring them to learn any skills in advance or spend any time in setting up the
notation.
It is also important that a relatively close mapping is available between the domain (the
appearance of a graphical user-interface) and the notation (paper sketches) so that the
users can easily learn to read the notation. On the other hand, one stated benefit of
Amir M Naghsh 50
paper-prototypes is that because the detail of colours, precise positioning, fonts etc. is
not available from a sketch, this avoids users focussing attention on such low level
details of design. This might be interpreted as an argument against too close a
mapping. Some authors have highlighted limitations in paper-prototyping in terms of
its ability to faithfully present certain aspects of interaction. Rettig (1994) highlights
the problem of representing aspects such as system response times. O’Neill et al.
(1999) discuss the lack of an explicit representation of the navigational structure,
which may result in users suggesting modifications to the layout of individual screens,
but being unable to modify or critique overall dialogue structure. Hence, paper-
prototyping offers a closeness of mapping only in respect to certain aspects of the
system being developed.
Since a member of the design team is required to ‘play the computer’ to enable users to
interact with the paper-prototype, it would be difficult for users to keep track of actions
that could result in displaying a sketched screen. In addition, the potential end users
cannot interact with the paper-prototype without the help of someone who knows the
system inside out to arrange a correct order of sketches to set up a scenario. This
hidden dependency between sketches restricts the end-user to interact only with the
layout of one sketch at a time rather than the navigational structure of the whole
system.
Further it can be argued that paper-prototyping to some extent has a hard mental
operation. The lack of dynamic representation closely mapped to the finished system
could force the user and designer to work out the dynamic behaviour of the finished
system in their heads. The design needs to consider all the different possible orders for
sketches to allow the user to interact with the prototype. The user has to try to work
out the overall navigational structure of the system from what the designer presented.
The use of squiggles or other vague lines in paper-prototyping can give a paper-
prototype a high degree of provisionality. It is easier to create marks whose meaning is
undefined or deliberately ambiguous. Such provisionality can act as a positive resource
for the paper-prototypes to be explored during the early design stage with the intention
of stimulating new ideas. In this case, there is often a clear view that the prototype is
not intended as a representation of any finished system.
Paper prototyping offers secondary notations in the form of scribbles on the paper, or
added ‘post-it’ notes. In paper-prototyping, post-it notes and similar secondary
notational devices can be used to indicate reasons for particular design choices,
critiques of particular elements, or indications that further work is required or planned.
Amir M Naghsh 51
Juxtaposability may be very important both during exploration – to consider alternative
designs for the same thing, and during evaluation – to allow for evaluations of
consistency in design. Paper prototyping allows for juxtaposability of different parts of
the prototype in a way that may be difficult to achieve with some software prototyping
systems. In addition, the difficulty in finding a page of interest in pile of sketched
pages makes the paper-prototype to offer a low level of visibility.
Using pencil and paper to draw sketches is an everyday familiar activity which makes
Paper-prototype to have role-expressiveness and consistency. The process of sketching
on paper is an easy task which does not require the user to perform many actions which
makes the paper-prototype have a low diffuseness.
As explained earlier it is unlikely that use of pencil and paper force the designer to
make errors by sketching an interface which was not intended, in particular when the
designer had a clear idea of the interface prior to sketching it. However, being unable
to execute the paper-prototype could make it difficult to design solutions for the
dynamic behaviour of the final system. Also the difficulty in finding a page of interest
in a paper-prototype could force the designer to make errors when simulating the
dynamic behaviour of the prototype. Therefore the paper-prototype has some degrees
of error-proneness.
Green and Petre (1996) express the importance of the possibility of progressive
evaluation as essential for end users and other stakeholders who are not experts. As
explained in chapter 2 paper-prototyping supports iterative design and allows users and
designers to interact with the prototype throughout the design process. The possibility
of evaluating the paper-prototype iteratively enhances the paper-prototyping with a
high progressive evaluation.
The low viscosity and high provisionality of paper-prototyping makes it simple and
cheap to use at early stages of design. This encourages more user involvement by
enabling users to make choices without the worry of committing to these choices. This
facilitates the paper-prototyping with a lower degree of premature commitment within
the wider design process.
4.3.2. DENIM
Reviewing DENIM using the framework above, it is clear that for changing the content
or layout of individual pages, DENIM will have a similar level of viscosity in
comparison to the paper-prototyping. Further development of DENIM (Lin et al.,
Amir M Naghsh 52
2002) allows abstraction to reduce the repetition viscosity. The later version of
DENIM includes component to support recurring page elements.
Components provide a mechanism for the designer to create and use reusable
widgets and fragments of interface designs. (Lin et a.l, 2002, p 4)
The later version of DENIM relies on providing abstraction by enabling users to
modify the notation to define new components. Such components could be used in the
design of menu or navigation bars. Providing abstraction in DENIM makes it easier
for later modification to the use of pencil and paper. However, this later version of
DENIM is abstraction tolerant and does not force the users to learn or define anything
before being able to use the tool.
While offering abstraction reduces the repetition viscosity, it has the trade-off of
increasing the hard mental operation and the hidden dependency. An unfortunate
consequence of using components is that it becomes difficult to identify where in the
model these components have been used and, if a component requires to be changed,
which part of the model is going to be effected.
In terms of the visual appearance of the DENIM page sketches, they offer a higher
level of closeness of mapping than a paper-prototype by supporting some aspects of
evolution of the prototype to make it look more like the domain (the eventual web-
pages). For example, the stamp tool can be used to replace a scribble with a typed text-
field. Also initial suggestions for relations between pages declared in the ‘site map’
level, can be refined to identify particular scribbles as the ‘hotspots’ that are the source
of page links. Links can be refined by adding specified time delays (for example to
indicate that some processing will be taking place) or by mapping them to specific
mouse actions (e.g. requiring a double-click to activate the link). A further
development of DENIM called Quill (Hong et al., 2002) allows for additional
interaction devices to be added to a design such as drop-down menus, thus it allows a
higher closeness of mapping for the visual appearance.
As explained earlier, paper-prototyping has limitation in terms of its ability to present
certain aspects of interaction. Being able to execute DENIM in a limited functionality
browser offers a major advantage in comparison to paper-prototyping, since it is no
longer necessary for the designer to ‘play the computer’ during the evaluation sessions,
which allows the designer to focus more on the users’ responses. The execution of the
model also provides a closer mapping to the interaction that users may experience in
the completed system, than is possible using pencil and paper. Similarly, adding in
Amir M Naghsh 53
time delays and specifying particular mouse actions to execute a link can also lead to a
closer mapping.
DENIM supports iterative design and provides the same level of progressive
evaluation as the paper-prototype. Being able to execute the prototype created by
DENIM at every opportunity allows the user and designer to evaluate the prototype
frequently while building it bit by bit. This process allows the stakeholders who are not
experts to become involved in the evaluation process.
DENIM permits the kinds of provisional marks that can be made with paper and
pencil. Hence, DENIM offering a high degree of provisionality may represent a design
direction that is well suited to the initial exploration of possible designs.
DENIM offers a limited support for communication of findings. DENIM does not
provide any form of secondary notation either within the design view or during
execution of the model. The only marks that can be made are scribbles, which cannot
be distinguished from the intended content of the design itself. In particular, this
approach does not allow users and other stakeholders to add their own comments to the
design. Landay & colleagues have developed 'The Designers Outpost' (Klemmer et al.,
2001) which uses a tangible user-interface based on a whiteboard, employing post-it
notes as the primitive interaction objects. Using the Designers Outpost annotations can
be associated with the display of a DENIM model, but the currently published version
of DENIM does not allow such annotations to be imported.
Use of gestures as commands gives DENIM a lower level of role-expressiveness than
paper-prototyping. For example, when a wrong form of stroke is used to indicate a
gesture, DENIM does not recognise the stroke and cannot match it with any of its
gestures, however DENIM response does not help the user to understand what would
be the correct stroke that is associated to the interested gesture. In addition, the
procedure of creating a page in DENIM does not offer role-expressiveness. The
designer has to scribble a page name on to the DENIM sheet at the storyboard level to
create a page. The created page always has a default size. However, at the page level
DENIM provides a more appropriate way to create a page by allowing the designer to
draw a rectangle that represents the size of a page and then associate a name to the
created page.
DENIM also has a lower level of consistency than the paper-prototype. The designer is
required to perform different actions in order to use same method at different zoom
levels in DENIM. An example would be the way that pages are created in DENIM as it
Amir M Naghsh 54
is explained in an earlier paragraph. Another example is the ability to select a group of
pages in DENIM that is only provided in some of zoom levels (storyboard and above).
It becomes difficult to find an individual page when the design model is large and has
too many sketched pages. To find a page of interest the designer has to repeatedly
zoom in, zoom out, and pan the design sheet to view pages that are placed far from
each other. Thus, it increases the level of DENIM diffuseness. Difficulty in finding a
page also decreases the level of visibility in DENIM.
DENIM offers similar level of error-proneness and premature commitment to the use
of pencil and paper. However, it can be argued that. DENIM allows the user to move
and juxtapose individual pages and maintain the level of juxtaposability offered by
paper-prototyping. This can be useful when attempting to evaluate consistency across a
large design.
4.3.3. INDESIGN
InDesign relies on the low viscosity of paper-prototyping. However, once the image
has been scanned into InDesign, the tool does not provide any mechanism for editing
which makes it suffer from a high level of viscosity and premature commitment. Once
the sketches are scanned, the designer is committed to them and for any amendments to
the sketches, the designer has to go back to the paper version to make those changes
and re-scan them. Designers may also try to apply standard graphical editor to modify
individual pages or states to overcome this problem. This process also makes it
difficult to execute the prototype frequently after each small change since the changed
sketches have been scanned first. Thus InDesign has a lower degree of progressive
evaluation in comparison to DENIM and paper-prototype. InDesign supports a similar
model for evaluation to that provided by DENIM, although the evaluation device does
not match that of a browser as closely – for example, no back button is provided. The
execution of the scanned images means that the dynamic behaviour of the prototype
created in InDesign can have a closer mapping than paper-prototype to the eventual
system.
The ability to add pencil marks to images before they are scanned allows for a similar
degree of provisionality to that provided by paper-prototyping.
InDesign, does not support abstraction. InDesign offers same level of role-
expressiveness, visibility and consistency compared to the use pencil and paper since it
is a direct use of scanned paper sketches. The images that are scanned in may also be
printed, or imported into a graphics editor to allow for juxtaposability during
Amir M Naghsh 55
evaluation. However, the InDesign environment itself does not provide methods to
juxtapose screens directly.
The use of navigational links at the storyboard level in InDesign can make it easier to
identify relation among pages thus reduces its hidden dependency to some extent
compared to a paper-prototype. However, since the hyperlinks are not glued to the
scanned content of pages, any alteration of the content requires further actions of
adjusting the position of hyperlinks according to the updated scanned content.
Therefore, it increases the hidden dependency much more in comparison to the paper-
prototype. In addition, to update the hyperlinks and relate them to the updated content
a higher level of thinking is required by the designer, which increases the hard mental
operation.
InDesign also has a high level of diffuseness. To associate hyperlinks to a page, firstly,
the designer has to identify the relationships between the pages at the storyboard level,
and then the links can be created between them. However, since the created hyperlinks
are not attached to actual content, the designer has to zoom in to each individual page
to adjust the position of the hyperlink trigger on top of the relevant content within the
page.
The difficult procedure of associating a hyperlink to the content of a page also
increases the error-proneness. Particularly, since InDesign does not provide
meaningful names that would represent the content of a page, there is a high possibility
that the designer may place the triggers - which are identified with name of the pages
that they link to - on top of an incorrect content within the page especially when there
are too many of them within a page.
The InDesign environment does not support any form of secondary notation and it
only relies on the annotation and comments made on the sketches before they were
scanned. InDesign could be viewed as an environment for executing a paper-prototype
by scanning each individual sketch. However, scanning sketches make design
iterations longer and more difficult in comparison to the paper-prototyping.
4.3.4. FREEFORM
As with DENIM, FreeForm exploits the low viscosity of pen-based interaction to
support design exploration. It allows users to select a set of marks and copy, move or
paste them to another part of the design, which can be considered as a low level of
abstraction.
Amir M Naghsh 56
Forms can be associated with navigation links which can be positioned within the
storyboard view. Evaluation is then possible in FreeForm in a specialised mode, in
which navigation links are indicated by specially coloured target areas. This provides
progressive evaluation as well as a close mapping to the dynamic behaviour that users
may experience in the completed system. In terms of the visual appearance of forms,
FreeForm offers a higher level of closeness of mapping by using a recognition system
to map the pen marks to specific user interface components. FreeForm emphasises the
evolution of the design by recognising gestures that correspond to particular Visual
Basic (VB) interaction components such as tables, drop-down menus, text fields etc. A
button is provided to convert the finished design into a set of VB forms. Once the pen
marks are transformed to the VB components, the user can change them by using the
provided toolbox in VB. However, previously too close mapping has been discouraged
to prevent users focusing attention on low-level details of design.
FreeForm permits provisional marks similar to ones made by pencil and paper.
However, because FreeForm is using such a recognition system, a mark that was
intended as provisional may later be interpreted by FreeForm and replaced with a (non-
provisional) user interface widget. Thus, it can be argued that FreeForm offers a lower
degree of provisionality to paper-prototyping or DENIM. For example, a provisional
mark can be used for a text-field or text-box, but when it comes to recognition the user
is required to make a choice and commit to one of them, while it is possible that the
user still does not have sufficient knowledge to make such a decision.
Similar to InDesign, FreeForm does not support meaningful names associated to the
pages in design. It also does not attach the hyperlink trigger to the page content. Thus,
FreeForm also suffers from a high hard mental operation, hidden dependency and
diffuseness.
Once the recognition process is finalised, the stakeholders are required to go through
the whole design to validate every individual recognised pen mark. Mistakes made as
the result of the recognition process means a higher level for error proneness” and
premature commitment. The user is required to think ahead to use marks which can be
mapped to specific user interface components.
FreeForm also supports some level of juxtaposition of forms by dragging them within
the storyboard view, however the very difficult procedure of moving pages in the
storyboard also increases its diffuseness. FreeForm offers same level of visibility and
consistency as InDesign does.
Amir M Naghsh 57
When FreeForm is in run mode, it is possible to add pen marks onto an overlay of each
screen. Plimmer & Apperley (2003) describe the use of this facility as a way to
simulate the user filling in text fields etc. Such an approach could be used to give users
a secondary notation with which to support communication about the design.
However, the marks are held on a single overlay, and a button is provided to clear the
overlay. Therefore, the marks cannot be treated as separate 'comments'. In addition,
when FreeForm returns to the 'design' mode, the marks are not visible.
4.4. DISCUSSION
The electronic paper-prototyping tools reviewed in this chapter are designed in a way
that facilitates the works of designers more than other stakeholder groups. Although, it
may be argued that these tools have been designed to encourage users' participation (by
realising paper-prototyping benefits), yet in the way they are presented, they do not
fully support direct participation of the members of different stakeholder groups to
have hands on involvement in generating a prototype and developing the design.
This section considers the analysis of the reviewed tools and suggests a profile of the
desirable levels of each of the cognitive dimensions for an electronic paper-prototyping
tool in order to support a participatory design in a distributed setting.
Viscosity: As explained earlier, to encourage more user participation the design tool
has to allow users to modify any part of the design in the repeated iterations from the
early stages of the process. Therefore, it is essential to maintain a low level of viscosity
in order to reduce the cost of change at any one time, and to make it easier for users to
make such changes.
Abstraction: The analysis suggested that to offering some levels of abstraction could
reduce the viscosity. However, providing a high level of abstraction in form of a
abstraction hungry tool can cause difficulties for less skilled users to become involved
in the design process. Therefore, it is important to provide a degree of abstraction
tolerant to reduce the viscosity, as well as not discouraging the users who might not
want to use them for any reason.
Closeness of Mapping: The results suggested that it is important to provide a relatively
close mapping between the prototype and the final system, so that the users can easily
learn to understand the prototype. However, it was also noted that too close a mapping
could divert users’ attention to low-level details that is not suitable at early stages of
the design. Therefore, it is essential to provide some level of closeness of mapping
Amir M Naghsh 58
which allows users to understand the prototype while not encouraging users to focus on
low-level details.
Consistency: There should be consistency in the marks used in notation and features
provided by the design tool. Providing a high level of consistency make it easier for
users, in particular the users with lower level skills, to learn the tool.
Diffuseness: Providing a low-level of diffuseness can make it easier to learn the tool.
Having less number of actions to remember when performing a task makes it much
easier for end users to learn the tool.
Error proneness: A low level of error proneness is desirable. Not being able to provide
a low level of error proneness means that the notation forces the users to make more
errors. Therefore, users become less encourage to participate to avoid making too
many mistakes.
Hard mental operation: It is important to offer a low degree of hard mental operation.
This can reduce the level of thinking required, hence makes it easier for users to use
the tool and as well as allowing users to focus on the design rather than how the tool
can be used.
Hidden dependency: A high level of hidden dependency can increase the level of hard
mental operation and viscosity by making it more difficult for users to make changes
or understand the relation between different components. On the other hand, providing
abstraction was suggested as a way to reduce the viscosity but using abstraction can
also add to the level of hidden dependency. Therefore, it is desirable to maintain a low
to medium level of hidden dependency that allow for some level of abstraction and has
the minimum increase in the level of viscosity or hard mental operation.
Premature commitment: It is important for the tool not to force users and designers to
commit to the initial choices that are made too early in design. A high level of viscosity
increases level of premature commitment by forcing users to look ahead in order to
make early decisions that will not require changes in later stages of design. Therefore,
it is essential to provide a low level of premature commitment.
Progressive evaluation: Providing a high level of progressive evaluation enable users to
interact with the design at different iterations. Allowing users to interact with the
design at any time can encourage more users’ participation by increasing their
understanding of the design.
Amir M Naghsh 59
Role-expressiveness: The tool should allow users to understand what each of the marks
and features offer. Providing a high level of role-expressiveness makes it easier for
novice users to learn to use the tool.
Secondary notation: A high level of secondary notation is essential. When the design
itself is to be used as a means of communication, providing extra information attached
to design as a form of a secondary notation can assist such communication by allowing
users to attach the critiques, choices and other responses to the design elements.
Provisionality: It is important to allow users to use marks with undefined meaning
when exploring the design at early stages. These marks are used to simulate new ideas
while may not provide a representation of any finished system. Providing a high level
of provisionality encourage users participation by allowing them to create their own
marks when designing.
Juxtaposability / Visibility: A high level of visibility and juxtaposability is desirable to
allow users to view any part of the design at any time as well as being able to view any
two components side by side.
Table 4-1 presents the results of the review of electronic paper-prototyping tools using
the cognitive dimension framework. The results show many differences as well as
similarities among the three compared to paper-prototyping.
For instance, the FreeForm has the highest level of closeness of mapping compared to
other two. This can make FreeForm a valuable tool for helping users to understand
how the finished system may look like in early design.
The lower level of hidden dependency, hard mental operation, diffuseness and
premature commitment in comparison to FreeForm and InDesign, makes DENIM an
easier tool for use by designers as well as users. DENIM provides a lower level of
consistency and role-expressiveness in comparison to other two tools, which can make
it difficult for the novice users (either designers or end-users) when beginning to use
DENIM.
Amir M Naghsh 60
Paper-
Prototype
DENIM InDesign FreeForm Suggested
Profile
Viscosity Low: knock-on
High: Repetition
Low: knock-on
High: Repetition
Medium: knock-on
High: Repetition
Low: knock-on
High: Repetition
Low
Abstraction Hating Tolerance Hating Tolerance Tolerance Closeness of Mapping High:
Layout
Low: Navigation
High High Higher Medium to High
Consistency High Low Medium Medium High Diffuseness Low Medium High High Low Error proneness Low Low High High Low Hard Mental Operation Low/Med Medium High High Low Hidden Dependencies Medium Medium High High Low Premature Commitment Medium Low High High Low Progressive Evaluation High High Low Medium High Role-Expressiveness High Low Medium Medium High Secondary notation Yes No No No Yes Provisionality High High High Medium High Juxtaposability Higher High Medium Low High Visibility Medium Medium Low Low High
Table 4-1 Summary of 14 cognitive dimensions in relation to electronic paper-prototyping tools
4.5. CONCLUSION
As explained earlier, the ability to add arbitrary additional material is an essential
aspect of human communication around documents and prototypes and it is important
to provide secondary notation to enable users to understand the design and participate
in authoring and evaluating the design by communicating and collaborating around the
prototype.
The analysis showed that electronic paper-prototyping tools (such as DENIM,
FreeForm and InDesign) are similar in regards to many of the dimensions. However,
none of the listed electronic paper-prototyping tools support secondary notation. These
tools do not permit users or other stakeholders to add extra information to give
feedback directly through the medium of the prototype. Instead, any comment or
feedback must be held separately (for example in an audio recording or minutes of a
meeting), resulting in a difficulty in identifying the items to which any comment refers.
This problem will be particularly acute if some stakeholders are not co-located with the
designers.
Amir M Naghsh 61
The importance of communication and interactions among stakeholders is emphasised
in the introduction chapter as a key dimension in the design process, which would
become a major difficulty and challenging when stakeholders are dispersed. Green and
Petre (1996) explained that secondary notation could be used as the communication
channel for different stakeholders to express extra information (e.g. comments in
code).
In a similar way, annotations are used around text documents for adding and showing
extra information. Wojahn et al. (1998) described making annotations as a common
practice to communicate about collaboratively produced artefacts. Digital annotation is
defined as the production of additional information related to a document as a whole or
to some parts of it (Bottoni et al, 2004).
Therefore, an electronic paper-prototyping tool should support possible forms of
secondary notations such as annotation, in order to enhance communication among
different stakeholders. The lack of the ability to annotate the design can severely limit
the ability of such prototyping tools to support communication between different
stakeholders at the early stages of design, particularly in distributed settings.
The next chapter reviews various studies investigating the use of annotation in
textbooks to digital documents, to inform design of annotation facilities in electronic
paper prototyping tools.
Amir M Naghsh 62
Chapter 5. ANNOTATION
5.1. INTRODUCTION
The traditional notion of annotation has recently evolved from paper document to
digital documents such as multimedia, 3D models and websites. This Chapter gives an
introduction to annotation and presents various studies investigating the use of
annotation in text books and digital documents. Further the chapter reports on the
forms and functions of annotations in textbooks and digital documents, presenting
them into two categories of personal and public annotation. Finally, the chapter reports
on studies that investigated characteristics for annotation tools to support asynchronous
communication around digital documents.
5.2. ANNOTATION
“Add explanatory notes to, furnish with notes” (Pearsall and Trumble, 2002).
Annotation refers to a note that describes, explains, or evaluates a particular document
or as extra information associated with a particular point in a document. Annotation is
a natural human activity that is used in day-to-day life. Brockmann et al. (2001)
conducted a research on examining some general aspects of “humanities” and found
annotation as one of the elements. Brockmann et al. refer to annotation as a support for
reading and explain that:
“scholars produce extensive marginal notes, annotating photocopies or
personal copies or attaching adhesive notes to a text” (Brockmann et al., 2001,
p. 7).
In general pen annotation on paper is a common practice of everyday life. Making
annotations is a natural activity that is performed in reading and reviewing processes.
When reading text books, a reader often makes notes, underlines important parts,
places asterisks, highlights or sketches upon the document. In a design environment,
designers often use post-it notes and other removable materials to annotate an artefact.
Amir M Naghsh 63
Since annotation is a natural activity performed in reading and reviewing text-
documents, the way people annotate on paper and mostly text-documents have
influenced most of the studies that have been carried out on above items. In 1984
Nielsen suggested that, in future, literature would only exist in an online form, as part
of an online journal or textbook. He concluded, therefore that computer systems for
such online literature should support the possibility of annotation for each individual
reader. Nielsen (1984) explained that there are several forms of annotations, and it
might be difficult for computer systems to support all of them.
Annotation is often for personal use but in many contexts even personal annotation
could be used to support collaboration even when it is not done with others in mind.
Personal annotations can be shared, for example by circulating annotated paper
documents at work. Marshall explained (1997) that when a marked-up document or
used book is shared the sharing is serendipitous.
Cadiz et al. (2000) found that the way that an annotation system is used is affected
when users are aware that their annotations will be shared and viewed by others and
they were more concerned when making annotation. Cadiz et al. (2000) reported that
users expressed that they did not make annotation if they felt that it was a minor or
“nitpicky” comment they were making. Their results also showed that users were
concerned while making a comment if it could be misunderstood by others or it could
misrepresent them as a harsh or unkind person. Cadiz et al. (2000)'s findings indicated
that users might not repeat the same annotation, if that comment was already made
earlier by another user. However, Wojahn et al. (1998) reported in earlier research that
users annotate the annotations. Their results indicated that users are more likely to
annotate an existing annotation that has been made by another user or reviewer rather
than annotating on an issue which has not already been annotated and discussed by an
initial reviewer. Such results were also found in Marshall’s (1997) research on second-
hand college text-books when students were highlighting comments made by the
previous owner of the text-book.
O’Hara and Sellen (1997) found annotation on paper convenient in reading and
reviewing activities, since paper provides a “malleable” surface and it is easy sharing
through its “ready-mobility”. Marshall (1997) explained that pen annotations are more
natural to people than annotation made by mouse and keyboard. She referred to
annotation as an “unselfconscious activity” and found that many annotation tools make
interruptions to the higher task that the user is carrying out. The development of tablet
computers has motivated researchers in digital annotations to learn from the way in
which people annotate paper documents and apply that in the digital world. The tablet
Amir M Naghsh 64
technology makes it possible for users to annotate on a digital document in a similar
way to annotating on a printed one.
In addition to tablet technology Morris et al. (1999) explained that people use other
digital technologies to overcome time and space constraints. They compared new
technologies to old communication mediums. For example some of the advantages of
email over letter writing were referred to such as speed, automatic filing, easy and
quick duplication and distribution. And also disadvantages such as email spamming.
The fast development in technology has made it possible to have asynchronous
communication around digital documents through annotating them. A study conducted
by Jung et al. (2002) also indicated that annotating digital documents could generate a
valuable history of dialogues which could be used for future retrieval.
5.3. EXAMPLES OF ANNOTATION TOOLS IN USE
As explained in the previous section annotation can be made either for a personal or
public purpose. Also annotation can be generated in synchronous or asynchronous
ways. Recent years have been witness to a number of research projects on annotations
focusing on various items in the digital world. Some of these items are:
• Text documents, such as digital books and articles: Quilt (Fish et al. 1988), Microsoft word and Prep editor (Neuwirth et al. 1990).
• Shared world-wide web :MADCOW (Bottoni et al. 2004), AnnoteImage (Brinkley et al. 2002), PAIS (Lober et al. 2000), Annotea (Kahan and Koivunen, 2001)
• Data visualisation and models in 3D: Redliner (Hanson and Wernert 1997), Space Pen (Harmon et al. 1996)
• 2D images: I2Cnet (Chronaki et al. 1997), AnnoteImage (Brinkley et al. 2002)
• Other multimedia environment, such as movies and audios:VideoAnnEx (Naphade et al., 2002), Madeus (Muriel et al. 1998)
• Structured data, such as databases and programming syntaxes
5.4. FORMS AND FUNCTIONS OF ANNOTATION
Although many researchers explored the use of annotations for different items in the
digital world, there have not been that many studies exploring the characteristics of
using annotation for most of the items mentioned in the previous section. The item that
has received more attention than others is the use of annotation in digital text
Amir M Naghsh 65
documents as well as paper text books. Many studies have been carried out focusing on
how readers annotate text documents and the practices which creates these annotations.
To aid better understanding of different forms of annotations, this section summarises
the results of some of these studies which focus on personal and public annotations.
5.4.1. PERSONAL ANNOTATIONS
People use annotation for different purposes in everyday life. One of the particular
situations that most people have experienced is annotating textbooks and articles.
Every individual has a unique style in making annotations, and several investigations
have been made to understand how people in general annotate textbooks. Nielsen
(1984) conducted a study to investigate how readers annotate textbooks and manuals.
He performed “a more anthropologically flavoured investigation of the notes made by
computer science students in their own personal textbooks during normal study
activities.” (Nielsen 1984, p. 3)
The results of his studies were used to present a taxonomy of annotations. He
explained that such taxonomy could help to have a better understanding of how
computer systems should support annotation for digital (on-line) literatures. Nielsen
(1984) divided the annotations in traditional textbooks into five major types. They are
Highlighting, Structuring, Cross References, Formal additions and Informal additions.
Each type was also subdivided further.
Highlighting “is the setting apart of some parts of the original text from other parts of
this text” which can serve to “distinguish parts of the text of perceived special
importance or the keywords to stand out for the quick scanning of the pages.” (Nielsen
1984, p.3)
Underlining and over-marking by pen or colour markers are two traditional ways of
making highlights in text documents.
Nielsen found it possible to have all types of highlighting supported by computer
systems. Some of the advantages of using highlighting in digital documents are
explained by him, which are: the possibility of hiding the highlighted parts when they
are not needed. This would stop readers from being distracted while reading the text.
Also such systems can use highlighting to filter the digital document. An example
could be using a colour marker for highlighting all the keywords and then using the
system to retrieve all the parts with such keywords. It also could be used to retrieve all
the important parts with a particular colour which are related to a specific assignment.
Amir M Naghsh 66
Structuring “is the explicit annotation of some structure in the original text.” Nielsen
referred to list numbering in the margin, or keywords instead of numbers as a
traditional way of structuring highlighting. An example is when a student finds only
some parts of the text relevant to an assignment and wants to read only those parts.
Instead of structuring the notes by using different highlighting colours, the student can
use key words in the margins to distinguish parts of the text for later study.
Cross references “is reference to some other text.” Nielsen explained that it is possible
to distinguish between references that are to the same page, another page in the same
textbook or another textbook. A current example of what Nielsen described as cross-
references can be the cross-reference facility in Microsoft word for index pages and
also references within the documents. Such cross-references in a digital document
allow user to go to a relevant section of text without needing to leaf through the
document.
A Formal Additions “is an addition having some specific structure.” Nielsen pointed
to a few examples of this type that are: quantitative diagrams such as bar charts, formal
diagrams such as flowcharts and formal tables. Nielsen explained that a computer
system can help the reader by supporting editors for generating annotations.
Informal additions “are annotations without any specific structure. Typically they
consist of natural language text or free hand drawings (Nielsen 1084, p. 7)”.
Furthermore Nielsen divided informal additions into the following categories:
• Corrections of errors in the text are often needed in traditional books, such
errors can easily be updated once they have been discovered in a digital form.
• Word explanations are required when the author uses not very well known
words or the document is in a foreign language.
• Exclamations and question marks in the margin are often used to show support
of or disagreement by the reader to the text. It could also be used for the parts
which are difficult to understand or need remembering for future use.
• Answers to exercise which are usually written in gaps between paragraphs or in
the margins close to the actual question for easy referencing.
• Interpretations and elaborations of existing text that are tied to a specific point
in the text.
• Informal drawings in the form of objects and sketches.
Amir M Naghsh 67
• And new independent text that could be included in the new version of existing
text.
A more recent study focusing on forms and functions of personal annotations in
textbooks was conducted (Marshall, 1997; Marshall and Brush, 2002 and 2004)
through performing extensive field studies. Marshall’s (1997) focus of research was to
discover common practices and patterns when creating personal annotations. In
addition to this, she studied the practices used for making personal annotations and the
implications that they can bring to digital libraries once they are transformed. She also
studied the value that personal annotations can have for later readers when they are
shared. Marshall (1997) conducted her research through a study on annotation in paper
books, since annotation tools and practices around paper have been well developed.
She used students of a major American university for her study around college
textbooks.
Results by Marshall (1997) along with O’Hara and Sellen (1997) showed that
annotations not only serve one function but often multiple functions. Therefore,
Marshall (1997) tried to “enumerate the most evident functions” used by personal
annotations made in college text books and categorised them into six groups. Later she
mapped all the annotation forms into the identified functions from her studies.
Procedural signals, this form of annotation was used by students to give clues and
explanations for different sections of the text book. For example it was used to indicate
which sections of the text book were relevant to an assignment or indicating important
parts of the text book which needed more than one revision.
Placemarks, Student used annotations to indicate the exact phrases from the text book
for later use. Mainly students used placemarks as an aid to memory. Some of the uses
were when students wanted to use a specific quote from the text book in the final
report or for definitions which were needed to be memorized for the exam.
In-situ locations for problem-working, Students used annotations to work out
problems once they came across them, rather than postponing it to some time later.
Marshall indicated that this sort of annotation occurred more often in the copies of
Organic Chemistry and Calculus, where students could work out the problems in
margins next to theorems and equations.
Interpretive activity, students used marginal notes, scribbles and free hand notes to
record their interpretations from the text book. Marshall, based on her findings across
different types of textbooks, categorised this function into three types. The first
Amir M Naghsh 68
identified type was interpretations explaining the meaning of a single world or phrase
within the text. These were used more often where the subject was a foreign language.
The second identified type was interpretation of the structure. And the third identified
type was the interpretation of a section or part of the page. Marshall found out that
students used annotation to interpret the content of a page more than the other two
types. This is was the most common type used by students for different subjects. It was
noted that this type of annotation could be more valuable to future readers (Marshall
1997).
Trace of the readers attention, students used annotation such as highlighting or
underlining when they were reading a difficult subject which formed a series of
sequences. Such annotations traced a reader’s attention while he/she was reading such
materials.
Incidental reflections, students made annotations which did not have any relation to
the textbook’s subject but more reflected the circumstances of time and place when
that student was reading the textbook. Students used the textbook’s paper as a way to
reflect on what was happening at the time in the student’s surroundings. An example
given by Marshall was the phrase “I Love You” in the early pages of a calculus book.
The taxonomy that was introduced by Nielsen (1984) covered both functions and
forms of the annotation but a visible distinction between form and function of
annotations in a textbook were not identified. Unlike Nielsen, Marshall (1997)
distinguished between forms and functions of annotations and presented a table to map
from forms to functions. However, she noted that it is not always possible to have a
single one-to-one map between a form and function since a form (e.g. highlighting)
could be used for more than one function. Table 5-1 is an extension to Marshall’s table
for mapping from forms to functions which also includes annotation taxonomy that
was introduced by Nielsen (1984).
Amir M Naghsh 69
Forms Functions
Nielsen Marshall Nielsen Marshall
Note
Highlighting / Underlining and over-marking
Underlining / Highlighting higher level structure, symbols
Hierarchical and differential highlighting
Procedural signals
Gives structure for future attention and use (very abstract)
Marginal symbols
Short highlights/ over-marks such as circles/ short marginal symbols
Exclamations and question marks
Place-marks and aiding memory
Indicating an exact point or phrase within the text to: Memorise, use it in later reports or to show support/ disagreement
Annotations in gaps or margins
Marginal annotations near figures or tables
Answers to exercises
Problem working
Answering questions / working out problems near figures and theorems for easy referencing
List numbering, keywords
Word or phrases between lines or in margins
Structuring Interpretation of structure (type two)/(Procedural signals)
Similar to first row but form of the annotation is changed and makes this form a more appropriate way for structuring
Annotations in form of text or scribble in the margins and maybe mapped to a word of interest
Word explanations
Interpretation to a single word (type one)
Translations of words in the margins
In addition to short and long notes, Informal drawing in form of objects or sketches
Short and long notes, in margins or as added as a new independent text (post-it)
Interpretations and elaborations
Interpretations (type three)
Interpretation to a section or part
Extended highlighting or underlining Simple highlighting
Trace of reader’s attention
Using only one means to highlight/ or being able to replace them with another
Scribbles / short notes
None Cross reference
None Referencing to other texts
None Note, drawing and other marking
None Incidental reflections
Reflections of the surroundings
Table 5-1 Mapping annotations form into functions
5.4.2. PUBLIC ANNOTATIONS
Marshall (1997) explained that personal annotations can become public and shared
through the reuse of the second hand textbooks. Marshall continued her investigation
on personal annotation in textbooks to recognize their values once they become public
and shared by others. Finally she suggested a strong set of implications for design of
annotation tools to facilitate readers work in a digital library setting and noted that an
Amir M Naghsh 70
implemented personal annotation tool for digital libraries is necessary to carry such
investigation forward (Marshall 1997). Some of these facilities which need to be
supported by digital libraries are as follow:
In situ annotation, distinguishable from the source, supports readers in annotating
while they read in such a way that their annotation will be in a format distinguishable
from the actual material that they are reading.
Non-interpretive markings, readers use many annotation forms which are non-
interpretive, including highlighting and underlining.
Fluidity of form, maintain the fluidity that annotation has on the paper by supporting
freeform capabilities.
Informal coding, readers usually have their own way of coding when they annotate,
for instance the way that different colour is used for highlighting could only make
sense to the reader. It is important to add enough to an annotation for digital setting to
maintain such an activity.
Smooth transitions between public and private annotations, to enable readers to
decide if they want to share their annotations. It was noted that even private annotation
can have value for the future readers of the second hand textbook. Also supporting the
reader to choose or remove annotations made by others as well as the ones that reader
has made.
Integration of reading as an activity, it should not distract the reading activity.
O’Hara and Sellen (1997) found annotating on paper is an integral part of reading
which can easily be integrated as one activity while online annotations are distractive
to reading.
Further research by Marshall and Brush (2004) found that only a small number of
private annotations can become public and shared. They found that users made some
changes in the way they made online annotation in comparison to the way that they
made personal annotations. Marshall and Brush (2004) suggested that online
annotation tools have to support two more required facilities.
Amir M Naghsh 71
Finding and filtering, helping users to find and filter annotations which the users want
to re-use and share with others. Such a facility is influenced mainly by the
effectiveness of the relevant user interface.
Support the transition from personal to shared annotations, Marshall and Brush
(2004) found that better user interfaces are required to manage and modify the anchor
of selected personal annotation to facilitate such a transition.
5.5. CHARACTERISTICS FOR ANNOTATION TOOLS
5.5.1. OPERATIONAL CHARACTERISTICS
A large number of studies (Brush et al., 2001; Cadiz et al., 2000; Neuwirth et al., 1990)
have investigated the use of annotation in supporting asynchronous collaboration and
communication around digital text documents. These investigations were mainly
focused on the capabilities that annotation systems need to adopt in order to enhance
asynchronous communication and collaboration around text documents through
developing and investigating such annotation tools.
Cadiz et al. (2000) along with Brush et al. (2001) have developed and tested systems
for asynchronous collaboration around digital text documents and both reported
notification and anchoring of annotations as two important areas which require further
research and should be supported in asynchronous electronic annotation systems.
While Brush et al.'s (2001) investigations were mainly focused only on two areas of
notification and anchoring of annotations from the user’s point of view, Cadiz et al.'s
(2000) research reported a more complete but abstract list of factors which influence
the use of an asynchronous electronic annotation system and the need for further
investigations.
Cadiz et al. (2000) conducted a large case study of digital annotations in a software
specification group focusing on writing specification documents. They investigated the
strengths and weaknesses of the existing collaborative annotation facility provided in
Microsoft Office 2000 and identified existing capabilities and possible improvements.
The major problems identified by Cadiz et al. include:
5.5.1.1. TECHNICAL ORPHANING
Cadiz et al. reported that the primary reason that the users stopped working with the
tool in their study was caused by orphaning of annotation.
Amir M Naghsh 72
Because the annotations system anchors annotations by computing a unique
signature for each paragraph, the system can fail to match an annotation to the
correct location when the text is edited. When this happens, the annotation is
“orphaned” and displayed at the bottom of the browser (Cadiz et al., 2000, p.
315).
They explain orphaning is a frustrating problem, since generated annotations become
meaningless once they are out of context.
5.5.1.2. PROGRESS AWARENESS
The results (Cadiz et al., 2000) indicated that the users were not satisfied with the way
that notification worked. It was not possible for users to easily determine who has done
what from the notification email. Instead they needed to log in to the annotation system
used to check every single annotation which was made. Brush et al. (2002) found the
same results which indicated that, it is preferable to have more information in the
notification messages which could support multiple communication channels, as well
as allowing users to customize the content of the notification messages.
5.5.1.3. RESPONSIVENESS
Cadiz et al. reported that shortcomings of notification systems also caused a lack of
responsiveness in users. Users did not find the iteration time (turn around) quick
enough for short responses and therefore they would not use it in such cases. However,
slow responses did not stop the generation of useful dialogues between users.
5.5.1.4. SHARED ANNOTATIONS
Cadiz et al. reported that once users were aware that their comments would be shared
by others they would change their approach in making annotations. They were less
likely to make small and not very important annotation. Also they tried not to make
comments which could be interpreted wrongly.
5.5.1.5. RICHNESS
Users also expressed the view that they would not use annotation for reporting
complex and critical problems. Instead they preferred to use more iterative and
interactive tools which would allow them to express themselves properly (Cadiz et al.
2000).
5.5.2. USER INTERFACE CHARACTERISTICS
In addition to the studies that were more focused on the practice involved in
asynchronous annotation tools, research (Morris et al. 1999, Neuwirth et al, 1990 and
Wojahn et al. 1998) has also been conducted on the characteristics of user interfaces
Amir M Naghsh 73
that can affect the way users collaborate and annotate digital documents. Wojahn et al
(1998) investigated how various interfaces effect annotation and communication and
proposed a set of requirements for annotation interfaces:
� There should be a minimum of motion required to make an annotation.
Annotation interfaces vary considerably in how many actions the user must
take to create an annotation.
� The document text should be easily distinguishable from the annotation. This
allows other team members, who may not have seen either the original
document or the annotations, to adjust themselves to the document quickly.
� The annotations should easily become visible and invisible while reading the
document. They should be accessible.
� The relationship between the primary documents and the annotations should be
easy to determine. It would allow readers to recognise which annotations refer
to a particular part of the document.
� Different annotation should be distinguishable to aid team members in
interpreting annotations from different authors and at different stages of the
process.
5.6. CONCLUSION
This chapter has reviewed the research carried out on personal annotation, with a
specific reference to the annotation made in textbooks. Previous research has shown
that annotating text is a common activity when reading (Marshall 1997). O’Hara and
Sellen (1997) found that readers use annotation to pinpoint important parts of a text
document and to help themselves to understand the text. Such remarks are also useful
for future tasks. Marshall (1997) found that annotations were used in many ways in
college textbooks including highlighting or underlining the most important parts to aid
the memory, book marking important sections and rephrasing the information into the
readers’ own words to aid interpretation. These annotations are often used as a
personal way of tracking references and keeping notes.
Reported studies on personal annotation and annotation in textbooks in this chapter
(Marshall 1997, Marshall et al. 2004, Nielsen 1984) has given a good understanding of
the form and functions that users can adopt while making annotation in textbooks.
Amir M Naghsh 74
Annotations are often beneficial to others as well, even when they are not purposely
made to be shared. Further research by Marshall et al. (2004) investigated the
possibilities of transferring personal annotation into public annotation. They reported a
list of capabilities which an online annotation system in a digital library context can
adopt to enhance such transition activity.
This chapter has also described the research conducted on the use of annotation in
asynchronous collaboration systems around text documents. These studies (Cadiz et
al., 2000; Morris et al., 1999; Neuwirth et al., 1990; Wojahn et al., 1998) have
investigated the use of shared annotation and reported on the characteristics that an
asynchronous system should adopt to support communication through annotations. The
next chapter considers these findings and reflects on them to identify a possible design
for a desirable annotation tool that can be used in designing interactive systems.
Amir M Naghsh 75
Chapter 6. GABBEH
6.1. INTRODUCTION
This chapter discusses the design and implementation of an electronic paper
prototyping tool with an annotation facility. To support the participatory design
approach in designing such an annotation facility a number of design sessions with
other researchers were conducted.
From the tools that were reviewed in chapter 2 and 3, InDesign, FreeForm and DENIM
were the only tools supporting electronic paper prototyping. The analysis in chapter 4
indicated the support for pen-based interaction technology and allowing stakeholders
to interact and modify the design model directly as one of the key dimensions in
encouraging more user participation. Therefore, InDesign was rejected since it does not
support pen based interaction technology.
DENIM and FreeForm were the only two tools which support pen based interactions.
Cognitive Dimension framework was applied to compare the characteristics of DENIM
and FreeForm as explained in chapter 4. The theoretical analysis results showed that
FreeForm in contrast to DENIM, suffers from a higher error proneness, hard mental
operation, diffuseness, hidden dependency and premature commitment, while DENIM
in contrast to FreeForm, benefits from a higher progressive evaluation, provisionality
and juxtaposability.
In addition, DENIM has the benefit of an open-source licence and source code
available online. DENIM also is supported with in-depth documentation of its design
and implementation which is vital for extending the tool. Therefore, DENIM was
chosen to be extended to support communication and collaboration in designing
interactive systems at early stages of design.
It was decided that the extension of DENIM should be called Gabbeh which is a
Persian hand-woven rug. Gabbeh was chosen to follow the fabric nature of naming of
Silk, DENIM and Satin. Also in some ways, Gabbeh the rug reflects exploratory and
participatory design (see appendix A for more detailed information on Gabbeh).
Amir M Naghsh 76
This chapter firstly reports on a possible design for an electronic paper prototyping tool
with annotation facility and follows it by describing DENIM and the technology used
by DENIM in more detail. And finally it explains the design and implementation of
Gabbeh.
6.2. DESIGNING AN ANNOTATION TOOL
Members of the research team (the researcher and two supervisors) conducted two
brainstorming design sessions which were video recorded. The materials used in these
sessions were mainly pencil, pen, paper and other office materials as well as printed
screen shots of DENIM tool.
Prior to the design sessions, the research team reviewed video recording of previous
interviews conducted in the Communication and Computing Research Centre at
Sheffield Hallam University on designing interactive systems, in particular web sites.
Reviewing the video recording revealed some of the uses of the comments in designing
web sites. The findings of previous studies in addition to the findings of the theoretical
studies in chapter 4 and literature review of annotation in chapter 5 were fed into the
two brainstorming design sessions to generate discussions.
The video recording of brainstorming sessions were reviewed and analysed. Based on
the findings and reviewed discussions the following list of requirements was identified
as one possible design model for a desirable annotation tool. However further
evaluation is required to validate such a design model.
6.2.1. CONSISTENCY TO CURRENT DESIGN AND EASY TO USE
Support users with a simple tool to add, edit and remove an annotation. As explained in
chapter 5, there should be a minimum of action required to make, edit or delete an
annotation (Wojahn et al., 1998). While the tool should be as simple as possible to be
used by different stakeholders, it also should be consistent with the design of the
selected electronic paper prototyping environment.
6.2.2. FLUIDITY OF FORM
Annotation is explained in chapter 5 as a common activity when using paper
prototypes. The fluidity of the use of pen on paper allows stakeholders to use
annotation as a way of communication when using paper prototypes. O’Hara and
Sellen (1997) found that annotating on paper is an integral part of the reading which
does not distract reading activity. Therefore, it is essential that annotation should not
distract the actual design activity and exploring and evaluating the proposals and
Amir M Naghsh 77
requirements. It has to be designed in such a way as to maintain the same fluidity that
annotation has on the paper and enable users to make comments as an integral part of
the design process which easily can be integrated as one activity.
As Marshall (1997) explained, it is essential to support freeform capabilities offered by
pen-based interaction technologies in addition to text comments to maintain the fluidity
that annotation has on paper.
6.2.3. IN SITU ANNOTATION
As reviewed in chapter 5, one of the facilities requiring support in digital libraries was
explained by Marshall (1997) as providing support for readers to annotate while they
read in such a way that their annotation can be distinguished from the actual material
that they read. Therefore, in a design activity stakeholders should also be able to
annotate design at all stages, whether they are creating and exploring the design or
evaluating it.
In addition, the user interface of such an annotation facility should have the following
characteristics that were proposed by Wojahn et al. (1998):
• The actual design or prototype should be easily distinguishable from the
annotation itself. This allows other stakeholders, who may not have seen either
the original design or the annotations, to adjust themselves to the design
quickly.
• Different comments should be distinguishable to aid stakeholders in
interpreting comments from different team members at different stages of the
design process.
• The relationship between the design component (or number of design
components) and the comment should be easy to determine. It would enable
stakeholders to recognise which annotations refer to a particular part of the
design.
6.2.4. INFORMAL USE
Stakeholders usually have their own way of coding when they annotate. They use a
combination of different pens with different colours, post-it notes and other office
materials which may only make sense to the members of the team which have created
them (Marshall 1997). It is important to add enough to a commenting tool to maintain
such an activity. Some examples could be enabling users to choose between different
colours, font size or freeform styles when creating a comment.
Amir M Naghsh 78
6.2.5. RETRIEVING AND FILTERING
More evolution and iteration of prototypes means more communications and therefore
more comments attached to the prototype. To enable stakeholders to track responses to
their comments and be aware of the design progress of a specific part of the design
through the attached comments, it is important to allow stakeholders to retrieve and
filter comments once they have been created to re-use and share them with others
(Marshall 1997). Or they can retrieve comments to refer to them in later decision
making. It is essential to support a filtering tool to filter all retrieved comments which
are related to a specific problem at any time. Such facility is mainly influenced by the
effectiveness of the relevant user interfaces.
6.2.6. AVAILABILITY
The design sessions found that annotations should be supported in all representations
offered by a prototyping tool. In particular, when different stakeholders interact with
different representations of a design it is vital to maintain consistency in making
comments available in all of the offered representations.
The results of design sessions also indicated that there should not be any boundaries
for creating annotation. This includes the ability of the user to annotate limitlessly
anything and anywhere on the paper prototype as well as being able to add comment
notes when there is no room for annotation in a specific area of the design.
Therefore, it is important to maintain such unlimited annotation by enabling the
comments to be associated with any arbitrary number of components in a design
model. Some specific examples are:
• To annotate any component in the design model (such as links, pages, scribbles,
etc);
• To annotate any collection of components, for example a comment which could
refer to a collection of pages;
• Any number of components share one comment;
• Any number of comments refer to one component;
• Comment on a specific sequence of components (for example a sequence of
pages in an e-commerce website: home -> shop -> basket -> check-out);
• To attach a sketch or a file to a comment;
Amir M Naghsh 79
• To use the comment to link the design model to other representations.
6.3. DENIM
Reviewing the way DENIM works was recognised as essential to being able to
implement Gabbeh. This section explains DENIM user interface and its underlying
architecture which later has been extended into Gabbeh.
6.3.1. THE DENIM USER INTERFACE
As explained earlier in chapter 2 DENIM is a Java based application which has been
developed in Berkeley University under an open-source licence. The primary DENIM
window is shown in Figure 6-1. The window has three main areas:
The centre area is a canvas where the user creates Web pages, sketches the contents of
those pages, and draws arrows between pages to represent their relationship to one
another.
On the left is a slider that reflects the current zoom level and allows the level to be set.
The bottom area is a toolbox that holds tools for drawing, panning, erasing, and
inserting text fields. Users “pick up” a tool by tapping on it, “drop” a tool by tapping in
an empty area in the toolbox, and “swap” tools by tapping on another tool. There are
several tools: a hand tool for panning, a pencil and eraser for sketching and erasing the
design, and a text field stamp for inserting text fields.
Figure 6-1 DENIM window
Amir M Naghsh 80
In DENIM users can sketch out the overall structure of a web site (a collection of
pages), sketch the contents of the pages as a set of ‘scribbles’, draw strokes (in form of
a arrow) to define a relationship between two pages. DENIM provides navigational
and organizational arrows to demonstrate relationships between pages. Navigational
arrows represent hyperlinks in the HTML sense hyperlinks from scribbles in one page
to another page. Organizational arrows are used to represent a conceptual relationship
between two pages (see Figure 6-2). There are five main zoom levels in DENIM,
which are identified on the zoom slider with icons representing the type of view
available at that level. The slider bar allows the site to be viewed from a site overview
that simply identifies the pages included, through a navigation view where the overall
navigation can be examined, down to a detailed view where fine details of individual
pages can be manipulated (see Figure 6-3).
Figure 6-2 DENIM design mode at storyboard view
Amir M Naghsh 81
Figure 6-3 DENIM design mode at page view
DENIM can execute the resulting hypertext in a reduced functionality browser. The
browser displays one page at a time, like a real Web browser, except the pages
displayed are the sketches that the designer has created (see Figure 6-4). If a
component within a page is the source of a hyperlink, it is rendered in blue. Clicking
on these design components displays the page that is the target of the link in a similar
way to normal browsers. The browser also has Back, Forward, and Refresh buttons.
Figure 6-4 DENIM in run mode
Amir M Naghsh 82
6.3.2. ARCHITECTURE OF DENIM
DENIM has been developed as an application on top of Satin and extends many of its
features. Satin (Landay et al.) is also a Java based application that was largely co-
developed with DENIM at Berkeley University. Satin makes heavy use of Java2D
libraries and some use of Swing GUI toolkit. Although Satin provides an abstraction of
Java2D, developers can use the Java2D instead when it is desirable (see Figure 6-5).
Satin was developed mainly to apply natural activities such as sketching and writing in
an electronic prototyping environment. DENIM uses Satin’s toolkit support for
informal ink applications.
Figure 6-5 DENIM relies on Satin
Satin provides a tree-like structure for manipulating and rendering graphical objects
which is called scenegraph. The scenegraph manages the way that graphical objects
and strokes are processed and displayed. Sheet is the root of the scenegraph and a node
in the Satin’s scenegraph is called “GraphicalObject”. There are several sub-classes
that extend the GraphicalObject. The simplest GraphicalObject is TimedStroke which
is set of points and time that are drawn. Satin assembles TimedStroke from mouse or
pen movement and provides reusable mechanisms for handling and processing strokes.
DENIM uses the same scenegraph used by Satin as its data structure. The tree
scenegraph allows DENIM to know what is on the screen. Figure 6-6, Figure 6-7 and
Figure 6-8 display three different representations for the DENIM scenegraph which
contains DenimSheet, DenimPanel, DenimLabel, DenimSketch, TimedStroke and
other components and instances used within DenimSketch. DenimSheet extends Sheet
class from Satin as the root of the DENIM scenegraph. DenimPanel is what represents
the screen in GUI or the webpage in a website. DenimPanel is the parent to
Amir M Naghsh 83
DenimLabel and DenimSketch and it keeps them together when the design is zoomed
in and out, or when one of them is moved and the other one needs to move with it too.
Figure 6-6 UML diagram of DENIM scenegraph
Amir M Naghsh 84
Figure 6-7 Tree data structure of DENIM scenegraph
Amir M Naghsh 85
Figure 6-8 A visual representation of DENIM scenegraph. DenimSheet boundary is highlighted with a red line, DenimLabel with a black line, DenimSketch with a green line and TimedStroke
with a blue line. Also TypedText and ScribbledText are highlighted within the sketch
Each GraphicalObject has one or more views which know the visual boundaries of the
object and how to render the object. Each GraphicalObject also has one or more
interpreters for handling strokes. Interpreters can be added to any graphical region
displayed on the sheet and process the ink strokes directed to that region before they
are rendered to the screen. Each GraphicalObject has two sets of interpreters to handle
strokes in two categories for gesture and ink.
When strokes are drawn they are dispatched to the root GraphicalObjects at the tree
(which is the sheet) and recursively traverse down the tree, processed only by the
GraphicalObjects that contain the complete stroke. This method allows the stroke to be
processed first as the gesture before processing it as ink. It also provides global
behaviour for gestures. (See Figure 6-9).
Amir M Naghsh 86
Figure 6-9 Diagram of how strokes are handled as gesture and ink
Amir M Naghsh 87
6.4. GABBEH
Early design of Gabbeh prior to its first version was presented in the HCI2004
conference (Naghsh and Özcan, 2004). This section reports on the user interface of the
first version of Gabbeh. It further describes the implementation of that version of
Gabbeh in detail and reports the reasons for selecting some of the requirements
mentioned to be developed in this version.
To address the issues raised by sessions for designing an electronic paper prototyping
tool that has annotation facilities, Gabbeh as an extension to DENIM was developed to
support communication and collaboration among stakeholders by annotating the
design. The core innovation in Gabbeh is that it allows different stakeholders to add
arbitrary annotations in the form of comments when the model is being designed or
when the model is being executed. It is important to note that this version of Gabbeh
(version 1) does not meet all the requirements recognised in section 6.2.
6.4.1. THE GABBEH USER INTERFACE
Gabbeh offers annotation features in both, ‘design mode’ and ‘run mode’. The ‘design
mode’ consists of the design sheet and a limited functionality browser. The ‘design
mode’ enables the designers to create design proposals by using available features in
the toolbox and multiple representations offered through zooming. In addition, it
allows designers to interact and evaluate such design proposals by using the limited
functionality browser.
Figure 6-10 Login box: allowing stakeholders to select different versions of Gabbeh
The ‘run mode’ consists of a limited browser. Since the end-user might not have access
to a graphic tablet, the design sheet is excluded from the ‘run mode’ version to allow
Amir M Naghsh 88
end-users to work only with a simple browser with annotation features that can be run
on a standard PC. A stakeholder depends on being an end-user or a designer can
choose one of the modes by selecting an option in the log in dialogue box (see Figure
6-10).
Figure 6-11 Gabbeh toolbox, Commenting pencil for creating and editing comments (fifth from left), pad for filtering the
comments (sixth from left)
6.4.1.1. THE DESIGN MODE
6.4.1.1.1. CREATING COMMENTS
Gabbeh allows designers to add arbitrary scribbles (free-hand notes) to a comment
using a commenting pencil (see Figure 6-11) similar to the free-hand writing tool used
to create elements in a web page in ‘design mode’.
The designer can create a comment at any of five zoom levels. There are two ways that
a designer can use the commenting pencil to create comments. The first way is to use
the commenting pencil to write words in free-hand style on the canvas. Gabbeh uses
the Satin toolkit to group the ink strokes that are close to each other into words and
phrases. Then Gabbeh automatically creates a comment using the phrase (see Figure
6-12). The defined background colour for a created comment is blue which can be
changed easily by using same commenting pencil tool (explained later). Comments are
given a background colour. This is intended to allow development teams to distinguish
between different types of comments, or perhaps between comments from different
speakers. The usage of colour is deliberately left open.
Figure 6-12 Comments can be created by writing words or phrases directly on the design sheet at any zoom levels
Amir M Naghsh 89
The second way of creating a comment is to use the commenting pencil to open a
dialogue box which allows the designer to type in the comment and insert it on the
canvas (see Figure 6-13).The commenting pencil has to be clicked once on an area on
the design sheet where the designer wants to insert the comments. When the
commenting pencil is clicked once, Gabbeh opens a dialogue box and allows the
designer to type in the comment text as well as selecting the background colour of the
comment.
Figure 6-13 the comment dialogue box allowing the designer to select the colour as well as entering the typed text to create a
comment
6.4.1.1.2. ASSOCIATING COMMENTS TO OBJECTS
A comment in Gabbeh can be associated with any arbitrary number of design
components, such as panels, labels, texts and scribbles. Arrows have been used to
associate comments with design components. An arrow between a comment and a
design component represent a relationship which is called annotation arrows.
Annotation arrows indicate that the particular design component has comments
attached to it.
To create an annotation link, the designer draws a stroke by using the pencil or the
commenting tool between the comment and design component. Gabbeh checks if it is
an arrow. To be an annotation arrow, it has to start from a comment and end on a
design component. Once Gabbeh recognises the stroke as an arrow, it will change its
colour similar to the comment background. This process can be repeated as many times
as the designer wants in order to share the same comments for different components
(see Figure 6-14).
Amir M Naghsh 90
Figure 6-14 Using the drawing or commenting pencil to associate a comment with one or many design objects
Gabbeh also allows a comment to be associated with a different number of pages or
design components located within different pages (see Figure 6-15).
Figure 6-15 Representing a comment shared between components from two pages in the design
6.4.1.1.3. EDIT COMMENTS
The commenting pencil can be used to change the colour of a comment regardless of
its style being freehand or typed text. The designer can use the commenting pencil to
click on the interested comment, and then Gabbeh will open the dialogue box similar to
the one shown in Figure 6-13 and allow the designer to change the colour to a different
Amir M Naghsh 91
one for the comment. Also if the comment is in typed text format, the same dialogue
box allows the designer to change the text in addition to the colour.
The commenting tool can also be used to add more strokes to a phrase used in
comment by writing relatively close enough to the comment. However, it is not
possible to edit the phrase in freehand style once it is created. To edit a freehand
comment the designer can delete the comment and rewrite the comment with new
changes, or convert it into typed text using same dialogue box that has been used to
edit the comment’s colour.
6.4.1.1.4. DELETE COMMENTS
The eraser tool from toolbox can be used where required to delete a comment. Once a
comment is deleted all of the associated links going out from that comment are deleted
too. However, it is possible to use the eraser tool to only delete association arrows and
not the comment. For example in Figure 6-15 the eraser could be used to delete only
one of the arrows which is not required any more.
In addition to the eraser tool, the delete gesture could also be used. Another alternative
would be using the delete option from the pie menu.
6.4.1.1.5. FILTERING COMMENTS
Using Gabbeh indicated that as the prototype evolves the number of comments
attached to it increases rapidly. Preliminary exercises indicated that the design could
become crowded with a large number of comments and it is vital to be able to make
comments invisible and only allow the interested comments to be visible and
displayed. Figure 6-16 shows a design case which became crowded with a large
number of comments at early iterations. The comments displayed in Figure 6-16 are
displayed in the storyboard view of Gabbeh. These comments are attached to a
prototype of a website which has only four pages. Therefore, it was suggested that it is
essential to be able to make comments visible or invisible and filter them to allow
designers to concentrate on the actual design and comments relevant to that part of the
design.
Amir M Naghsh 92
Figure 6-16 Design sheet crowded with comments
The designer can use two tools to manage the visibility of the comment in the ‘design
mode’. The commenting pencil is one of the tools that can be used to hide comments or
make them visible. The designer can use the commenting tool over any individual
comment to open the dialogue box to manage the visibility of the selected comments
(see Figure 6-17). The dialogue box allows the designer to choose from two options of
showing and hiding a comment. When the comment has been set to be invisible, only
the starting point of the arrow that associates the comment to a design component
remains visible (see Figure 6-18).
Figure 6-17 Using the commenting pencil for managing the visibility of comments
Amir M Naghsh 93
Figure 6-18 A coloured node represents the comment once it is set to invisible
Gabbeh also supports a simple filtering tool in the design view to offer a way of
managing the visibility of a larger number of comments. The current design gives two
options. One way is to click the filtering tool somewhere on the design sheet to manage
the visibility of all comments attached to the design sheet; the other option is to use the
filtering tool by clicking on the label of an interested page to manage only the
comments attached to that one page (see Figure 6-19). The filtering tool also allows
the designer to select the colour of comments that their visibility status should change.
Figure 6-19 Managing comments in Gabbeh
Amir M Naghsh 94
6.4.1.2. THE RUN MODE
End users may execute Gabbeh using a separate limited functionality browser to
review a design. To make Gabbeh easier for end-users, the design sheet is excluded
from the version of ‘run mode’, which is installed at end-users site (see Figure 6-20). It
allows the end-users to work only with a simple browser with annotation features (see
Figure 6-21).
Figure 6-20 The run mode interface for Gabbeh. File button is to open a design file, Run button to start the execution of the design model and save button to save the added comments to
the design model
Figure 6-21 Limited functionality browser with commenting features
Gabbeh allows end-users to view and add comments while they are reviewing the
design in ‘run mode’. This functionality is intended to allow end-users to give
feedback through the prototyping medium. The end-user can choose to view the
comments by selecting the view comments button from the menu bar (see Figure 6-21)
and Gabbeh displays comments in a side window adjacent to the side of the page (see
Figure 6-22).
Amir M Naghsh 95
Figure 6-22 Viewing comments in the 'run mode'
Figure 6-23 Adding comments when Gabbeh is executed
Gabbeh displays the comments location within the page using coloured numbers on the
page (see Figure 6-22). If the comment is only associated with the page, Gabbeh only
displays the comment in the side window.
Users are able to add comments when Gabbeh is being executed, which can then be
visible in the ‘design view’. Figure 6-23 shows an example of how users can add a
comment in ‘run mode’ by clicking on the "Add Comment" button which opens a
comment dialogue box similar to the one in the ‘design mode’. Once comment is
created it will be displayed in the side window as well. In this version, comments
Amir M Naghsh 96
created in ‘run mode’ are associated with a page, but not with a specific position on the
page.
6.4.2. IMPLEMENTING GABBEH
The architecture of DENIM is decomposed and spread into many fine-grained
packages as can be seen in appendix D. Therefore, understanding the overall system is
challenging. For example, if one decides to identify all the related classes to
DenimPanel it would be very difficult through the current packaging of DENIM. In
addition to the difficulty of identifying such relationships DENIM classes have a high
coupling which makes it very difficult to modify or extend. Once one class is modified
it is more likely that several classes are affected by that change in other classes and
packages in DENIM. Figure 6-24 shows classes related to the DenimPanel which are
affected once a DenimPanel is modified.
Amir M Naghsh 97
Figure 6-24 Dependency Graph for DenimPanel
Amir M Naghsh 98
For example, the io package (see Figure 6-24) uses a predefined form for DenimPanel
to convert it to a document object model (DOM) element and save it to a file. DOM
uses a predefined way for accessing and manipulating DENIM components such as
DenimPanel to save them into a file that can be accessed later and read to load the
DENIM model. The structure used to convert DenimPanel is:
<DenimPanel>
<ID>…</ID>
<AffineTransform>…</AffineTransform>
<BoundingBox>…</BoundingBox>
<DenimLabel>…</DenimLabel>
<DenimSketch>…</DenimSketch>
</DenimPanel>
Any modification to DenimPanel structure implies a revision of all the methods for
converting the DenimPanel to DOM element and creating DenimPanel from DOM
element.
DENIM, through the use of the Satin toolkit supports the development of new
interpreters and recognisers. DENIM also supports the creation of custom components
to be added within DenimSketch. But the high coupling of DENIM and its concrete
architecture makes it difficult to modify the scenegraph and structure of DENIM
shown in Figure 6-6. For example to add a new object within DenimSheet similar to
DenimPanel, all the interpreters, some of the classes in the command, view, io, action
and component packages will need modification. Such modification is required
because all the classes related to DenimPanel shown in Figure 6-24 are designed and
developed based on DENIM scenegraph (see Figure 6-6) which means that they only
support the use of DenimPanel as a recognised object to be added to DenimSheet with
specific references to DenimPanel from each class and package.
Therefore, the development of the commenting facility is a difficult task, as
introducing a new instance into DENIM scenegraph requires major modification of its
structure. So, it is time consuming and costly to implement a quick prototype of the
proposed design to validate the identified requirements and explore the design of the
commenting tool further.
Consequently, due to time constrains and the complexity of DENIM it was decided to
extend DENIM based on what is already available within the DENIM structure to
Amir M Naghsh 99
support the commenting facility and allows stakeholders to annotate the prototype
during the design process.
This section describes the development of Gabbeh and indicates the areas of design
which were not developed further due to the reasons mentioned above.
6.4.2.1. INTRODUCING COMMENT AND COMMENT-MARK
Since it was recognised that modifying the DENIM structure is difficult and not
practical with the given resources during this research project, available entities in
DENIM structure were investigated and DenimLabel was identified to be extended
further to implement the requirements described in section 6.2.
DenimLabel was chosen instead of DenimPanel or DenimSketch, since it is the only
entity in DENIM that can be created by using scribbled text (free ink) or typed text
which supports fluidity of form. It also supports the requirements explained in sections
6.2.3 and 6.2.6 by allowing the DenimLabel to be created at any zoom level and added
to any location on the sheet, while DenimSketch can only be created at particular
zoom-levels like page or detail.
On the other hand DenimPanel, as explained before, is a rectangle which keeps
DenimLabel and DenimSketch together. DenimPanel is created when a rectangle is
drawn on the sheet. There are three interpreters which allow DenimPanel to be created
directly and once it is created, a default DenimLabel “page” is associated with
DenimPanel and it is only possible to modify it by using the text editor. DenimPanel is
also created when a DenimLabel is created. This way it is DenimLabel which is
created first by using a typed text or scribbled text and then associated to a new
DenimPanel.
As explained it is not possible to create a new object as an extension of DenimLabel
due to the concrete structure of DENIM. Therefore, a flag as a variable is added to
DenimLabel which allows two sets of behaviours for DenimLabel depending on
whether the flag is on or off. By default once the DenimLabel is constructed the flag is
unchecked and the normal methods of DenimLabel can be used by other classes. Once
the flag is checked, DenimLabel would be treated as a comment and in addition to the
DenimLabel methods the new set of methods added to DenimLabel could be called as
well. This can be seen as extending DenimLabel into DenimComment without altering
the DENIM scenegraph.
In addition to the modification of DenimLabel, a DenimCommentMark object was also
created to be added to the DenimSketch to highlight the location which the comment is
Amir M Naghsh 100
referring to on the DenimSketch. By default, DenimCommentMark is set to invisible
since the arrow points at the position that the comment is associated with (see Figure
6-17 and Figure 6-18). The current structure of the “run mode” does not allow the
arrows to be displayed and therefore DENIM-Comment-Mark is used to indicate a
comment’s position by displaying the comment ID on DenimSketch (see Figure 6-22).
DenimCommentMark is created as an extension to DenimIntrinsicComponent since an
intrinsic component can consist of components rather than the GraphicalObjects.
Figure 6-25 DenimComponent extending DenimIntrinsicComponent
To support the use of comments, in addition to the modification to DenimLabel and
creating the DenimCommentMark, a number of tools and interpreters have to be
modified or created to allow stakeholders to use such tools to draw strokes which can
be handled by interpreters to create comments.
6.4.2.2. COMMENT TOOLS
ToolsArea is the container for all the tools in DENIM. Tools are extensions of the java
swing JPanel which are placed in the ToolsArea and can be selected with a mouse click
or a tap of pen on the tablet. Once a tool is selected the mouse cursor changes to the
selected tool and the tool in the ToolsArea is set to invisible. To drop the tool back into
the ToolsArea, the user can click on an empty area of the ToolsArea or on another tool
to exchange them, for example exchange pen with hand. Once a tool is grabbed, a set
of interpreters associated to that tool will be activated to handle strokes created by that
tool.
Amir M Naghsh 101
Figure 6-26 Introducing new tools in Gabbeh
Two tools are added (see Figure 6-11) to the toolbox in addition to the original tools
introduced in DENIM. The new two tools for annotation extend class tool (see Figure
6-26). The comment pen is added for creating and editing comments while the
comment view and filter tools are created for managing the visibility of comments.
CommentInterpreter and CommentFilterInterpreter are the two main interpreters
created to enable each tool to perform their actions. The following sections describe
these interpreters in addition to the modified ArrowInterpreter.
6.4.2.3. COMMENT INTERPRETER
To provide the comment facility in Gabbeh a number of interpreters have been added
to DenimSheet in addition to the ones from DENIM. CommentInterpreter is added to
DenimSheet once Gabbeh is started and became enabled once the black pencil for
creating comment is grabbed.
Once the CommentInterpreter is enabled it can handle drawn strokes to recognise if the
user is using the comment pencil tool to create a comment in a free-ink style or wants
to open the comment dialogue box to create a text comment. As explained before
DenimLabel is used to create a comment which consists of ScribbleText or TypedText
(see Figure 6-6).
Amir M Naghsh 102
If the comment pencil tool is used to create comments in a free-ink style the
CommentInterpreter determines whether strokes drawn on the DenimSheet should be
treated as a new comment or should be added to an existing comment. The
CommentInterpreter contains DENIM’s ScribbledTextInterpreter which converts
TimedStrokes into one ScribbleText object which then can be used to create a label or
comment in Gabbeh. For example “Blue” in Figure 6-12 is a ScribbleText object
constructed of two TimedStrokes “B” and “lue”. The ScribbledTextInterpreter also
determines whether bordering strokes are close enough to an existing ScribbleText to
become part of it or not. In the case of the “Blue” example the second TimedStroke
was drawn close enough to the first one to create one ScribbleText.
Once the ScribbleText is created it is not possible to edit it. The current structure of
DENIM does not allow editing TimedStroke and therefore it is not possible to edit
comments in free-ink format created in Gabbeh. Any modification to StrokeAssembler
in Satin which generates TimedStroke to enable editing of strokes would change the
underlying structure of Satin and therefore DENIM. This in turn would require major
modifications to both of them which is far beyond this work.
Therefore in this version of Gabbeh to edit a comment that is created in free-ink style,
it has to be deleted and created again. However, if the comment tool is used to create a
TypedText comment, it is possible to use the same tool to open a dialogue box to edit
the text or colour for that comment.
The structure of DenimText as defined earlier only allows one of the forms of scribble
or typed text. In addition, interpreters handling TimedStroke in DENIM and Gabbeh
can only process them as part of a ScribbleText object. Therefore, it is not possible to
construct a comment using a ScribbleText and TypedText.
It is not possible to merge the two types of scribble and typed text into one comment
but it is possible to convert the scribble text into typed text at any time. The comment
pencil tool can be used to open the comment dialogue box and therefore
CommentInterpreter can convert a ScribbleText comment (free-ink style) into
TypedText comment or edit a TypedText comment as well as editing the colour of
comments of both types.
6.4.2.4. ARROW INTERPRETER
Gabbeh provides annotating arrows in addition to navigational and organisational
arrows introduced in DENIM. The annotating arrows represent a link between a
comment object and a GraphicalObject or a location within the visual boundaries of a
Amir M Naghsh 103
GraphicalObject. Therefore, DENIM’s ArrowInterpreter has been extended to
determine whether or not to recognise a stroke created by a comment pencil tool or
normal DENIM pencil as an arrow and further to interpret it as an annotating arrow if
it is originated from a comment. If the stroke is drawn from a GraphicalObject for
example a label and ends in a comment, the ArrowInterpreter does not interpret it as an
arrow. When the stroke is recognised as an annotating arrow, the arrow’s colour will
be changed to match the comment’s colour and also a DenimCommentMark instance is
inserted at the destination point that the arrow is pointing at. The DenimCommentMark
instance is only visible in the “run-mode” to allow the user to identify the point that a
comment is referring to.
As explained in section 6.2.6 the comment facility should allow for an annotation to be
added to any component (e.g. links) in the design model. Since the arrow itself is not a
GraphicalObject, it is not possible to associate a comment to an arrow by using an
annotation arrow. The arrow is the extension of TimedStroke which is redrawn every
time DenimSheet is rendered (for example when it is panning or zoomed). Modifying
arrow structures is not possible at this stage as it effects too many classes and therefore
the requirements for adding comments on links is omitted from the list of feature
which are needed to be developed for this version of Gabbeh.
6.4.2.5. COMMENT FILTER INTERPRETER
To support filtering and retrieving comments it is essential to be able to make all the
comments invisible and only display the comments that a user is interested in.
Therefore to manage the visibility of comments the CommentFilterInterpreter was
implemented. The CommentFilterInterpreter manages the visibility of comments
through a dialogue box implemented for this purpose.
The CommentFilterInterpreter is associated with the filter tool which has already been
described. This version of Gabbeh allows stakeholders to use the filter tool to manage
the visibility of all comments associated to the DenimSheet or only a collection of
comments associated to a selected DenimPanel. Once the tool is used to dispatch a
stroke, CommentFilterInterpreter determines the location that stroke originated from. If
it originated within a DenimLabel then Gabbeh allows the stakeholder to manage the
visibility of comments for the parent Panel of that Label, otherwise if it has been
originated somewhere on the DenimSheet then it would be allowed to control the
visibility of all comments attached to the sheet. If the stroke originated within a
DenimSketch the interpreter does not process it and passes it to the next interpreter in
the queue or attaches it to the child GraphicalObject.
Amir M Naghsh 104
The effectiveness of the CommentFilterInterpreter is highly influenced by the user
interface of the control dialogue box. The current interface only allows filtering
comments based on their colours since no other identifier is associated with comments.
In addition to time and resource limitation, the requirement for the filter and view tool
was shallow and abstract. Therefore, a simple interface was implemented to introduce
such a possibility as part of the commenting facility and allows the stakeholders to
explore the requirement for such a tool by using it when it is needed.
6.4.2.6. RUN MODE
To allow comments to be displayed in the run mode, DenimRunSheet and
DenimRunWindow are both modified. As can be seen in Figure 6-21 two buttons are
added into the menu bar of DenimRunWindow. One of the buttons is to enable
stakeholders to add new comments and another is to hide and view comments when
needed. Since the DenimPanel associated to a collection of comments needs to be
visible at all times, a JPanel has been added beside the DenimRunSheet to display
comments. Using two panels to display comments allow the DenimRunSheet panel to
display a static view for the DenimPanel at all times while the comment panel could be
scrolled to view all the comments associated with a specific DenimPanel.
However, using two separate JPanels does not allow drawing the arrows to link
comment to a position within the DenimPanel. To enable arrows in “run mode” a third
panel must be placed over two panels to map a comment to its associated location by
drawing arrows. However, the current focus is not on the usability and beautification of
Gabbeh and such modifications require more resources and time. Therefore, the
possibility of such implementation is considered as future work when the core design
of Gabbeh is validated.
Since it is assumed that end-users only have access to a simple desktop, the free ink
capabilities are not supported in the run mode. Therefore it is not possible to create
ScribbleText comments. The add button only allows stakeholders to create a
TypedText comment in the run mode. Also use of two separate panels to display the
DenimPanel and comments makes it difficult to draw arrows from comments to
position on DenimSketch and therefore to indicate the location of comments on the
sketch DenimCommentMark has been used. The comment marks become visible for
the DenimPanel which is displayed, once the view comment button is selected.
Comments are displayed based on their creation time, new comments are displayed
first. The comment ID is the main source for sorting comments. This version of
Gabbeh only allows creating comment at the page level and it does not allow adding
Amir M Naghsh 105
comments at the detailed level and within the DenimSketch. However adding
comments at the detailed level and sorting and filtering comments in the run mode are
essential requirements which should be included in future versions.
6.5. CONCLUSION
This chapter has explained the implementation of Gabbeh to be able to validate the
proposed design for an electronic paper prototyping tool and refining the design
criteria and improving them as part of a formative evaluation. This chapter has
extended the previously introduced design of DENIM further to allow different
stakeholders to add arbitrary annotations in the form of comments when the model is
being designed or executed.
While the time and resource constraints were the main limitations, the development
mainly focused on implementing a prototype to investigate and explore the design
criteria of an electronic paper prototyping tool such as Gabbeh and its role in
facilitating communication and collaboration among dispersed stakeholders. It has
been noted that to investigate the usability and enhance the beautification of Gabbeh
more time and resources are required which is out of the scope of this work and can be
addressed in later stages after the design criteria is investigated and validated.
The features such as annotation on transitions, displaying arrows in the run, a mix style
of text and free-ink for comments and other features which were omitted from this
version are important and should be implemented once the resources are available, but
it is not essential at this stage as the aim is to discover to what extent Gabbeh could
facilitate the communication among dispersed stakeholders.
Amir M Naghsh 106
Chapter 7. OBSERVATIONAL STUDIES
7.1. INTRODUCTION
Once a working prototype of Gabbeh with the essential criteria was implemented it
was vital to evaluate Gabbeh to be able to validate the proposed design with the co-
operation of participants as stakeholders, as well as refining the design criteria and
improving them as part of a formative evaluation.
This chapter reports on the evaluation of the Gabbeh version 1, explaining the
objectives, evaluation approach, layout of the study and participants involved in the
study. Finally, it presents the outcome of the evaluation study and discusses the results
of the observations.
7.2. OBJECTIVES
Gabbeh was developed as a rapid prototype to discover to what extent an electronic
paper prototyping tool can facilitate asynchronous communication among distributed
stakeholders at the early stages of design.
The evaluation of Gabbeh was conducted to validate the design criteria as well as
finding ways to evolve and refine them. This emphasised the need for a formative
evaluation of Gabbeh to verify if it met stakeholders’ needs and also to explore the
design further. The use of field study as the formative evaluation was rejected due to
the limited resources that were available to this research. The cooperative approach
was also rejected, since it was possible that the facilitator influence the way that
participants use the tool and consequently the results would reflect such influences.
Therefore, to minimise the possible control and influence of facilitators and
collaborators, it was decided to conduct observational studies to explore and refine the
design. However, it should be noted that the results of observational studies can not be
generalised to all users since it is not a controlled experiment.
A summative evaluation was undesirable at this stage since the development of
Gabbeh was at its early stages of exploring and validating requirements which are
higher priorities compared to evaluating the efficiency of the design. In addition,
Amir M Naghsh 107
considering the time and resource constraints and the high cost of a usability test, it
was decided to not include usability testing to evaluate the efficiency of Gabbeh’s
interfaces. However, it is recognised that once the design proposal is validated by
stakeholders it is vital to improve the efficiency of Gabbeh’s user interface.
Members of the research team3 conducted two design sessions to identify objectives
for the evaluation study as well as arranging the layout of the study.
Three main objectives were identified for this evaluation study.
• The main and most important objective was to determine if the comments were
going to be used in Gabbeh or not.
• Communicating design ideas among stakeholders is one of the important tasks
at early stages of design. Therefore, the second goal was to discover if a tool
like Gabbeh would be used to generate any form of communication such as
discussions or dialogues between different stakeholders. Particularly, would
stakeholders use the comment facility to respond to each other in form of a
discussion?
• As explained in chapter 5, little is known on the use of annotation in designing
interactive systems while the attention has been focused on the use of
annotation in textbooks and multi authoring text documents. The final objective
for this evaluation was to discover how comments would be used when
designing a website.
7.3. LAYOUT
The evaluation study was conducted by simulating design exercises. DENIM was used
in the evaluation studies as an environment without a comment facility against Gabbeh
to explore the use of the commenting facility once it is available to stakeholders. As
previously explained this work is mainly interested on asynchronous communication
among stakeholders in a dispersed setting. Therefore, the layout was adopted in a way
to support a dispersed setting where different stakeholders were situated in two
separate locations (see Figure 7-1).
3 Research team in this thesis refers to the author of the thesis and the three supervisors of the advisory group.
Amir M Naghsh 108
Figure 7-1 Illustration of the layout of the evaluation exercises in two different location. The only communication channel
was through the email and prototype while there was no face-to-face meeting
Three participants were involved in each exercise. Participants were divided into
groups of two “designer-users” and one “end-user”. The designer-users and end-users
were located at two separated offices. The only communication channel between them
was through email and the prototype itself (see Figure 7-1).
The two designer-users worked together on a Graphic Tablet connected to a laptop.
They had access to two displays (Graphic Tablet and Laptop screen), keyboard and an
extra mouse (see Figure 7-2 and Figure 7-3). A video camera was used to record the
designer-users activities as well as discussions between them. The end-user had access
to a desktop PC to review the prototype and give feedback to the designer-users.
Another video camera was installed at the end-user’s location to record the participant
activities.
Each design exercise consisted of two sessions. Each session was between 1.5 and 2.5
hours long. Only one session was allocated to a day and each exercise was split over
two days. A design task was allocated to each of the sessions. Designer-users were
asked to perform the assigned task in the first session by using DENIM and the task
assigned to second session by using Gabbeh and same order was used throughout the
evaluation.
Each individual “designer” went through two training sessions and received
approximately one hour training on using DENIM and one hour on using Gabbeh.
Designers received training on DENIM prior to the design task allocated to DENIM
and they received training on Gabbeh prior to the allocated task to Gabbeh. The
Amir M Naghsh 109
purpose of the training was to help designer-users to become familiar with each tools’
features and with using a Graphic Tablet, as this was a new device to some
participants. The training was provided well in advance, two to three days before each
session of design exercise to allow the designer-user to arrange more training to use the
tool if the designer-user felt it was necessary.
The order of using DENIM first then Gabbeh allowed designers to use DENIM without
being aware of the features offered in Gabbeh. Designer-users became familiar with
the strengths and weaknesses of DENIM and adopted their way of working with what
it had to offer. Next, designer-users were introduced to Gabbeh and were introduced to
its comment facilities. At this stage whilst designer-users were familiar with DENIM,
they had the opportunity to decide if the new features introduced in Gabbeh are useful
to them and can improve their working method.
Figure 7-2 This is a picture taken in the pilot study demonstrating the designer-users working together using a graphic tablet and laptop connected to each other. Both designer-users were contributing to the design by using the tablet, keyboard and the mouse. Halfway through the
design, two designer-users swapped places
Amir M Naghsh 110
Figure 7-3 Picture taken from the first design exercise (group1). The picture is demonstrating one designer (left) in charge of the tablet while the other designer (right) is more distanced and works with the computer. The video record showed both designer-users discussed the problem
but only one (left) was creating the design. While the other one (right) was mainly giving feedback and making sure that the agreed design was developed and in some occasions give
assistance by typing on the keyboard
The other possible order of using Gabbeh first would allow designer-users to
experience the same weaknesses that they have discovered by using Gabbeh when
using DENIM, in addition to the shortcoming that they could have experienced due to
the absence of the commenting facility. Being aware that certain features are stripped
from the tool could have influenced the natural way that designer-users would use
DENIM. Therefore, designer-users were not introduced to Gabbeh until they had
received training on Gabbeh prior to the second session. However, designer-users were
informed at the beginning of the layout of the evaluation study and the fact that the
design tool would be different in the two sessions.
As previously explained, the main objective of this evaluation study was to discover if
the comment facility was going to be used by participants once it was available to
them. The evaluation study did not intend to investigate the effectiveness of the
comment facility on enhancing the communication. In other words, the study was not
investigating if the introduction of the comment facility might have resulted in
generating more communications or comments compared to a tool such as DENIM that
Amir M Naghsh 111
does not support such a facility. Therefore, it was decided that is not required to swap
the order of design tasks between the two sessions.
At the starting point for each exercise the designer-users were given a task sheet
describing requirements for a website and were instructed that they were to develop a
preliminary design and gather feedback from the user throughout the design session.
Design sessions lasted between two and three hours. Participants in the study were paid
a small fee for their time. After each session an informal debriefing session was held
with the designer-users, to ask for their feedback and discuss any issues. This was also
recorded.
7.4. PILOT STUDY
The first exercise was a pilot study. The designer-users for the pilot study consisted of
a member of staff of a computer science department with some experience in web
design, a professional interactive system developer and the end-user was a member of
staff from the business school. They used DENIM to design a ‘research-group’ website
and Gabbeh to design a ‘personal’ website (see Appendix B for the task sheets).
The task sheet was a brief description of the requirement for a website printed on one
side of an A4 paper. Since the tasks were brief requirements, it was vital to enable the
end-user to give sensible feedback in the form of adjustments and clarification of the
requirements to the designer-users. Therefore, the end-user was required to have a
good understanding of the described problem in the task sheet.
Prior to each session the task sheet relevant to that session was given to the end-user to
adopt the task. The first task ‘research-group’ website was generated by the research
team, considering that the end-user is a member of academic staff and should be able
to relate the task to her knowledge in the work environment. For the second task a
‘personal’ website, the end-user was asked to generate the task by writing a description
of what she would want to have in her personal website. Both tasks were chosen in a
way to relate to the end-user knowledge and allow the end-user to influence the design
by making interpretations of the task.
At the beginning of each session, the designer-users were given a task sheet, briefing a
website requirement and were instructed that any further questions regarding the task
should be forwarded to the end-user. However, they were encouraged to generate a
preliminary prototype based on the given task sheet and make inquiries about the
requirements while forwarding the first version of the prototype. Designer-users were
Amir M Naghsh 112
instructed to adopt an iterative design approach to generate a prototype in the 1.5 to 2
hours given time. They were instructed to keep the end-user informed and seek for
end-user’s feedback when appropriate.
The results of the pilot study suggested that the end-user was not engaging strongly in
the design process. In particular, the end-user made very few comments about the
proposed design. Although, there were comments made by the end-user but they were
more in form of agreements and accepting the design proposal the way it was
presented. More importantly, it took more time than it was expected for the end-user to
send her feedback to the designer-users. The time allocated to the end-user was 10
minutes for each iteration but it was exceeded every time to up to 20 minutes. The end-
user found that the given time to review the design and give feedback was not
sufficient for understanding and giving feedback on the design proposal and answering
designer-users’ questions. On the other hand due to the resources constraint it was not
possible to extend the length of the session, or to conduct the experiment over number
of days instead of one day.
7.5. REVISED STUDIES
Difficulties with response time of using a real end-user were explained in the previous
section. Therefore, it was decided that the author should play the end-user for the later
exercises. Since the author had already worked with both design tasks and was
involved in generating them, thus he had a more in depth insight to the given tasks and
better understanding of the problem. This artificial customer was intended to offer a
better simulation of an end-user.
For the remaining three exercises, six participants were recruited and trained
individually. Six participants were divided into three pairs of ‘designer-users’. The first
pair consisted of two members of staff of the School of Computer Science who, were
experienced in web design. The remaining two pairs were final year undergraduate
students of a Software Engineering Degree, who were returning from industrial
placement and were experienced with designing websites. Each group was asked to
design a ‘visit Sheffield’ site using DENIM and a ‘research group’ website using
Gabbeh (See Appendix B for the design tasks). The same order of tasks was
maintained throughout the study.
Amir M Naghsh 113
7.6. RESULTS
This section reports on the result of the design exercises which have also been reported
and presented at the DSVIS 2005 conference (Naghsh et al, 2005).
7.6.1. WERE COMMENTS USED?
The results showed that all participants used the commentary feature of Gabbeh. Also,
the content of emails was reduced to a few lines informing the end-user of a new
update of the design and in some cases informing the end-user of which comments
they needed to check within the design. For example, one of the emails posted by
designer-users was only a few lines:
“We have made the amendments....We were slightly unsure about couple of
points. Please read the comment in red”.
The same designer-users sent longer emails when using DENIM:
Just need to clear up a few issues.
1. We dont understand point 5 of your requirements
Can you please explain this?
2. Can you please provide us with more details about requirements 4 and 5 on
the original "Website Design Task Brief". We have provided a page with a list
of group projects. We also provided links to these projects from researchers
pages. Does this meet the requirement?
To investigate the use of comments it was required to calculate the number of
comments created in each session. To calculate the number of comments created in
Gabbeh, the number of individual comments attached to the sheet was calculated. For
example, Figure 7-4 shows five comments in Gabbeh. In addition, to calculate the
number of comments attached to emails, the content of each email was broken down
based on the subject to identify individual comments. For example, the above email
which was sent when using DENIM was divided into four comments as follows:
• We dont understand point 5 of your requirements
• Can you please provide us with more details about requirements 4 and 5 on the
original "Website Design Task Brief"
• We have provided a page with a list of group projects
Amir M Naghsh 114
• We also provided links to these projects from researchers pages. Does this
meet the requirement?
Sessions (Gabbeh – DENIM) Group1 Group2 Group3 Total of groups
Gabbeh: End-User 10 6 6 22
Gabbeh: Designer-Users 14 10 11 35
Gabbeh: 24 16 17 57
DENIM: End-User 7 4 6 17
DENIM: Designer-Users 4 1 3 8
DENIM: 11 5 9 25
Total (of one group) 35 21 26 82
Table 7-1 This table presents the number of comments created for each session as well as presenting the overall comments
made while using Gabbeh in comparison to DENIM
The results presented in Table 7-1 provide a clear answer to the first question that
commentary tool is likely to be used when it is available (also see Figure 7-4). The
results in Table 7-1 shows that designer-users made over four times more comments
when using Gabbeh compared to DENIM.
Amir M Naghsh 115
Figure 7-4 The comments were used in all exercises. This figure is demonstrating the use of comments as Gabbeh was
used to develop a prototype (Group 2, Session 2)
Reviewing video recordings revealed that the Gabbeh commenting facility allowed
designer-users to create comments once they recognised the need for it during the
design process. In DENIM, designer-users did not keep any record of comments during
the design process, and they would wait until the design reached an appropriate point
to send to the end-user and write the comments in an email. It was observed that at the
time of sending the email it was not an easy task for designer-users to remember all the
design issues during the design. For example after the first version of design was ready
by group 1, one of the designers started typing an email to the end-user and half-way
through the email he asked the other designer to help him to remember the design
issues that they had already discussed and to confirm what they needed to decide to
send to the end-user.
The results showed that although designer-users created more comments using
Gabbeh, it was not the case that the commentary tool was always used when it might
have been appropriate to do so. For example, analysis of the video recordings revealed
the following discussion:
Designer 1: … that would link to a research page which is then going to have
all the researchers name, I think that’s what they say at the moment. Does this
Amir M Naghsh 116
make sense, what do you recommend? Do you want to take a different
approach?
Designer 2: I would go a different way to represent this, the only thing that I
think of is to have all the researchers name on the home page … … But put it
like this and we will see what they (refereeing to user) going to say. We are
going to get some feed back anyways. … (group 1, session 1)
In this case, the designer-users went ahead without adding any comments, but assumed
that the user would respond if they were particularly dissatisfied. However, when
working with DENIM, one of the same two designer-users was heard saying: “Now we
need a commenting tool to ask user what they think [referring to a specific part of
design]”.
7.6.2. WERE DISCUSSIONS SUPPORTED?
To answer this second question all the comments within emails and the prototypes
were reviewed. As explained in chapter 6, each comment has a reference ID (see
Figure 7-5). This ID is an automated number, which is generated in Gabbeh when each
comment is created. The larger ID number means that comment was created later in
design process, which helped to identify the possible threads.
In addition, to considering the context of comments for each page, the recorded videos
were also reviewed to learn about the time and situation that the comment was created.
This made it possible to identify threads of comments in the form of discussion.
Results presented in Table 7-2 shows that there were six designer-initiated discussions
made in Gabbeh (see Figure 7-5 and Figure 7-6) and two in DENIM. For example, the
following thread of comments was made in group 2, session 2 when participants were
using Gabbeh. The thread consists of three comments starting with the designer-users:
Designer-users: “We provided a photograph of each researcher together with
links to their profile and a quick overview of their interests at here”
User “can we have a group pic instead of individual ones here?”
Designer-users: “We provided an area for some text about the group as well
as a group photograph.”
Amir M Naghsh 117
Figure 7-5 This thread of two was identified in the design developed by the group 2. The designer-users have changed
part of the design in response to users comment. Here the design-users are explaining the new design which receives the
user’s approval
Figure 7-6 This thread was identified in the design from the pilot study. The designer-users explain the design while
requesting the user’s approval
Sessions (Gabbeh – DENIM) Group1 Group2 Group3 Total of groups
Gabbeh: End-User 2 2 3 7
Gabbeh: Designer-Users 2 2 2 6
Gabbeh: 4 4 5 13
DENIM: End-User 2 1 1 4
DENIM: Designer-Users 1 0 1 2
DENIM: 3 1 2 6
Total (of one group) 7 5 7 19
Table 7-2 List of identified discussions for each session divided in two main groups of Designer-initiated or User-initiated discussions
Amir M Naghsh 118
Reviewing the video recording and the results of the analysis presented in table Table
7-2 clearly answered the second question and showed that the comment tool was used
to generate distinct discussions where one or more comments were made in response to
another. In addition, designer-users used the colour scheme to separate their comments
from the end-user comments. For example group 1 used red colour for all responses
whilst group 2 used different colours (but not the same as the end-user) for different
iterations.
7.6.3. HOW WERE COMMENTS USED?
To answer this question each individual comment was identified and the recorded
videos were reviewed to retrieve the time that the comment was made, the author(s) of
the comment and the key points of the discussion that was happening at the time of the
creation of the comment. The recorded videos were also used to break comment
objects, that had multi sentences, down into smaller comment-phrases which
represented a single specific issue.
Each individual comment was printed out on a piece of paper and all the information
related to that comment such as the time, author and other key points was written on
that paper. This was done to allow comments to be arranged in different ways. A big
surface was used to display all the comments. The comments were then clustered to
identify categories of interest. The instructive comments such as “please ignore the
first comment” were not considered in this clustering since they were used to overcome
some of the functional limitations of Gabbeh.
7.6.3.1. CLARIFYING
The results showed that a number of comments were made particularly to explain the
design. The designer-users made comments to explain the functionality or the dynamic
behaviour of a part of design. This avoids the designer-users developing the design in
detail, whilst indicating how the design may be developed. Figure 7-7 shows an
example of this category. The two comments in brown and blue are used to clarify the
design. The brown comment is simply explaining what the end-user should be
expecting from this page:
“This page holds an image of the researcher and their interests in more detail,
also included on this page is a list of their interests and current work. The
courses that they teach on are also included here “
While the blue comment, is mainly explaining the functionality of a specific part of the
design that is indicated by an arrow pointing to that specific area of design. Designer-
Amir M Naghsh 119
users instead of designing a new page have used the comment to explain what the end-
user should expect to see after clicking on the link.
“Link will grab all publications by name and year supplied as an attribute of
the link”
Figure 7-7 Examples of clarifying comments (group 2, session 2). The blue comment is clarifying the design. Designer-users have made comment to explain what would happen once
the user clicked on one of the items in the publication lists. It has allowed designer-users to avoid designing a new page to simulate the dynamic behaviour and its functional output in the
form of a new page.
7.6.3.2. VERIFYING
The results showed that in addition to the use of comment for clarifying the design,
designer-users made comments to report on specific parts of a design and request for
end-user verification. Also they might ask for the confirmation to changes that have
been done based on a previously created comment by the end-user.
The results suggested that verifying comments are concentrated on specific issues
which require end-user feedback. Figure 7-8 shows an example of a verifying
comment. The end-user has requested to have link to publication and be able to view
them.
“End-User: Have a link to website and the location of publication, to have view
of the paper(s).”
Amir M Naghsh 120
In response, the designer-user have changed the design to include what the end-user
has requested and made a comment highlighting the change as well as seeking the end-
user’s confirmation on the change.
“Designer-User: Here we provide links to each paper, is this what you
wanted?”
Figure 7-8 An example of verifying comments as part of a thread
7.6.3.3. EXPLORING
Designer-users used comments to ask questions that would help them to obtain more
detail about the end-user’s needs or desires. They used comments to explore the
design. Exploring comments were made to obtain feedback on design on a higher level
or when designer-users could not understand the problem to propose any solution for
it, whilst verifying comments were mostly used to gain users confirmation on a
specific part of the design. For example designer-users who could not decide on what
they have to design for a given requirement, when they sent the first version of their
work, they also asked the end-user in the email:
“Can you please provide us with more details about requirements 4?”
Comments on clarifying, verifying and exploring categories were all made by the
designer-users.
Amir M Naghsh 121
7.6.3.4. ALTERING
Observation showed that the end-user made comments to alter the design through the
design process. Altering includes comments on a change or correction of the existing
design or an instruction to include a new element in the design. Figure 7-9 shows a
comment made by the end-user requesting the designer-users to add a new page to the
design. The red comment shows the end-user confirming the need for having a list of
publication but wants a separate page to display the publications.
Figure 7-9 The end-user requesting a new page to be added - An example of altering the design
7.6.3.5. CONFIRMING
Feedback provided by end-users approving or disapproving of aspects of the design. It
was mainly used in response to verifying comments made by designer-users. The
yellow comment in Figure 7-8 is a response to the red comment by design-users
requesting end-user feedback. The confirmation is a short “yes, that’s what I wanted”
comment.
7.6.3.6. UNDERSTANDING
The results showed that the end-user cannot always make sense of a design proposal
sent to him/her. Since there could be a lack of clarifying comments about different part
of design, end-user can often find it difficult to understand or relate the design to a
problem. Therefore, comments were used in form of questions to help end-users to
understand the design better to relate it to his/her needs.
The red comment in Figure 7-10 shows an understanding comment. The first line of
the comment is simply asking the designer-users what this box with the cross in it is.
After investigating the creation time for this comment and other comments that were
created afterwards, the designer-users responded by adding a comment (comment ID
Amir M Naghsh 122
32) referring to the box as “pic/graphic” as well writing the word “picture” on the box
itself.
Figure 7-10 Red comment shows an example for understanding comments helping the end-user to understand
the provisional part of design
Altering, confirming and understanding comments were made by end-users.
7.6.3.7. ORGANISATION
In addition to above categories, a few comments, two in total were used to discuss the
organisation of the design process itself. For example, designer-users created a
comment to instruct the end-user what to check in the new version of their design:
Designer-user: “Please read the comment in red.”
7.6.3.8. OVERVIEW OF COMMENTS IN EACH CATEGORY
Table 7-3 shows number of comments in each category for the revised evaluation
studies. There were two organisation comments that are not included in this table.
Since the author of the thesis simulated the end-user in the revised evaluation studies,
the results of the pilot study was also considered to test whether it was possible to
categorise them using the suggested classification. Table 7-4 shows the number of
comments associated to each category for the pilot study. The result shows that it was
possible to categorise the comments created by the real end-user in the pilot study
using a same classification (see appendix D for all the comments in each category).
Amir M Naghsh 123
Designer-users Author as the End-User
Clarifying Verifying Exploring Altering Confirming Understanding
DENIM 1 2 4 13 1 3
Gabbeh 22 10 3 15 3 3
Total 23 12 7 28 4 6
Table 7-3 Total number of comment-phrases associated to each category for the three evaluation studies
Designer-users Recruited subject as End-User
Clarifying Verifying Exploring Altering Confirming Understanding
DENIM 1 1 2 2 1 2
Gabbeh 5 4 1 4 4 1
Total 6 5 3 6 5 3
Table 7-4 Number of comment-phrases associated to each category for the pilot study
7.7. DISCUSSION
Analysis of the video recordings and debriefing sessions showed a number of concerns
with the current design of Gabbeh. Three important issues are considered below.
7.7.1. MANAGING LARGE NUMBERS OF COMMENTS
As a prototype evolves, the number of annotations and comments attached to it also
increases and it becomes more difficult to manage them. Observations showed that of
the four groups of designers-users only two groups used the filtering tool provided.
One group of designer-users reported that they found it difficult to deal with the
positioning of the comments in the ‘design-mode’. Instead, they preferred to have all
the comments shown at one place. To overcome the problem these designer-users
made all the comments invisible on the design sheet and instead used the ‘run-mode’
Amir M Naghsh 124
browser to view the comments for each page, before returning to the design mode to
continue their work.
In the debriefing discussion following the exercises designer-users explained they used
the “run mode” to view comments because they could identify the sequence of
comments by going through comment IDs. The review of video recording revealed that
all three groups of designer-users used the ‘run mode’ to view comments after the first
iteration, once they had received the end-user comments.
7.7.2. IDENTIFYING AND MANAGING DISCUSSIONS
Analysis of the video tapes and of the comments made, showed that there were distinct
discussions in which one or more comments are made in response to another.
However, it was difficult to identify these threads without referring back to the video
recordings to clarify the relationships between the comments. Clearly, there is a
requirement to identify comments as replies to previous comments. The colour scheme
of comments was used in a variety of ways, particularly for separating designer
comments from end-user comments. One group of designers used different colours for
different iterations of the design. Thus, their second email to the end-user consisted of
the single instruction: “Please read the comment in red”.
7.7.3. SUPPORTING PROGRESS AWARENESS AND TRACKING
One problem that became apparent after a number of iterations was the need to develop
some form of version control over the prototype and the comments. For example, an
end-user might make a comment requesting a change, which is then implemented by
the designer-users. If the comment is not removed from the design record, then it may
not refer to anything in the current design. Clearly, there ought to be some way of
managing the history of the comment to show that it has been addressed.
This suggests that each comment could be regarded as having a life-cycle starting with
its creation. This life-cycle might be different for comments from different categories.
Understanding such life-cycles will be an area for further work.
7.8. CONCLUSION
The exercise showed that the comment facility is used once it is available to designer-
users. It also has suggested a classification for the comments made in Gabbeh and
DENIM. The evaluation study has suggested that comments created in designing
interactive systems such as website could be divided into seven categories as follows.
Designers-users made comments for:
Amir M Naghsh 125
• clarifying the design detail;
• exploring unspecified and not well understood requirements;
• verifying the parts that were already designed.
End-users used comments for:
• understanding by asking questions about the design;
• altering the design;
• confirming the changes in the design.
Additionally, some comments were used to discuss the organisation of the design
process itself.
The evaluation identified that it is not possible to discover to what extent Gabbeh could
enhance the communication among of stakeholders. To be able to make strong
conclusions on the use of discussions through annotations in designing interactive
systems, the need for a longitude study with involvement of a real end-user was
recognised. However, the evaluation suggested that this version of Gabbeh requires
enhancement in important areas of the management of large collections of comments,
identifying discussions and understanding the life-cycle of comments within a larger
design process. The following chapter reports on further design studies and
investigations on enhancing Gabbeh to meet some of the findings mentioned in this
chapter.
Amir M Naghsh 126
Chapter 8. SUPPORTING DISCUSSIONS IN
GABBEH
8.1. INTRODUCTION
As explained previously, Gabbeh was developed to discover to what extent an
electronic paper prototyping tool with commenting facility could enhance
communication and collaboration among different stakeholders. The formative
evaluation of Gabbeh version 1 showed that the new comment facility was used once it
was made available to users and designers. However, the observational studies
revealed the difficulties involved in simulating the design task and recognised the need
for a longitudinal evaluation study.
Particularly, the observational studies were found insufficient to discover the extent
that commenting facility could have been used to enhance communications through
supporting dialogues and discussions between dispersed stakeholders. The analysis of
observational studies identified three main areas in Gabbeh which made it difficult to
be used in a larger design task with many more iterations.
• progress awareness,
• identifying and managing discussions (supporting discussions),
• managing large numbers of comments.
To maintain the reliability of Gabbeh through the evolutionary process, it was essential
to keep the number of changes to minimum whilst improving the prototype. This
chapter explains the Gabbeh version 2 that was used in the longitudinal study.
Although, there are other areas of interest such as usability issues and features that the
researcher would liked to incorporate into Gabbeh, due to the time and resources
limitation a decision was made to not consider them at this stage.
Amir M Naghsh 127
8.2. ENHANCING THE DESIGN OF THE COMMENTING TOOL
8.2.1. PROGRESS AWARENESS
Designing interactive systems such as websites is an iterative process allowing a
prototype to evolve through many iterations during the design process. The
observational studies showed that the number of comments associated to a design grow
large in just one or two iterations. Such a growth would be even bigger when the size
of the design and the number of stakeholders increases. This means stakeholders need
to deal with a large number of comments whether created by them or in response to
them to be able to track the progress of the design.
Version control was identified as one essential requirement which has to be supported.
There are a number of researchers (Appan et al., 2005; Bottoni et al., 2004; Brush,
2002; Cadiz et al., 2000; Jung et al., 2002; Leland et al., 1988; and Weng et al., 2004)
investigating the version control for systems supporting asynchronous annotations in
dispersed settings. To enable the stakeholders track the design process, they need to be
aware of the comments created in different iterations. They are also required to be
aware of how their comments were received by other stakeholders, if there is any
response to their comments or if their comments have been incorporated into the
design.
Therefore, it is essential to provide additional information associated to a comment to
be able to identify when and at what iteration a comment has been created, who has
created the comment, and if it has been addressed or responded to. It should be
possible to distinguish new comments from the old comments. The progress awareness
should allow a stakeholder to track comments to identify whether there are responses
to a comment or if it has been incorporated into the design or it has not been dealt with
yet.
The observational studies also indicated that designer-users and end-users needed to be
able to associate comments that were created in the run mode to a specific position on
the page. The stakeholders needed to map the comment to a location in order to direct
other stakeholders’ attention to a specific part of the design that they were providing
feedback on, or a part of the design that had been changed.
8.2.2. SUPPORTING DISCUSSIONS
As explained in chapter 5, the conducted studies in supporting annotation, for example:
Quilt (Fish et al. 1988); MADCOW (Bottoni et al. 2004); Space Pen (Harmon et al.
Amir M Naghsh 128
1996) and AnnoteImage (Brinkley et al. 2002) have paid little attention to supporting
discussions through annotations.
However, the results of the evaluation study in chapter 7 identified threads of
comments in the form of distinct discussions. However, it was very difficult to identify
these discussions and threads since the first version of Gabbeh did not allow for a
comment to be associated to another comment. Therefore, video recordings were used
to clarify the relationships between the comments. For example Figure 8-1 shows a
collection of comments created by designers and users associated to one page in a
design model from the evaluation studies. Reviewing the video recording revealed that
there is a relationship between comments 5, 25 and 28 in the form of a discussion. The
investigation showed that end-user created comment 25 in response to designers’
comment 5, and comment 28 was created by designers as a reply to comment 25.
Figure 8-1 A discussion example from observational studies
A commenting tool should allow stakeholders to create comments as replies to other
comments. Two such comments should be linked to each other to allow the
stakeholders to identify the relationship between them in later stages of the design.
Amir M Naghsh 129
In addition, it was noticed occasionally that responses to a comment consisted of few
words of agreeing or disagreeing. Comment “Yes, that’s what I wanted” in Figure 8-2
is one example of using a short comment to approve the design. Therefore the need for
a simple way of supporting or disagreeing with a comment was recognised to allow
stakeholders to respond to a comment without the need of creating a new comment.
The design of the tool should allow each stakeholder to agree or disagree with any
comment and further it should be able to determine the number of agree and disagree
votes for an interested comment.
Figure 8-2 Short comments were used to approve the design
8.2.3. MANAGING LARGE NUMBERS OF COMMENTS
The number of comments and annotations associated to a prototype increases rapidly
as the prototype evolves and therefore it becomes difficult to manage the comments.
The design of an electronic paper prototyping tool such as Gabbeh should enable
stakeholders to retrieve comments which share some common properties during the
design process.
The comment’s properties can be explained in the form of additional data associated to
comments. For example, the name(s) of the person who has made the comment, the
date and time that comment was created at the iteration that comment was created in
and the colour of comment can be considered as some of the properties that can be
associated to comments. For instance, in one of the previous observational studies,
different stakeholders used different colours to distinguish between each other
comments during the design process.
Therefore, a more featured filtering tool is required to allow the stakeholders to
retrieve comments based on one or a collection of common properties. The use of such
a filtering tool is strongly influenced by its supported features as well as the
effectiveness of its user interface. However, as explained earlier, issues regarding the
usability of the filtering tool is out of this work’s scope.
Amir M Naghsh 130
8.3. GABBEH VERSION 2
The observation of the evaluation studies indicated that designer-users were more
comfortable with the positioning of the comments in the run mode, and the run mode
was used more often to review the comments in one screen whilst the other screen
(graphic tablet) was used to alter the design. Since all of the designer-users were
interacting with two screens at the time, one to alter the design (design view) and one
to interact with comments (run view), it was decided to design Gabbeh version 2 by
assuming that designers will use two screens at the same time. The design mode is
displayed on the graphic tablet and the run mode is displayed in the standard screen
allowing designers and users to interact with comments. This prevents the design view
from getting crowded with a large number of comments which would cover the actual
prototype.
Furthermore, this section explains the further development of Gabbeh and describes
the implementation of essential design criteria to enable Gabbeh to facilitate
discussions in the run view.
8.3.1. PROGRESS AWARENESS
8.3.1.1. NAME DIALOGUE BOX
To distinguish comments from different stakeholders a new feature is introduced into
Gabbeh to allow a user to enter his/her name. Every time Gabbeh starts up an input
dialogue box will pop up requesting the user to input a name (see Figure 8-3). The
current version of Gabbeh does not check if it is a returning user or not and the users
should make sure to use same user name every time using the Gabbeh. The provided
name by the users will be associated with the comments that are created by that user.
At this stage of the development more attention has been given to adding the necessary
features into Gabbeh in the simplest and quickest way instead of exploring the best
suitable design for such features.
Amir M Naghsh 131
Figure 8-3 The dialogue box for allowing the users to provide a name
8.3.1.2. REVISION
The structure of Gabbeh version 1 does not support a client-server application that can
support dispersed stakeholders either to allow them to connect to a server and work
with a shared version of the prototype model, or to be able to connect to the server to
upload their version that can be merged into the original version available on the
server. Therefore it was assumed that stakeholders would take turns to use the Gabbeh
and modify the prototype. A simple version counter is also provided into Gabbeh to
enable the stakeholders to identify at what stage in the design a comment was created.
Every time Gabbeh is started up, a dialogue box is displayed to confirm the correct
revision. It is also possible to change the revision number through the pie menu. The
revision number allows the users to decide if a new iteration or revision has started and
will change the revision number once the prototype is loaded into the Gabbeh before
altering the design. This new revision number will be associated with all the generated
comments by the user at that stage.
The usage of the revision option is left open in this version of Gabbeh to allow the
stakeholders to adapt it to their way of working.
8.3.1.3. MAPPING COMMENTS IN THE RUN MODE
The observational study indicated the importance of being able to associate comments
to a point within a page in run mode. Further development of Gabbeh allows users to
select a comment and then associate the comment to a position on the page.
Amir M Naghsh 132
To associate a comment to a point on the page, first a comment has to be selected. This
can be an existing comment or a new comment that just been created using the add
comment button as explained in chapter 6. Once the comment is selected, then the link
option in the tool bar should be selected and enabled. When the link button is set to
enabled, the user can use the mouse cursor or the pen on the graphic tablet to indicate
the position that comment has to be associated with. After selecting the position on the
page, a coloured number same as the comment ID will be displayed at that position
(see Figure 8-4).
Figure 8-4 The new associated point to a comments is highlighted on the page
Figure 8-4 shows a selected comment as part of a discussion with the comment id “9”.
The comment is associated to an area on the page which is highlighted by the coloured
number. However, since the comment also belongs to a discussion, the areas associated
to the first comment that has initiated the discussion are displayed too. In this case it is
the areas highlighted by “2” associated to the first comment in the discussion with the
red colour. This is intended to allow stakeholders to identify where initially the
discussion related by clicking on any comments belonging to the discussion.
Amir M Naghsh 133
8.3.2. SUPPORTING DISCUSSIONS
8.3.2.1. SELECTING A COMMENT
A click of the mouse on the comment selects a comment. When a comment is selected,
it becomes highlighted with a blue border. In addition, the comment id, author name,
revision number and the date that the comment was created become visible at the top
of the comment (see Figure 8-6 and Figure 8-5).
The location marks associated to all comments are displayed when no comment is
selected in the run view (see Figure 8-5). However, as it is shown in Figure 8-6, when
a comment is selected, only marks associated to that comment are displayed on the
page. This allows the stakeholders to identify which part of the design the comment is
referring to. The design of the CommentMark in the second version of Gabbeh consists
of the comment id surrounded with a coloured line representing the comment colour.
An alternative design could include coloured numbers with different sizes and maybe
animated.
Figure 8-5 Comments displayed in the run-view when no comment is selected.
Amir M Naghsh 134
Figure 8-6 Displaying a comment when it is selected
8.3.2.2. REPLY TO COMMENTS / EDIT COMMENTS
Double clicking on any comment would open a comment dialogue box which allows a
user to either edit the comment or reply to the comment (see Figure 8-7). When a user
double clicks on a comment, two types of information associated to the comment are
used to compute whether the user can edit the comment or reply to it. The two types of
information are: the name of the person who has created the comment and the revision
number of the comment.
When a user double clicks on a comment, if it has been created by him/herself within
the same revision then it is possible for him/her to edit the comment (see Figure 8-7).
Otherwise, if the comments has been created by someone else or it has been created in
an older revision then the dialogue box would be used to allow the user to reply to the
comment. This allows a user to use same functionality to edit his/her recent comments
whilst being able to use it to reply to other users’ comments.
Figure 8-7 Editing an existing comment
Amir M Naghsh 135
The comment created as a reply will be displayed under the source comment within a
grey background to indicate that this comment was a response to source comment (see
Figure 8-8).
Figure 8-8 comment inserted as a reply to an existing comment
8.3.2.3. INTERACTING WITH DISCUSSIONS
To decide on how to relate the comments in the form of a discussion in Gabbeh,
community forum systems which support discussions such as vBulletin4 were
considered. Consequently, a tree hierarchy model similar to that commonly used in
community forums was selected.
Related comments in the form of replies are grouped together with the first comment at
the top and the rest of the replies grouped together under the first comment with a grey
background. This group of comments is called a discussion in this thesis. A discussion
consists of at least two comments. The first comment in the discussion, the one which
has initiated the discussion is always visible at the top of the group.
Two views are supported by Gabbeh for a discussion. In the closed view, the first and
latest comments are displayed. Figure 8-9 shows the closed view of a discussion. The
“+” sign next to the first comment in a discussion indicate the closed view for a
discussion can be expanded.
4 See http://www.vbulletin.com/
Amir M Naghsh 136
Figure 8-9 Closed view of a discussion
To expand the discussion body, the user can click on the “+” sign and open the
discussion. Figure 8-10 shows an expanded view for a discussion with the rest of the
comments between first and last comment. The “+” sign changes to “-” to indicate the
expanded view for the discussion. To close the discussion back, the user has to click on
the “-”.
Figure 8-10 Expanded view of a discussion
Amir M Naghsh 137
8.3.3. MANAGING LARGE NUMBERS OF COMMENTS
8.3.3.1. FLAG COMMENTS
It was noted that the commenting tool should allow each stakeholder to agree or
disagree with a comment. In addition, as explained earlier, the current structure of
Gabbeh does not allow integration of progress awareness to enable stakeholders to
track the design process and indicate if their comments have been responded to.
Due to time and resource constraints which have made it not possible to implement the
whole proposed design, it was decided to provide a flag option in this version of
Gabbeh to allow the stakeholders to highlight a comment when it is required. For
example, if a stakeholder wants to show his/her support towards a comment, or
indicate that a comment has not been read. The usage of the flag option was left open
to user to adapt with their way of work.
To flag a comment, the user can right click on the comment. Once a comment is
flagged, the letter “F” in colour red is displayed next to it (see Figure 8-11). It is
possible to set or clear the flag for any comment created at any stage of design
regardless of who has been its author.
Figure 8-11 A comment with flag
8.3.3.2. FILTERING TOOL
A more featured filtering tool than the one available in the first version of Gabbeh is
supported to allow stakeholders to retrieve comments based on the colour, name, flag
and revision number. Gabbeh set the default value of the version to current and colours
to all. The “current” value for the revision as it can be seen in Figure 8-11 only retrieve
and display comments and discussions created or updated in the current revision or the
one before it. This way Gabbeh displays the most recent comments and discussions.
However, the user can click on a version button to change the revision value to blank
which means all comments and discussions will be displayed. Also comments created
in specific revisions can be retrieved by entering the revision number into the revision
Amir M Naghsh 138
box then clicking on the update button which is indicated with a face icon (see Figure
8-12). Revision numbers can be entered as a collection of numbers such as “2,4,7” or
in the form of “3-6” meaning all revisions from 3 to 6.
Figure 8-12 filtering toolbar
Gabbeh allows comments to be filtered and retrieved by its creator as well. Stakeholder
name can be entered into the name text box and then click on the update button. In
addition, comments can be filtered based on their colours. When the colours option is
selected in the filtering toolbar, it means that all the colours will be displayed. To select
specific colours, the colours check box should be deselected. This will enable the other
colour checkbox and allow the user select the interested ones.
An option is also supported to allow the users to filter the comments with a flag. An
extra option provided is the “all” button, that when it is selected, all the created
comments associated to the design model will be displayed into the comment panel.
This is intended to allow the stakeholders to view all comments in the run view.
8.4. CONCLUSION
This chapter has explained the further implementation of Gabbeh mainly to improve
Gabbeh to be used as part of a longitudinal evaluation study. This chapter has extended
the previously introduced design of Gabbeh further to reflect some of the required
enhancement to the design of Gabbeh in three main areas of identifying and managing
discussions, progress awareness and managing large numbers of comments.
Whilst the time and resource constraints were the main issues, only some of the
enhancements, essential to support discussions have been incorporated into the second
version of Gabbeh. However, it is important to emphasise that features such as being
able to support comments, having a voting mechanism to facilitate decision making
and other identified design issues in Chapter 6, as well as usability issues of these
designs, are important and should be implemented once the resources are available but
are not essential at this stage.
Amir M Naghsh 139
Chapter 9. LONGITUDINAL STUDY
9.1. INTRODUCTION
As explained previously, the observational studies revealed the difficulties involved in
simulating the design task and in discovering to what extent the commenting facility
could be used such that the need for a longitudinal evaluation study was recognised.
This chapter reports on using Gabbeh version 2 in a longitudinal case-study in a
dispersed setting. This chapter presents the outcome of the case-study and discusses the
results of the evaluation.
9.2. OBJECTIVES
The main objective in developing Gabbeh was to build a rapid prototype to discover
how an electronic paper prototyping tool might facilitate asynchronous communication
among distributed stakeholders in the early stages of design.
Previous evaluation studies suggested that stakeholders would use the comment facility
for communicating ideas and that discussions might arise. However, it was not possible
to reach strong conclusions based on the results of the observational studies because of
difficulties involved in simulating a real design task with real end-user and the small
number of design iterations for a given design task. The analysis presented in chapter 7
suggested that it was essential to conduct an evaluation over longer period of time that
involves frequent iterations of the same design task.
In addition, the analysis of observational studies identified areas of Gabbeh which
needed further improvement to allow Gabbeh to be used in a larger design task with
many more iterations. To enable Gabbeh to be used in a longitudinal study Gabbeh was
developed further as explained in Chapter 8 to facilitate communication by enabling
users and designers to manage comments and discussions.
Once the Gabbeh version 2 was available, it was possible to conduct a longitudinal
study with three main objectives identified to extend previous studies.
Amir M Naghsh 140
The first objective was to discover if participants would use the enhanced comment
facility to communicate design ideas in a dispersed setting in form of a discussion.
The second objective was to explore how such discussions were used in the design
process.
The third objective was to investigate if it was possible to categorise comments using
the suggested classification developed in Chapter 7 for comments.
9.3. LAYOUT
The longitudinal study involved two participants. One of the participants was a
potential end-user, a 30 year old Chemical Engineer who had used software systems
such as Microsoft Office and internet explorer on a day-to-day basis. The other
participant was a professional designer, a 27 year old website designer, with BA and
MA qualifications in product and user interface design. The end-user was located in
Sheffield whilst the designer was located in Cardiff at the time of the evaluation.
At the time of the study, the user was looking for some consultancy for an idea of an e-
commerce web site to design and sell t-shirts online. Since the results of the
observational study indicated the importance of using real life design tasks, it was
decided to conduct the longitudinal evaluation by using the t-shirt website idea as the
case-study. Using the t-shirt idea had two main advantages. Firstly it would encourage
the end-user to be more involved in the design process since the user was a direct
beneficiary of the study. Secondly, the possibility of developing the case-study into a
live website would also increase the designer’s interests in the case-study.
The end-user was asked to provide a document explaining the t-shirt idea and all the
features which the end-user was expecting to have in such a website. The end-user
provided one page (A4 size) explaining the idea, explaining some motivation and
giving an abstract idea of what was expecting of the website (see Appendix C).
The end-user received approximately one hour’s training on how to use the Gabbeh in
the run mode. Also, the run mode of Gabbeh was installed on the end-user’s portable
computer and provided with a couple of prototypes generated by Gabbeh from
previous design exercises to allow the end-user to become familiar himself the tool.
The designer also received one hour’s training on how to use Gabbeh and the design-
mode of Gabbeh was installed on the designer’s computer. The designer-user was
provided with a 15” Graphic Tablet in addition to the portable. As explained in chapter
Amir M Naghsh 141
8, Gabbeh version 2 was enhanced by using two screens, the graphic tablet screen to
allow the designer to sketch the web pages, and the other regular screen (the portable
computer’s) to allow the designer to review the design as well as annotating it and
managing the discussions.
The user and designer had met prior to the study at a couple of social gatherings and
had some basic familiarity with each other. However, it was explained to both of them
that they were not supposed to have any contact with each other either by telephone or
face to face during the study, and the only permitted communication channel was
through exchanges of emails and exchanges of Gabbeh models. The communication
was restricted for two main reasons. Firstly, the case-study was designed to investigate
the asynchronous communication among different stakeholders. Secondly, it was not
possible for the author to control and record communications through other possible
channels.
Once the designer and user were comfortable in using Gabbeh the requirement
document created by the end-user was sent to the designer. The designer was reminded
that the document had been created by the user and any queries should be directed to
the end-user. As mentioned before, the designer was also reminded that there could be
a possibility of further work to implement the actual website based on the
achievements of the case-study.
In addition, the user and designer were informed that there was no deadline involved
for finishing the case study. They were also briefed that there was no time limitation
imposed to perform any one of the iterations and they could spend as much time as
liked before sending the design to the other party.
Each of the participants was asked to forward any communication that they had with
each other to the researcher. To reduce the chance of missing any part of this
communication, a dummy email address was created for the user to communicate with
the designer. The email account was also accessible by the researcher.
Both participants received an informal debriefing interview over the phone at the end
of the case-study which was recorded on audio tape.
9.4. RESULTS
The case-study lasted 25 days starting from the date that the design task was sent to the
designer. During the 25 days of the case-study the prototype was exchanged between
the designer and user. The time that was spent by the designer or user to contribute to
Amir M Naghsh 142
the design prior to each of the exchanges was considered as one session. In total, the
design process consisted of 10 sessions where 5 of them were designer sessions and the
other 5 were user sessions.
Each iteration in the design process was considered to be the designer and user
contributions made through a designer session followed by a user session. Through
each of the iterations an updated version of the design was produced. Since the design
process consisted of 5 iterations, therefore 5 revisions of the prototype were generated.
By end of the design process, the prototype had 10 pages (see appendix C).
To report the results of the case-study all 95 comments made by designer and user
were imported into an spreadsheet with all the relevant information such as date,
author, colour, version number, the comment id and the discussion number (if the
comment was part of any discussion).
A discussion was considered to be a group of comments where each comment was a
reply to another comment. In other words, either the designer or the user could reply to
each others’ comments that had been made in prior sessions. Therefore, the discussion
was considered to have consisted of at least two comments from two different
stakeholders where the first solo comment was considered to be the initiator and the
reply comment was considered to be the response. However, it was possible to reply to
an individual comment many times in one session. Although each individual comment
could have received many replies, each individual comment as reply could only be
associated with one comment. Therefore the discussions would grow in the form of a
tree, where an initiator comment could receive multiple responses and each response
could receive further multiple responses and so on.
In total, the designer and user created 87 comments to generate 28 discussions attached
to 9 pages of the design model. In addition, results indicated that only 8 comments
from the total of 95 comments did not receive any response (see Table 9-1).
Discussion Comments in
Discussions
Solo
Comments
Total number
of comments
Total 28 87 8 95
Table 9-1 Results for the comments associated to the whole design
Amir M Naghsh 143
Table 9-2 shows some results for each one of the 10 pages in the prototype. The first
column shows the number of discussions associated with each page. The second
column shows the total number of comments that are part of a discussion for each page
and the third column shows the number of solo comments for each of the pages.
Page
No. of
discussions
No. of comments in
discussions
Solo
comment
home 4 20 0
t-shirt 5 13 1
design 6 18 1
design(2) 1 5 1
basket 1 2 3
new user 2 4 0
about us 6 18 1
news 0 0 0
contact us 2 5 1
feedback 1 2 0
total 28 87 8
Table 9-2 Number of discussions associated with each page of the design
The results indicated that each discussion was the result of comments which were
generated in a number of designer and user sessions. A discussion consists of a number
of comments that are created in different sessions; therefore a discussion is associated
with a number of sessions whilst a comment is associated to only one session. Table
9-3 groups the discussions based on the number of sessions that are associated with it.
For example, the last row shows that only 5 of the 28 discussions consisted of
comments which were generated in 5 different sessions. The outcome of the evaluation
Amir M Naghsh 144
shows that about a quarter of the discussions were associated with at least four sessions
or more.
No. of sessions No. of discussions
2 13
3 8
4 2
5 5
Table 9-3 Discussions grouped based on the number of sessions
In addition, the results also indicated the number of comments associated with each of
the discussions. Table 9-4 shows that 8 of the discussions consisted of at least four or
more comments. Furthermore, the results suggest that both user and designer used the
comment facility to conduct conversations and discuss design issues throughout the
design process.
No. of
comments in
discussion
No. of
discussions
Initiated
by the
Designer
Initiated by the
User
2 13 2 11
3 7 5 2
4 3 1 2
5 3 1 2
6 1 1 0
7 1 0 1
Total - 28 10 18
Table 9-4 Discussions grouped based on number of the comments involved
Amir M Naghsh 145
The results displayed in Table 9-4 show that most of the discussions were initiated by
the user. 18 discussions were initiated by the user whilst only 10 discussions were
initiated by the designer. However, it is important to note that 11 out of 18 discussions
initiated by the user consisted of two comments created in two sessions. The majority
of these discussions consisted of a comment created by the user requesting an
alteration to the design and a response by the designer clarifying or acknowledging the
changes. For example, the following comment by user is a request to an alternation by
requesting more options to be added to the design, and the designer clarifies that such
options will be added into the future design:
User: “Can you add an option for sending news emails to the user email
address, for promoting our new designs or products?” (Session 4)
Designer: “yes it will be added” (Session 5)
In another example, the user has requested some alterations to the order of the menu
bar and the designer has responded back by acknowledging that the user’s request has
been incorporated into the design.
User: “I think the order should be like this: Home, t-shirt, Design, contact us,
corporation, I suggest you to visit good websites such as amazon!” (Session 2)
Designer: “ok” (Session 3)
Results clearly showed that designer and user have both used the commenting facility
to communicate ideas and to support discussions. Reviewed discussions suggest that
both designer and user used the comment facility to respond to each others comments.
9.5. HOW WERE DISCUSSIONS USED?
Further analysis identified four particular ways that user and designer used discussions
during the design process.
9.5.1. PROGRESS AWARENESS
Analysis suggests that both the user and designer have used the comment facility and
the discussions as a way of tracking the design progress and indicating the status of the
design in each of the iterations.
Both participants stated in the debriefing interviews that on receiving the design model
they would begin by reviewing the discussions and new comments to learn about the
Amir M Naghsh 146
design and then look into the design to find the issues that comments were referring to.
For example the user stated that:
“I would check the [new] comments first, then I would check the design and if I
was not happy I would look into the old comments and then make a new
comment to explain what I was expecting to see.”
New comments created by each participant were treated as a way of tracking the
design progress. The designer explained modification to the design by creating a new
comment to inform the user of the changes. The designer mentioned that the comments
were used to either inform the user that the requested changes were incorporated into
the design or the design was modified and it required user’s feedback. In both cases the
designer used the comments to notify the user of changes.
Figure 9-1 and Figure 9-2 display the page “design” from two different versions of the
generated prototype. The page shown in Figure 9-1 was created as part of the first
version and evolved through the design process to become what it would look like in
Figure 9-2 in the last version. Through the process the designer and the user used the
comments to instruct each other to what each of them needed to look for in the design.
Figure 9-3 shows a discussion which took place during the sessions 2 to 4. The
discussion was initiated by the user for the page “design” (see Figure 9-1). In the next
session, the designer has modified the design to incorporate the user’s request. In
addition to changes to the page as can be seen in Figure 9-2, the designer also has
responded to the user’s comment by acknowledging the changes and instructing the
user to review new parts of the design. Furthermore, the user has responded to confirm
that new additions were reviewed and instructs the designer with more feedback that
can be found in the relevant page.
Amir M Naghsh 147
Figure 9-1 Page "design", sent to user in iteration one
Figure 9-2 Page "design", sent to user in iteration 9, evolving through 8 iterations
Amir M Naghsh 148
Figure 9-3 A discussion that is representing comments from user and designer giving instruction to each other
The results showed that the designer also used the flag facility to support progress
awareness by highlighting the comments which were not yet incorporated into the
design. In session three, the designer indicated in a email that it was not possible to
incorporate all the comments created by the user into the design.
Designer: “I didn’t have the time to go through all of your comments. I have
flagged the ones which I have not done yet” (Session 3: Email)
Figure 9-4 shows a discussion which was initiated in session one and was ended in
session eight. The first two comments in the discussion were created in session one and
two. Then the designer flagged the second comment without making any changes to
the design model. Later in the fifth session the designer came back to incorporate that
issue into the design model and cleared the flag of the second comment and
acknowledged the changes incorporated into the design by creating the third comment.
Amir M Naghsh 149
Figure 9-4 Showing a discussion, there has been a two iterations gap between second and third comment.
9.5.2. DISCUSSING DESIGN ISSUES
The analysis of the results and interviews showed that discussions were used to
generate dialogues on design issues.
One of the ways that discussions were used was to allow designer and user to debate
design problems. Particularly, discussions were used in the cases where they could not
agree on a solution. Using the discussions allowed the designer and the user to express
different points of view and reach a common ground towards a specific design issue.
Being able to discuss design issues would allow the designer to first explore the design
with the user before making too many changes to the design.
Figure 9-5 and Figure 9-6 show two examples where the designer and the user debated
a design problem. In both cases they used the comments to explain their point of view
while justifying it by directing each others attention to a specific part of design or to
other external resources, such as existing websites.
Figure 9-5 shows a conversation between the designer and the user, discussing the use
of the term “Have fun” on the front page of the prototype. The discussion was
conducted by the user comment from the first session and was continued in the
following two sessions with designer and user exchanging their points of view. The
discussion allowed the designer to understand the user’s point of view and incorporate
the changes into the design after discussing it in four sessions.
Figure 9-6 shows another example of the designer and the user debating a design
problem. However, in this case it is the user who agrees with the designer by the end of
the discussion. This is an example where the designer did not make any changes to the
design through the sessions that they were debating, and only made one final
modification once the a decision was made by the end of the discussion.
Amir M Naghsh 150
Figure 9-5 An example of debating a design issue
Figure 9-6 Another example of debating a design issue
Amir M Naghsh 151
9.5.3. DIRECTING ATTENTION
Whilst discussions were used by the designer and the user to direct each other’s
attention when debating design issues (as explained in the previous section), they were
also used to direct the user’s attention to a specific design issue, in order to encourage
the user to provide feedback on the relevant issue. For example Figure 9-7 shows a
discussion initiated by the designer clarifying a part of the design whilst exploring the
design further by asking for the user’s feedback. The analysis of the interviews
suggested that being able to preview the design while going through the discussions
helped the user to have a better understanding of the design and therefore could
provide instructive feedback. The user stated in the debriefing interview that:
“I could see how the design was and then relate the comments to the design…..
… .. Previewing the design and reading the comments helped a lot in imagining
how the final thing would be.”
Figure 9-7 User’s feedback
The results also indicated that discussions were used to allow the designer to provide
information about design issues. The discussions allowed the designer to clarify the
design where the user’s responses indicated that the user had not understood a specific
part of the design. Figure 9-8 shows a discussion where the designer is clarifying the
design without the need to incorporate the new changes into the design. The responses
created by the user in Figure 9-8 were already reflected into the design and designer
had considered them prior to the user’s latest feedback. The discussion then had been
used to allow the designer to direct user’s attention to a specific part of the design in
order to help him understand the issue.
Amir M Naghsh 152
Figure 9-8 Clarifying design through discussion
9.5.4. RECORDING DESIGN ISSUES
The analysis suggests that both the user and designer used the discussions as a source
of information associated with the design. Reviewing recorded interviews showed that
the designer and user would check their old comments before making new comments.
The users stated:
User: “I would look back at [the designer] and my old comments to see what I
have said before. If I was using emails I could not go back to check other old
emails every time, but this way it was easy to go back to check what we were
talking about.”
The analysis also showed that discussions were used to extend the design model by
recording design issues that had not been possible to incorporate into the design model
at this stage. Figure 9-9 shows a discussion which expresses a part of the design which
is not possible to implement at this stage. The discussion allows the designer to explain
to the user that such a feature would exist in the finished system. Further when
implementing the actual website, these comments, in addition to the design model, can
be used to demonstrate how the finished system will be.
Amir M Naghsh 153
Figure 9-9 A discussion, recording a design issue
Figure 9-10 shows that discussions allow the designer and user to communicate
technical issues related to a part of the design and make decisions during the design
process to be retrieved at later stages.
Figure 9-10 A discussion on technical issues
9.6. REVIEWING AND REFLECTING THE COMMENT CLASSIFICATION
To extend the identified classification for comment from Chapter 7, it was important to
apply the classification for all the comments associated with the case-study and
identify any comment which did not fit into the previously identified categories. In
Chapter 7, seven categories were identified based on the analysis of the observational
studies.
To test the classification, all the comments created by the user and designer were
printed out on the paper to allow a better way of clustering them. Since many of the
comments created by participants were multi-sentence comments, some of them were
broken down into comment phrases. A comment phrase represents only one specific
Amir M Naghsh 154
issue. It can include all the sentences from a comment object in Gabbeh or include only
part of a comment object.
User: “It is very good. I think we are making progress. However, I think the
photo should be in the middle of the page and the comment will come beneath
it.” (Iteration 3)
For example, the above comment was broken into two comment phrases; one comment
phrase is indicating that user is confirming the changes in the design and the second
comment phrase is suggesting further alterations to the design.
Comment phrase one: “It is very good. I think we are making progress.”
Comment phrase two: “However, I think the photo should be in the middle of
the page and the comment will come beneath it.”
Clarifyi
ng
Verifyi
ng
Explori
ng
Alteri
ng
Confirmi
ng
Understand
ing
Organisi
ng
Justifyi
ng
Design
er 33 8 3 0 6 0 1 7 58
User 0 0 0 31 11 7 1 4 54
total 33 8 3 31 17 7 2 11 11
2
Table 9-5 Number of comment phrases associated to each category
Table 9-5 shows the number of comments associated with each category. One new
category called “Justifying” was identified in the analysis which was added to the
previously defined seven categories. In addition, the definition of the confirming
category was revised since it was primarily used by the user in the previous study
while the results of the case-study showed it had been used by both the designer and
the user. In total 112 comments were identified, of which, only 11 did not fit into the
identified classification for comments from Chapter 7. (See appendix C for a complete
list of comments).
Amir M Naghsh 155
9.6.1.1. JUSTIFYING
The results showed that both the user and designer used the comments to give reasons
and explanations on specific design issues. When a disagreement occurred over a
design issue, the user and designer used comments to justify their way thinking as the
correct and appropriate solution to the design issue. The ‘justifying’ comments usually
represented the points of the view of an individual towards a design issue and the
reasons behind it. Some examples are provided here from the case study.
Designer: “i dont want to put 'coporation' again, its off putting, a coporation
sounds like a PLC, when really we want to get the mass market - about us,
helps the customer to 'relate' to us (its a pyschological marketing
attempt)”(page 7, iteration 2)
User: “People should think that a professional group who are doing marketing,
finance, delivery etc. are behind this website. So those who have concern about
the company will come to this page. Therefore this page should be very
professional with strong message insurring visitors.”
Designer: “still to me coporation sounds "we are bigger than you" when that is
not the essense of the site, unless there is a way (like amazon) to allow the
customers' name to appear, therefore they will privaliged An alternative
suggestion "company" .“
9.6.1.2. CONFIRMING
In addition to the feedback by the user to confirm or disagree with parts of the design,
the analysis showed that the designer also used comments to acknowledge or confirm
changes to design in response to ‘altering’ comments made by the user.
Therefore, ‘confirming’ comments are the feedback provided by the user or designer to
agree or disagree with a part of the design that is implemented or will be considered in
the finished system. Figure 9-11 and Figure 9-12 show two ‘confirming’ comments
created by the designer in response to changes that were requested by the user.
Figure 9-11 Confirmation by designer
Amir M Naghsh 156
Figure 9-12 Confirmation by Designer
Figure 9-13 shows another confirmation comment which was created by the user in response to a discussion over a design issue.
Figure 9-13 Confirmation by user
9.7. DISCUSSION
Reviewing the recorded interviews showed that the user and designer used the
commenting tool, instead of the body of text in emails, to communicate ideas during
the design process. They both showed their interest in the asynchronous
communication. The designer indicated that it was easier to manage the time and
therefore spend better quality time in the design process. While the user said it made it
possible to spend more time on reviewing the design and “think over it”.
Furthermore, the analysis of the results and the debriefing interviews identified a
number of concerns with the design of Gabbeh which need to be considered in future
enhancements.
9.7.1. PROGRESS AWARENESS
The analysis showed that the designer created many comments to inform the user of
the progress made in the design model. The results suggested the importance of being
able to identify which comments had been incorporated into the design and which had
Amir M Naghsh 157
not. The designer used the flag feature as a way of tracking which comments were
incorporated in to the design.
In addition, to enhance the progress awareness it was recognised that it should be
possible to allow the user and designer to review different versions of design at the
same time. It was noted that reviewing old comments in discussions is not sufficient to
enable a designer or user to understand the whole picture regarding a design issue. It
was recognised that some comments become meaningless once the design is changed.
Therefore, since a discussion consists of a number of comments created in different
iterations it is essential to be able to access relevant versions of the design for each
comment when going through the discussion.
As the design model evolves, the number of pages of the design increases and it
becomes more difficult to track which page has been updated by going through all of
them. Both the user and designer indicated that it would be useful to be able to see
what pages in the design model had been updated.
9.7.2. RICHNESS OF COMMUNICATION CHANNEL
The results of debriefing interviews indicated the need for a richer communication
channel. The user stated that it would be useful to be able to use ‘smiley faces’ to show
emotions. The user mentioned it was difficult to say I don’t like a part of a design at all
and it could be easier if I could show my disagreement in form of a ‘smiley face’.
The results also showed that the user and designer used the commenting tool to instruct
each other to visit other websites which they found useful. It was realised that it is
important that the comment content could be linked to other external sources and
representations. In addition, it should be possible to attach a picture or file to a
comment or discussion to be used by other stakeholders.
9.7.3. EMPOWERING USERS
The analysis showed that the user was interested in contributing the design by getting
involved in making changes in the sketches in addition to providing feedback through
comments. In the debriefing interview the user mentioned that “I wanted to make
marks on the actual page”. The user indicated that in few occasions it seemed to be
easier to draw on an existing sketch in order to express a point of view instead of
explaining it as a long comment. Therefore, it is important to allow users to annotate
the prototype while reviewing it in the run-mode.
Amir M Naghsh 158
9.8. SUMMARY
This chapter reported the results and analysis of a longitudinal case-study in a
dispersed setting. The results showed that the comment facility could be used when
available to communicate design ideas and issues, and to support discussions.
More importantly, the findings of the analysis indicated that the comment facility was
not used only used to support discussions. It was also used to:
• support a progress awareness to track the design changes;
• discuss and debate design conflicts to reach common ground;
• direct attention to external resources or to a specific part of the design to
enhance understanding and encourage more instructive feedback;
• record design decisions as the design is progressing.
In addition, the results of reviewing the comments classification complemented the
findings of Chapter 7 and in addition to that, it introduced a new category called
justifying and it modified the definition of confirming category to include both the user
and designer comments.
Amir M Naghsh 159
Chapter 10. CONCLUSION
10.1. SUMMARY
In the preceding chapters, the thesis has examined existing electronic paper
prototyping tools that support participatory design at early stages of design. To
encourage participation of distributed stakeholders, the thesis has investigated the use
of annotations to facilitate asynchronous communication in electronic paper
prototyping tools. This chapter reflects on the contributions of the thesis in this area
and presents the contribution order of their importance. Furthermore, the chapter offers
suggestion for future work.
10.2. CONTRIBUTIONS
10.2.1. GABBEH
The thesis has demonstrated the feasibility of supporting communication through
providing annotations in electronic paper prototyping tools. Chapter 6 and 8 reported
the design and implementation of Gabbeh, an electronic paper prototyping tool that
allows different stakeholders to add arbitrary annotations in the form of comments
when the prototype is being designed or executed.
The thesis reported the results of two studies that were conducted to evaluate Gabbeh.
Chapter 7 presented the results of an observational study and Chapter 9 showed the
results of a longitudinal study. Results of both studies showed that the comment
facility was used once it was available to stakeholders. More importantly, the results
showed that supporting annotation in electronic paper prototyping tools can facilitate
communication since the results highlighted that stakeholders used the comment
facility to respond to each others’ comments in the form of discussions.
10.2.2. USE OF ANNOTATIONS
10.2.2.1. A CLASSIFICATION OF ANNOTATIONS
The results presented in chapter 7 and 9 were considered to identify a possible
classification of the uses made of annotations in designing interactive systems. The
findings of the observational study followed by the longitudinal study suggested that
Amir M Naghsh 160
comments created in designing interactive systems such as websites could be divided
into eight categories as shown below. Stakeholders made comments for:
• clarifying design detail,
• exploring unspecified and not well understood requirements,
• verifying the parts that were already designed.
• understanding by asking questions about the design,
• altering the design,
• confirming the changes in the design.
• Justifying their way of thinking and giving reason for a design issue.
• Organising the design process.
The results of the studies showed that stakeholders created different types of comments
when responding to each other as part of a discussion. For instance, an altering
comment created by the end-user requesting a change generated a discussion where
both the designer and end-user used comments to justify their way of thinking as well
as asking questions to explore the design. Finally, the designer clarifies the changes to
be incorporated into the design and asks for the end-user’s verification where the end-
user confirms the changes.
10.2.2.2. USE OF DISCUSSIONS
The findings of the observational and longitudinal studies indicated that the comment
facility was also used to generate discussions:
• to support progress awareness in order to track the design changes;
• to discuss and debate design conflicts to reach common ground;
• to direct attention to external resources or to a specific part of the design in
order to enhance understanding and encourage more instructive feedback;
• to record design decisions as the design progresses
10.2.3. COGNITIVE DIMENSIONS
The thesis demonstrated that the traditional single dimension frameworks such as
concept of fidelity are not sufficiently fine-grained to analyse the important differences
Amir M Naghsh 161
and similarities between electronic paper prototyping tools and it is essential to apply
multi dimensional frameworks. Furthermore, the thesis showed that it is feasible to use
Cognitive Dimensions to compare existing electronic paper prototyping tools.
Chapter 4 applied the Cognitive Dimensions framework to explore the characteristics
of prototyping approaches. The results of the theoretical analysis suggested a profile of
desirable levels of each of the cognitive dimensions for an electronic paper-prototyping
tool in order to support a participatory design approach in a dispersed setting. It has to
maintain a low level of viscosity, diffuseness, hard mental operation, hidden
dependencies error-proneness and premature commitment. On the other hand, offering
a medium degree of closeness of mapping and tolerant level of abstraction was
suggested. The results of the analysis also indicated the importance of providing high
level of provisionality, progressive evaluation, role-expressiveness and consistency in
order to support a highly iterative design process which allows stakeholders to easily
learn how to interact with the prototype from the early stages in design.
In addition, chapter 4 argued the importance of providing possible forms of secondary
notation such as annotation in order to enhance communication among different
stakeholders. However, the results of the theoretical analysis have distinctively
highlighted that none of the existing electronic paper prototyping tools supported
secondary notation. The lack of the ability to provide secondary notation can severely
limit the ability of such prototyping tools to support communication between different
stakeholders at the early stages of design.
10.2.4. RECOMMENDATIONS FOR SUPPORTING ANNOTATION IN ELECTRONIC PAPER
PROTOTYPING TOOLS
Based on the findings of the theoretical studies in Chapter 4 and literature review of
annotation in Chapter 5, the thesis suggested a list of recommendations for designing
annotation tools.
Chapter 6 presented this list of recommendations in the form of a possible design that
could be used in designing electronic paper prototyping tools that support annotations.
The proposed design was considered in the implementation of the first version of
Gabbeh. Furthermore, the results of the evaluation studies contributed to the list of
recommendations of proposed design and consequently further enhancing the second
version of Gabbeh which was reported in Chapter 8.
Amir M Naghsh 162
Based on the suggested list of recommendations and results of the two evaluation
studies, the thesis suggests the following list of requirements to be considered when
designing annotation facilities in electronic paper prototyping:
10.2.4.1. CONSISTENCY TO CURRENT DESIGN
The annotation tool should be consistent with the current design of the electronic paper
prototyping tool while it also has to be designed in a way which requires a minimum
number of actions and skills for stakeholders to use it.
10.2.4.2. FLUIDITY OF FORM
Using the annotation tool should not distract the actual design activity where the
stakeholders are exploring and evaluating the proposals and requirements. It has to be
designed in a way that maintains the same fluidity that annotation has on the paper and
enable users to make comments as an integral part of the design process. In other
words, it should be possible to amalgamate the design activity and creation of
comments as one activity.
10.2.4.3. IN SITU ANNOTATION
The design of annotation tools should allow the stakeholders to annotate the prototype
while performing the design activity in such a way that their annotations are easily
distinguished from the actual design or the prototype. It should also be possible to
differentiate the annotations from one another, for instance it should indicate that they
were created at different stages of design. The relationship between the design
component (or number of design components) and the annotation should also be easy
to determine.
10.2.4.4. INFORMAL USE
The design of the annotation tool has to consider that stakeholders usually have their
own way of coding when they annotate. The design should provide a rich medium to
maintain such an activity.
10.2.4.5. RETRIEVING AND FILTERING
The annotation tool should allow for additional information (e.g. author’s name or
date/time of creation) to be associated with annotations as their properties. The tool
should allow the stakeholders to distinguish different annotations through these
properties. In addition, a filtering tool is required to manage large number of
annotations and which enables stakeholders to retrieve annotations that share some
common properties.
Amir M Naghsh 163
10.2.4.6. AVAILABILITY
The annotations should be available in all representations of a prototype. The
annotation tool should allow the stakeholders to annotate anything without limit and
anywhere on the prototype by enabling the annotation to be associated with any
arbitrary number of components in a prototype.
10.2.4.7. PROGRESS AWARENESS
The design of annotation should allow the stakeholders to be able to identify when and
at which iteration an annotation has been created, who has created it, and if it has been
addressed or responded to or not. It should be possible to distinguish new annotations
from old ones. The progress awareness should allow a stakeholder to track comments
to identify whether there are responses to a comment or if it has been incorporated into
the design or has not yet been dealt with.
In addition, since different annotations are created in different iterations of the design
process, they can refer to different versions of the same prototype. When stakeholders
review annotations created in earlier iterations they should be able to review previous
versions of a prototype where the annotation was created.
10.2.4.8. SUPPORTING DISCUSSIONS
To facilitate communication the annotation tool should support discussions. The tool
should allow stakeholders to create annotations as reply to other annotations. Such two
annotations should be linked with each other to allow the stakeholders to identify the
relationship between them in later stages of the design.
The tool should allow the stakeholders to easily identify annotation associated with a
discussion. They should be able to determine which annotation has initiated the
discussion and what the latest responses in the discussion are.
10.3. FUTURE WORK
10.3.1. ENHANCEMENT TO THE CURRENT VERSION OF GABBEH
10.3.1.1. INTRODUCING FURTHER FEATURES
Because of the time and resource constraints it was not possible to provide all the
features suggested in the list of recommendations of proposed design into Gabbeh. The
remit of future work is to incorporate outstanding requirements from the proposed
design and the enhancements that were identified during the two evaluation studies
into the next version of Gabbeh.
Amir M Naghsh 164
The results of the longitudinal study in chapter 9 referred to the end-user explaining
that in some cases, instead of explaining the design issue in a comment, it would have
been easier for the user to amend the sketched pages to reflect his viewpoint.
Therefore, it is essential to enable the stakeholders using Gabbeh in run-mode to make
changes to the sketched pages, or be able to sketch in comments and attach them to the
prototype.
The results of evaluation studies indicated the need for a richer communication
channel. The stakeholders should be able to agree or disagree with a comment without
the need to create new comments. The end-user in the longitudinal study stated that it
would have been useful to be able to use smiley faces to indicate whether the user liked
or disliked a part of the design. Use of smiley faces was considered to be an easier way
to show disagreements in particular situations.
Progress awareness is another area which requires further enhancement to meet the
suggested requirements in an earlier section. Essentially, it should allow stakeholders
to identify which comment is new, and which comments have been incorporated into
the design. Further enhancements should enable the stakeholders to go through
different versions of the design when reviewing comments. Additionally further
development might consider introducing a method of informing the stakeholders of the
pages that have been updated without the need of going through all the sketched pages.
10.3.1.2. EFFICIENCY ISSUES
Another area of future work that should be investigated further is evaluating the
efficiency of Gabbeh. The limitation of time and resources did not allow this research
to investigate the usability issues involved in designing the annotation tool. However,
the effectiveness of the design of the annotation tool and filtering features are largely
dependent on the provided user interfaces. To improve user participation the interfaces
of the facilities such as filtering tools, and the displaying panel for discussions have to
be enhanced though further usability studies.
10.3.1.3. COGNITIVE DIMENSIONS PROFILE
As explained in earlier section 10.2.3, the Cognitive Dimensions framework was used
to explore the characteristics of prototyping approaches. The results of theoretical
analysis presented in the discussion section of Chapter 4 suggested a profile for an
electronic paper-prototyping tool to support a participatory design approach in a
dispersed setting. This thesis focuses on the importance of providing secondary
notation as suggested by this profile. Gabbeh was implemented as an extension to
Denim to provide forms of secondary notations. Apart from secondary notation
Amir M Naghsh 165
dimension, Gabbeh shares the same levels of all the cognitive dimensions as Denim.
Therefore, an area of future work should consider using the suggested profile of
cognitive dimensions in section 4.4 to improve Gabbeh. Some of the dimensions that
need to be improved according to the suggested profile for Gabbeh (similar to Denim)
as presented in table 4.2 are: consistency, diffuseness, hard mental operation, hidden
dependencies, role-expressiveness and visibility
10.3.2. DESIGN RATIONALE
In designing any computer system, many decisions are made as the product
goes from a set of vague customer requirements to a deliverable entity. Often it
is difficult to recreate the reasons, the rationale, behind various design
decisions. Design rational is the information that explains why a computer
system is the way it is (Dix et al., 1993, p. 248).
One of the areas that can be explored as advancement in this research is the use of
annotation, particularly discussions, to support some form of design rationale. The
results of the longitudinal study presented in chapter 9 showed examples of a designer
using the discussions to refer to an earlier comment provided by the user to clarify the
changes that were incorporated into the design. Therefore, there is a potential to use
recoded discussions in the form of a history associated with different parts of the
design. The comments associated with a discussion can reflect the rationale of the
changes incorporated into that specific part of design which the discussion is
associated with.
To investigate the use of discussions in the design rationale, features such as being able
to support comments, or having a voting mechanism to facilitate decision making can
be considered as further enhancements.
10.3.3. SUPPORTING ANNOTATION IN SITUATED SETTINGS
To encourage a greater user involvement, participatory design techniques make use of
stakeholders’ vast experience of collaborating around traditional tables by using tools
and materials that offer family resemblance to what they use in their ordinary
workplace. Recent research in CSCW (Masoodian et al., 2007; Morris et al., 2006;
Ryall et al., 2004) has considered interactive tabletops an ideal environment for co-
present collaborations for small groups. An area of future work can be considered on
introducing an electronic paper prototyping tool (such as Gabbeh) into a tabletop
environment, intended to increase stakeholders’ participation and mutual learning
when collaborating synchronously.
Amir M Naghsh 166
Supporting direct input from multiple users was identified as the essential enhancement
into Gabbeh in order to enable it to be used for further investigations in this area of
research. Therefore, in collaboration with the I-MAG laboratory at Grenoble
University, multi user entry has been introduced into Gabbeh to prepare the base for
future work in this direction (Naghsh and Bailly, 2005).
10.3.4. DISTRIBUTED PARTICIPATORY DESIGN
A number of researchers have investigated the use of participatory design in distributed
design teams (Jemtrud et al., 2006; Danielsson and Danielsson, 2005; Naghsh and
Ozcan 2004; Everitt et al., 2003; Kensing and Blomberg, 1998). A Series of workshops
on Distributed Participatory Design (Naghsh et al., 2008; Danielsson et al., 2006) have
been organised on this emerging area of research to overcome the challenges of
performing participatory design in distributed design teams and to understand the
usefulness and constraints of this approach. Gabbeh is a first step in opening up this
important area of research by providing support as a distributed participatory design
tool. Furthermore, the design of Gabbeh can be considered in the design of other tools
that may support distributed participatory design.
Amir M Naghsh 167
REFERENCES
Agarwal, R., Prasad, J., Tanniru, M. and Lynch, J. (2000). Risks of rapid
application development. Communications of the ACM, 43 (11), ACM Press,
pp. 1.
Andriole, S.J. (1989). Storyboard prototyping: A new approach to user
requirements analysis. QED Information Sciences, Inc., Wellesley, MA,
United States.
Andy, D., Naghsh, A.M. and Ozcan, M.B. (2004). Support for participation
in electronic paper prototyping. In: The Proceedings of the Participatory
Design Conference, Toronto, Canada. pp. 105 – 108.
Appan, P., Shevade, B., Sundaram, H. and Birchfield, D. (2005). Interfaces
for networked media exploration and collaborative annotation. In: IUI '05:
Proceedings of the 10th International Conference on Intelligent User
Interfaces, San Diego, California, United States. pp. 106-113.
Audy, J., Evaristo, R. and Watson-Manheim, M.B. (2004). Distributed
analysis: The last frontier? In: HICSS '04: Proceedings of the 37th Annual
Hawaii International Conference on System Sciences (HICSS'04) - Track 1,
pp. 100-102.
Bastide, R. and Palanque, P.A. (1995). A petri net based environment for
the design of event-driven interfaces. In: Proceedings of the 16th
International Conference on Application and Theory of Petri Nets, Torino,
Italy, pp. 66-83.
Baumer, D., Bischofberger, W.R., Lichter, H. and Zullighoven, H. (1996).
User interface prototyping—concepts, tools, and experience. In: ICSE '96:
Proceedings of the 18th International Conference on Software Engineering,
Berlin, Germany. pp. 532-541.
Beck, K. and Andres, C. (2004). Extreme programming explained: Embrace
change. 2nd ed., Addison-Wesley Professional, Boston, MA, United States.
Amir M Naghsh 168
Bellotti, V. and Bly, S. (1996). Walking away from the desktop computer:
Distributed collaboration and mobility in a product design team. In: CSCW
'96: Proceedings of the 1996 ACM Conference on Computer Supported
Cooperative Work, Boston, Massachusetts, United States. pp. 209-218.
Blackwell, A.F., Britton, C., Cox, A.L., Green, T.R.G., Gurr, C.A., Kadoda,
G.F., Kutar, M., Loomes, M., Nehaniv, C.L., Petre, M., Roast, C., Roe, C.,
Wong, A. and Young, R.M. (2001). Cognitive dimensions of notations:
Design tools for cognitive technology. In: BEYNON, M., NEHANIV, C.L. and
DAUTENHAHN, K. (eds.). The 4th International Conference on Cognitive
Technology, Warwick, UK pp. 325-341.
Blinman, S. and Cockburn, A. (2005). Program comprehension:
Investigating the effects of naming style and documentation. In: AUIC '05:
Proceedings of the Sixth Australasian Conference on User Interface,
Newcastle, Australia. pp. 73-78.
Bodker, S., Gronbaek, K. and Kyng, M. (1993). Cooperative design:
Techniques and experiences from the scandinavian scene. Lawrence
Erlbaum Associates,.
Bodker, S. and Gronbaek, K. (1991). Cooperative prototyping: Users and
designers in mutual activity. Int.J.man-mach.stud., 34 (3), Academic Press
Ltd, pp. 453-478.
Bottoni, P., Civica, R., Levialdi, S., Orso, L., Panizzi, E. and Trinchese, R.
(2004). MADCOW: A multimedia digital annotation system. In: AVI '04:
Proceedings of the Working Conference on Advanced Visual Interfaces,
Gallipoli, Italy. pp. 55-62.
Brinkley, J., Jakobovits, R. and Rosse, C. (2002). An online image
management system for anatomy teaching. In: The American Medican
Informatics Association Annual Symposium, Washington, DC, United
States, pp. 983.
Brockman, W., S., Neumann, L., Palmer, C., L. and Tidline, T.J. (2001).
Scholarly work in theHumanities and theEvolving InformationEnvironment.
Washington DC., Digital Library Federation, Washington DC, United States.
Brush, A.J.B., Bargeron, D., Gupta, A. and Cadiz, J.J. (2001). Robust
annotation positioning in digital documents. In: CHI '01: Proceedings of the
SIGCHI Conference on Human Factors in Computing Systems, Seattle,
Washington, United States. pp. 285-292.
Amir M Naghsh 169
Button, G. and Sharrock, W. (1996). Project work: The organisation of
collaborative design and development in software engineering.
Comput.supported coop.work, 5 (4), Kluwer Academic Publishers, pp. 369-
386.
Cadiz, J.J., Gupta, A. and Grudin, J. (2000). Using web annotations for
asynchronous collaboration around documents. In: CSCW '00: Proceedings
of the 2000 ACM Conference on Computer Supported Cooperative Work,
Philadelphia, Pennsylvania, United States. pp. 309-318.
Chronaki, C.E., Zabulis, X. and Orphanoudakis, S.C. (1997). I2Cnet
medical image annotation service. International journal of medical
informatics, 22 (4), Elsevier, pp. 337-347.
Clarke, S. (2001). Evaluating a new programming language. In: KADODA,
G. (ed.). The Thirteenth Annual Meeting of the Psychology of Programming
Interest Group, pp. 275-289.
Coar, K. (2004). The sun never sits on distributed development. Queue, 1
(9), ACM Press, pp. 32-39.
Cooper, D., Stepney, S. and Woodcock, J. (2002). Derivation of Z
refinement proof rules. University of York, York, UK.
Crosby, M.,E., Scholtz, J. and Wiedenbeck, S. (2002). The roles beacons
play in comprehension for novice and expert programmers. In: KULJIS, J.,
BALDWIN, L. and SCOBLE, R. (eds.). The 14th Annual Workshop of the
Psychology of Programming Interest Group, London, UK. pp. 58-73.
Curtis, G. and Vertelney, L. (1990). Storyboards and sketch prototypes for
rapid interface visualization. In: CHI Tutorial, Seattle, Washington, United
States.
Damm, C.H., Hansen, K.M. and Thomsen, M. (2000). Tool support for
cooperative object-oriented design: Gesture based modelling on an
electronic whiteboard. In: CHI '00: Proceedings of the SIGCHI Conference
on Human Factors in Computing Systems, The Hague, The Netherlands.
pp. 518-525.
Danielsson, K. and Danielsson, U. (2005). Distributed participatory design:
A case study. In: HCI International Conference, Las Vegas, United States.
Danielsson, K., Naghsh, A.M. and Dearden, A. (2006). The first workshop
of distributed participatory design. In: NordiCHI 2006, Oslo, Norway.
Amir M Naghsh 170
Davis, A.M. (1992). Operational prototyping: A new development
approach. IEEE softw., 9 (5), IEEE Computer Society Press, pp. 70-78.
Dearden, A.,J., Siddiqi, J.I. and Naghsh, A.M. (2003). Using cognitive
dimensions to compare prototyping techniques. In: The First Joint
Conference of EASE & PPIG (15th Annual Meeting of the Psychology of
Programming Interest Group), Keel, UK.
Dix, A., Finlay, J.E., Abowd, G.D. and Beale, R. (2004). Human-computer
interaction. 3rd ed., Prentice-Hall, Inc, Upper Saddle River, NJ, United
States.
Duyne, D.K.V., Landay, J.A. and Hong, J.I. (2002). The design of sites:
Patterns, principles, and processes for crafting a customer-centered web
experience. Addison-Wesley Longman Publishing Co., Inc, Boston, MA,
United States.
Ehn, P. (1993). Scandinavian design: On participation and skill. In: In
Schuler, D. and Namioka, A., (Eds.) Participatory Design. Principles and
Practices, pp. 41-78.
Ehn, P. (1990). Work-oriented design of computer artifacts. Lawrence
Erlbaum Associates, Inc, Mahwah, NJ, United States.
Ehn, P. (1988). Playing the language-games of design and use-on skill and
participation. In: Proceedings of the ACM SIGOIS and IEEECS TC-OA 1988
Conference on Office Information Systems, Palo Alto, California, United
States. pp. 142-157.
Ehn, P. and Kyng, M. (1991). Cardboard computers: Mocking-it-up or
hands-on the future. Lawrence Erlbaum Associates, Inc, pp. 169-196.
Engelberg, D. and Seffa, A. (2002). A framework for rapid mid-fidelity
prototyping of web sites. In: Proceedings of the IFIP 17th World Computer
Congress - TC13 Stream on Usability, pp. 203-215.
Everitt, K.M., Klemmer, S.R., Lee, R. and Landay, J.A. (2003). Two worlds
apart: Bridging the gap between physical and virtual media for distributed
design collaboration. Human factors in computing systems, 5 (1), pp. 553-
560.
Fish, R.S., Kraut, R.E. and Leland, M.D.P. (1988). Quilt: A collaborative
tool for cooperative writing. In: Proceedings of the ACM SIGOIS and
Amir M Naghsh 171
IEEECS TC-OA 1988 Conference on Office Information Systems, Palo Alto,
California, United States. pp. 30-37.
Floyd, C., Mehl, W., Reisen, F., Schmidt, G. & Wolf,G. (1989). Out of
scandinavia: Alternative approaches to software design and system
development. In: Human-Computer Interaction 4, pp. 253-350.
Fuchs, N.E. (1992). Specifications are (preferably) executable. Software
engineering journal, 7 (5), Michael Faraday House, pp. 323-334.
Green, T.R.G. (1983). Learning big and little programming languages. In:
WILKINSON, A.C. (ed.). Classroom Computers and Cognitive Science, .
Green, T.R.G. and Blackwell, A.F. (1998). A tutorial on cognitive
dimensions. Available on-line at:
http://www.ndirect.co.uk/~thomas.green/workStuff/Papers/index.html
Green, T.R.G. and Petre, M. (1996). Usability analysis of visual
programming environments: A 'cognitive dimensions' framework. Journal
of visual languages and computing, 7 (2), pp. 131-174.
Gronbaek, K. (1991). Prototyping and active user involvement
in system development: Towards a cooperative prototyping approach.
Ph.D. Thesis. Computer Science Department, Aarhus University.
Gronbaek, K. (1989). Rapid prototyping with fourth generation systems:
An empirical study. 5 (2), OFFICE: Technology and People, pp. 105-125.
Hanson, A.J. and Wernert, E.A. (1997). Constrained 3D navigation with 2D
controllers. In: VIS '97: Proceedings of the 8th Conference on Visualization
'97, Phoenix, Arizona, United States. pp. 175-ff.
Harmon, R., Patterson, W., Ribarsky, W. and Bolter, J. (1996). The virtual
annotation system. In: VRAIS '96: Proceedings of the 1996 Virtual Reality
Annual International Symposium (VRAIS 96), pp. 239.
Heeks, R., Krishna, S., Nicholson, B. and Sahay, S. (2001). Synching or
sinking: Global software outsourcing relationships. IEEE softw., 18 (2),
IEEE Computer Society Press, pp. 54-60.
Henderson, K. (1991). Flexible sketches and inflexible data bases: Visual
communication, conscription devices, and boundary objects in design
engineering. Science, technology & human values, 16 (4), pp. 448-473.
Amir M Naghsh 172
Hong, J.I., Landay, J.A., Long, A.C. and Mankoff, J.C. (2002). Sketch
recognizers from the end-user's, the designer's, and the programmer's
perspective. In: American Association for Artificial Intelligence, Stanford,
CA, United States. pp. 73-77.
Hong, J.I., Li, F.C., Lin, J. and Landay, J.A. (2001). End-user perceptions of
formal and informal representations of web sites. In: CHI '01: CHI '01
Extended Abstracts on Human Factors in Computing Systems, Seattle,
Washington. pp. 385-386.
J., W. and R. D. (1988). Rapid prototyping for user interface design. In:
M., Helander (ed.). Handbook of human-computer interaction. Elsevier,
Amsterdam, North-Holland, pp. 859-875.
Jemtrud, M., Nguyen, P., Spencer, B., Brooks, M., Liu, S., Liang, Y., Xu, B.
and Zhang, L. (2006). Eucalyptus: Intelligent infrastructure enabled
participatory design studio. In: WSC '06: Proceedings of the 38th
Conference on Winter Simulation, Monterey, California. pp. 2047-2054.
Jourdan, M., Layaida, N., Roisin, C., Sabry-Ismail, L. and Tardif, L. (1998).
Madeus, and authoring environment for interactive multimedia documents.
In: MULTIMEDIA '98: Proceedings of the Sixth ACM International
Conference on Multimedia, Bristol, United Kingdom. pp. 267-272.
Jung, T., Gross, M.D. and Do, E.Y. (2002). Annotating and sketching on 3D
web models. In: IUI '02: Proceedings of the 7th International Conference
on Intelligent User Interfaces, San Francisco, California, United States. pp.
95-102.
Kahan, J. and Koivunen, M. (2001). Annotea: An open RDF infrastructure
for shared web annotations. In: WWW '01: Proceedings of the 10th
International Conference on World Wide Web, Hong Kong, Hong Kong. pp.
623-632.
Kant, M.v.d., Wilson, S., Bekker, M., Johnson, H. and Johnson, P. (1998).
PatchWork: A software tool for early design. In: CHI '98: CHI 98
Conference Summary on Human Factors in Computing Systems, Los
Angeles, California, United States. pp. 221-222.
Kensing, F. and Blomberg, J. (1998). Participatory design: Issues and
concerns. Comput.supported coop.work, 7 (3-4), Kluwer Academic
Publishers, pp. 167-185.
Amir M Naghsh 173
Khazaei, B. and Triffitt, E. (2002). Applying cognitive dimensions to
evaluate and improve the usability of Z formalism. In: SEKE '02:
Proceedings of the 14th International Conference on Software Engineering
and Knowledge Engineering, Ischia, Italy. pp. 571-577.
Klemmer, S.R., Newman, M.W., Farrell, R., Bilezikjian, M. and Landay, J.A.
(2001). The designers' outpost: A tangible interface for collaborative web
site. In: UIST '01: Proceedings of the 14th Annual ACM Symposium on
User Interface Software and Technology, Orlando, Florida. pp. 1-10.
Kutar, M., Britton, C. and Barker, T. (2002). A comparison of empirical
study and cognitive dimensions analysis in the evaluation of UML diagrams.
In: The 14th Annual Meeting of the Psychology of Programming Interest
Group Workshop, Uxbridge, UK.
Landay, J.A. (1996a). Interactive sketching for the early stages of user
interface design. PhD. Computer Science Department, Carnegie Mellon
University.
Landay, J.A. (1996b). SILK: Sketching interfaces like krazy. In: CHI '96:
Conference Companion on Human Factors in Computing Systems,
Vancouver, British Columbia, Canada. pp. 398-399.
Landay, J.A. (1995). Interactive sketching for user interface design. In:
CHI '95: Conference Companion on Human Factors in Computing Systems,
Denver, Colorado, United States. pp. 63-64.
Landay, J.A. and Myers, B.A. (2001). Sketching interfaces: Toward more
human interface design. Computer, 34 (3), IEEE Computer Society Press,
pp. 56-64.
Landay, J.A. and Myers, B.A. (1996). Sketching storyboards to illustrate
interface behaviors. In: CHI '96: Conference Companion on Human Factors
in Computing Systems, Vancouver, British Columbia, Canada. pp. 193-194.
Landay, J.A. and Myers, B.A. (1995). Interactive sketching for the early
stages of user interface design. In: CHI '95: Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems, Denver, Colorado,
United States. pp. 43-50.
Leland, M.D.P., Fish, R.S. and Kraut, R.E. (1988). Collaborative document
production using quilt. In: CSCW '88: Proceedings of the 1988 ACM
Amir M Naghsh 174
Conference on Computer-Supported Cooperative Work, Portland, Oregon,
United States. pp. 206-215.
Lin, J. (2003). Damask: A tool for early-stage design and prototyping of
cross-device user interfaces. In: UIST'03, Vancouver, Canada. pp. 13-16.
Lin, J., Newman, M.W., Hong, J.I. and Landay, J.A. (2000). DENIM: Finding
a tighter fit between tools and practice for web site design. In: CHI '00:
Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems, The Hague, The Netherlands. pp. 510-517.
Lin, J., Thomsen, M. and Landay, J.A. (2002). A visual language for
sketching large and complex interactive designs. In: CHI '02: Proceedings
of the SIGCHI Conference on Human Factors in Computing Systems,
Minneapolis, Minnesota, United States. pp. 307-314.
Liu, L. and Khooshabeh, P. (2003). Paper or interactive?: A study of
prototyping techniques for ubiquitous computing environments. In: CHI
'03: CHI '03 Extended Abstracts on Human Factors in Computing Systems,
Ft. Lauderdale, Florida, United States. pp. 1030-1031.
Lober, W.B. and Brinkley, J.F. (2000). XML-based image annotation: PAIS -
a personal annotated image server. In: Slice of Life, .
M. Naphade, C.-Y. Lin, J. R. Smith and B. Tseng, S.B. (2002). Learning to
annotate video databases. In: SPIE Symp. on Electronic Imaging: Storage
and Retrieval for Image and Video Databases, San Jose, CA, United States.
Marshall, C.C. (1997). Annotation: From paper books to the digital library.
In: DL '97: Proceedings of the Second ACM International Conference on
Digital Libraries, Philadelphia, Pennsylvania, United States. pp. 131-140.
Marshall, C.C. and Brush, A.J.B. (2004). Exploring the relationship between
personal and public annotations. In: JCDL '04: Proceedings of the 4th
ACM/IEEE-CS Joint Conference on Digital Libraries, Tuscon, AZ, United
States. pp. 349-357.
Marshall, C.C. and Brush, A.J.B. (2002). From personal to shared
annotations. In: CHI '02: CHI '02 Extended Abstracts on Human Factors in
Computing Systems, Minneapolis, Minnesota, United States. pp. 812-813.
Martin, J. (1991). Rapid application development. Macmillan Publishing
Company, New York.
Amir M Naghsh 175
Masoodian, M., McKoy, S. and Rogers, B. (2007). Hands-on sharing:
Collaborative document manipulation on a tabletop display using bare
hands. In: CHINZ '07: Proceedings of the 7th ACM SIGCHI New Zealand
Chapter's International Conference on Computer-Human Interaction,
Hamilton, New Zealand. pp. 25-31.
Morrey, I., Siddiqi, J.I., Hibberd, R. and Buckberry, G. (1998). A toolset to
support the construction and animation of formal specifications.
J.syst.softw., 41 (3), Elsevier Science Inc, pp. 147-160.
Morris, J.H., Neuwirth, C.M., Regli, S.H., Chandhok, R. and Wenger, G.C.
(1999). Interface issues in computer support for asynchronous
communication. ACM comput.surv., 31 (2es), ACM Press, pp. 11.
Morris, M.R., Paepcke, A., Winograd, T. and Stamberger, J. (2006).
TeamTag: Exploring centralized versus replicated controls for co-located
tabletop groupware. In: CHI '06: Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems, Montral, Qubec, Canada. pp. 1273-
1282.
Muller, M.J. (1991). PICTIVE—an exploration in participatory design. In:
CHI '91: Proceedings of the SIGCHI Conference on Human Factors in
Computing Systems, New Orleans, Louisiana, United States. pp. 225-231.
Naghsh, A.M. (2004). An electornic prototyping tool to facilitate
annotations in the early sages of design. In: Doctoral Consortium at
Participatory Design Conference, Toronto, Canada.
Naghsh, A.M. (2004). Towards computer supported collaboration in
electronic paper prototyping. In: Doctoral Consortium at CSCW'04 the ACM
Conference on Computer Supported Cooperative Work, Chicago, United
States.
Naghsh, A.M. and Andy, D. (2005). Supporting collaboration in electronic
paper prototyping. In: The Workshop on Cognition and Collaboration
Analyzing Distributed Community Practices for Design at CHI 2005,
Portland, Oregon, United States.
Naghsh, A.M. and Andy, D. (2004). A demostration of GABBEH: A tool to
support collaboration in electronic paper prototyping. In: CSCW'04 the ACM
Conference on Computer Supported Cooperative Work, Chicago, United
States.
Amir M Naghsh 176
Naghsh, A.M. and Bailly, G. (2005). Denim-Gabbeh multi-user technical
report. Communication & Computing research Centre, Sheffield Hallam
University, Sheffield, UK.
Naghsh, A.M., Danielsson, K., Gumm, D. and Warr, A. (2008). The second
workshop of distributed participatory design. In: CHI 2008, Florence, Italy.
Naghsh, A.M., Dearden, A. and Ozcan, M.B. (2006). Investigating
annotation in electronic paper-prototypes. In: GILROY, S.W. and
HARRISON, M.D. (eds.). DSV-IS'05 Interactive Systems, Design,
Specification, and Verification,12th International Workshop, Newcastle
upon Tyne, UK. pp. 90-101.
Naghsh, A.M. and Ozcan, M.B. (2004). GABBEH - A tool for computer
supported collaboration in electronic paper prototyping. In: A., D. and L.,
W. (eds.). HCI'04 the Annual Conferance of British HCI Group: Design for
Life, Leeds, UK. pp. 77-80.
Naslund, T. (1997). Computers in context –But in which context? In: Kyng,
M. & Mathiassen, L. (Eds). Computers and Design in Context, Cambridge,
MA., United States. pp. 171-200.
Neuwirth, C.M., Chandh, R., Charney, D., Wojahn, P. and Kim, L. (1994).
Distributed collaborative writing: A comparison of spoken and written
modalities for reviewing and revising documents. In: CHI '94: Conference
Companion on Human Factors in Computing Systems, Boston,
Massachusetts, United States. pp. 202.
Neuwirth, C.M., Kaufer, D.S., Chandhok, R. and Morris, J.H. (1990). Issues
in the design of computer support for co-authoring and commenting. In:
CSCW '90: Proceedings of the 1990 ACM Conference on Computer-
Supported Cooperative Work, Los Angeles, California, United States. pp.
183-195.
Newman, M.W. and Landay, J.A. (2000). Sitemaps, storyboards, and
specifications: A sketch of web site design practice. In: DIS '00:
Proceedings of the Conference on Designing Interactive Systems, New York
City, New York, United States. pp. 263-274.
Newman, M.W., Lin, J., Hong, J.I. and Landay, J.A. (2003). DENIM: An
informal web site design tool inspired by observations of practice. Human
computer interaction, 18 (3), pp. 259-324.
Amir M Naghsh 177
Nielsen, J. (1993). Usability engineering. Academic Press, New York.
Nielsen, J. (1992). Finding usability problems through heuristic evaluation.
In: CHI '92: Proceedings of the SIGCHI Conference on Human Factors in
Computing Systems, Monterey, California, United States. pp. 373-380.
Nielsen, J. (1984). How readers annotate textbooks and manuals. 182
Computer Science Dept., Aarhus University,.
O'Hara, K. and Sellen, A. (1997). A comparison of reading paper and on-
line documents. In: CHI '97: Proceedings of the SIGCHI Conference on
Human Factors in Computing Systems, Atlanta, Georgia, United States. pp.
335-342.
O'Neill, E. (2001). User-developer cooperation in software development :
Building common ground and usable systems. Springer, London, UK.
O'Neill, E. (1998). User-developer cooperation in software development:
Building common ground and usable systems.
O'Neill, E., Johnson, P. and Johnson, H. (1999). Representations and user-
developer interaction in cooperative analysis and design. Human-computer
interaction, 14 (1), pp. 43-91.
Ozcan, M.B., Parry, P.W., Morrey, I. and Siddiqi, J.I. (1998). Visualisation
of executable formal specifications for user validation. Selected papers on
services and visualization: Towards user-friendly design, 1358 Lecture
Notes In Computer Science, Springer-Verlag, pp. 142-157.
Palanque, P. and Bastide, R. (1997). Synergistic modeling of tasks, users
and systems using formal specification techniques. Interacting with
computers, 9 (2), Elsevier, pp. 129-153.
Palanque, P.A. and Bastide, R. (1995). Formal specification and verification
of CSCW using the interactive cooperative object formalisms. In: HCI'05
People and Computers, Huddersfield, UK. pp. 213-232.
Parry, P. W. and Ozcan, M. B. (2000). Development of a visual
requirements validation tool. In: Limerick, Ireland.
Pearsall, J. and Trumble, B. (2002). Oxford english reference dictionary.
Oxford University Press, 2ed edition; Oxford, UK.
Amir M Naghsh 178
Perry, M. and Sanderson, D. (1998). Co-ordinating joint design work: The
role of communication and artefacts. Design studies, 19 (3), Elsevier
Science Ltd, pp. 273-288.
Pilatti, L., Audy, J.L.N. and Prikladnicki, R. (2006). Software configuration
management over a global software development environment: Lessons
learned from a case study. In: GSD '06: Proceedings of the 2006
International Workshop on Global Software Development for the
Practitioner, Shanghai, China. pp. 45-50.
Plat, N., Katwijk, J.V. and Toetenel, H. (1992). Application and benefits of
formal methods in software development. Software engineering journal, 7
(5), Software Engineering Journal, pp. 335-347.
Plimmer, B. and Apperley, M. (2003). Evaluating a sketch environment for
novice programmers. In: CHI '03: CHI '03 Extended Abstracts on Human
Factors in Computing Systems, Ft. Lauderdale, Florida, United States. pp.
1018-1019.
Preece, J., Preece, J., Rogers, Y. and Sharp, H. (2002). Interaction design:
Beyond human-computer interaction. John Wiley & Sons, Inc, New York,
NY, United States.
Pycock, J. and Bowers, J. (1996). Getting others to get it right: An
ethnography of design work in the fashion industry. In: CSCW '96:
Proceedings of the 1996 ACM Conference on Computer Supported
Cooperative Work, Boston, Massachusetts, United States. pp. 219-228.
R., B., D., N. and P., P. (2003). A tool-supported design framework for
safety critical interactive systems. Interacting with computers, 15 (3),
Elsevier, pp. 309-328.
Rettig, M. (1994). Prototyping for tiny fingers. Communications of the
ACM, 37 (4), ACM Press, pp. 21-27.
Roast, C., Khazaei, B. and Siddiqi, J.I. (2000). Formal comparison of
program modification. In: J., S. and J., P.J. (eds.). IEEE Symposium on
Visual Languages, pp. 165-171.
Roast, C.R. (1998). Modelling unwarranted commitment in information
artefacts. In: CHATTY, S. and DEWAN, P. (eds.). Engineering for Human-
Computer Interaction, pp. 70-90.
Amir M Naghsh 179
Roast, C.R. and Siddiqi, J.I. (1997). Usability requirements as specification
constraints: An example ofWYSIWYG. Software engineering. IEE
proceedings, 144 (2),.
Rosson, M.B., Kellogg, W. and Maass, S. (1988). The designer as user:
Building requirements for design tools from design practice.
Communications of the ACM, 31 (11), ACM Press, pp. 1288-1298.
Rosson, M.B., Maass, S. and Kellogg, W.A. (1987). Designing for
designers: An analysis of design practice in the real world. In: CHI '87:
Proceedings of the SIGCHI/GI Conference on Human Factors in Computing
Systems and Graphics Interface, Toronto, Ontario, Canada. pp. 137-142.
Rudd, J., Stern, K. and Isensee, S. (1996). Low vs. high-fidelity
prototyping debate. Interactions, 3 (1), ACM Press, pp. 76-85.
Ryall, K., Forlines, C., Shen, C. and Morris, M.R. (2004). Exploring the
effects of group size and table size on interactions with tabletop shared-
display groupware. In: CSCW '04: Proceedings of the 2004 ACM
Conference on Computer Supported Cooperative Work, Chicago, Illinois,
United States. pp. 284-293.
Sharp, H. and Robinson, H. (2004). An ethnographic study of XP practice.
Empirical softw.engg., 9 (4), Kluwer Academic Publishers, pp. 353-375.
Siddiqi, J.I., Morrey, I.C., Roast, C.R. and Ozcan, M.B. (1997). Towards
quality requirements via animated formal specifications. Ann.softw.eng., 3
J. C. Baltzer AG, Science Publishers, pp. 131-155.
Siddiqi, J.I., Morrey, I., Hibberd, R. and Buckberry, G. (1999).
Understanding and exploring formal specifications. Ann.softw.eng., 6 (1-4),
J. C. Baltzer AG, Science Publishers, pp. 411-432.
Triffitt, E. and Khazaei, B. (2002). A study of usability of Z formalism based
on cognitive dimensions. In: KULJIS, J., BALDWIN, L. and SCOBLE, R.
(eds.). The Fourteenth Annual Meeting of the Psychology of Programming
Interest Group, pp. 15-28.
Tullis, T.S. (1990). High-fidelity prototyping throughout the design. In:
Human Factors Society 34th Annual Meeting, pp. 266.
Uceta, F.A., Dixon, M.A. and Resnick, M.L. (1998). Adding interactivity to
paper prototypes, proceedings of the , 506-511. In: Human Factors and
Ergonomics Society 42nd Annual Meeting, pp. 506-511.
Amir M Naghsh 180
Walker, M., Takayama, L. and Landay, J.A. (2002). High-fidelity or low-
fidelity, paper or computer medium? In: Human Factors and Ergonomics
Society 46th Annual Meeting: HFES2002, pp. 661-665.
Weng, C. and Gennari, J.H. (2004). Asynchronous collaborative writing
through annotations. In: Proceedings of the 2004 ACM Conference on
Computer Supported Cooperative Work, Chicago, Illinois, United States.
pp. 578-581.
Wilson, S., Bekker, M., Johnson, P. and Johnson, H. (1997). Helping and
hindering user involvement — a tale of everyday design. In: CHI '97:
Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems, Atlanta, Georgia, United States. pp. 178-185.
Wing, J.M. (1990). A specifier's introduction to formal methods. Computer,
23 (9), IEEE Computer Society Press, pp. 8-23.
Wojahn, P.G., Neuwirth, C.M. and Bullock, B. (1998). Effects of interfaces
for annotation on communication in a collaborative task. In: CHI '98:
Proceedings of the SIGCHI Conference on Human Factors in Computing
Systems, Los Angeles, California, United States. pp. 456-463.
Wong, Y.Y. (1992). Rough and ready prototypes: Lessons from graphic
design. In: CHI '92: Posters and Short Talks of the 1992 SIGCHI
Conference on Human Factors in Computing Systems, Monterey, California.
pp. 83-84.
Amir M Naghsh 181
APPENDIX A
This appendix presents a more detailed background on Gabbeh, a hand-woven Persian
rug.
Amir M Naghsh 182
Amir M Naghsh 183
Amir M Naghsh 184
Amir M Naghsh 185
APPENDIX B
This appendix presents the design tasks used in the observational study, as well as the
results of the study which was used for the analysis of Chapter 7.
RESEARCH GROUP WEBSITE TASK BRIEF
Please could you design a website using the Denim tool.
The website is intended to provide information about a group of researchers at
Sheffield Hallam University.
Contents should include:
• An attractive home page
• A page for each individual researcher to describe their work - this should
include personal information about the researcher and links for students to find
out about courses that the researcher teaches
• Pages describing research projects that the group is involved in
• A page listing publications for the whole group (by year)
• Pages for individual researchers publications, perhaps organised according to
projects or titles
• A page of events in which people in the group are involved
• Links to the host institution & research institute
• Photos of the people involved in the research & other images related to the
research (e.g. of prototypes etc.)
You may have some additional suggestions that could enhance the site, given its aims.
Amir M Naghsh 186
VISIT SHEFFIELD WEBSITE TASK BRIEF
Your design should take into account the fact that many of the intended users may be
using slow dial-up lines.
Please could you design a website for www.visitsheffield.co.uk, using the DENIM
tool.
The website is intended to attract tourists and other visitors to Sheffield.
• Contents should include:
• An attractive home page
• Information about how to get to Sheffield (by road, rail, air etc.)
• Information about the region around Sheffield
• Information about how to get around Sheffield & the region
• Information about accomodation with suitable links to providers
• Information about Leisure activites in Sheffield, e.g. sports, nightlife,
attractions etc.
• Information about restaurants / eating & drinking etc.
• A news area that can be regularly updated.
You may have some additional suggestions that could enhance the site, given its aims.
Your design should take into account the fact that many of the intended users may be
using slow dial-up lines.
Amir M Naghsh 187
List of comments collected from the observational study:
No Comment Group Session Creator Classification
1 A better and more descriptive "Title" for home page such as Sheffield Hallam University Research Group (SHURG)
1 DENIM end-user Altering
2 Add a picture on top of the page which shows all the members of the team
1 DENIM end-user Altering
3 Second page, changing title to Team of Researchers
1 DENIM end-user Altering
4
In Projects, we need a page with 3 boxes that identifies the research subjects: (i.e., Software Development, Networking and possibly have room for adding one or more research area to the page)
1 DENIM end-user Altering
5 Having a scurrility option on research projects
1 DENIM end-user Altering
6 We dont understand point 5 of your requirements
1 DENIM designer-
user Exploring
7 Can you please provide us with more details about requirements 4 and 5 on the original "WebSite Design Task Brief".
1 DENIM designer-
user Exploring
8 We have provided a page with a list of group projects.
1 DENIM designer-
user Clarifying
9 We also provided links to these projects from researchers pages. Does this meet the requirements?
1 DENIM designer-
user Verifying
10
In option 4: We want to have 4 selections, rather than 3 (for our future research) and we prefer to have them in a boxes to makes them more attractive
1 DENIM end-user Altering
11
in Option 5: We need security on each project subject as we want each researcher to have security on their research area. Please let me know if it is not still clear?
1 DENIM end-user Altering
12 Interactive set of photos 1 Gabbeh designer-
user Clarifying
13 if this is link to shu search engine I need to have box to input the keyword to search for.
1 Gabbeh end-user Understanding
14 link opens the search engine in a new page
1 Gabbeh designer-
user Clarifying
15 Add Shu logo on top of this page 1 Gabbeh end-user Altering
16 please ignore the first comment 1 Gabbeh end-user Organising
17 add a list of publication without abstract in this pgae
1 Gabbeh end-user Altering
18 list of publication grouped by year 1 Gabbeh designer-
user Verifying
19 have link to website and the location of publication to have view of paper
1 Gabbeh end-user Altering
20 here we provide links to each paper. Is that what you wanted?
1 Gabbeh designer-
user Verifying
21 yes that’s what I wanted 1 Gabbeh end-user Confirming
Amir M Naghsh 188
No Comment Group Session Creator Classification
22 add two schedule for two semesters 1 Gabbeh end-user Altering
23 timetables are added to this page for each semester
1 Gabbeh designer-
user Verifying
24 news related to projects and success stories
1 Gabbeh designer-
user Clarifying
25 Can you please provide us with more details about consultancy page?
1 Gabbeh designer-
user Exploring
26 Here we have provided a list of consultancy options. Does this meet your need?
1 Gabbeh designer-
user Verifying
27 each of these options should be a link to another page
1 Gabbeh end-user Altering
28 here we provide a link and short summary for each option.
1 Gabbeh designer-
user Verifying
29 no more changes is required 1 Gabbeh end-user Confirming
30 maybe you can separate the news page so it can be more interactive and updated now and then?
1 Gabbeh end-user Understanding
31 a link to publication page is added 1 Gabbeh designer-
user Verifying
32 here we provided a list of your projects and experiments
1 Gabbeh designer-
user Clarifying
33 abstracts are moved to the publication page
1 Gabbeh designer-
user Clarifying
34 new pages are added for these options 1 Gabbeh designer-
user Verifying
35 list of case studies with a brief overview 1 Gabbeh designer-
user Clarifying
36 On the home page, there seems to be a lot of text, can you make it more interactive?
2 DENIM end-user Altering
37 I didn't fine any information on accommodation?! Can you add one?
2 DENIM end-user Altering
38 how to get to Sheffield should cover all the means of getting to Sheffield
2 DENIM end-user Altering
39 add more information about the activities in Sheffield
2 DENIM end-user Altering
40 please check the updates 2 DENIM designer-
user Organising
41 this home page consists of images and links to each individual page that describes each ac
2 Gabbeh designer-
user Clarifying
42
we provide a photograph of each researcher together with links to their profiles and a quick preview of their interests at here.
2 Gabbeh designer-
user Clarifying
43 An introduction on what researchers do on day to day basis here at Sheffield Hallam university
2 Gabbeh designer-
user Clarifying
44 This page will generate the publications by year depending on where the time line is set to by the user of the site
2 Gabbeh designer-
user Clarifying
Amir M Naghsh 189
No Comment Group Session Creator Classification
45 We provide an area for some text about the group in addition to group photo
2 Gabbeh designer-
user Verifying
46 can we have a group link instead of individual one
2 Gabbeh end-user Altering
47 could you add links to host university and other research centres in this page?
2 Gabbeh end-user Altering
48 links to sponsored research inst 2 Gabbeh designer-
user Verifying
49
this page hold an image of the researcher and their interests in more details, also include don this page is a list of their interests and current work. the courses that they teach are also included here
2 Gabbeh designer-
user Clarifying
50 can you put the researcher name instead of about me?
2 Gabbeh end-user Altering
51 There should be a page listing all publications for this researcher.
2 Gabbeh end-user Altering
52 link will grab all publications by name and year supplied as an attribute of the link
2 Gabbeh designer-
user Clarifying
53 current projects that are carried out by post graduate students at Sheffield Hallam university
2 Gabbeh designer-
user Clarifying
54 can we have a small introduction on each project and also list of the people who are walking on that project
2 Gabbeh end-user Altering
55 This page will generate the publications by year depending on where the time line is set to by the user of the site –
2 Gabbeh designer-
user Clarifying
56 Good idea... the time line 2 Gabbeh end-user Confirming
57 What do you mean by other visitors? 3 DENIM designer-
user Exploring
58 Do you want to search the facilities as well as browsing?
3 DENIM designer-
user Exploring
59 We have not used animation sounds frames for the dialup line limitation. Is there feedback on this?
3 DENIM designer-
user Verifying
60
Other users would include all the people who need to find some information regarding Sheffield. Students, businessmen and women and so on
3 DENIM end-user Understanding
61 I think we should not have problem with the speed
3 DENIM end-user Confirming
62 Can we have some pictures of key places on the home page?
3 DENIM end-user Altering
63 what do you mean by search and browsing?
3 DENIM end-user Understanding
Amir M Naghsh 190
No Comment Group Session Creator Classification
64 add a page for how to get to Sheffield 3 DENIM end-user Altering
65 what's the search box for in the accommodation page?
3 DENIM end-user Understanding
66 We leave the news area in the homepage.. how regularly they will be updated?
3 Gabbeh designer-
user Exploring
67 Do you need any automation for the news area?
3 Gabbeh designer-
user Exploring
68 what is the big box? 3 Gabbeh end-user Understanding
69 can you add e about us page or an space on this page to give a overview of the research group?
3 Gabbeh end-user Altering
70 a group picture 3 Gabbeh designer-
user Clarifying
71 this links to a new page 3 Gabbeh designer-
user Clarifying
72 graphic/pic 3 Gabbeh designer-
user Clarifying
73 list of staff names and link to their homepages
3 Gabbeh designer-
user Clarifying
74 can we separate these people in different groups like researchers, staff , post graduates and so on?
3 Gabbeh end-user Altering
75 this is your profile 3 Gabbeh designer-
user Clarifying
76 can I have a link to my publications and one to show the projects that I am working on?
3 Gabbeh end-user Altering
77 requested sections were added under my research.
3 Gabbeh designer-
user Verifying
78 information regarding a course, e.g. web based
3 Gabbeh designer-
user Clarifying
79 can we have a summary for each project? 3 Gabbeh end-user Altering
80 This page list the publications by year and by project
3 Gabbeh designer-
user Clarifying
81 can you give more details for each publications?
3 Gabbeh end-user Altering
82 the project page, project overview and people involved
3 Gabbeh designer-
user Clarifying
Amir M Naghsh 191
APPENDIX C
This appendix presents the design task used in the longitudinal study, as well as the
results of the study which was used for the analysis of Chapter 9. This appendix also
presents the final version of the prototype that was generated by end of the longitudinal
study.
A T-SHIRT WEBSITE
Today popularity of online purchasing is increasing. Customers prefer to buy their
needs online. Internet has provided a limitless opportunity to shop online. An internet
user can surf online, finds its own needs, compares prices and decides in time. There is
no pressure on the customer by the shop assistant to decide whether they want the
product or not. People can shop in the middle of night. Internet is not limited to where
you are, who you are or time. The other advantage of online shopping is buying and
sending present to others. This service is not offered by the most of the shops in high
street. Therefore there is a need for a rapid and user friendly website offering wide
ranges of services for its users. As clothing is a big market including all classes of the
people, we are looking forward to selling high quality t-shirts on line. The website
should be powerful and easy to use. Every individual apart from its age and gender
should be able to use it and shop online. It is needed that the website provides an
excellent service for those professional who are looking for unique services. On the
other hand, the website should offer strong designing tool for those who are not
satisfied with ready products and desire their own design. Therefore the website covers
two categories:
• Ready t-shirts category
• Design tool
READY T-SHIRTS
This category contains t-shirts designed by the Company. The category should be
subdivided in different classes depends on the users interest.
Amir M Naghsh 192
DESIGN TOOL
This section of the website offers the user a powerful tool to design its own t-shirt. This
is the main feature of the website. The user can select type of the t-shirt, colour and
size. Also the user can add its own logo, sign or by using website toolbar can create a
one. Also the users should be able to select position of its own logo on the t-shirt.
After selecting the t-shirt, the user can save its own favourite design or t-shirt; or
proceed to purchase it. Each user can create an account. The account shows billing
history, order history, purchase history and users information. Also there should be a
section for tracking an order.
Amir M Naghsh 193
RESULTS
Page Author ID Reply Comment Classification
designer 85 This will be in colour and more attractive! Clarifying
user 117 85 It is very good. I think we are making progress. Confirming
However, I think the photo should be in the middle of the page and the comment will come beneath it. Altering
designer 134 117 Ok, I have moved the picture to the middle, Clarifying
is it what you want? Verifying
user 148 134 A recent survay shows a new visitor only Justifying
spends 20 sec on the first page, deciding
to continue surffing or leaving! It is very
important to impress the visitor at first
sight. So this page is very important.
Words should be selected wisely to attract
any group.
designer 3 Will there be just a 'tshirt' image or 't-shirt on 'cool guy' and/or 'cool girl'? Exploring
user 23 3 It should attract people in first click, Understanding
so on this page Several photos could be used, Altering
each photo can represent a catogery.
designer 86 23 Maybe something like this? I think Verifying
it is good to have top sellers here. Clarifying
user 118 86 It is good Confirming
the photo change randomly. Altering
designer 135 118 yes, it would be more effective and it will be like that in the Clarifying
actual website. Keep this in mind that this is only a
simple prototype and it is does not support that kind of annimation
but once we agree on the design, I can begin
the development of the actual site which would support
that sort of annimation.
user 150 135 I look forward to that feature. Confirming
Hom
e
Amir M Naghsh 194
user 24 I don't like "Have Fun!" Confirming
designer 49 24 have fun is a much more customer Justifying
oriented way to get them to create!
see sites like www.innocentdrinks.com
they do this, and it works very well
as a marketing tool
user 64 49 Have fun is a common phrase which Justifying
everyone uses it.I am looking for
something especial, something that
no oneelse has used it before.
Such as:"select, Altering
design, buy & wear!"
designer 74 64 I have changed it incorporating your suggestion Clarifying
user 119 74 We are getting close! Confirming
user 69 49 Yes you are right for drinks, for example Justifying
no one buys drinks online from that website,
that might works for that kind of the company
but I want to sell t-shirts online. That company is
selling its products in stores not online. The website
is part of their customer service. Thus, we want
to sell online, so should attract people as well as
it should be professional.
designer 73 69 I see what you mean, but its about identifying Justifying
with the customer, and creating an
"online customer service" -
anyways, if you dont want 'have fun' Exploring
please send through an alternative phrase
user 126 It would be nice if the website shows name Altering
of the returning user on the top of the page,
such as: "Welcome back....". Also there should be
an option to reset the name when other user
is using same computer!
designer 138 A welcome message was added to Clarifying
the design page for the returining users,
but I also have added that to here.
Is this what you wanted? Verifying
Hom
e
user 151 It would better to say: Altering
Amir M Naghsh 195
If you are not Name" please click here".
user 159 How about the idea of creating a category Altering
named "innocent t-shirt" which the user can
add a logo or a message related to a charity
or peaceful group?
designer 166 159 sure, here I am only providing a simple Clarifying
prototype, you can change the content
when I begin the designing of the actual
website.
designer 16 One section is "catagories" what other sections would you like here Exploring
Male/female?
Size?
Colour?
Anything else?
user 32 16 On this page we need to state only main catogeries: Altering
The user should be able to find her/his
favourite catogery, for example :
adult t-shirt, under 18, Fun tshirt, plain t-shirt
Romance t-shirt etc
designer 55 32 Ok, the catagories can be added in, Clarifying
and these will probably be represented
in pictures of the catagories
user 70 Users should directed to a new page after Altering
selecting their category. Then they should
be able to select size, colour etc. Please visit
tshirthell.com for more ideas.
designer 78 70 yes, this will be the case Clarifying
user 128 Where are the new pages for each category? Understanding
designer 168 128 not designed yet Clarifying
user 120 I want the photos change randomly and be updated Altering
with the news designs.
designer 140 120 it will be like that in the actual site Clarifying
T-s
hirt
user 152 140 So would you please make links from Altering
Amir M Naghsh 196
this
page to the actual site.
designer 169 152 I have to remind you that it is only a Clarifying
simple prototype by actual site I mean the website which I will develope later from this prototype
user 153 It will be great if the name of the user be shown Altering
on each page or make an option for the
new users to make account if they decide
T-s
hirt
to do so.
user 72 I was thinking about returned users. How the Understanding
page will be shown for them.
designer 101 72 they can log in using a username and Clarifying
password.
(click on the link to see how it would be when users log in) Organising
user 121 101 Please see my comments on the new pages. Organising
user 129 Can the website allow all format of the photos Understanding
to be uploaded?
Is there any limit for file size?
designer 143 129 well, it is a more of a technical issue, Clarifying
yes it would allow all the formats, but in regards
to the limit, it would deponds on your hosting
contract. I might be able to get more information
on different available deals in the market and give you some prices
and then you can decide on the limit.
user 30 Here, the user should be able to choose Altering
t-shirts models and designs such as:
Coulor, Size, Patterns etc.
By choosing each section the preview window
should be updated.
Could you please put a toolbar for the design Altering
tools.
designer 58 30 The middle box here is the design tools - Clarifying
my apolgies thats was not clear
Desig
n
Amir M Naghsh 197
user 68 How about changing the heading to: Altering
Design & Buy
designer 91 68 I don't think that sounds good for Justifying
a heading but I have changed it! Clarifying
user 123 91 Would you please synchronise all the headings. Altering
designer 142 123 I have changed the Design & Buy, to "Design" Clarifying
The user is not really buying anything in this page, Justifying
User can design a pattern here and then add the
finished design in to the basket, and that's where
user can buy the t-shirt.
user 155 142 Agree! Confirming
designer 20 Here is the page for customers to customise their choices Clarifying
designer Will there be an option for them to upload thier own pictures? Verifying
user 29 20 Yes, we need that feature Confirming
user 31 Also we need a feature for the user to Altering
be able to open an account so he could save
his work in case of the future use.
designer 57 31 I have added a log on for existing customers Clarifying
user 63 57 How about new users? They should be able to open a new account. Altering
designer 79 63 there should be a button for that, yes. Clarifying
user 158 Can you change "save" to "save as..." Altering
Desig
n
designer 108 A list of pictures uploaded by user Clarifying
I think it is important to have,
make it quicker for the user to reuse
a logo, if he does not have a fast internet
connection to upload it everytime
user 124 108 We can give the users ability to name their Altering
designs and save it under their own name!
designer 144 124 yes, I think that's what I meant. as you can Clarifying
see there are two options, one is saved designs,
D
esig
n a
nd B
uy
and the other is saved pictures.
Amir M Naghsh 198
Saved design is a list of desing names.
user 156 144 Is it possible to have a preview window next Understanding
to the designed names, just to remind the user
a taste of the designed t-shirt.
designer 164 156 Yes it is possible, the preview screen Clarifying
in middle of the page is designed
to allow a user to preview previously
designed t-shirts as well as new ones.
designer 107 A list of saved to designs, Clarifying
user can re-use them, or
Desig
n a
nd
Buy
edit them further
user 131 The user should be forwarded to a secure page Altering
for entering credit card number.
designer 172 131 yes it will be like that in the actual website Clarifying
user 130 The user should be able to track old orders or Altering
purchases as well as a complete history of
all transactions.
user 132 Also it would be great if the system keeps Altering
a record of the user's card number in case
for speeding up the purchase (if and only if
the user wish) which can appear as an
Basket
option.
designer 110 This option is for Clarifying
user 157 Can you add an option for sending news emails Altering
to the user email address, for promoting
our new designs or products?
designer 165 157 yes it will be added Confirming
user 122 Before saving the user should accept the company Altering
New
User
policy.
designer 146 122 Ok Confirming
user 28 Please change page name from "about us" to Altering
Amir M Naghsh 199
Corporation
designer 51 28 i dont want to put 'coporation' again, Justifying
its off putting, a coporation sounds
like a PLC, when really we want to get
the mass market - about us, helps
the customer to 'relate' to us
(its a pyschological marketing attempt)
Abo
ut U
s
user 66 51 People should think that a professional Justifying
group who are doing marketing, finance, delivery etc.
are behind this website.
So those who have concern about the company
will come to this page.
Therefore this page should be very
professional with strong message insurring visitors.
designer 75 66 still to me coporation sounds "we are bigger than you" Justifying
when that is not the essense of the site, unless there
is a way (like amazon) to allow the customers' name
to appear, therefore they will privaliged
An alternative suggestion "company"
user 125 75 ok, "company" is good in that case. Confirming
designer 5 > doing good things? Clarifying
A page highlighting your companies 'donation' of 10p per shirt to a good charity
user 25 5 Does this mean a new page? Understanding
Have you designed anything for it?
designer 52 25 Yes, three more pages are needed Clarifying
user 160 How about saying "innocent t-shirts"? Altering
designer 170 160 it can go in to the Clarifying
designer 15 Do you want a page on 'personnel' within the company? Verifying
user 27 15 No, I don't think that would be needed Confirming
designer 76 27 ok Confirming
designer 4 > news Clarifying
this will feature a separate page of latest news
Is this something that you want? Verifying
Abo
ut U
s
user 26 4 I need a link here to a new page Altering
Amir M Naghsh 200
containing
company's news.
designer 77 26 news is also inclusive of company news Clarifying
designer 6 > our story Clarifying
The story behind the 'company' formation
user 67 I think the order should be like this: Altering
Home
t-shirt
Design
contact us
corporation
Abo
ut U
s
I suggest you to visit good websites such as
amazon!
designer 99 67 ok Confirming
designer 19 Should you provide a page on "good links" Verifying
user 33 19 No, This feature is not necessary. Confirming
But what do u mean by good links? Understanding
designer 17 Do you want a page where 'customers' can leave their comments? Verifying
user 34 17 yes, feedback feature is important for the Confirming
company. But I was thinking to have it in Altering
a seperate page.
designer 59 34 yes, i agree, feature should be on Confirming
a separate page, titled "comments" Clarifying
designer 60 I think you have to feature address, Justifying
otherwise people wont feel confident
C
onta
ct
Us
to buy from "unknown" companies,
especially as it is a new company
user 161 Can the system send a thanking message to the Altering
user after sending his cooments to show
Com
ment
s
our appreciation for his/her feedback?
designer 171 161 Yes Confirming
Amir M Naghsh 201
FINAL DESIGN
Amir M Naghsh 202
Amir M Naghsh 203
Amir M Naghsh 204
APPENDIX D
Figure D DENIM UML diagram
4