usability studies and designing navigational aids for the world wide web

Computer Networks and ISDN Systems 29 (1997) 1489-1496

Usability studies and designing navigational aids for the World Wide Web

D. Bachiochi a M. Berstene b,l, E. Chouinard a, N. Conlan b, M. Danchak a.*, T. Furey b, 7 C. Neligon b, D. Way b

URensselaer Polytechnic Institute, Troy, NY 12180-3590, USA ‘Aetna. Inc, Hartford, CT 06156, USA

Abstract

Web designers are constantly searching for ways to improve their works. Recently published books provide such recommendations, but their quality varies greatly. This paper describes how usability testing was used to validate design recommendations. The results show a need for navigational aids that are related to the particular Website and located beneath the Browser buttons. Furthermore, usability criteria were established that limit page changes to 4 and search times to 60 seconds for information retrieval. 0 1997 Published by Elsevier Science B.V.

Kqvwords: Usability; Navigation; World Wide Web, Web design

1. Introduction

The undisputed popularity of the World Wide Web has gained the attention of everyone! The Web is no longer reserved for the technical champions; the technically challenged are also asking for equal rights and access. But regardless of their capabilities, all users of the web are faced with a common task that will determine their success or failure. That is the task of finding your way along the information superhighway, or wayfinding

Wayfinding is defined as the “purposeful, oriented movement during navigation” [2]. There are four basic questions one must ask when traversing the

* Corresponding author. E-mail: [email protected] ’ E-mail: [email protected]

peaks of the Grand Tetons, the caverns of New York City or the wonders of the World Wide Web:

Where am I? Where do I want to go? Am I on the right path? Am I there yet?

Navigation is then the process of determining a path to be traveled through the chosen environment [I]. In other words, navigation addresses the question of “Where do I want to go?“. Much of the current wayfinding research literature deals with virtual real- ity. However, the Web will quickly become a major focus of wayfinding research as users become more frustrated with poor designs.

Many users do not see a navigational problem with the Web. They are content with the features provided by the Browser, such as “Back” and “For-

0169-7552/97/$17.00 Q 1997 Published by Elsevier Science B.V. All rights reserved PII SO169-7552(97)00027-5

1490 D. Bachiochi et al. /Computer Networks and ISDN Systems 29 (1997) 1489-1496

Fig. 1. Generic page design. Horton’s navigation buttons comprise seven functions: Content, Map, Index, Home, Up, Previous and Next, We grouped the last three together and called them local navigation buttons. These three allow you to maneuver within the Website structure based on your current position. A defined hierarchy or organization is very important to this design. “Up” takes you one level up in the site hierarchy. “Previous” takes you to the web page that resides to the left of the current page in the organization, whereas “Next” takes you to the page to the right.

ward’, and the hypertext links found within Websites to get where they want to go. Browser navigational aids are particularly misunderstood, even by experienced and sophisticated users. A recent study [4] showed that computer scientists do not recognize the fact that Browser historical lists are based on a push- down stack concept rather than a temporal list. As a result, users end up at surprise locations. This is not a problem if you have time to waste. After all, “surf- ing” is a past-time. As the Web matures, particularly the Intranet, user efficiency will become more of an issue. Nielsen [5] points out the problems of navi- gating large spaces with hypertext links. This study concentrates on the navigational aids, provided by the Website designer, that complement the Browser functionality. Inter-Website maneuvering is typically done via the Browser, intra-Website maneuvering must be the responsibility of the designer. We call this “structural wayfinding” since it is predicated on the structure of the Website, not the content.

2. Background

A number of good design guidelines have become available in the recent months. William Horton et al.‘s book entitled “The Web Page Design Cook-

book” [3] is an excellent reference and contains very specific recommendations on structural wayfinding. They present a generic web page design and advo- cate using text and icon navigation buttons as well as path buttons to make life easier. These navigational aids are abstracted in Fig. 1. Clicking on these buttons take you to different places in the Website organization or structure.

The remaining four functions take you to a specific page in the organization. ‘Contents” accesses a Web page that resembles a table of contents for the Website. Hierarchical organization is shown using indentation, much like a textbook. Hypertext links allow the user to go to the desired page directly from this table of contents. The “Map” page is a graphical representation of the table of contents; it shows the structural layout using spatial relationships and each page can be accessed from the map. The “In- dex” function presents a list of words in alphabetical order, again similar to a textbook, linked to their occurrence in the site. Lastly, the “Home” function takes the user to the home page of the particular Website.

Taken together, these four buttons provide a pow- erful and complementary navigational scheme that appeals to different users. This appeal is extended to the form of the buttons at the top and bottom of each

D. Bachiochi et al. /Computer Networks and ISDN Systems 29 (1997) 3489-1496 1491

web page. Text is used to represent the functions at the top of the page. This provides functionality without interfering with the Website graphics. Icons are used at the bottom of the page to provide re- dundancy and another representational scheme for diverse users.

The Path buttons provide an alternate means of traversing vertically in the hierarchy. As one pro- gresses down the hierarchy, a link label for the page just departed is added to list. One can move up one page in the hierarchy using the “Up” button or the link in the path button list. Path buttons also allow you to jump more than one level by merely clicking the page link label and to maintain an orientation relative to the site hierarchy.

As with many guidelines, one is never sure if the recommendations given are based on research, experience, or merely author preference. The authors of this book have excellent reputations, but research evidence was lacking. The recommendations were very logical and attractive. but needed validation through formal testing. Not that testing must be done on everything - but certainly on major points that affect user efficiency. A lost user is one who will never return to an intemet Website or who wastes valuable time on an intranet site.

3. Testing

To validate the recommendations discussed in the previous section, we used Aetna’s Usability Labo- ratory and employees from that company as evaluators. The Laboratory is state-of-the-art, incorporating multiple TV cameras, recording devices and obser- vation areas. Since the questions being asked were important to Aetna’s corporate-wide Web page design effort, we configured the test room to emulate conditions normally found in an Aetna workspace. This included a 486 processor with VGA display and Netscape Navigator 2.0.2. The test Website consisted of 45 separate pages, with a hierarchical structure, and was located on a remote file server. This was done to optimize access and concentrate on the actual navigational features of the design. The test was conducted in two phases. Each phase consisted of an explanation of the navigational aids available, an opportunity for the evaluators to explore the Website

using these features, and then actual testing using a series of questions related to the Website. In some instances the answers to these questions were found on a single page and others required synthesis of information from a number of pages. Time to locate and answer each question was recorded, as was the number of page changes required to find that answer. Debriefing sessions solicited comments from each evaluator in each phase.

The original intent was to test the navigational concepts on the test Website (Phase 1) and then ap- ply the results to an existing Aetna Website (Phase 2) for a before and after test. This plan was aban- doned during Phase 1 for a number of reasons, most importantly the lack of a completed Website that was sufficiently complex to adequately test the concepts. Thus, the basic Website was used in both phases, but some redesign of pages and navigational features occurred between the two phases.

4. Phase 1 design

In the first Phase we wanted to look at specific aspects of the navigational aids; individual aids, placement and format of aids, and finally the appropriate combinations of these aids. Three different tasks were devised, each looking at one of these issues, as shown in Table 1. In this Phase, all Browser functions were disabled.

Task 1 dealt with the individual buttons. One group of evaluators was just given Local Navigation buttons plus a “Home” button. They served as the basic control. For the second group we added Struc- ture Buttons to the four just mentioned, whereas Path buttons were added for the third group. The next task looked at placement and form of navigation buttons. Each group had Structure and Local Navigation but-

Table 1 Phase 1 test conditions

Condition Condition Condition

Task 1 Local Navigation Structure buttons Path buttons buttons

Task 2 Text - top, Text - bottom, - icons - top icons - bottom -

Task 3 Text - top. Icons - top. -

icons - bottom text - bottom -


tons as well as Path Buttons. However, the Structure + Local Navigation buttons were placed at either the top or bottom of each page and were represented using either text or icons. The last task included all forms of buttons and varied the combinations, as shown in Table I.

The design of Phase 1 required at least 24 evaluators, so as to have 6 evaluators per variable condition (25 evaluators actually took part in this phase). The evaluators were drawn from the general Aetna employee population and basically reflected that mix. Most were over thirty years of age, female, worked at Aetna for more than five years, used computers more than ten hours per week and logged onto the Web once a day.

5. Phase 1 results

No statistically significant results were found in this Phase relative to time or number of page changes. However, evaluator comments were especially rich in problems and ideas. The lessons learned can be summarized as follows:

First and foremost, evaluators wanted navigation buttons fixed at the top, in the region of the Browser buttons. Scrolling to button locations is the user’s least favorite operation. Locating buttons at the top and bottom (Structure + Local Navigation) and the middle (Path Buttons) were a convenience, but not a solution. They merely reduced the amount of scrolling rather than elimi- nating it. Users quickly become patterned by the Browser used. Evaluators with even minimal Web experienced often remarked “I’d use “Back’ here”, or similar comments. This may also reflect the new- ness of our navigational aids to this population. “Home” is a very important concept. When evaluators became lost, they typically returned to the home page to reorient themselves. The problem is which home! A typical session displays the user’s home via the Browser button, and the Website home using text or icons. Users do not read blocks of text; they jump around and react to links. In many instances, the evaluators scrolled right by an answer because it was on the second or third line of a paragraph. Outline

form seemed to be the best organization of textual material.

l The Local Navigation buttons were very confusing, especially Previous and Next. For experienced web users, there was a lot of mental con- flict with the “Back” and “Forward’ concept of the Browser. Novice users just did not understand the concept.

l Tables of Contents, Maps, and Indices are valuable additions to a Website, provided they reflect the true and detailed structure of the Website. The map used provided a nice graphic, but did little to help the evaluators navigate. Detailed Indices made searching very easy. They also provided a context or boundary for the search that is not available with search engines. Seeing the possible choices is much easier than trying to construct a choice from memory. These results provided a starting point for Phase 2.

There were still some unanswered questions concem- ing Local Navigation and Path buttons, placement of Structure buttons, and the interaction of Website navigational aids with Browser functions. These were addressed in the next Phase.

6. Phase 2 design

Given these results, we designed another test to look at two distinct issues: the value of Structure buttons and fixing the buttons at the top near the Browser buttons. Table 2 summarizes the design of this Phase.

The control condition was the standard Browser button configuration plus a “Home” link at the bottom of each page. The second condition basically repeated the recommendations of Horton et al. [3]. The last condition used text navigation aids fixed at the top, as shown in Fig. 2. By fixed, we mean that these buttons remain in the same physical location even when the user scrolls the web page. The only way

Table 2 Phase 2 test conditions

Condition 1 Condition 2 Condition 3

Browser + Browser + Structure + Browser + Structure + Home Local Navigation + Path Local Navigation

(fixed at top)

D. Bachiochi et al. /Computer Nehvorks and ISDN Systems 29 (1997) 1489-1496 1493

Fig. 2. Fixed structure buttons.

Table 3 Phase 2 performance measures

Average number Average search of page changes time (set)

I. Browser + Home 6 92 2. Browser + Structure +

Local Navigation + Path 4 77 3. Browser + Structure +

Local Navigation 4 63 (fixed at top)

to do this currently is by creating an HTML frame for the buttons and another frame for the rest of the Web page. Path buttons were not included in the last condition. We also redesigned the map to provide a detailed, but graphical, view of the web site. This was available in both conditions two and three.

Fifteen new evaluators were recruited from the Aetna employee population. Fourteen were over thirty years of age, 11 were female, all had more than 5 years experience at Aetna and used computers more than ten hours per day, but only five logged onto the Web once a day. Eight of the remaining evaluators had no Web experience whatsoever. A real testing opportunity!

7. Phase 2 results

Table 3 summarizes the performance measures for this Phase. Significant statistical differences were found between Conditions 1 and both Conditions 2 and 3 relative to the number of page changes. Further

analysis of the data indicated that this difference was attributed to novice users only. We concluded that navigational aids definitely enhanced performance for novices, precisely the pool of users we are trying to attract to the Website.

No statistical significance exists for the measured times. Although the averages appear to be very different, the variability (i.e. the standard deviation) of performance of evaluators affects the statistical results. Variability was very large for Condition 1 (55 seconds), less so for Condition 2 (41 seconds) and low (9 seconds) for Condition 3. The large variability for Condition 1 dominated the other results. These results tell us that Condition 1 evaluators were very inconsistent while Condition 3 evaluators were very consistent in performance. A substantial finding itself! l These performance results and the comments

made by the evaluators yield the following lessons learned for Phase 2:

l Fixing navigation buttons reduces scrolling and search times, thereby improving productivity. Much time is wasted just maneuvering to the location of navigation buttons.

l Browsers are for browsing. They are intended to facilitate inter-Website navigation, but they do not help much within the Website itself. This is the responsibility of the Website designer, and is entirely appropriate.

l People rely on memory to find information. This comes from the fact that typical Website do not have an inherent structure. Hence, users have to remember where they saw the information. Even when given good navigational tools, experienced web users revert to retracing certain paths to find answers. Given more experience with these navigational tools, this should change.

l Good Structure aids are invaluable to navigation. These buttons and their associated web pages improve performance, particularly for novice users. We feel that experienced users will also benefit, once they get rid of bad habits developed using sites that have no structural aids. The redesigned map was a substantial improvement, based on comments.

l Path buttons are of questionable value. Given fixed Structure buttons, they add little to performance or preference.


l Local Navigation buttons are too confusing for most users, especially novices. While potentially a good idea, they remained the least used of the aids.

generates new validated design ideas, and provides a usability measure with which to test subsequent designs. Hopefully we can further reduce the time and effort it takes for users to find information on the World Wide Web.

8. Recommendations

Fig. 3 illustrates the generic page layout that resulted from this usability study. We can summarize by saying that: l Browser tools are not sufficient for effective

searching. They must be augmented with: - a logical design structure, - structure buttons and their associated Web

pages, - a Website home button labeled appropriately, - fixed navigational aids at the top, beneath the

Browser tools. l To efficiently find information, users should not

have to: - make more than 4 page changes, - take longer than 60 seconds to find informa-

tion. These results show how usability studies can not

only validate or invalidate recommendations, but can also drive the design. Web designers are crying for help in establishing good designs. The quality of the help available varies and each must be validated through testing. If the author’s recommendations were not validated, designers should be aware of that fact and use the recommendations accordingly. This study validates some ideas, invalidates others,

I Browser-

Fig. 3. Recommended navigation aids.

References

[II

121

[31

[41

[51

Darken, R.P. and Siebert, J.L., A toolset for navigation in virtual environments, in: UIST ‘93 Proceedings, p. 157. Darken, R.P. and Siebert, J.L., Wayfinding strategies and be- haviors in large virtual worlds, in: SIGCHZ ‘96 Proceedings, p. 142. Horton, W., Taylor, L., Ignacio, A., Hoft, N.L., The Web Page Design Cookbook, John Wiley & Sons, New York, 1996 Jones, S. and Cockbum, A., A study of navigational support provided by two World Wide Web browsing applications. Hypertext ‘96, p. 161 Nielsen, J., Multimedia and Hypertext, The Internet Beyond, AP Professional, 1995, 1

Daniel J. Bachiochi has recently graduated Phi Beta Kappa, with a BA in Computer Science and Mathematics, from Boston College in Chestnut Hill, MA (USA). He worked as an Intern during school breaks with Human Factors Engi- neering at Aetna, Inc. in Hartford, CT (USA) over the span of one year. During his time at Boston College, Mr. Bachiochi pursued interests in Compilation Optimization of Func-

tional Programming Languages as a participant in the Boston wide Church Project. He will begin a career with the United States Government during the summer of 1997.

Michael C. Berstene is an Informa- tion Technology professional with 25 years of experience. He has been the manager of the Human Fac- tors Engineering group at Aetna, Inc. in Hartford, CT (USA) for the last five years. He has a BS degree from the University of Con- necticut in Management Science and an MS from Rensselaer Polytechnic Institute in Computer Science. Mr. Berstene leads a team that provides

user interface design guidance in the development and evaluation of PC, voice, Internet and Intranet based applications. His team focuses on human-centered design and usability to ensure that applications are both useful and usable in meeting business

D. Bachiochi et al. /Computer Networks and ISDN &sterns 29 (1997) 1489-1496 1495

needs. He is currently working on the usability of Web sites and the development of guidelines and standards for designing Web sites.

t

Elaine F. Chouinard received her Master of Arts degree in Experi- mental Psychology in 1992. She has worked on education and human factors projects in social service, mili- tary and corporate environments. For the current project, Ms. Chouinard served as lab technician and statisti- cian.

Nancy M. Conlan has been a Hu- man Factors Engineer at Aetna, Inc. in Hartford, CT (USA) for 11 years. She has a BS degree from Cen- tral Connecticut State University and an MA degree from Trinity College, Hartford, CT. Her focus on human- centered design helps ensure that applications are useful and usable, meeting both business and users’ needs. Ms. Conlan has consulted on numerous projects, providing guid-

ante in the development and evaluation of user interfaces for PC. voice, Internet and Intranet based applications. She has recently focused on researching usability issues of Web sites, and is a member of Aetna’s Web Development Guidelines group, which is developing guidelines for designing Web sites.

Michael M. Danchak is presently Professor of Computer and Informa- tion Science at The Hartford Grad- uate Center, having previously been Dean of the School of Engineering and Science and directed the Com- puter and Information Science program at The Center. He was for- merly Supervisor of Display Sys-

I terns, Instrumentation and Control Engineering, Nuclear Power Sys- tems, Combustion Engineering, Inc.

Dr. Danchak received his B.S.E. from Princeton University and Ph.D. from Rensselaer Polytechnic Institute.

Dr. Danchak is a member of various professional groups in- cluding the IEEE, ACM, the Human Factors and Ergonomics So- ciety, the American Society for Engineering Education and the Society for Information Management. His main professional interests are computer graphics and human computer interactions and he has written and reviewed various human factors/computer graphics related papers for several publications. He has been an active researcher and consultant in these areas since 1974. He has also developed expertise in the training and educational needs

of working professionals. He was member of the State of Con- necticut Governor’s High Technology Council and is currently a member of the Board of Connecticut Innovations, Inc., a quasi- public agency that deals with fostering technological development with the State. He was the principal investigator on an NTIA grant that established the Connecticut Distance Learning Consortium and currently serves as the Chair of that organization.

Dr. Danchak has over 400 hours of teaching experience in Distance Learning. He has taught courses using satellite, mi- crowave (ITFS), interactive compressed video (ICV) and student response units (SRU), sometimes all in one course. Recently he completed Phase I of the Certified Distance Learning Instructor program at the Teletraining Institute, Stillwater, OK. He is also involved in research with other technologies, such as computer based systems. for Continuing Professional Education.

Thomas Furey has been a Hu- man Factors Engineer at Aetna, Inc.(USA) for the past 11 years, and also serves as the Usability Lab Manager. Tom has an Associate De- gree in Mass Communications. In addition to providing technical support for all Lab initiatives, Tom’s current focus is usability evaluations of Web applications. Tom also con- sults with specially challenged employees and provides adaptive technology solutions.

Christine A. Neligon has been a Hu- man Factors Engineer at Aetna, Inc. in Hartford, CT (USA) for six years. Ms. Neligon’s focus is on the design and usability of GUI, hypertext, and Web applications. She has spent the last two years as a participant on teams developing the user interface design and measuring the usability of Web applications. Ms. Neligon initiated, and continues to facilitate, Aetna’s Web Development

Guidelines group, which is developing guidelines for designing Web sites. She is a member of Aetna’s Intranet Working Group and facilitates the group’s Tools team, which helps to identify the requirements for Web authoring and development tools at Aetna.

Donna Way is a Human Factors Engineer with nine years experience in usability evaluation. In addition to usability consulting, Donna has participated in the development of the Human Factors Engineer- ing Guidelines and provided input into the GUI Design Standards and Aetna’s Web Design Guidelines. Her current responsibilities include on-


Foing project evaluation and design and development of HFE’s Web site for Aetna’s Intranet. Donna has a degree in nursing

which has helped her with the ergonomic aspect of her duties Human Factors Engineering.

usability studies and designing navigational aids for the world wide web

Documents