EATIS 2007

Design Guidelines for Web Applications Based on Local Patterns

César I. Llanos Universidade Federal de Minas Gerais Belo Horizonte, Minas Gerais, Brasil

CEP 31270-290 cesarllanoss@gmail.com

Marcia N. Muñoz Universidad Católica de la Santísima

Concepción Alonso de Ribera 2850 Concepción, Chile

marciam@ucsc.cl

Abstract We present the results of a project to determine web design guidelines bases on user interface design patterns present in Chilean web applications. The scope of the work is restricted to web applications in the .cl domain. We consider three types of web applications: web sites, search engines and blogs. First, we chose a study sample of 178 web sites, 4 search engines and 16 blogs. Then, a typical web application’s elements were identified, and their most common page locations were determined. Usability tests were performed on a representative sample of web sites so as to evaluate them qualitatively and thus determine which elements are relevant to the users from a usability point of view. Finally, we derive design guidelines for the most representative features for each type of web application, based on analysis of the study sample and the usability test results. Keywords: User interfaces, usability, web applications 1. Introduction Tim Berners-Lee working at CERN in Europe first created the World Wide Web around 1990. Since then, it has experienced an exponential growth that shows no signs of abating. Some factors explaining this explosion are the availability of cheap web site development tools, low network entry costs, and the relative ease of web publishing, all of which aid the generation of web sites by non-specialists. The first Netcraft survey in August 1995 found 18,957 hosts, while the latest Netcraft survey in November 2006 records more than 100 million web sites [11]. The first web sites were the products of sloppy HTML coding and haphazard planning. Since then, many new Web technologies that simplify and speed up web site development have been introduced. However, in many cases, these same technologies favor aesthetics over web site efficiency. For these and other reasons, many corporate web sites in the late '90s showed signs of poor design [2]. Recently, human-computer interaction specialists have been paying more attention to software usability, so that

nowadays it has become an integral part of software development culture. For example, in 1997 the World Wide Web Consortium (W3C) created the Web Accesibility Initiative (WAI) [17], which develops guidelines widely regarded as the international standard for Web accessibility for people with disabilities and in 2001, the European Union funded UsabilityNet [5], a project to provide resources and networking for usability practitioners, managers and European Union projects. In Latin America, there are an increasing number of events and conferences focused on human-computer interface issues, many of them with specialized tracks discussing usability, such as the Latin American Conference on Human-Computer Interaction (CLIHC), which held its second meeting in Mexico in 2005. In 2004, the Chilean government published its ``Guía Web 1.0 para el desarrollo de sitios Web del gobierno de Chile'', a web site development guide for government web sites, which places special emphasis on web site usability and acknowledges its importance [3]. Our research is based mainly on the idea that a Web site's ease of use will be enhanced if its design applies web site design templates to which web users are accustomed already, so as to increase intuitive web site use and decrease the effort required to perform common tasks. As intuition is related to language and culture, any web site usability study must take these factors into account. Most published studies examine English-language web sites [9]. This research studies web sites by and for inhabitants of Chile, a spanish-speaking Latin American country with a population of about 16 million and an Internet penetration of 40% [16]. As such, it is restricted to web applications in the .cl domain and was done during the first semester of 2006. 2. Web Usability Software usability is a broad concept that encompasses many aspects of software development and design. It usually denotes the ease with which people can employ a particular tool, such as a computer program or a web site, in order to achieve a particular goal. However, a broad consensus on the precise definition of web usability is still

EATIS 2007

lacking [1]. To that purpose, it is instructive to review the definitions of usability given in several relevant ISO standards and other publications. The ISO 9241-11 standard, Guidance on Usability, defines usability as the extent to which a product can be used by specified users to achieve specified goals with effectiveness, efficiency and satisfaction in a specified context of use [7]. This definition is used in many subsequent related ergonomic standards. In the software engineering community, the term usability has been more narrowly associated with user interface design. The ISO/IEC 9126 standard, Software Product Evaluation - Quality Characteristics and Guidelines for their Use [6], was developed separately as a software engineering standard. It defines usability in association to the design and evaluation of the user interface and interaction as: a set of attributes that bear on the effort needed for use, and on the individual assessment of such use, by a stated or implied set of users. The ISO/IEC 9126 standard was superseded in 2000 by the ISO/IEC FDIS 9126-1 standard [8], which defines usability as the capability of the software product to be understood, learned, used and attractive to the user, when used under specified conditions. In Usability Meanings and Interpretations in ISO Standards [1], Abran et al. analyze the conflicting viewpoints and related requirements that have led to the different definitions of usability in these ISO standards. Finally, Jakob Nielsen, a respected usability consultant, defines usability as a quality attribute that assesses how easy user interfaces are to use [14]. This definition is used throughout this work. 2.1. Web applications Following Conallen [4], we define a Web application as a Web system that allows its users to execute business logic with a Web browser. Thus, it evolves from a static web site to become a dynamic, distributed, multimedia and interactive platform for user interaction [18]. Common examples are e-commerce sites, search engines, wikis, discussion boards, search engines, weblogs, etc. Though many variations are possible, a Web application is commonly structured as a three-tiered application. In its most common form, a Web browser is the first tier, an engine using some dynamic Web content technology (e.g., CGI, PHP, Java Servlets or Active Server Pages (ASP)) is the middle tier, and a database is the third tier. The Web browser sends requests to the middle tier, which services them by making queries and updates against the database and generating a user interface. Web application users behave differently from other software users. They barely read the content, but mostly browse it, looking for keywords and markers that would lead them onto their goal. They are impatient and crave immediate response. They are averse to scrolling and

searching, and will leave a web site if they cannot quickly find what they are looking for [10][12]. As such, the usability of the web application's user interface is critical to user retention and return rates. 3. Usability evaluation for web applications There are many possible approaches to web usability evaluation. The usability of Web applications has been traditionally measured through qualitative tests. Some of the most common are:

• A formal usability inspection: a group of usability experts judges the user interface and the services offered, with one of the participants acting as a moderator, pointing out the Web application's strengths and weaknesses.

• End user usability testing: end users try out the web application and their performance is recorded for posterior analysis. This activity may be done under controlled conditions in a laboratory environment or directly in situ.

• Expert heuristic evaluation: usability experts use a heuristic usability checklist to evaluate the web application.

• Participatory heuristic evaluation: this test is similar to an expert heuristic evaluation but is extensive to end users so as to include their concerns and work context into the usability heuristics.

• Listening test: the tester guides the end user through the web application to be evaluated, asking questions about the different site elements and indicating specific tasks to be performed, recording all end user comments and reactions.

Alternatively, usability questionnaires developed for web and/or regular software applications can also be used. Finally, there are several automated validation applications available online that can be useful in evaluating different usability and accessibility aspects, such as:

• W3C markup validator (http://validator.w3.org) • W3C Tidy (http://cgi.w3.org/cgi-bin/tidy) • Bobby (http://webxact.watchfire.com) • Hera (http://www.sidar.org/hera)

4. Web site selection In order to identify those web site design patterns prevalent in Chilean web applications, we chose a set of search engines, blogs and web sites to be studied using the methodology discussed below, and shown in Figure 1. In all, a study sample of 178 web sites, 4 search engines and 16 blogs was selected.

EATIS 2007

Figure 1. Web site selection methodology

4.1. Web sites Searches for Chilean web sites were done using the search engines http://www.google.cl and http://www.todocl.cl, which classify web sites into categories and then further subdivide these categories into areas. A list of the chosen categories, along with their membership cardinalities, is shown in Table 1.

Table 1: web site categories

Category Number Arts 14 Science and Technology 8 Education 13 Government 13 News media 16 Sports and recreation 17 Health 15 Tourism and maps 8 Society 16 Internet 15 Commerce 12 Universities 15 Business 16

Web sites were then chosen according to the following rule: for each major category, the four areas with the greatest number of web sites are chosen. At most 6 web sites were chosen for each area, according to the following criteria:

a) All of the web site's pages must be in the .cl domain.

b) We choose those six web sites with the highest page ranking value and which are accesible via both search engines.

c) If we cannot identify six web sites complying with the previous conditions, priority is given to

those web sites deemed most representative by comparing the outputs of both search engines.

If an area is divided further into subareas, only those 4 subareas with the greatest number of web sites are chosen. Then, a web site sample is chosen as described above. A total of 178 websites were selected using this scheme. 4.2. Search engines As their design is usually very different from other web sites, they are studied as a separate category. The 4 search engines shown below were selected based on national and regional preferences.

• Todocl (http://www.todocl.cl), a search engine indexing Chilean web sites exclusively.

• Google.cl (http://www.google.cl), concentrating on .cl web sites.

• Huifa (http://www.huifa.cl), another Chilean search engine.

• 8va (http://www.8va.cl), a Chilean regional search engine.

4.3. Blogs As blogging are not limited to any geographic area, determining which blogs are written by and read by Chilean users is a difficult issue. However, a weblog search using the search engines mentioned in the previous section showed that most Chilean blogs use the same blog providers, and that, in general, blog design is determined by the blog templates available for each provider. Therefore, it is necessary to first identify the most common blog providers used by Chilean blogs so as to access their blog template libraries, and then to study them to identify possible design patterns. The most common blog providers are shown in Table 2.

Table 2: Common blog providers

Name URI Number Blogger http://www.blogger.com 3 Blogia http://www.blogia.com 7 Bitácoras.com http://www.bitacoras.com 3 Ya.com http://blogs.ya.com 3 All except for Blogger are spanish-language blog providers based in Spain. For this study, a sample of 16 popular weblogs from all 4 blog providers was chosen. 5. Web design patterns The generation of web design guidelines for Chilean web applications requires the identification of the most commonly used user interface design patterns. This is done by a three-part process: first, the web application's

EATIS 2007

home page is captured; next, several common design elements such as website identifier, logo, search area, etc. are identified through web site analysis and their relative location is recorded, and finally these data are analyzed to determine the chosen design elements’ relevance to the user interface design. At the same time, a qualitative usability evaluation on a web site sample is performed to further assess the importance of selected design elements as explained in section 5.3. 5.1. Home page capture Web application home pages are captured via a standardized screen capture process, using the two most popular web browsers at the time (Dec. 2005 to Feb. 2006): Internet Explorer v6.0 and Mozilla Firefox v 1.0.7, both loaded with all browser plugins needed for the pages to display correctly. Screen resolution was standardized to 1024x768 pixels and no scrolling is allowed. 5.2. Web page division into areas Each home page is divided into 9 variable-sized areas, as shown in Figure 2. If any design element spans a row or column, its location is recorded also as horizontal or vertical, respectively. In other words, the location of a design element can take 15 possible values.

Figure 2: Home page screen areas

5.3. Qualitative usability evaluation The goal of a qualitative usability evaluation is to assess the importance of selected design elements in Web application design so as to be able to emit

recommendations based not only on well-known user interface design patterns but also on upcoming trends. Moreover, qualitative evaluations have been found to yield good insights and be more credible and trustworthy than quantitative studies, and less likely to break under the strain of a few methodological weaknesses [15]. Also, according to Nielsen [13] and Krug [10], it is possible to do qualitative usability tests with as few as 5 users and still get meaningful results. As mentioned in section 3, there are several possible approaches to web usability evaluation. We have chosen to use both a qualitative listening test [10] and a usability questionnaire, as both methods emphasize the user's behavior and practical experience when using a web application. Also, only the most representative web site for each category, shown in Table 3, was used for this test. The Listening Test comprises the following stages: 1. The user logs into the web site. 2. The user explores the website, trying to understand

its logic and identify its main sections and logical structure. During this process, the evaluator asks questions and registers all user comments.

3. The user traverses the main sections of the web application, orally indicating the steps he is following, which are recorded by the evaluator.

4. The evaluator gives the user a task to complete, which is tailored for that specific web site. The user performs the task, thinking aloud, that is to say, mentioning the actions he is considering will help him perform the task.

5. Finally, the evaluator records the user's opinions and recommendations for the website.

Table 3: Representative web sites

Category Website Arts http://www.scd.cl Education http://www.preutech.cl Government http://www.gobiernodechile.cl News media http://www.lun.cl Sports and recreation http://www.canal13.cl Health http://www.consalud.cl Society http://www.rn.cl Internet http://www.goweb.cl Commerce http://www.almacenesparis.cl Universities http://www.uchile.cl Business http://www.redsercotec.cl

After the Listening Test, the user is handed a Usability Attributes Evaluation questionnaire, which comprises 117 questions divided into 11 sections, as shown in Table 4.

EATIS 2007

Table 4: Questionnaire sections

Section Questions Site ID 8 Logo 4 Colors 2 Tagline 4 Navigation 22 User registration 7 Links 13 Content 18 Structure 14 Multimedia 7 Accesibility 18

5.4. Data analysis Both the listening test and questionnaire results are then analyzed to determine which web site design elements were used or deemed relevant by the users. Also, the web site study sample is examined to detect common web design elements and their locations. For each of the 178 websites and 16 blogs studied, the location of 40 common design elements was recorded as per Figure 2. In the case of the 4 search engines, the location of only 25 design elements was recorded. In order to determine the relevance of a particular design element for each category, we preselect all those design elements that appear in a particular location in at least 30% of the websites for that category. Also, a particular design element is deemed relevant for general web application interface design if it is located at a particular location in at least 5% of all websites studied. Then, the most frequently seen elements are classified as conforming a design pattern (70 - 100% presence), a weak pattern (60 - 70% presence) or marking a trend (50 - 60% presence). 6. Web site element locations As mentioned before, only those design elements that are present in at least 30% of the web sites of a category are considered relevant. Then, for each category we derive a table that summarizes the location of the most frequently seen design elements. As an example, Table 5 shows the 7 most common design elements present in a blog home page, along with its frequency of occurrence, and its frequency of occurrence at a certain location of the page. For example, ads are present in 56% of all blog home pages. In 66% of these cases, ads are placed in a horizontal upper position, while 22% web sites present ads in the lower left corner. It may be noted that some design element location figures add up to a value greater than 100%. This usually means that some design elements appear more than once in or span different locations.

Table 5: Common blog elements and their locations

Element Presence Location % Upper left 63% Site ID 100% Upper center 56% Upper left 38% Upper center 38% Navigation 50% Middle left 38% Middle center 69% Content 100% Lower center 56% Upper center 45% Site description 69% Upper left 36% Upper right 33% Upper left 25% Search box 75% Middle left 25% Lower right 56% Sections 100% Middle right 44% Horizontal Upper 67% Ads 56% Lower left 22%

This analysis is performed for all three types of web applications: web sites, search engines and blogs. As mentioned before, we recorded the locations of 40 different design elements present in the studied web sites. We present the results of this analysis graphically, divided by element types and frequencies of occurrence.

Figure 3: Most common design elements in web sites

Figure 3 shows the most common design elements detected in the studied web sites’ home pages, along with its percentage of occurrence. All design elements shown are present in at least 40% of the web sites. Also, the figure indicates at most the two most common locations,

EATIS 2007

indicating the percentage of occurrences in said location in italics. Thus, it can be seen that a site ID is present in 94% of all web sites, of which 60% are located in the upper left corner of the page, and 24% are located in the upper center of the page. It can also be seen that the upper left and upper center areas are the most used, and are shared by several design elements. Other design elements present in web sites are shown in Figure 4. These elements are present in fewer web sites, between 10% and 26%. However, some design elements are strongly associated with specific locations, indicating, for example, that whenever a copyright notice is present, it is to be found in the lower center or lower right of the page.

Figure 4: Less common design elements in web sites

Figure 5 shows the most common links found in web sites home pages. They are not as common as the elements shown in Figures 3 and 4, as their presence varies between 13% and 49%. Also, no trends regarding their locations can be established, as no strong preferences for a given location can be detected. In effect, the most common location associated with the most common link, Contact, is the upper right area of the page, with only a 36% of occurrence. It is instructive to compare these results with those shown in Figure 6, which leads us to conclude that home links are more common than home icons, for example. Figure 6 shows the most common icons found in the web sites studied. As can be seen, these icons are found in only a rather low fraction of all web sites. However, strong trends in the location of the sitemap and e-mail icons can be detected.

Figure 5: Most common links in web sites

Figure 6: Most common icons for web sites

Figure 7 shows other common web site elements, such as frames, services, multimedia, etc. whose presence in web sites vary between 7% and 97%. The diverse location percentages lead us to conclude that only a few elements such as the page name and the use of white space are common, and have clearly defined positions.

EATIS 2007

Figure 7: Other common website elements

Our results show that only 9% of the web sites surveyed have an introductory page, of which 56% include multimedia animation, and 38% include audio. The previous analysis was also done for the chosen search engines and blogs. However, space limitations preclude us from showing these results here in a graphical form. 7. Web design guidelines The results shown in section 6 form the basis of the web design guidelines for web applications presented henceforth. Only those web design elements present in at least 10% of all web sites and strongly associated with a particular location (> 50%) were considered. Figure 8 presents the derived design guidelines for web sites. The trend-marking design elements are divided into three groups, depending on their frequency of occurrence at a particular location. Group 1 corresponds to those design elements present at the given position in at least 70% of all cases, Group 2 elements have a percentage of occurrence of between 60% and 70%, while elements in Group 3 mark a trend by being present in between 50% and 60% of cases. Also, areas in the figure are shaded to indicate that there is at least one design element strong associated with that position. It can be said then that the upper left area of a web site page is the most crowded, strongly associated with a logo and/or a page name, and less strongly associated with a site ID. Taglines can less commonly be found here. Web site content is usually in the mid-center of the page, along with headlines, but these can also be found in the lower

center of the page, sharing the space with a copyright notice. Finally, an e-mail icon can usually be found in the upper right corner, while the vertical right area is mostly blank space. This last observation meshes with the Spanish language character of the domain under study, which is read left-to-right.

Figure 8: Design guidelines for web sites

The design guidelines for search engines are shown in Figure 9. The trend in search engine design is toward sparse sites, with plenty of white space, which direct the user’s attention to the middle center of the page, where the search box usually resides.

Figure 9: Design guidelines for search engines

Finally, Figure 10 shows the design guidelines for blogs. As most blogs use templates provided by the blog provider, it is necessary to examine these templates so as to choose the one that most closely resembles these guidelines.

EATIS 2007

Figure 10: Design guidelines for blogs

8. Conclusions The guidelines presented above were derived by taking into account those web design elements most commonly seen in Chilean web applications. Thus, their use in web application design will improve usability, as users can easily locate common web site elements intuitively in specific areas of the page. The design elements were chosen by means of qualitative usability evaluations: a listening test and a usability evaluation questionnaire. These tests also allow us to corroborate Neilsen’s thesis that it is possible to evaluate a web site using a reduced number of test users [13]. The web applications studied present great variability in the elements present and their locations. In contrast, as was expected, our results show that most search engines and blogs use a limited number of designs: most search engines use sparse web sites where important design elements are easily identified, and most blogs follow predefined templates whose elements are easily recognized by users. This study considered only Chilean web applications. The Chilean web site design patterns detected turn out to be roughly similar to those reported of English-language websites by other authors [9][10]. Future studies may further compare these patterns to identify specific differences. 9. Acknowledgments This research was funded in part by UCSC FAD project 10/2004 and is also part of C. Llanos's senior thesis. We also want to thank Mario Medina for his thoughtful

comments and assistance in the preparation of this document. 10. References [1] A. Abran, A. Khelifi, W. Suryn, and A. Seffah, “Usability Meanings and Interpretations in ISO standards”, Software Quality Journal, Springer New York, Nov. 2003, pp. 325-338. [2] A. N. Badre, Shaping Web Usability: Interaction Design in Context, Addison-Wesley, Boston, MA, 2002. [3] J. C. Camus Gajardo, editor. Guía para Desarrollo de Sitios Web, Ministerio Secretaría General de Gobierno, Gobierno de Chile, Santiago, Chile, 2004. [4] S. Conallen, Building Web Applications with UML, 2nd. Ed., Addison-Wesley, Boston, MA., 2002. [5] European Union, Usabilitynet, available at http://www.usabilitynet.org, 2006. [6] International Organization for Standardization, ISO/IEC 9126: Software Product Evaluation – Quality Characteristics and Guidelines for their Use. Geneva, Switzerland, 1991. [7] International Organization for Standardization, ISO 9241-11: Guidance on Usability, Geneva, Switzerland, 1998. [8] International Organization for Standardization, ISO/IEC FDIS 9126-1: Software Engineering – Product Quality – Part 1: Quality Model, Geneva, Switzerland, 2000. [9] M. Ivory, R. Megraw, Evolution of Web Site Design Patterns, ACM Transactions on Information Systems, 23(4), Oct. 2005, pp. 463-497. [10] S. Krug, Don’t Make Me Think: A Common Sense Approach to Web Usability, New Riders Publishing, Berkeley, CA, 2000. [11] Netcraft, Web server survey, available at http://news.netcraft.com, 2006. [12] J. Nielsen, Designing Web Usability: The Practice of Simplicity, New Riders Publishing, Berkeley, CA, 2000. [13] J. Nielsen, Why you only need to test with 5 users, http://www.useit.com/alertbox/20000319.html, 2000. [14] J. Nielsen, Usability 101: Introduction to Usability, http://www.useit.com/alertbox/20030825.html, 2003. [15] J. Nielsen, Risks of Quantitative Studies, http://www.useit.com/alertbox/20040301.html, 2004. [16] World Internet Project Chile, Uso de Internet en Chile 2006, http://www.wipchile.cl, 2006. [17] World Wide Web Consortium, Web Accesibility Initiative, http://www.w3.org/WAI, 2006. [18] L. Xu, B. Xu, and J. Jiang, Testing Web Applications focusing on their Specialties, ACM SIGSOFT Software Engineering Notes, 30(1), Jan. 2005.