Usability Testing Report of College of

Liberal Arts & Sciences (CLAS) Website

Submitted to: Ceily Hamilton, Director of Information Technology

Alex Chapin, Executive Director of Academic Technology

Compiled by: Andrea Patawaran-Hickman, Jess Cope, and Kailan Smith

February 2015

February 26, 2015

Ceily Hamilton, CLAS Director of IT

Alex Chapin, Executive Director of Academic

Technologies

College of Liberal Arts & Sciences

UNC Charlotte

Dear Ms. Hamilton & Mr. Chapin,

The attached report presents results of a usability test focusing on faculty and staff’s

exchanges with the UNC Charlotte College of Liberal Arts & Sciences (CLAS) website for

Dr. Wickliff’s ENGL 5181 Writing User Documents course. The goal of the usability test

was to observe and analyze how well the website functioned for the UNC Charlotte

community. This report includes specifics from the observation of four faculty members from

departments within the College of Liberal Arts and Sciences (two professors of Philosophy,

one associate professor of Geography & Earth Sciences, and one professor of English) who

agreed to participate in a 30 minute usability test. These tasks included navigating and

locating UNC Charlotte’s CLAS website, History department’s CLAS page, an Ecology

research topic, the CLAS Connections log-in, the help page for editing and updating profiles,

Exchange Online to submit news or events, and a blog, which in this case was a blog titled

“Class Dismissed”.

In completion, the results, observations, and responses by the Participant show that UNC

Charlotte’s CLAS website is functioning, but this report discusses suggestions and/or

recommendations that will further make the CLAS website a success. Effectively, the CLAS

website was easily found and all participant were generally satisfied with the lay-out of the

website. Most participants were particularly frustrated with locating the help page, research

topics, and finding/logging onto CLAS Connections. In fact, the usability test indicated that

the participants did not know the exact purpose of CLAS Connections. These observations

indicate that the UNC Charlotte CLAS website is usable, but is not entirely efficient in

accessibility of resources in reaching its audience.

Recommendations include more obvious methods to present resources such as live links

within paragraphs with larger font size and bold face, the three main attributes of CLAS

website (CLAS Connections, Thinking Matters, and Exchange Online) relocated to left side

of screen so that users can recognize them as links instead of photos, and conducting more

workshops about CLAS website for faculty and staff.

Sincerely,

Andrea Patawaran-Hickman, Jess Cope, Kailan Smith

UNC Charlotte Graduate Students

Contents

Executive Summary .................................................................. 4

Introduction ............................................................................... 5

Methodology ............................................................................. 5

Pre-Interview & Participants ...................................................... 5

Recommendations .................................................................. 11

Conclusion .............................................................................. 12

Appendices ........................................................................ 14

Appendix A: Participant Task Times .................................. 14

4

Executive Summary

To analyze the functionality of the UNC Charlotte College of Liberal Arts & Sciences

(CLAS) website for target audience of faculty, staff, alumni, donors, and students, a sequence

of usability tests were conducted to exam if the UNC Charlotte CLAS website is a useful

resource to the UNC Charlotte community. This usability test targeted faculty and staff.

During the usability test, we observed that participants had most difficulty with:

Locating research topics

Locating and logging into CLAS Connections

Locating help page for updating profiles

Submitting news/events to Exchange Online

The participants expected for these tasks to be effortless, but were often deterred because the

information needed was concealed beneath other parts of information or not easily accessible.

To improve the UNC Charlotte’s CLAS website and fulfill faculty and staff’s needs, primary

recommendation includes clear, concise, and readily available resources to appear on

homepage of CLAS website and within compartments: CLAS Connections, Thinking

Matters, and Exchange Online.

This report includes specifics from the observation of four faculty members from departments

within the College of Liberal Arts and Sciences (two professors of Philosophy, one associate

professor of Geography & Earth Sciences, and one professor of English) who agreed to

participate in the usability test. These participants were given a pre-questionnaire to confirm

that they fit within the criteria of the targeted audience. The usability testing was conducted at

convenient locations, suggested by the four participants, where they would feel most

comfortable. For each test, the participants were asked to complete seven tasks while

navigating the UNC Charlotte CLAS website. Participant activity was audio and video

recorded with an online screen recorder, Screencast-O-Matic, and notes were taken by two

observers. After the test, each participant was asked a series of follow-up questions to

acknowledge their preferences on functionality and presentation of the CLAS website.

The participants were able to complete the test within a 30 minute timeframe, however, this

does not mean that each task was completed. The CLAS website was positively and

effortlessly found and all participants were generally satisfied with the lay-out of the website.

Yet, most participants were particularly frustrated with locating the help page, research

topics, and finding/logging onto CLAS Connections. In fact, the usability test indicated that

the participants did not know the exact purpose of CLAS Connections. All four participants

had to move on from at least one specific task because it was either too difficult or too

ineffective. This indicates that the UNC Charlotte CLAS website works well, but is not

entirely efficient in presentation and accessibility of resources.

5

Introduction

In order to test the usability of the UNC Charlotte’s CLAS website, a series of tests were

conducted that specifically focused on the CLAS website’s CLAS Connections, Thinking

Matters, and Exchange Online webpages. The usability test included pre-questionnaire

screening, consent forms, tasks, and follow-up questions for each participant. The objective

of the usability test was to observe how well the CLAS website connects faculty members to

other faculty, staff, and community. Other aspects test how easily faculty members could

navigate between sections of CLAS website and contribute to exchanges. The follow-up

questions focused specifically on participants’ layout and design preferences.

This report consists of findings and recommendations from results that analyze four faculty

participants (two professors of Philosophy, one associate professor of Geography & Earth

Sciences, and one professor of English). Although the faculty participants identified

themselves as frequent internet users, they admitted to having limited experience with the

CLAS website. The participants’ efforts in this study have allowed us to gain a better

understanding of user needs, locate problematic areas, and offer suggestions for

improvement.

Methodology

Pre-Interview & Participants Targeted audience of UNC Charlotte’s CLAS website was primarily faculty and staff. After

examining the names of faculty and staff, a list was compiled of possible participants. To

contact these faculty members, an email blast was sent stating objectives and goals for

testing. Possible participants were given a pre-questionnaire to confirm that they fit within the

criteria of the targeted audience (see Figure 1). The following information requested

included:

Name

Gender

Years at UNCC

Average internet use per week

Blogging experience

Experience with the CLAS website

Preferred meeting day and time

6

Participant

Professional Title

Gender

Years at

UNCC

Average

Internet

Usage

per week

Active

blogger

Experience

with CLAS

website

Participant A

Professor

Department of Philosophy

& Department Chair

F 7 months Daily No Limited

Participant B

Associate Professor

Department of Geography

& Earth Sciences

F 10 years 10-20 hrs.

weekly

No Limited

Participant C Lecturer of Philosophy

Department of Philosophy

M 9 years 30 hrs.

weekly

No Limited

Participant D Associate Professor

Department of English

M 25 years Daily No Limited

Figure 1: Pre-questionnaire results. The participants selected for usability testing included

faculty members who were targeted as having limited experience with the CLAS website.

Usability Test

All four participants were asked to perform a 15-30 minute usability test while navigating and

completing tasks on UNC Charlotte’s CLAS website. These tasks included navigating and

locating:

UNC Charlotte’s CLAS website

History department’s CLAS webpage

An Ecology research topic

CLAS Connections log-in

Help page for editing and updating profiles

Exchange Online to submit news or events

A blog titled “Class Dismissed.”

Participants were informed that the test was for Dr. Wickliff’s ENGL 5181Writing User

Documents course and directors of the website, and will only be seen by Dr. Wickliff, fellow

ENGL 5181 students, and directors of the CLAS website for further research and

improvements. The participants were then asked to sign a consent form, stating that they

agree to the terms of the usability test. The participants’ performance was audio and video

recorded through an online screen recorder, Screencast-O-Matic. Lastly, the participants were

reminded that the CLAS website was being tested, not the participants, and were encouraged

to verbalize their thought process and actions performed while the observers silently analyzed

and took notes.

7

The participants were all frequent users of the internet with an average internet usage per

week ranging from daily to 30 hours a week, however, all four participants had difficulty

performing tasks given for usability test (see Figure 1). Although these participants use

internet regularly, they did state that their experience with CLAS website was limited. This is

evident by results from usability test of each participant. Analyses were compiled of each

participant’s task times by minutes and the results show that logging into Class Connections,

locating departmental CLAS pages and specific research topics acquired the longest time (see

Appendix A). Observations concluded that participants are likely to give-up on task

completion after three unsuccessful attempts or after three minutes of time. Participants

identify their next step after failing to complete a task as seeking help through email or

telephone communications.

Post-Test: Follow-Up Questions

The participants were vocal during the usability test, stating their opinions on what was

difficult or easy to use. Therefore, follow-up questions focused on key components in regards

to the design of UNC Charlotte CLAS website and asked for preferential choices about the

functionality and presentation. There was a total of eighteen questions programmed for quick,

reflexive answers about website design by having participant choose one of two options for

each question. These set of questions provided valuable findings and were analyzed for

recommendations of further improvement (see Appendix B and C).

Findings

This section addresses the main areas of strength and recommended revisions that were

identified in both the usability tests and the follow-up questions for the CLAS website. This

report has prioritized and condensed information from follow-up questions and provide

recommendations when appropriate.

Landing Page

The CLAS website’s rolling graphics are preferred to competitive universities’ non-

rolling graphics and selectable graphics.

Participants prefer UNC Charlotte’s current method of only embedding text and

pictures with no videos.

Participants indicated not realizing that CLAS Connections, Thinking Matters, and

Exchange Online graphics are links. While these links are colorful and visually

appealing, they resemble a picture and not readily identifiable as links. Their right

column position also makes them less noticeable to users. We recommend changing

the link images and moving them to the left-side navigation column of CLAS website.

8

Layout and Design

A majority of participants prefer the current CLAS website three column layout to

paragraph or tab format.

The link hover color is more visible and clear than competitive universities.

CLAS website’s current method of grouping the Liberal Arts and Sciences’

departments under the “Academics” link is preferred to separating them.

Participants did not find links in the right-side column of the CLAS Connections

page: “Department & Programs”, “Research”, and “Connection Update” (see Figure

2). The right column gives minimal attention to these important links, which is

contributing to lack of noticeability. The font size and color also diminishes their

noticeability. Moving these links to the left-side column and giving them a darker and

larger font would make website layout more user-friendly.

Figure 2: CLAS website right-side navigation.

Moving the research topics to the left-side of the page may help users realize that

these are navigation links. Figure 3 demonstrates a mock design of how research

topics on left-side could appear.

9

Figure 3: Mock design of CLAS website research topics on left-side column of CLAS

website.

Participants preferred having only sanserif in the CLAS website. Most participants

described serif font as “old” or “outdated” while sanserif is considered more

“modern” and “cleaner looking”.

Participants did not realize that the “Exchange Online” link would change “We Want

Your News!” after clicking on it, nor did they immediately notice the change (see

Figure 4). In addition to changing the “Exchange Online” link, adding “We Want

Your News!” with its own link on the landing page might be beneficial to users since

left-side of website is preferred search area for all participants. This change would

reduce the amount of clicks to reach the news submission and increase visibility (see

Figure 5).

Figure 4: Change from “Exchange Online” to “We Want You News!”

10

Figure 5: “We Want Your News!” on left-side column of CLAS landing page.

Content and Language

The current CLAS website’s “Academics” page can be navigated more quickly than

those of competitive universities; the list form is preferred to block form or only a

jump-to bar. However, adding a jump-to bar to our current list form would be

preferred by two of the participants (see Figure 6).

Figure 6: UNC Charlotte CLAS “Academics” list versus jump-to menu from competitive university

website.

Participants were not aware that a department page is different than a “CLAS

Connections” department page. All participants were instructed to go to the History

department’s CLAS page during the usability test and went straight to the department

page and never visited the department CLAS Connections page. In follow-up

discussion those participants said they were not aware that “CLAS Connections” page

11

existed. Highlighting during future instructions to faculty and staff that “CLAS

Connections” consists of departments pages on the landing screen might be beneficial

to user and website utility.

The current “Faculty and Staff” page has no links to departments, people, or research

topics. Participants are not unified in a preferred method of linking to these things,

but some method is desired. One participant preferred having faculty finder search or

a jump-to bar, while two participants preferred having links. Further investigation into

methods of linking to faculty, staff, and research will accommodate these issues. This

could be an opportunity to link to the CLAS Connections’ department pages and

research links (see Figure 7).

Figure 7: UNC Charlotte’s CLAS Faculty and Staff page versus Faculty and Staff links from competitive

university website.

Three out of four participants were confused on whether they were searching the

entire UNC Charlotte website or only the CLAS website and preferred to be able to

manually switch between searching the UNCC website and the CLAS website.

Adding a button that would allow users to select their preference would be effective.

UNC Charlotte’s search function did not yield results in many cases, which led

participants to leave the CLAS page and search through Google.

Recommendations

This section provides a roadmap for the areas of improvement that were identified in the

usability tests to advance the function, efficiency, and desirability of the UNC Charlotte

CLAS website. Recommendations are prioritized on the basis of how much participants were

affected by these issues.

Finding a Department’s CLAS Connection Page

Make the “CLAS Connections” link look less like a graphic and move to the left-side

column of the CLAS landing page.

12

Have a link to department’s “CLAS Connections” page on each department page.

Move department links on the “CLAS Connections” page to the left-side and give

them darker color.

Finding a Research Topic

Provide a link to the “CLAS Connections” research topics under the research

navigation link.

Move research links on the “CLAS Connections” page to the left-side column and

give them a darker color (see Figure 3).

Logging in and Updating a Profile

Make the login link on the “CLAS Connections” page bigger and either move it to

the left-side column or the upper-right portion of the screen where it is located on

faculties’ profiles.

Submitting News or Events

Make the “Thinking Matters” and “Exchange Online” links look less like graphics

and move them to the left-side column of the CLAS landing page.

Have a “We Want Your News!” link placed permanently next to the “Thinking

Matters” and “Exchange Online” links (see Figure 5).

Conclusion

The results and recommendations of the usability assessments referred to in this report

examined whether the UNC Charlotte CLAS website’s principles in layout and design proved

usable for faculty members, and whether faculty members could perform basic and valuable

tasks that would enable them to actively participate in the content of the website. Major links

are not currently being seen or recognized by faculty members which has a negative impact

on their ability to add or search new knowledge. Admittedly, other factors beyond the scope

of this assessment may be impacting on the overall effectiveness of the website and hindering

faculty engagement. In summary, the results of this usability test demonstrate that the CLAS

website functions well in its overall layout and design. Some of the best practices that were

identified include:

Column style layout

Side navigation

Distinct color changes for links when hovered over

Rolling graphics

13

However, there remain areas of improvement if the website desires to have more active

faculty participation. The greatest needs for consideration include:

Create links for department’s “CLAS Connections” page on each department

academic page.

Move research links on the “CLAS Connections page” to the left-side column and

darken the color.

Make the login link on the “CLAS Connections” page a larger font size and move the

link to the left-side column or the upper right-side of CLAS website.

Create a “We Want Your News!” link next to “Thinking Matters and “Exchange

Online”. Increase the font size and bold the links that exist within the paragraphs and

consider moving the link to the top of the submission paragraph.

These recommendations may help faculty members perform basic tasks in a timely manner

which will increase the CLAS website’s efficiency. By implementing the recommendations,

faculty members may quickly learn how to login, find help, and contribute more effectively

to the platform. In addition, the exact purpose of the website should be clearly defined to

users.

14

Appendices

Appendix A: Participant Task Times

0 0.

5

1 1.

5

2 2.

5

3 3.

5

4 4.

5

5

CLA

S w

ebsi

te

Dep

artm

ent

CLA

S p

age

Res

earc

h T

op

ic

Logi

n

Hel

p p

age

Sub

mit

new

s Blo

g

Task

Tim

es

by

min

ute

s

Par

tici

pan

t A

Par

tici

pan

t B

Par

tici

pan

t C

Par

tici

pan

t D

15

Appendix B: Follow-Up Questions and Graph Key:

4 out of 4 3 out of 4 2 out of 4 2 out of 3 Not applicable

Preference Participant

A

Participant B Participant C Participant D Results

Column or Paragraph Paragraph Column Column No

preference

2 out of 3:

Column

Column or Tab Column Column Column Tab 3 out of 4:

Column

Paragraph or Tab Paragraph Tab Paragraph Both

Sanserif with serif or

sanserif only

No

preference

Sanserif only Sanserif only Sanserif with

serif

2 out of 4:

Sanserif only

Hover color or hover

shading

Hover color Hover color Hover color Hover color 4 out of 4: Hover

color

Faculty info or faculty

finder and jump to

Faculty

finder and

jump to

Both Faculty finder,

jump to, and

keyword finder

for research

Faculty info

Faculty info or

department links

Faculty info Department

links

Department

links

Faculty info 2 out of 4: each

Department links or

faculty finder and jump

to

Faculty

finder and

jump to

Department

links

Department

links

Department

links

3 out of 4:

Department

Links

Academics grouped or

academics separated

Academics

separated

Academics

grouped

No preference Academics

grouped

2 out of 3:

Grouped

List for academics or

picture blocks of

academics

List for

academics

Both List for

academics

List for

academics

List for academics or

jump to for academics

List for

academics

List for

academics

Jump to Both

Block pictures or jump to Block

pictures

Block pictures Jump to Jump to 2 out of 4: each

Rolling graphics or non-

rolling graphics

Rolling Rolling-if

updated

Rolling-if not

too quickly

Rolling-If

updated

4 out of 4:

Rolling

Rolling graphics or

selectable graphics

Rolling Rolling Selectable Rolling 3 out of 4:

Rolling

Search with no option or

search with options

Search

options

Search

options

Search options No options 3 out 4: Options

Only embedded text and

pictures or embedded

videos

Embedded

text and

pictures

Embedded

text and

pictures

No preference Embedded

text and

pictures

3 out of 4:

Embedded

text and pictures

Side navigation or top

navigation

Side

navigation

Both Side navigation Side

navigation