greenguardian.com website evaluation submitted by: marketline research, inc. june 8, 2004

GreenGuardian.com Website GreenGuardian.com Website EvaluationEvaluation

Submitted by:

MarketLine Research, Inc.

June 8, 2004

June 8, 2004 MarketLine Research

Background & Objective

Background In 2003, the SWMCB launched a major public information initiative with

the Green Guardian logo and GreenGuardian.com as the centerpieces. The website is designed to communicate SWMCB’s environmental

messages to a targeted audience of educated women in the 25 to 54

age bracket.

Objective The reported research was designed to both quantitatively and

qualitatively evaluate usability of the GreenGuardian.com web site

with the intended target audience.


Research Design & Process

Step 3

Step 2

Step 1

Step 4

107 respondents are qualified and surveyed for search engine use on the Internet. 42 agree to visit and evaluate site.

42 respondents are emailed instructions for performing 5 disposal related searches on GreenGuardian.com

6 new participants are recruited and attend usability lab testing sessions at Leeds Research facility.

42 respondents are contacted after they visit site and complete tasks. They participate in an 11 min survey.

Telephone Survey

n = 42


Methodology – survey research

107 participants completed a short telephone survey

39% (42) agreed to visit the Green Guardian web site to complete five search related tasks and participate in a follow-up telephone survey to evaluated their experience

– 7 participants from each of the six SWMCB counties visited the site and performed the five assigned tasks

– the average site visit was 11 minutes– Visits ranged in length from 4 to 45 minutes

Each person completing the follow-up telephone survey was mailed $15 for their participation


Respondent Profile

Age 25 to 34 yrs. 21% 35 to 44 yrs. 38% 45 to 54 yrs. 41%

Education Some technical school 12% Technical school graduate 2% Some college 29% College graduate 40% Post graduate 17%

All work full or part-time


Profile of Computing Behavior

Comfort Using Search EngineVery comfortable 74%Somewhat comfortable 24%Not very comfortable 2%

Frequency of Search Engine UseDaily 69%Weekly 26%Monthly 2%

Most Used Search EngineGoogle 59%Yahoo 22%MSN 7%Other 12%

98% use computer to look for environmental information

67% search for solutions to household problems

78% use computer to buy or order products

48% have used computer to find information on environmental activities such as recycling or composting

5% (2 persons) had previously visited GreenGuardian.com


First Impressions of Green Guardian Site

Comments on Appearance

Pleasant looking Friendly calming effect Clear/colorful Not cluttered Attractive to eye Not real flashy/glitzy Professional Straight forward Green good to eye Cute/catchy Not many pictures Drab background color Monotone

Comments on Content Not overwhelming Seemed simple Informative Place for environmental information All about recycling Associated with lawn care Confusing Looked intimidating with all the words Guy with shield nice


First Impressions of Green Guardian Site - continued

Comments on Organization

Relevant categories Easy access to information Clear layout Indexed site Easy to use Neat Easy to read


First Impressions of Mascot Logo

Comments on Appearance Great/cute/light/fun Pleasing to eye/calming Not powerful but friendly More like javelin than sword Eyes and mouth move Not good, not bad Underdone/doesn’t stand out Not eye catching / not memorable Unimpressive Lacks color Doesn’t grab you/nothing stuck out Kinda goofy/kind of dorky Didn’t pay attention to it Don’t remember details

Comments on Message Illustrates what site is about Realized it was for recycling Related to environmental concerns Makes sense – he guards everything Fits website name Could tell I was in right place Doesn’t demand that you recycle Clever Funny/humorous Don’t think it is a site to help me out

with recycling


Importance of Various Site Attributes

Attribute % Rating ‘5’ or ‘4’ % Rating ‘3’ % Rating ‘1’ or ‘2’

Easy navigation 98% - 2%

Can get to information quickly 93% 7% -

Informative 91% 9% -

Fast loading 91% - 7%

Home page well organized 91% 9% -

Easy to understand links 86% 9% 5%

Helps me solve problems 83% 14% 2%

Provides direction or information on what to do

83% 17% -


Importance of Site Attributes - continued


Consistent features 74% 19% 2%

Provides answers to FAQ’s 64% 36% -

Good balance text & graphics 60% 33% 7%

Provides technical information 57% 29% 14%

Uses photo/illustration for communicating ideas

52% 29% 19%

Overall site is graphically appealing

52% 29% 19%

Offers hot links to other information sites

43% 38% 19%


Rating of Green Guardian Site Attributes


Home page well organized 100% - -

Easy navigation 98% 2% -

Provides direction or information on what to do

98% 2% -

Informative 98% 2% -

Helps me solve problems 98% 2% -

Fast loading 93% 5% -

Easy to understand links 93% 5% 2%

Offers hot links to other information sites

93% 7% -



Consistent features 91% 7% -

Can get information quickly 91% 9% -

Provides answers to FAQ’s 91% 2% 2%

Provides technical information 88% 7% -

Good balance text & graphics 83% 17% -

Overall site is graphically appealing

83% 12% 5%

Uses photo/illustration for communicating ideas

62% 38% -

Rating of Green Guardian Site Attributes - continued


Able to Find Disposal Information on GreenGuardian.com

Disposal item % respondentsOld computer monitor 98%

Partially filled aerosol can of 98%

oven cleaner

Partially filled can latex paint 95%

Used fluorescent light bulbs 93%

All participants found location of their county hazardous waste drop off site.


Ease of Completing Searches on GreenGuardian.com

Rating

Very easy 74%

Somewhat easy 24%

Neither easy nor difficult 2%


Reasons Given for Easy Searches

Format

One place, up front Format easy to follow Categories made sense Differentiation between house

items and yard items Major categories on left hand

side and subcategories on the right

Search engine

Content

Concise clear headings of where to go

Topics were relevant Used terminology I would use Straightforward simplistic text County listing and links helpful


Rating Amount of Information

Rating

Not enough 0%

Just right 95%

Too much 5%


Rating Clarity of Information

Rating

Very clear 76%

Clear 22%

Somewhat clear 2%


Overall Site Rating

Very good (5) 71%

(4) 29%

(3)

(2)

Very poor (1)


Reasons for Positive Ratings of Site

Navigation Easy to find information Easy to click onto other sites Got what I needed Got information very quickly Answers quicker than telephone

Content Informative Gave a lot of

information Completeness of

information Well organized Categories you

needed Appreciate maps

Appearance Clean Easy to read Easy on eyes Typeface right size Subtle, but good


Website Value Statements

Information Related Source of information on disposal of anything A metro-area resource for information on “reduce” and “reuse” Excellent resource for conserving and recycling Excellent resource for household hazardous waste Great site to find out about environmental issues Can check to dispose of hazardous things Useful site especially when it comes to spring cleaning and outdoor

stuff

User Benefits Will save you time and money Has everything in one place – don’t need four or five other websites Don’t have to wait for a telephone person – most stuff self explanatory


Suggestions for Site Enhancement

Design Add more categories to home

page Better navigation back to home

page More graphics on home page Make ‘TV computer monitor’

icon a clickable link Better name that fits what I am

looking for Make search engine more

noticeable

Content More child-friendly More interesting for

13 yr. old List prices for disposal

of things Put in local community

recycling listings

Communications Get the word out

it’s available Tell where you will

promote site Announce site in

garbage or electric bills

Email sign up

Usability Lab Testing

n = 6


Methodology – usability lab testing

6 usability sessions were conducted at Leeds Research facility in Plymouth

Each session consisted of discussions and specific tasks performed on the GreenGuardian.com web site

Each session lasted about 30 minutes Sessions were videotaped for inclusion of participant comments in this

report Each person was paid $75 for their participation


Participant Profile

Moderately-to-very web-savvy, college-educated women

Working full or part-time

Between the ages of 25-54

Who had not been to this site before the session


Session Format

Participants logged onto the site and with the facilitation of the

moderator did the following:

Took quick look at home page and gave 1st impressions as to look and site purpose

Discussed expectations for each of the content categories on home page

Performed three search scenarios to locate information Examined the ‘Know What to Throw’ section in detail Revisited the home page and discussed organization Provided reactions to the Green Guardian mascot


Green Guardian Name / Purpose

The name Green Guardian has a logical association with the environment as well as grass/lawncare.

– In a top of mind guess after hearing just the name “lawn and garden” is often listed first.

– It becomes obvious that the site ties into healthy environmental habits after a glance or two at the home page. Recycling is often the first term that people mention once they look around the home page and describe its content.

But, a clear statement of purpose does not jump out at users. – The story of the Green Guardian is not likely to be read in full by many. Even

when forced to view it, most only perused the text.– More could be done to emphasize the mission of the site when a person first

gets on the site.– This statement should talk about the “quick how to” as much as the “good

reasons why” as it relates to proper disposal.


Participant Reactions – 1st impressions


Getting to the Site

People will likely search using words like “recycling”, “environment”, or “disposal” if trying to find information on the Internet about how to properly dispose of items:

– It would be important that words like these lead people to this site if people come in through Google and the like.

– Increased external awareness that a site like this exists would also be crucial if the purpose of the site is to increase the number of people/households who exhibit good behavior.

Some people might think about checking with their county for disposal rules and options, but it is not top-of-mind for all. Some do not expect there to be differences in policies or processes from county to county. People simply want to know how to dispose of items or to get a list of possible places they can go in the metro area that will accept their items.

Home Page


Mascot

The Green Guardian mascot is perceived as cute and inviting to some whereas, goofy and juvenile to others.

– The recycle sign on his shield ties him to the site.– To some his story is informative without being preachy. But, most ignore it.– People noticed movement in the eyes and the revolving ball. But, few noticed

the text descriptors change. Movement is fine, if not good. It signals to some that the site is working and has not locked up.

– A few people want to tweak things, of course: remove the armor because of its connection with violence. face him the other way so that the text descriptors can face left versus

right, where they will be more noticeable.

While another mascot or logo may speak more profoundly to some, it is difficult to find one that pleases all. This mascot seems to be noticeable and unique to most. It should help people remember the name of the site, if nothing else.


Participant Reactions – mascot


Color/Aesthetics

The green color on the GreenGuardian.com site is seen as a logical choice

and some find it calming. But, there may be too much green, making the appearance kind of drab.

While it need not be bright, some color would help to:– accentuate and emphasize key issues and areas– make the site look more exciting

Some feel visual images might be nice on the home page, too. These

photos and images could be used to highlight hot topics.


Participant Reactions – aesthetics


Categorization

The main categories that branch off the home page are fairly straight forward. But the following were confusing to some:

At Work: – People seem to assume that this is a site for households. – Some would never think to go to a site for work-related disposal.– Companies tend to have established policies for the disposal of

commonly discarded items (food, chemicals, bottles). – The introductory page for the work section highlights a pie chart that

shows where “our” garbage goes. The lead in sentence references businesses, but at a glance the content does not jive with the title.

Not all know that “FAQ” stands for “frequently asked questions” even though it is used quite often in other places (and other websites).

News & Events could/should contain global news about environmental issues or tragedies not just Green Guardian sponsored/affiliate events.


Other Ways to Categorize

Some had suggestions on how to streamline the existing categories to something like this: Know what & how to throw (best ways to dispose of or donate

items) Know what & how to buy (things to consider when buying products) Find out how to get involved (events in the community, activities for

kids, etc.) Find out more about us and our mission (FAQ, important links,

etc.) A user-based classification method was introduced to some ( home, work,

kids, etc.) but did not seem to improve on the current approach, if at all.


Participant Reactions – alternative categories


Home Page Recap

Use some color and images to highlight things

Possibly to streamline these categories

Make the overall purpose jump out more clearly

These could be more prominent

Inside “Know What to Throw”


Content

The content on the site is useful, eye-opening in some cases, and comprehensive. Several learned something they did not already know during the session. They said things like:

– “Yuk. I didn’t realize monitors have lead in them”– “I didn’t realize you had to know how to compost” – “I didn’t realize there were these non-profits who take/want items others

dispose of”

Although people were OK with the mix of education and action, and expected both, they were not expecting the information/education part to be so prominent.


Some Education is Critical

Even those who considered themselves quite aware of the reasons why proper disposal is important were enlightened in some way.

Those less aware can use it to defend a request (of kids) or to see why a change in behavior would be good.

Those less aware might be turned off if the message interferes with their ability to act, but may need a few tidbits of information to gradually change their behavior in a positive direction.


Participant Reactions – education

Even those who considered themselves quite aware of the reasons why proper disposal is important were enlightened in some way.

Those less aware can use it to defend a request (of kids) or to see why a change in behavior would be good.

Those less aware might be turned off if the message interferes with their ability to act, but may need a few tidbits of information to gradually change their behavior in a positive direction.

– “You are much more likely to get me to do something if there is a good reason for it. And, not only that I’ll do it willingly and with enthusiasm.”

– “I don’t want to know about legislation about it? No. I just want to know how to get rid of it.”


Layout

The site is easily navigable, but not very efficient when trying to get to

an answer.

– There is too much text

– Getting to the answer is the first priority (“How do I x?”)

– Education is nice and can happen in manageable doses


Participant Reactions – layout


Amount of Text

Research has shown that people do not read web pages, they scan them1 typically from top to bottom, left to right:

– They read web pages more like a billboard than a book.– They might see something like this when searching for something specific:

1J. Nielson, “How Users Read on the Web”. Alertbox 1997

How to get rid of TVs


Participant Reactions – text


One Woman’s Redesign Ideas

What to throw

What to buy

FAQ Get involved

Who we are

What kind of item do you want to get rid of? o Household items o XX

Did you know? Blah, blah, blah

Search


Organization

Things are not always where they are expected.– Information on TV disposal is easy to find. – Information on fluorescent lights it not.

The actual trail to is: Know what to throw > toxins & poisons The most intuitive path is: Know what to throw > household items

In the case of household items for one, many people want/expect to find a laundry list of bulleted items to click on. This list can be long if alphabetized since it is easy to search through. People can also review and make mental notes of other items that need special disposal in the future.

People want to make obvious progression with each click and ideally get to things in just a few clicks. Almost no one experienced this when looking for fluorescent light disposal.

Search is often a last resort and can save the experience if they get there successfully, so it is important that it work well. {Some felt the results that show up under search were not in the format they expected (bullets or phrases to take them to a specific “action” verses a topic “area”}


Participant Reactions – organization

“I was expecting light bulbs/lights to jump out at this point. “The first one I saw was batteries so I assumed light bulbs would have been under this…If it is not here, I’d expect it under a miscellaneous category.”

“Why are batteries listed here but lights are not? It is misleading. I would just assume if it is not here it is not a problem issue. That is I can throw it in the trash.”

“If I don’t see an item listed under household items then I assume it is not a big issue. The ones that are listed must be the most important. If not, then all items should be listed – even if the list is long – that’d be ok.”


Signaling Progression:

When within the “know what to throw” section the page layout looks very similar as a person clicks into the sub-categories.

– At a glance the layout is similar enough that some people do not realized they have moved on from the initial page.

– While consistency is good in many ways, a change in color or some change in the layout might make it easier to notice.

See example on the next slide

Note: Several people had confusion determining the difference between categories, such as “household hazardous waste” and “toxins & poisons”.


Progression Example

At a glance these two pages look very similar (not like one has moved deeper into a new set of information)


Audience

With the exception of possibly the mascot, most feel the site is geared for adults.

This site would be appropriate for kids but not enticing to kids.

– The kids’ section on the site was a reasonable start, if a child wanted something to do and ended up there.

– But, if a goal of the organization is to cultivate a future audience there may need to be a more visible attraction for kids on the home page.

– “Kids are more picture oriented or short phrase oriented versus big paragraphs.”


Overall Enhancement Considerations

Communicate more clearly the focus of the site on the home page. Make home page statement of purpose more eye catching. Review position of search engine on home page. Examine the length of scroll down menus with possibility to shorten. Review the hierarchy of item placement for any possible confusion

e.g. fluorescent bulbs. Examine the need for navigational aides for returning to the home page

from county pages. If site is to be used by kids, consider shortening text and adding element

of visual excitement to a section devoted to them. Have more material for teachers.

greenguardian.com website evaluation submitted by: marketline research, inc. june 8, 2004

Documents

site easy

reported research

recycling slide

overall site

green guardian web site

environmental information

parttime slide

participation slide