when too many is just enough: citizen engagement and federal government websites

35
When Too Many Is Just Enough: Citizen Engagement and Federal Government Websites Jeffrey Ryan Pass | IA Summit 2013 @jeffpass #ias2013 #Usabilityfo rGov

Upload: jeffreyryanpass

Post on 12-May-2015

395 views

Category:

Documents


0 download

TRANSCRIPT

Page 1: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

When Too Many Is Just Enough:Citizen Engagement and Federal Government Websites

Jeffrey Ryan Pass | IA Summit 2013

@jeffpass#ias2013

#UsabilityforGov

Page 2: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

I am:

It starts with a guy introducing himself…

Jeff Pass A first-time IA Summit speaker A “User Experience Consultant” Tenaciously fighting the good fight Here to talk about Citizen Engagement

through large-scale online IA-focused usability studies

Hi

Page 3: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

President Obama supports:

Next he talks about this guy…

Innovation Transparency Information Feedback We the People Portuguese Water Dogs

For all of these reasons (but one)…

Obama from “Change” to “Engage”

Page 4: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

On May 23, 2012 the White House, CIO and CTO released:

He commissioned a strategy & signed a memo…

Presidential Memorandum: Building a 21st Century Digital Government

Digital Government: Building a 21st Century Platform to Better Serve the American People

These are better known as the Digital Government Strategy

Page 5: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

The Presidential Memorandum:

The memo was a memo (and not very exciting)…

Introduced the Strategy Put departments/agencies on notice Established a 12-month roadmap

Page 6: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Introduced by Federal CIO and CTO:

The Strategy was something altogether different…

“Federal Government must be able to deliver information and services to the American people anytime, anywhere and on any platform or device”

Four over-arching principles (wait for it…)

12-month agency milestones (keep waiting…)

Federal CIO, Steven VanRoekel Federal CTO, Todd Park

Page 7: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Principle Addressed Through1 – “Information-Centric” Content Syndication

Data via APIsTaxonomy/Metadata

2 – “Shared Platform” Agency GovernanceAgency InventoryEvaluate GSA vehiclesShared CMS / Open Source

3 – “Customer-Centric” Modern UXSite/content consolidationSEOMobileMeasure satisfaction

4 – “Security and Privacy” FISMA complianceData securityPersonal Information Privacy

Four Over-Arching Principles*

* The first and third are most relevant to today’s chat

Page 8: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

12-Month Agency Milestones*

3 Months 8/2012 6 Months 11/2012 12 Months 5/2013 Identify 2 services for

APIs Identify 2 services for

Mobile Agency Governance Launch Digital

Strategy Page

Device Inventory Customer Satisfaction

Implemented

2 APIs Implemented 2 mobile services

Implemented New systems adhere Compliance

verification with GSA standards

Evaluate new GSA vehicles

ü ü* No real bearing on today’s chat but important to know

Page 9: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Digital Content of, for, and by Citizens…*

Information-Centric includes: Presenting content “in the way that is most useful for the

consumer of that information”

Allowing “customers to shape, share and consume information, whenever and however they want it”

“Using modern tools and technologies [to gather] customer feedback to make improvements”

Customer-Centric includes:

* Not really, but bear with me

Page 10: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Sounds Like a Job For…*

Me! Us: Information Architects Content Strategists Usability Specialists Other User Experience Professionals

* At least in some significant part

Page 11: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

So Much We Can Do…

Many aspects of IA/UX/Content Strategist work can execute on the Digital Government Strategy

Specifically, large-scale un-moderated usability tests (focused on IA) can contribute to “citizen engagement”

So let’s rummage through the IA toolbox…

Page 12: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Rummaging Through the IA Toolbox…We have the technology (to perform large-scale usability testing and analysis)

Open Card Sorts (e.g. OptimalSort)

Closed Card Sorts (e.g. WebSort)

Reverse Card Sorts (Tree Sorts) (e.g. Treejack)

Un-Moderated Usability Tests (e.g. Usabilla)

One-Click Tests (First-Click) (e.g. ChalkMark)

Immediate Feedback Tests (e.g. FiveSecondTest)

I have this ultimate set of tools… I can fix it!

Page 13: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Still Rummaging…We have other tools too(to compliment large-scale usability testing and analysis)

Surveys (e.g. SurveyMonkey) Page-based Feedback Mechanisms (e.g. Voice of Consumer) Customer Satisfaction Tools (e.g. ForeSee) Click Analysis Tools (e.g. CrazyEgg) Heat Mapping Tools (e.g. ClickHeat) User Research Tools (e.g. Ethnio) Crowdsourcing Feedback Tools (e.g. UserVoice)

Page 14: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Case Study: Large-Scale Closed Card Sorts…

First the background:

IA design of a public-facing website for a government healthcare agency

Began with over 100 content collections Goal to end with no more than five domains under a

single, unified IA and residing in a WCMS Iterative testing as well as multiple rounds of wireframe

usability testing

Page 15: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Case Study: Large-Scale Closed Card Sorts…

Next, card sorting basics:

Technique for organizing and validating IA Dates back more than 100 years Can be performed in person, remotely, or online Several types:

Open Card Sorts Closed Card Sorts Reverse Card Sorts

Page 16: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Case Study: Large-Scale Closed Card Sorts…

An online (closed) card sorting looks like this:

Page 17: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Case Study: Large-Scale Closed Card Sorts…

Now the conventional wisdom (card sorting in general, but closed card sorting specifically):

You need a minimum sampling for validity No value in samplings bigger than 25-35 participants

Creates more analysis and reporting work

So, how many participants should you have? There are many (well reasoned and documented) opinions…

Page 18: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Case Study: Large-Scale Closed Card Sorts…

Legend: Minimum Optimum

Freed (2012): 15 - 20

Page 19: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Case Study: Large-Scale Closed Card Sorts…

Legend: Minimum Optimum

Gaffney (2000): 4 - 6

Page 20: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Case Study: Large-Scale Closed Card Sorts…

Legend: Minimum Optimum

Nielsen (2004): 16

Page 21: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Case Study: Large-Scale Closed Card Sorts…

Legend: Minimum Optimum

Paul (2008): 6 - 12

Page 22: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Case Study: Large-Scale Closed Card Sorts…

Legend: Minimum Optimum

Robertson (2001): 4 - 8

Page 23: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Case Study: Large-Scale Closed Card Sorts…

Legend: Minimum Optimum

Spencer & Warfel (2004): 7 - 10

Page 24: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Case Study: Large-Scale Closed Card Sorts…

Legend: Minimum Optimum

Tullis & Wood (2004): 20 - 30

Page 25: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Case Study: Large-Scale Closed Card Sorts…

Legend: Minimum Optimum

Tullis & Wood (2005): 30 - 40

Page 26: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Case Study: Large-Scale Closed Card Sorts…

Legend: Minimum Optimum

Wood & Wood (2004): 25 - 30

Page 27: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Case Study: Large-Scale Closed Card Sorts…

So, how many participants did the case study have?

?

Page 28: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Case Study: Large-Scale Closed Card Sorts…Multiple closed card sorts with 1,000+ participants!

Page 29: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Case Study: Large-Scale Closed Card Sorts…

And how did we engage the participants? Directly.

Social media was our recruiterA blog post was our screener

OptimalSort and TreeJack were our vehicles

Page 30: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Case Study: Large-Scale Closed Card Sorts…

And the result?

Gained valuable insight for IA improvements, plus confirmed that large-scale approach:

Serves as a user outreach/feedback mechanism Allows for qualitative data collection alongside

quantitative data (via free-text comment fields) Raises awareness of the contribution of usability studies to

the presentation and use of online content Supports the Digital Government Strategy Really doesn’t result in unnecessary analysis and reporting,

but…

Page 31: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Challenges and Lessons Learned…

To avoid being crushed by the weight of data, analysis, and reporting you must:

Have a clear, well-established methodology Have a clearly defined goal and scope Use an online card sorting tool that can handle large-

scale participation

Page 32: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

So what do you think?

Share your thoughts and experiences about large-scale usability studies and direct user engagement

I’m listening…

Page 33: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Thanks for your time and participation!

Jeffrey Ryan Pass Lead User Experience ConsultantAquilent (www.aquilent.com)

[email protected]@jeffpass

Didn’t get enough (I honestly cannot imagine)? Then check out our (with UserWorks colleague Weimin Hou) case study posters at #IAS2013!

Bye

Page 34: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Shameless Poster Plugs…

Page 35: When Too Many is Just Enough: Citizen Engagement and Federal Government Websites

Sources:Freed, E. (2012). How-To Guide for Intranet Card Sorting. The Social Intranet Blog (09/11/2012). Retrieved 03/12/2013 from http://www.thoughtfarmer.com/blog/2012/09/11/intranet-card-sorting/. Gaffney, G. (2000). What is Card Sorting? Information & Design, 2000. Retrieved 03/12/2013 from http://www.ida.liu.se/~TDDD26/material/CardSort.pdf. Nielsen, J. (2004). Card Sorting: How Many Users to Test. Jakob Nielsen’s Alertbox: July 19, 2004. Retrieved 12/21/2012 from http://www.nngroup.com/articles/card-sorting-how-many-users-to-test/. OptimalWorkshop (2011). How Many Participants Do I Need for My Survey? (And How Many Should I Invite?). Optimal Workshop Support Knowledge Base 11/14/2011. Retrieved 03/12/2013 from http://www.optimalworkshop.com/help/kb/remote-user-testing/how-many-participants-do-i-need-for-my-survey-and-how-many-should-i-invite.

Paul, C. L. (2008). A Modified Delphi Approach to a New Card Sorting Methodology. JUS Journal of Usability Studies, Volume 4, Issue 1, November 2008. Retrieved 03/12/2013 from http://www.academia.edu/150978/A_Modified_Delphi_Approach_to_a_New_Card_Sorting_Methodology. Robertson, J. (2001). Information Design Using Card Sorting. Step Two Designs, 02/19/2001. Retrieved 03/12/2013 from http://www.steptwo.com.au/papers/cardsorting/index.html. Sachs, J. (2002). Aristotle's Metaphysics. Green Lion Press, Santa Fe, NM.

Spencer, D., & Warfel, T. (2004). Card Sorting: A Definitive Guide. Boxes and Arrows 04/07/2004. Retrieved 03/12/2013 from http://www.boxesandarrows.com/view/card_sorting_a_definitive_guide. Tullis, T. S., & Wood, L. E. (2004). How Many Users Are Enough for a Card-Sorting Study? UPA 2004 Conference, Minneapolis, NM. Retrieved 12/21/2012 from http://home.comcast.net/~tomtullis/publications/UPA2004CardSorting.pdf. Tullis, T. S., & Wood, L. E. (2005). How Can You Do a Card-sorting Study with LOTS of Cards? UPA 2005 Conference, Montreal, Quebec, Canada. Retrieved 12/21/2012 from http://www.eastonmass.net/tullis/presentations/Tullis&Wood-CardSorting.pdf. Wood, J. R., & Wood, L. E. (2008). Card Sorting: Current Practices and Beyond. Journal of Usability Studies, Volume 4, Issue 1, November 2008. Retrieved 03/12/2013 from http://www.upassoc.org/upa_publications/jus/2008november/wood3.html. UserZoom (2011). Online Card Sorting: What, How & Why? UserZoom 01/20/2011. Retrieved 03/12/2013 from http://www.userzoom.com/online-card-sorting-what-how-why/.

Note: The Digital Government Strategy was announced on 05/23/2012 in the Presidential Memorandum: Building a 21st Century Digital Government (http://www.whitehouse.gov/the-press-office/2012/05/23/presidential-memorandum-building-21st-century-digital-government) and detailed in the actual strategy document Digital Government: Building a 21st Century Platform to Better Serve the American People (http://www.whitehouse.gov/sites/default/files/omb/egov/digital-government/digital-government.html).