六合彩

Measuring ROI on ACSI Surveys

Tim EvansTim EvansSocial Security AdministrationSocial Security AdministrationOffice of Electronic ServicesOffice of Electronic Services

[email protected]@ssa.gov

IntroductionIntroduction

ROI on your surveys not directly attributable to ROI on your surveys not directly attributable to your survey expenditures—you may be your survey expenditures—you may be validating prior workvalidating prior work

Ways of using ACSI data for ROI purposesWays of using ACSI data for ROI purposes– Identify user behavior that explicitly shows your site’s Identify user behavior that explicitly shows your site’s

valuevalue– Identify data that shows how you can improve your Identify data that shows how you can improve your

site to increase its valuesite to increase its value– Identify and focus your (limited) resource prioritiesIdentify and focus your (limited) resource priorities

SSA and ACSISSA and ACSI

Began Q1/04Began Q1/04Four current surveysFour current surveys– Main siteMain site– FAQ’s siteFAQ’s site– Retirement Planner siteRetirement Planner site– Internet Social Security Benefit Application--ISBA (not Internet Social Security Benefit Application--ISBA (not

published)published)

Two more surveys plannedTwo more surveys planned– Business Services On-line (employer wage reporting)Business Services On-line (employer wage reporting)– Disability (largest single group of surveyed visitors Disability (largest single group of surveyed visitors

say they came for disability info)say they came for disability info)

SSA ROI for ACSI Surveys--SSA ROI for ACSI Surveys--SummarySummary

Main survey shows high value + return on 2003 Main survey shows high value + return on 2003 site redesignsite redesign

FAQ survey shows clear reduction in use of FAQ survey shows clear reduction in use of high-cost support channels as direct result of high-cost support channels as direct result of FAQ’s availabilityFAQ’s availability

Retirement Planner survey shows high value of Retirement Planner survey shows high value of Planner + the opportunity to focus our resourcesPlanner + the opportunity to focus our resources

ISBA Survey helps clarify resource priorities in ISBA Survey helps clarify resource priorities in limited-budget environmentlimited-budget environment

Main Site Survey, Q3/04Main Site Survey, Q3/04http://www.socialsecurity.govhttp://www.socialsecurity.gov

OverallOverall 7575

ContentContent 8080

FunctionalityFunctionality 7676

Look & FeelLook & Feel 7979

NavigationNavigation 7373

SearchSearch 7474

Site PerformanceSite Performance 8383

Compare to ForeSee Results Compare to ForeSee Results BenchmarksBenchmarks

Overall Satisfaction: +5 Overall Satisfaction: +5 vis a visvis a vis Q2/04 Q2/04 Gov’t benchmark; +4 Gov’t benchmark; +4 vis a vis vis a vis overalloverall

Look & Feel: +4/+3Look & Feel: +4/+3

Content: +2/+2Content: +2/+2

Navigation: +5/+3Navigation: +5/+3

2003 Site Redesign2003 Site Redesign

Site-wide adoption of CSS Style Sheets Site-wide adoption of CSS Style Sheets for common look & feel (colors, fonts, etc.)for common look & feel (colors, fonts, etc.)

Added common header with top nav-linksAdded common header with top nav-links

Primarily implemented via common Primarily implemented via common DreamWeaverDreamWeaver template for easy template for easy propagation site widepropagation site wide

Context: ~80 SSA webmasters with Context: ~80 SSA webmasters with central, but informal governancecentral, but informal governance

SSA FAQ’s Survey, Q3/04SSA FAQ’s Survey, Q3/04http://ssa-custhelp.ssa.gov/cgi-bin/ssa.cfg/php/endhttp://ssa-custhelp.ssa.gov/cgi-bin/ssa.cfg/php/end

user/entry.phpuser/entry.php

Overall ScoreOverall Score 7474

ContentContent 8181




SearchSearch 7575


Custom Questions Results Reveal Custom Questions Results Reveal ROI--#1ROI--#1

If there were no FAQ’s, how would you have If there were no FAQ’s, how would you have contacted Social Security to ask your question? contacted Social Security to ask your question? (N=1273)(N=1273)– Send e-mailSend e-mail 16%16%– Browse web siteBrowse web site 8 8– Call 800 numberCall 800 number 3939– Call local SSA officeCall local SSA office 2222– Visit local SSA officeVisit local SSA office 1010– Write SSAWrite SSA 1 1– Would not have contacted SSAWould not have contacted SSA 4 4

Custom Questions Results Reveal Custom Questions Results Reveal ROI--#2ROI--#2

After reading our answer(s), what do you plan to After reading our answer(s), what do you plan to do next? (N=487)do next? (N=487)– Nothing; question answeredNothing; question answered 36%36%– Nothing; question not answered Nothing; question not answered 4 4– Send SSA e-mailSend SSA e-mail 6 6– Call SSA 800 numberCall SSA 800 number 1414– Call SSA local officeCall SSA local office 9 9– Visit local SSA officeVisit local SSA office 2222– Try different questionTry different question 4 4– Look elsewhere on siteLook elsewhere on site 3 3

Key Comparisons for 2 QuestionsKey Comparisons for 2 Questions

Send SSA e-mailSend SSA e-mail 16%16% 6% 6% -62% -62%

Call SSA 800 numberCall SSA 800 number 3939 1414 -65 -65

Call SSA local officeCall SSA local office 2222 9 9 -59 -59

Visit local SSA officeVisit local SSA office 1010 2222 +120+120

ROI—Reduction Use of High-Cost ROI—Reduction Use of High-Cost Support ChannelsSupport Channels

Forester Research Estimates Cost of ServicesForester Research Estimates Cost of Services– Knowledge-Based Self Service: $1.17/incidentKnowledge-Based Self Service: $1.17/incident– Telephone support: $33/incidentTelephone support: $33/incident– E-mail support: $10/incidentE-mail support: $10/incident

SSA Results, shown by ACSI dataSSA Results, shown by ACSI data– 65% reduction in calls to 800 number65% reduction in calls to 800 number– 59% reduction in calls to local office59% reduction in calls to local office– 62% reduction in e-mails62% reduction in e-mails– 120% increase in local office visits120% increase in local office visits

SSA Retirement Planner Survey, SSA Retirement Planner Survey, Q3/04Q3/04

http://www.socialsecurity.gov/r&m1.htmhttp://www.socialsecurity.gov/r&m1.htm

Overall ScoreOverall Score 7777

ContentContent 8282





Tasks/TransactionsTasks/Transactions 7878

SSA Retirement Planner Survey, SSA Retirement Planner Survey, Q3/04--FuturesQ3/04--Futures

Likely to ReturnLikely to Return 8585

RecommendRecommend 8383

File on-lineFile on-line 6161

What’s wrong with this picture?What’s wrong with this picture?

Is it good news or bad?Is it good news or bad?

Retirement Planner—Custom Retirement Planner—Custom QuestionQuestion

How do you plan to file for retirement benefits? How do you plan to file for retirement benefits? (7/12-8/9 data; N=1127)(7/12-8/9 data; N=1127)– On lineOn line 39%39%– Face to faceFace to face 3737– TelephoneTelephone 4 4– On line w/SSA helpOn line w/SSA help 2 2

More recent data shows ‘on line w/help’ running More recent data shows ‘on line w/help’ running ~13%, ‘on line’ ~30%--totaling ~13%, ‘on line’ ~30%--totaling more than 40%more than 40% (not enough data for conclusions)(not enough data for conclusions)

Pew Internet StudyPew Internet Studyhttp://www.pewinternet.org/PPF/r/128/report_display.asphttp://www.pewinternet.org/PPF/r/128/report_display.asp

How Americans Get in Touch with How Americans Get in Touch with GovernmentGovernment (May, 2004) (May, 2004)– TelephoneTelephone 40%40%– WebWeb 2424– In personIn person 1313– E-mailE-mail 1111– Write a letterWrite a letter 1010

Pew Internet Study—Cont’dPew Internet Study—Cont’d

Why Americans contact GovernmentWhy Americans contact Government– Conduct TransactionsConduct Transactions 30%30%– Get specific informationGet specific information 2525– Register opinionsRegister opinions 1919– Help w/specific problemsHelp w/specific problems 1111

For complex, problem-solving contacts, For complex, problem-solving contacts, 62% prefer phone/in person62% prefer phone/in person

What Does This Tell Us About ROI What Does This Tell Us About ROI for Retirement Planner?for Retirement Planner?

With ~40% interested in filing on line, SSA With ~40% interested in filing on line, SSA is doing quite well (compare Pew 30% is doing quite well (compare Pew 30% transactions figure)transactions figure)Futures Score shows opportunity to Futures Score shows opportunity to improve on-line application, as Boomers improve on-line application, as Boomers age and interest in on-line filing risesage and interest in on-line filing risesNew ‘On-line w/Help’ Numbers strong New ‘On-line w/Help’ Numbers strong indicator for future, bearing out Pew indicator for future, bearing out Pew conclusions on complex contactsconclusions on complex contacts

Internet Social Security Benefit Internet Social Security Benefit Application (ISBA)Application (ISBA)

Retirement/spouse applicants vastly more Retirement/spouse applicants vastly more satisfied (>2:1) than disability applicantssatisfied (>2:1) than disability applicantsFar more (5:1) retirement/spouse applicants Far more (5:1) retirement/spouse applicants successfully complete on-line applicationsuccessfully complete on-line applicationLowest scores in Functionality, Lowest scores in Functionality, Tasks/Transactions; cut across board, Tasks/Transactions; cut across board, especially especially among disability applicantsamong disability applicantsDisability applicants have far lower level of Disability applicants have far lower level of Internet experience (5:1 “beginner”), and may Internet experience (5:1 “beginner”), and may need additional helpneed additional helpExpectations count: level of Internet experience Expectations count: level of Internet experience inversely proportional to satisfactioninversely proportional to satisfaction

ISBA and ROI LessonsISBA and ROI Lessons

Retirement/spouse applicants >80% of ISBA Retirement/spouse applicants >80% of ISBA customers; universe will grow as Boomers age customers; universe will grow as Boomers age and demand service (Pew data cited earlier)and demand service (Pew data cited earlier)Disability applicants have lower Internet Disability applicants have lower Internet experience, need more support via high-cost experience, need more support via high-cost channelschannelsThe whole disability application process differs The whole disability application process differs substantially from retirement application processsubstantially from retirement application processWith resources limited, best use lies in improving With resources limited, best use lies in improving overall Functionality, Tasks/Transactionsoverall Functionality, Tasks/TransactionsAll applicants will benefit from general All applicants will benefit from general improvements in the on-line applicationimprovements in the on-line application

WrapWrap

ROI on your surveys not directly ROI on your surveys not directly attributable to your survey expendituresattributable to your survey expenditures

Ways of using ACSI data for ROI Ways of using ACSI data for ROI purposespurposes– Identify user behavior that explicitly shows Identify user behavior that explicitly shows

value of your sitevalue of your site– Identify data that shows how you can improve Identify data that shows how you can improve

your site to increase its valueyour site to increase its value– Identify and focus your resource prioritiesIdentify and focus your resource priorities

SSA ConclusionsSSA Conclusions

Main survey shows high value + return on 2003 Main survey shows high value + return on 2003 site redesignsite redesign

FAQ survey shows clear reduction in use of FAQ survey shows clear reduction in use of high-cost support channels as direct result of high-cost support channels as direct result of FAQ’s availabilityFAQ’s availability

Retirement Planner survey shows high value of Retirement Planner survey shows high value of Planner + the opportunity to focus our resourcesPlanner + the opportunity to focus our resources

ISBA Survey helps clarify resource priorities in ISBA Survey helps clarify resource priorities in limited-budget environmentlimited-budget environment

六合彩

Business

ssa roi

ssa email

ssa local office

ssa faqs survey

contacted ssa

ssa retirement planner

ssa webmasters

visit local ssa office