Measurement of eGovernment user satisfaction and impact

eGovMoNet

Sebastiaan van der Peijl (Deloitte)

Hungary, 26 March 2009

Introduction to the study approach and objectives

Results of the pilot study

Next steps: implementing user satisfaction measurement in eGovernment

AGENDA

- 3 -

Study approach and objectives

Measurement of user satisfaction & impact of eGovernment services

ASSIGNMENT: a multilayer user satisfaction and impact

measurement toolkit for inclusive public eServices

- 4 -

Study approach and objectives: the policy context

Information Society

• Information and communication technologies (ICT) to increase the value of services to citizens and business

Expectation management

• Consumers compare quality of public services with quality of private sector services

Delivery of eServices (sophistication)

• Effect of high-quality services on take-up and satisfaction

eGovernment

• Impact of social (em/a)pathy, trust in government institutions, social exclusion....

Measurement needs

• How can we measure citizen needs and expectations

- 5 -

Study approach and objectives: a five-step process

1

•State-of-the-art: in EU27 and abroad (Close collaboration with eGov subgroup)

2

•Design of survey instrument (Citizens/Business)

3

•Pilot in 10 member states:10.000 citizens, 4000 business

4

•Instrument evaluation: 2 levels of measurement (benchmark tool & eService evaluation tool)

5

•Assessment by experts and government contacts during Workshop and meetings & Policy recommendations

Pilot study results: testing the instrument

- 6 -

Selection•Belgium,•Netherlands•France•UK•Sweden•Austria•Italy•Spain•Poland•Germany

Survey sample•10.000 citizens•4000 companies

Population•Citizens: representative for the internet population of the

Member States (Eurostat data for gender and age)•Companies: representative mix of SME’s and large companies

via CEO (for SME’s) and Financial and HRM functions in Middle and top management for large companies

•Executed by online panel providers•Timing: first half September 2008

- 7 -

Pilot study results: supply-use gap for citizens

Health

-rela

ted

serv

ices

(Birt

h an

d m

arria

ge) C

ertifi

cate

s

Decla

ratio

n to

the

polic

e

Driver

's lic

ence

Pass

ports

Unem

ploy

men

t ben

efits

Publ

ic lib

rarie

s (c

atal

ogue

s, sea

rch

tool

s)

Stud

ent g

rant

s

Car re

gist

ratio

n

Applic

atio

n fo

r a b

uild

ing

perm

issio

n

Annou

ncem

ent o

f mov

ing

Enro

lmen

t in

high

er e

duca

tion

Inco

me

taxe

s

Job

sear

ch se

rvice

s0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

eGov Use eGov supply side

Gap between eGovernment supply and use differentiates between services

- 8 -

Pilot study results: supply-use gap for business

Socia

l con

tribu

tion

for e

mpl

oyee

s / C

orpo

rate

tax

/ VAT

Registra

tion

of a

new

com

pany

Subm

issio

n of

dat

a to

stat

istic

al o

ffice

s

Custo

ms d

ecla

ratio

n

Envi

ronm

ent-r

elat

ed p

erm

its

Publ

ic pr

ocur

emen

t0%

20%

40%

60%

80%

100%

eGov Use eGov supply side

Gap between eGovernment supply and use is less within a business environment

Pilot study results: Non-use of eGovernment by level of trust (Citizens)

- 9 -

Non-use of the Internet for contact with public administrations by level of trust in the Internet and in public administrations: when trust is low % of non-use is higher

Very little Little Mediocre Much Very much0%

5%

10%

15%

20%

25%

30%

35%

40%

45%

50%

Trust in InternetTrust in publicadministrations

Pilot study results: Reasons for non-use of the Internet (Citizens)

- 10 -

Creation of Awareness

Other reasons

Lack of skills

Too difficult to use

No ability to access

No expectation to find

Lack of trust using the Internet

No ability to find

Quality of public e-services

No willingness to use the Internet

No awareness of the existence

No need of the Internet

0% 5% 10% 15% 20% 25% 30% 35% 40% 45% 50%

Pilot study results: satisfaction with e-services (Citizens)

- 11 -

eCommerce eBanking Social networking

eGovernment information

eGovernment services

eParticipation0

1

2

3

4

5

6

7

8

9

10

8.99.5

8.17.4 7.5

7.0

eGovernment is lagging behind in use and satisfaction compared to non-government eServices

Pilot study results: satisfaction with e-services (Business)

- 12 -

eBan

king

Send

ing

e-invo

ices

eWor

king

Receivin

g e-

invo

ices

Send

ing

orde

rs

Voice

-IP

Video

conf

eren

cing

Receivin

g or

ders

eGov

ernm

ent i

nfor

mat

ion

eGov

ernm

ent s

ervic

es5

6

7

8

9

109.0

8.6 8.5 8.5 8.5 8.4 8.2 8.18.4 8.3

The difference within the business environment is less

Pilot study results: impact of level of sophistication (Citizens)

- 13 -

Satisfaction goes up with the level of interaction of the service

E-mail Information Downloading forms

Uploading forms Transaction

Highest level of interaction

5

6

7

8

9

10

7.0 7.17.3

7.8 7.9

Overall level of satisfaction

c

Pilot study results: impact of level of sophistication (Business)

- 14 -

E-mail

Information

Downloading forms

Uploading forms

Transaction

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

Yes, totally Partially No, not at all I can't say, my interactions with public agencies for this event are still ongoing

The higher the level of interaction the more objectives are achieved

Pilot study results: perceived benefits by citizens

- 15 -

Saving

tim

e

Mor

e fle

xibi

lity

Sim

plifi

catio

n

Saving

mon

ey

Bette

r con

trol

Mor

e tra

nspa

renc

y

Bette

r qua

lity

Incr

ease

of t

rust

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

77.9% 76.1%

63.2%57.8%

47.1% 43.8%38.9%

30.4%

Perceived impacts are time-saving, flexibility and simplification

Pilot study results: perceived benefits by businesses

- 16 -

Saving time

More flexibility

Simplification

Saving money

Better control

More transparency

Better quality

Increase of trust

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

73.0%67.1% 64.3%

58.7%51.9% 50.3% 48.0%

41.1%

Perceived impacts by business are equal

- 17 -

Pilot study results: key findings

1

•The first important element shaping the use of and satisfaction with eGovernment services is trust (in Internet, Government, administration...).

2

•The level of quality and the level of sophistication play both a highly important role in take-up and satisfaction.

3

•Lack of awareness of eGovernment services is an crucial barrier for the use.

4

•Perceived benefits are time-saving, flexibility and simplification.

5

•Measurement of user satisfaction and impact is an unmistakable part of keeping in touch with the quickly evolving world of inclusive eService expectations.

- 18 -

Next steps: implementation

• Toolkit :• 4 questionnaires• Survey instrument

manualDELIVERABLES

- 19 -

Next steps: implementation

User Satisfaction

Benchmark for Citizens (USB)

User Satisfaction

Benchmark for Business (USB)

eService Evaluation Tool

for Citizens (eSet)

eService Evaluation Tool

for Business (eSet)

- 20 -

Next steps: implementation

Multilayer research tool

USBMeasuring use

and user satisfaction

within a comparative

cross-national framework

eSETMeasuring

use and user satisfaction

for one service

Standard measurement framework

Citizens

Business

Customizable modular structure

User types Use Satisfaction Impact

Next steps: implementation

- 21 -

• Use and Satisfaction Benchmark/eService evaluation• Target group: citizens/business; city/region; Service….

Define objectives and scope

• Use professional advise (statistical agency, contractor in the research business)

Adapt the instrument towards your needs

• Fieldwork modus (online, telephone....)• Research sample

Prepare, organize the data gathering process

Analysing and reporting

Conclusions and recommendation

Dashboard (monitoring long-term perspective)