© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

Zagreb, 22-23 May 2014

(School of Public Administration)

Citizen involvement in service quality:

Citizen surveys, focus groups, & citizen

panels

Marcel Guenoun & Elke Loeffler, SIGMA Experts

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

How can you measure citizen satisfaction?

Qualitative

analysis

Quantitative

analysis

Quantitative/

qualitative

analysis

Focus

groups

Citizen

surveys

Citizen

panels

Understand

citizens’

views,

empathy

Representative

sample,

external

validity

Views of

different

stakeholders,

quick feedback

Reliability In-depth

analysis

Sounding

board

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

Strengths & weaknesses of survey methodologies

Speed Flexibility Interaction

with

service

users

Coverage

of service

users

Response

rate

Cost

Phone

survey + + + + ++ + - - Written

survey - - - - + - - + Online

survey + + + + - - - - ++ Face-to-

face

survey - + + + + + + - -

A context-specific choice of the approach is key!

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

Source: European Institute of Public Administration: European Primer on Customer

Satisfaction Managment, 2008, p. 52-53 (adapted).

The 5 – D Model for raising the right questions

D 1 Definition of the ‘problem’

D 2 Design

D 3 Data collection

D 4 Data analysis

D 5 Communication

D 6 Use of data and information gathered

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

Internal vs external perceptions

Public officers Citizens

In your view, what are the issues that

should be discussed in the local council

meetings to attract the interest of citizens?

1 Neighboorhoods

2 Traffic

3 Culture

4 Education & early

childhood

5 Environment

6 Green space

7 Public safety

8 Social cohesion

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

An important survey result from Besancon

57% of overall satisfaction

concerning the municipality is explained

by front-office satisfaction…

A cause-effect analysis

that makes you gain

politicians audience!

(and more if you balance

front-office budget with

overall budget)

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

What is a focus group?

A dialogue method whereby a discussion

group of about eight persons is assembled for the purpose of exploring participants‘ views and attitudes to a structured set of questions and issues. Normally chaired by a trained &

independant facilitator.

Source: The Consultation Institute: Effective Focus Groups. A Consultation Institute Best Practice Guide by Barry Creasy (adapted).

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

Strengths and weaknesses of a focus group

+ Allows to discuss more complex issues, useful to gain ideas for quality improvement + Do not cost much, even though in some cases a small financial reward should be given to the participants (e.g. a shopping voucher) - Are not statistically representative even though focus group members may represent different stakeholders - Role of faciliator is crucial, needs to be independent and professional

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

Case study: Assessing the quality of life and

governance in a housing estate in the UK

The Beacon Community

Regeneration Partnership is

a resident-led charity in

Falmouth, Cornwall which

started in 1996 and now

offers a range of services to

local people, including a

garden and handyman

Service.

http://www.bcrp.org.uk

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

Current

state

W

Quality of

governance

Tenants

Staff

Public

officials

Voluntary

sector

Local

business

Media

Private

contractors

Quality of

outcomes

Board

Trend

The Governance Test © Governance International 2014

Objective of the focus groups in

Carrick Housing Estate

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

The structure of the focus group sessions

Each focus group discussed:

1) The current STATE “How safe do you feel in this area?”

2) The TREND “Has safety improved or got worse in the last three years or so?”

3) PROPOSALS “How could organisations and groups working in this area deal with identified weaknesses and build on strengths?”

4) The WILLINGNESS of participants to get engaged in helping to implement the discussed proposals “What are you willing to do to make things better?”

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

Example of the composition of a

focus group

Issue to be discussed: Community safety

Focus groups of:

A) Citizens

B) Business

C) Media

D) Public sector

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

The result: A Balanced Scorecard for Public

Services on Carrick Housing Estates, UK Quality of Life Issues

Liveable

environment

Carrick

Housing Staff Board

members Voluntary

groups Carrick

District

officers

Public officials

Community safety Young

families Media

Health, social well-

being and disability

issues

Disab.

Tenants Board

members

Education and

Training

Young People

Business

Governance Principles

Transparency Young

families Board

members Media

Partnership

working

Carrick

Housing staff Board

members Public

officials Voluntar

y groups

Sustainability Carrick

District

officers

Young

people

Honest and fair

behaviour

Disab.

tenants Private

contractors Business

Further information: Bovaird, Tony and Löffler, Elke (2007), Assessing the Quality of Local Governance: A Case

Study of Public Services, Public Money & Management 7 (2), pp. 293-300.

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

The Besançon Citizens Panel

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

Recruitment of panel members

Database : 19.885

Random draw : 5.000

Contacted : 4.340

Answered the questionnaire : 1.092

Accepted to become a panel member : 800

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

Who are the panel members?

Besançon Échantillon

Agriculteurs 74 0

Artisans, commerçants, chefs d'entreprise 2.248 17

Cadres, professions intellectuelles supérieures 7.883 61

Professions intermédiaires 13.071 101

Employés 16.049 125

Workers 10.915 85

Retraités 17.391 135

Unemployed 15.353 119

Students 20.099 156

Total 103.083 800

Those 3 segments of population were

recruited “on site” because they

often do not have a phone number or

accept to answer! !

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

Public Service Quality: An analysis of citizens’ expectations

Satisfaction level Contribution to

satisfaction

Reliability

-0,3 77%

Transparency

-0,4 63%

Customer relations -0,3 58%

Inclusion -0,3 55%

Tangible characteristics +0,2 54%

Assurance +0,1 49%

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

(13 local policy issues)

A citizen survey of local policies

Policy Segmentation “Public Policies” Survey

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

Tetraclass analysis of public

policies Tetraclass analysis of a

specific policy

Importance/performance

matrix of a key policy

dimension

Contrasting internal & external

perceptions

Panel outlook

19

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

Group exercise:

Assessing customer satisfaction

Discuss in small groups (from the same organisation if

possible):

Do you know if your citizens are satisfied with the service(s)

you deliver?

If yes, how do you know (how do you collect information)? If

you don’t know, what are the questions would you like to ask

them?

From your point of view, what are the key elements in your

service which create satisfaction or dissatisfaction?

© OECD

A j

oin

t i

nit

iati

ve o

f th

e O

EC

D a

nd

th

e E

uro

pe

an

Un

ion

,

pri

nc

ipall

y f

ina

nced

by t

he

EU

Source: European Institute of Public Administration: European Primer on Customer

Satisfaction Managment, 2008, p. 52-53 (adapted).

Assignment:

Design a citizen survey, a focus group meeting

or citizen panel based on the 5 – D Model

D 1 Definition of the ‘problem’

D 2 Design

D 3 Data collection

D 4 Data analysis

D 5 Communication

D 6 Use of data and information gathered