co-design and deliberative engagement—what works? · 07 graph 1: co-design and deliberative...
TRANSCRIPT
02
DEM
OC
RAC
Y 20
25
CONTENTS
04 ABOUTDEMOCRACY2025–BRIDGINGTHE TRUSTDIVIDE
05 EXECUTIVESUMMARY
06 PREAMBLE
09 METHODOLOGY
11 STAGEONE:CASESTUDYREVIEWS
14 STAGETWO:THEORETICALSTUDYREVIEWS
17 STAGETHREE:VALIDATION
22 CONCEPTUALMODELFORASSESSINGTHE QUALITYOFENGAGEMENTWITHCITIZENS
27 CONCLUSION
28 REFERENCES
32 ABOUTTHEAUTHOR
03
DEM
OC
RAC
Y 20
25FIGURES, GRAPHS AND BOXES
06 Figure1:Definingspacesbetweendeliberativeengagementsandco-design
26 Figure2:Conceptualmodelforassessingthequalityofengagementswithcitizens
GRAPHS
07 Graph1: Co-designanddeliberativedemocracyliteraturefrom1980to2018
17 Graph2: Proportionofcasescitingrepresentationrelatedsub-variablesbyco-designordeliberative engagement
18 Graph3: Proportionofcasescitingnon-representationrelatedsub-variablesbyco-designordeliberative engagement
BOXES1 09 Box1: SystematicReviewFramework
16 Box2: Variablesandsub-variablesinfluencingpositiveoutcomesinco-designanddeliberative engagement
23 Box3: VariableandOutcomesMeasurementScale
04
DEM
OC
RAC
Y 20
25ABOUT DEMOCRACY 2025 – BRIDGING THE TRUST DIVIDE
AcrossAustraliatrustinourdemocracyisonthe
decline.Trustisthegluethatfacilitatescollective
actionformutualbenefit.Withouttrustwedon’t
havetheabilitytoaddresscomplex,long-term
challenges.Trustisalsocloselytiedtodemocratic
satisfaction.MoAD’s(MuseumofAustralian
Democracy)recentresearch,Trustand
DemocracyinAustralia,showsthatin2018
satisfactionindemocracyhasmorethanhalvedin
adecadeandtrustinkeyinstitutionsandsocial
leadersiseroding.By2025ifnothingisdoneand
currenttrendscontinue,fewerthan10percent
ofAustralianswilltrusttheirpoliticiansand
politicalinstitutions–resultinginineffective
andillegitimategovernment,anddecliningsocial
andeconomicwellbeing.
Thisproblemmustbeaddressedasamatterof
urgency.MoADistakingaction.Wearebringing
togethereverysectionofthecommunityand
ignitinganationalconversationonstrengthening
Australiandemocraticpractice.MoADandour
foundationpartner,theInstituteforGovernance
andPolicyAnalysisattheUniversityofCanberra
(UC-IGPA),haveembarkedonaboldnewinitiative,
Democracy2025,tobridgethetrustdivideand
re-engageAustralianswiththeirdemocracy.
MoADholdsauniqueposition,onthefrontlineof
democracy,civicagencyandchange,amuseum
notjustofobjectsbutofideas.Weempower
Australiansthroughexhibitions,schools’learning
programsandeventsthatbothstimulateand
inspire.Trustedbythepublic,government,public
serviceandbusinessalike,weadvancenational
conversationsaboutdemocracy,past,present
andfuture.
Democracy2025isdrivingaprocessofnational
reflectionandrenewalonhowwecanrebuildtrust
andstrengthendemocraticpracticeinAustralia.
Webelievethatthisambitiousgoaliscriticaltothe
healthofthenation.Nothinglesswilldo.
Daryl Karp Professor Mark Evans
Director, DirectorofDemocracy2025
MoAD UC-IGPA
05
DEM
OC
RAC
Y 20
25EXECUTIVE SUMMARY
ThisthirdDemocracy2025reportdocuments
findingsfromasystematicreviewintowhatworks
inpublicparticipation.
Uniquely,thereportintegratesexpertisefromtwo
distinctapproachestocitizenengagement,
co-designanddeliberativeengagement.Each
approachoffersdifferentyetcomplementary
insightsintothevariablesthatleadtoeffective
citizenengagement,providingusefulevidence
thatcaninformpublicsectorcapabilityinthisarea.
Thisreportcontributestothegrowingbodyof
evidenceoneffectivecitizenengagementby
developingaconceptualmodelforassessingthe
qualityandimpactsofrepresentationandnon-
representationrelatedvariables.
Theconceptualmodeldrawson33casestudies
and36theoreticalstudiestoidentifysixkey
variablesthatinteracttoinfluenceoutcomesin
co-designanddeliberativeengagement.
Theseinclude:
1. inclusiverepresentation
2. autonomyandequalityofallparticipants
3. pluralityofviewpointsandengagement
methods
4. qualityofprocessdesignandfacilitation
5. transmissionofcitizengenerated
recommendations,and
6. citizenparticipationasademocraticvalue.
Inaddition,theconceptualmodelidentifiesthree
outcomerelatedmeasuresforassessingthe
impactofeachofthesevariablesonthelegitimacy
ofpublicdecisions.
Theseinclude:
1. Participantsagreeingonthesolutionsor
recommendations,
2. Participantstrustinginthelegitimacyofthe
processtoinfluencedecisionmaking,and
3. Consequentialitydefinedasdecisionmakers
acceptingcitizengeneratedrecommendations.
Collectively,thevariablesandoutcomemeasures
formanevidencebasedconceptualmodelfor
assessingthequalityandimpactofcitizen
engagementprocesses,supportingpublicsector
capability,politicalaccountability,andultimately
thelegitimacyofpublicsectordecisions.
06
DEM
OC
RAC
Y 20
25PREAMBLE
Co-designanddeliberativeengagementsaretwo
citizenparticipationmethodsthatarerapidly
changingthewaygovernmentsaroundtheworld
aregrapplingwithcomplexpublicpolicyproblems.
Theseapproachesemergefromdistinctly
differenttraditions,theformermoredominantin
managementandpublic-sectorinnovation1and
thelatterbelongingtotheareaofpolitical
philosophy2.Furthermore,thepractice-based
focusofdeliberativeengagementhasdeveloped
onlyrecentlyfollowingasubstantialfocuson
theoreticalaspectsinthebroaderfieldof
deliberativedemocracy3.Thisisincontrasttothe
focusonapplicationandpracticetoolsthatlieat
thecoreofco-design.4
Giventhesedifferences,itisnotsurprisingthat
therehasbeennosignificantacademicattemptto
integratethesemethodsintoacohesive
frameworkforunderstandingboththetheoryand
practiceofcitizenengagement,makingthis
researchafirstinthisarea.
1 See:Bason,2010.2 See:Florida,2018.3 See:ElstubandMcLaverty,2014.4 See:EvansandTerrey,2016.
Theunderlyinghypothesisisthatleveragingthe
expertiseofbothapproachesmayleadtonew
insightsintothevariablesthatleadtoeffective
citizenengagement,providingusefulevidence
thatcaninformpublicsectorcapabilityinthisarea.
Bothapproacheshaverapidlyexpandedinthe
pastfourdecadesfromemergingconceptsto
entirefieldsofstudyandapplication.Toillustrate
thisgrowthinattention,Graph1(page7)provides
resultsofanadvancedGoogleScholarsearchof
literatureineachdisciplinebetween1980to2018
(accessedon11August2018).
Theseresultspaleincomparisontotheextensive
culminationofmaterialswrittenoneachsubjectin
variousotherformsofinvestigation,including
greyliterature,websites,andmedia.
GOVERNMENT PUBLIC SERVICE
POLITICS POLICY SERVICE/SYSTEMS
DELIBERATIVE ENGAGEMENTS CO-DESIGN
Figure1:Definingspacesbetweendeliberativeengagementsandco-design
07
DEM
OC
RAC
Y 20
25
What is a Deliberative Engagement?
Deliberativedemocracyasatheoreticaltradition
isnowinformingdemocraticengagements,most
commonly(thoughnotexclusively)intheformof
mini-publics.Democraticengagements,following
theprinciplesofdeliberativedemocracy,are
designedtoenableapluralityofvoices,toshinea
lightonthetensionsandconflictsbetween
differentactors,andtoenableasharedspacefor
forgingunderstandingoncollectiveproblems5.
What is Co-design?
Co-designprocessesinvolvecitizensinthe
iterativedevelopmentofinsights,prototyping,
evaluationandscalingofnewsolutions6.Design
thinkingenablesrepositioningofparadoxical
perspectivesintonewproblem-solvingframes7
withtheunderlyingbeliefthatengagementwith
citizensinthedevelopmentanddeliveryof
5 See:Baiocchi and Ganuza, 2017.6 See:EvansandTerrey,2016.7 See:Dorst,2011.
productsorserviceswillleadtobetter(andshared
responsibilityfor)outcomes8.
Legitimacy and Representation
Bothapproachesrecognisethatallcitizenshave
legitimatevoicesinpublicpolicyprocesses9.Tobe
legitimate;however,participationshouldenable
equitableopportunitiestocontributetodecision
making,regardlessofpersonalcircumstancesor
contexts.Inlargeandcomplexsocieties,itisnot
possibleforallaffectedpeopletobeinvolvedin
decisionmaking10,necessitatingsomeformof
representation.
8 See:ParkerandHeapy,2006.9 See:Holmes,2011;andErcanet.al,2018.10 See:Cohen,1997;Michelman,1997;andFlorida,2013.
0
5000
10,000
15,000
20,000
25,000
Deliberative Democracy Codesign
2010-182000-091990-991980-89
Graph1:Co-designanddeliberativedemocracyliteraturefrom1980to2018
08
DEM
OC
RAC
Y 20
25Powerfulgroupstendtodictatehowlesspowerful
groupsarerepresentedinpolicydebatesandhow
asocialgroupisrepresentedwilldeterminehow
legitimatetheirinterestsandperspectivesare
perceivedtobe11.Marginalisedgroupsand
individualcitizenshavealowercapacityfordriving
policynarratives,compellingarguments,and
voicingconcernscomparedtopowerful
stakeholdergroups,businessesandpoliticians12.
Sincerepresentationhasasignificantimpacton
howequallytheneedsofaffectedpeopleare
considered,itisimportanttounderstandhow
representationimpactsoutcomesincitizen
engagementprocesses.
11 See:HabibisandWalter,2015.12 See:Lowndes,2016.
Forthepurposeofthisreport,representation
referstothecollectionofvariablesthatdescribe
participantsincitizenengagementprocesses,
suchashowinclusivelyparticipantsrepresent
affectedpeople,howequallyparticipants
contributetodiscussionsanddecisions,andthe
diversityofviewpointstheyhold.
09
DEM
OC
RAC
Y 20
25METHODOLOGY
TheSystematicReviewFramework(hereafterthe
framework)presentedinBox1,wasdevelopedto
identifykeyquestionsanddefinitionsforthe
systematicreview,alongwithcriteriaforthe
selectionofrelevantliteratureandtheprocessto
beundertakentoidentifyandassessliterature.
Box 1: Systematic Review Framework
KEY QUESTIONS1. Whatvariablesinfluenceoutcomesinco-designanddeliberativeengagement?
2. Istherearelationshipbetweenthosevariablesandrepresentationinco-designanddeliberativeengagement?
Definitions for the purpose of this review (P=population | I=intervention | C=control group intervention | O=outcome)
P= Representatives Individuals acting on behalf of others in the context of public participation
I= Co-Design Citizen involvement in public participation processes where design thinking is used to drive innovation and creativity
C= Deliberative Engagements
Citizen involvement in public participation processes where deliberative reasoning is used to drive shared decision making
O= Legitimacy The genuine and effective involvement of citizens in decision making processes
Criteria for Literature Selection
1. Literaturemustdescribevariablesrelatedtorepresentationinco-designordeliberativeengagementtoaddressthekeyquestionsforthisreview
2. Literaturemayincludeboththeoreticalandcasestudiestocoverempiricalandnormativeperspectives
3. Literatureshouldbepeerreviewedwithsuitablecitationstoensurereliabilityoffindings
4. LiteraturemustbewrittenintheEnglishlanguagewithfulltextavailabletosupportindepthreview
Search Strategy
1. GoogleScholar,AnalysisandPolicyObservatory,WileyOnlineLibrary,JSTOR,WebofScience,SCOPUS,andAustralianPublicAffairsFullTextwillbereviewedtoidentifysuitableliterature
2. Abstractsreviewedinitiallytoconfirmsuitabilityagainsttheselectioncriteria
3. Selectliteraturereviewedindetailtoidentifyconceptualthemes
10
DEM
OC
RAC
Y 20
25Data analysis
Thedataanalysisinvolvedathree-stageprocess
toidentifyvariablesthatinfluenceoutcomesin
co-designanddeliberativeengagements.Both
representationandnon-representationrelated
variableswereassessedtodetermineif
representationmatterswhenengagingcitizens
Stage one: analysis of case studies
Areviewof13casestudieswasundertakento
identifyfrequentlyoccurringvariables.Noteswere
takenonvariablescitedineachcasetoenable
emergenceofthemeswhichwerethencodedinto
sub-variablesforcomparativeanalysis.
Inadditiontodescriptivevariablessuchas
location,duration,andformofrepresentation,any
benefitsornegativeimpactscitedinthecase
studieswerealsorecordedtoassessifparticular
variablesinfluencedoutcomes.
Stage two: analysis of theoretical studies
Buildingonthefindingsofstageone,areviewof
36theoreticalstudieswasundertakentoassessif
thevariablesidentifiedasinfluencingoutcomesin
stageone,werealsocommonlycitedasimportant
inco-designanddeliberativeengagementtheory.
Variablescitedbythemajorityofstudies
(50percentormore)wereconsideredofhigher
relevancethanvariablescitedbyfewerstudies.
Inaddition,stagetwoalsoidentifiedsub-variables
thatprovideadeeperunderstandingofhow
variablesinfluenceoutcomes.
Stage three: validation of review findings
Avalidationexercisewasundertakento
confirmtherelevanceandmeasurabilityofthe
sub-variablesidentifiedthroughstagesone
andtwo.
Thevalidationprocessfirstlyassessedthe
frequencyofeachsub-variablebeingreferredto,
eitherdirectlyorindirectly,in20additionalcase
studies(10co-designand10deliberative
engagementcases)toconfirmtheirrelevance
forpracticalapplication.
Secondly,thevalidationassessedtheapparent
achievementofeachsub-variableinorderto
confirmtheirmeasurability.
11
DEM
OC
RAC
Y 20
25STAGE ONE: CASE STUDY REVIEWS
Stageoneconsideredarangeofvariablesthat
mayhaveimpactedoncitizenengagement,such
asthenumberofparticipants,durationof
participation,selectionmethods,focusareas,and
locations,toidentifyifanyparticularvariables
appearedtoincreasethenumbersofpositive
benefitscitedbyeachcase.
Co-design cases
Arangeofbenefitswerecitedacrossthe
co-designcasesreviewed.Thehighestcited
benefitwastheconvergenceofdiverse
experiencesandexpertise13,followedbyincreased
understandingofissuesandsolutions14,and
increasedtrustandempathy15.
Ofthesixco-designcasesreviewed,fourcases
focusedonhumanservicespolicyissues16with
twocasesfocusingonissuesrelatedto
transport17.Whenfactoringinthenumberofcited
benefits,thehumanservicesfocusedcases
recordedahigherproportionofbenefitsat
75percentcitingthreetofourbenefits,compared
to50percentoftransportfocusedcasesciting
threetofourbenefits.
13 See:Lee,2007;EnserinkandMonnikof,2003;Brandtet.al,2010;Penuelet.al,2007;andBowanet.al,2013.
14 See:EnserinkandMonnikof,2003;Zimmermanet.al,2011;andBowanet.al,2013.
15 See:Lee,2007;Penuelet.al,2007;andBowanet.al,2013.16 See:Lee,2007;Penuelet.al,2007;Brandtet.al,2010;and,
Bowenet.al,2013.17 See:EnserinkandMonnikof,2003;Zimmermanet.al,2011.
Inaddition,75percentofcasesthattargeted
knownserviceusersasparticipants18and
75percentofcasesthatuseddesignworkshops
astheprimaryengagementmethod19bothcited
threetofourbenefits,higherthanopenselection
andprototypingdesignmethods.
Finally,bothcasesthatoccurredover12months
ormorecitedthreetofourbenefitswhichwasnot
achievedincaseswithashorterduration20.
Table1providesanoverviewofthenumberof
benefitsreportedbycaseswheretheabove
variableswerepresent.Thesmallsamplesizesfor
thesevariablesandinconsistencyinreportingmay
howeverimpactonthereliabilityoftheseresults
whichshouldbeinterpretedwithcaution.
18 See:Lee,2007;Penuelet.al,2007;Brandtet.al,2010;andBowenet.al,2013.
19 See:EnserinkandMonnikof,2003;Lee,2007;Brandtet.al,2010;andBowenet.al,2013.
20 See:Penuelet.al,2007;andBrandtet.al,2010.
12
DEM
OC
RAC
Y 20
25Table1: Numberofbenefitscitedinco-designcaseswheresub-variablewaspresent
1-2 benefits 3-4 benefits Total Cases
PotentiallyInfluentialVariables Number Percent Number Percent Number Percent
Focus
HumanServices 1 25% 3 75% 4 67%
Transport 1 50% 1 50% 2 33%
Selection Approach
TargetedSelection 1 25% 3 75% 4 67%
OpenSelection 1 100% 0 0% 1 17%
MixedSelection 0 0% 1 100% 1 17%
Co-design Approach
DesignWorkshops 1 25% 3 75% 4 67%
Prototyping 1 100% 0 0% 1 17%
Both 0 0% 1 100% 1 17%
Length of Participation
SeveralWorkshops 1 33% 2 67% 3 50%
Lessthan1Month 1 100% 0 0% 1 17%
12Months 0 0% 1 100% 1 17%
Over12Months 0 0% 1 100% 1 17%
Note: Benefits per centages refer to the proportion of cases where sub-variable is present (differs by variable). Total case per centages refer to the proportion of all cases (6).
Deliberative engagement cases
Aswiththeco-designcases,therewerearangeof
benefitscitedinthedeliberativeengagement
cases.Increasedknowledge,understandingand
empathy21wasthehighestcitedbenefit,followed
byenablingdiverserepresentation22.Participants
feelingrespectedandlistenedto23,increased
acceptanceintrustandlegitimacyofoutcomes24,
andenablingconvergenceandcompromise25were
alsocommonlycited.
21 See:McWhirteret.al,2014;Felicettiet.al,2015casestudiesaandb;Felicettiet.al,2012;CuratoandOng,2015;andNiemeyeret.al,2013.
22 See:McWhirteret.al,2014;andFelicettiet.al,2015casestudiesaandb.
23 See:McWhirteret.al,2014;Ercanet.al,2018casestudyb;andFelicettiet.al,2015casestudya.
24 See:McWhirteret.al,201;4andErcanet.al,2018casestudiesaandb.
25 See:McWhirteret.al,2014;Felicettiet.al,2015casestudya;andNiemeyeret.al,2013.
Whenconsideringbenefitscitedbyeachcase,
therewasnosignificantlyinfluentialfactors
presentinthedeliberativeengagementcases.
However,whileexpertswereinvolvedinsixofthe
sevencases,onecaseincludednoexpertsandstill
achievedfiveoftheninecitedbenefits26.
Thiscasewasopentodiverseviewpointsfromall
peoplewholivedinaparticularregionwhowere
invitedtodeliberateonissuesdirectlyconcerning
themratherthanapredeterminedtopicoffocus.
Itwasnotclearifeitherdiverseviewpointsor
enablingthefocustorelatespecificallyto
participant’slivedexperienceshadaninfluenceon
theseoutcomes.
26 See:Ercanet.al,2018.
13
DEM
OC
RAC
Y 20
25Furthermore,whilethreeofthefourAustralian
casescitedfivetoeightbenefitswhichwasnot
achievedinanyoftheotherlocations,thesmall
numberofcaseswouldrequirefurther
investigationintothepoliticaland/orsocial
contextsthatcouldhaveimpactedthisresult.
Table2providesanoverviewofpotentially
influentialvariablesbythenumberofbenefits
reportedinthecases.
Table2. Numberofbenefitscitedindeliberativeengagementcaseswheresub-variablewaspresent
1-4 benefits 5-8 benefits Total Cases
PotentiallyInfluentialVariables Number Percent Number Percent Number Percent
Representation
Descriptive 2 50% 2 50% 4 57%
Surrogate 1 100% 0 0% 1 14%
Geographic 1 50% 1 50% 2 29%
Location
Australia 1 25% 3 75% 4 57%
Italy 1 100% 0 0% 1 14%
Philippines 1 100% 0 0% 1 14%
UnitedStatesofAmerica 1 100% 0 0% 1 14%
Note: Benefits percentages refer to the proportion of cases where sub-variable is present (differs by variable). Total cases percentages refer to the proportion of all cases (7).
Key finding one: representation and non-representation related variables matter
Thecasestudyreviewidentifiedarangeof
variablesthatpotentiallyinfluenceoutcomesin
co-designanddeliberativeengagementcases,
severalofwhichrelatetoissuesofrepresentation.
Inparticular;theapproachtotheselectionof
representatives,ensuringdiverseviewpoints,and
involvementofthosewithlivedexperienceswere
identifiedaspotentiallyinfluencingoutcomes.
Therewerehowever,arangeofnon-representation
relatedvariablesthatalsoappearedtoinfluence
outcomes,including;thefocusoftheprocess,the
qualityandtypeofmethodsused,thedurationof
participation,andthepolitical/socialcontext.
Duetoinconsistenciesinreportingacrosscases,
theresultsofthisstageofanalysiswere
inconclusiveontheirown,providingabasisfor
furtherinvestigatingthenormativevalueplaced
oneachofthesevariablesinthetheoretical
literature.
14
DEM
OC
RAC
Y 20
25STAGE TWO: THEORETICAL STUDY REVIEWS
Stagetwosoughttodetermineifanyofthe
variablesidentifiedaspotentiallyinfluencing
outcomesinstageone,werealsocommonlycited
asimportantinthetheoreticalliterature.
Co-design theory
Inthe18co-designstudiesreviewed,fourofthe
variablesidentifiedinstageone,werehighlighted
inmorethan50percentofstudies.Fromthose
variables,fivesub-variableswerealsocitedin
50percentormoreofstudiescitingeachvariable.
Themostcommonlycitedvariableoverallwas
diverseviewpointswhichwascitedin78percent
ofstudies.Ofthosestudies,57percentplaced
particularvalueontheroleofbothserviceusers
andprofessionalsasequalparticipants27and
50percentstressedtheneedtoaddresspower
imbalancestoenableeffectiveco-design28.
Thenextmostcommonlycitedvariablewasthe
co-designmethodwhichwascitedin67percent
ofstudieshowever,itwasthequalityofprocess
designandfacilitationwhichwasmostcommonly
identifiedwithinthesestudiesratherthanspecific
co-designmethods29.
27 See:Binderet.al,2008;SzebekoandTan,2010;EvansandTerrey,2016;Trischleret.al.2018.
28 See:Binderet.al,2008;BuurandLarsen,2010;SzebekoandTan,2010;Holmes,2011;GreenbaumandLoi,2012;Harderet,al,2013;andTrischleret.al,2018.
29 See:BuurandLarsen,2010;SzebekoandTan,2010;Holmes,2011;Harderet.al,2013;Andersenet.al,2015;EvansandTerrey,2016;Trischleret.al,2018.
Inaddition,theco-designmethodwasfollowed
closelybypolitical/socialcontextwhichwascited
in56percentofstudies,withasignificantvalue
placedoncitizenparticipationbeingacceptedasa
democraticvalue30,highlightingtheimportantrole
thatco-designcancontributetothelegitimacyof
publicsectordecisions.
Finally,theimportanceofthefocusofthecase
wasreferredtoin56percentofstudieswhich
significantlyfavouredservicesandproductdesign
morebroadly31ratherthanhumanservices
contexts.
Table3outlinesthevariablescitedbythemajority
ofstudies,alongwiththeirmostcommonlycited
sub-variables.
30 See:Binderet.al,2008;BuurandLarsen,2010;SzebekoandTan,2010;Holmes,2011;GreenbaumandLoi,2012;EvansandTerrey,2016;andHuybrechtset.al,2017.
31 See:Leadbeater,2004;ParkerandHeapy,2006;Binderet.al,2008;SzebekoandTan,2010;GreenbaumandLoi,2012;Harderet.al,2013;Huybrechtset.al,2017;Bason,2018;andTrischleret.al,2018.
15
DEM
OC
RAC
Y 20
25Table3: Variablesandsub-variablesidentifiedmostfrequentlyinco-designtheoreticalstudies
Variable as percentage of total studies Sub-variables as percentage of studies citing related variable
1. DiverseViewpoints(78%) 1. Interdisciplinaryusersandprofessionalsasequalpartnersindesign(57%)
2. Equalisingpowerimbalances(50%)
2. Co-designMethods(67%) 3 Qualityprocessdesignandfacilitation(58%)
3. Political/Social Context(56%)
4. Citizenparticipationasanaccepteddemocraticvalue(70%)
4. Focus(56%) 5. Designofservicesandproducts(90%)
Note: Variable percentages refer to the proportion of all studies citing each variable. Sub-variable percentages refer to the proportion of those studies that also cited the sub-variable.
Deliberative engagement theory
Inthe18deliberativeengagementstudies
reviewed,threeofthevariablesidentifiedinstage
one,werehighlightedinmorethan50percentof
studies.However,onlyonesub-variablewascited
inmorethan50percentofstudiesreferringto
thosevariables.Subsequentlythehighestcited
sub-variableswereidentifiedfromeachofthe
majoritycitedvariables.
Thedeliberationmethodwasthehighestcited
variable,referredtoin83percentofstudies.
Ofthosestudies,thetransmissionofcitizen
generatedrecommendationstoformaldecision-
makingbodieswasthehighestcitedsub-variable,
referredtoin40percentofthosestudies32.
32 See:Bohman,1998;UhrandUhr,1998;Parkinson,2003;DryzekandNiemeyer,2008;Hendricks,2015;andDryzek,2016.
Diverseviewpointswasthesecondmost
commonlycitedvariablewhichwasreferredto
in56percentofstudies.Ofthosestudies,
80percentreferredtopluralityofbeliefs,
convictions,idealsandviewpoints33.
Finally,issuesrelatedtothepolitical/socialcontext
werecitedin50percentofstudies.Ofthose
studies,thehighestcitedsub-variableswerethe
recognitionthatlargeandcomplexsocieties
requireinclusiverepresentationduetothe
impossibilityofinvolvingallaffectedpeople34,
andtheimportanceofensuringparticipantsare
autonomousandcapableofformingaview
withoutexternalconstraints35.
Table4outlinesthevariablescitedbythemajority
ofstudies,alongwiththeirmostcommonlycited
sub-variables.
33 See:Cohen,1997;Bohman,1998;UhrandUhr,1998;SmithandWales,2000;DryzekandNiemeyer,2008;Dryzek,2016;Elstubet,al,2016;andBaiocchiandGanuza,2017.
34 See:Cohen,1997;Michelman,1997;andFlorida,2013.35 See:Cohen,1997;SmithandWales,2000;andFlorida,
2013.
16
DEM
OC
RAC
Y 20
25Table4: Variablesandsub-variablesidentifiedmostfrequentlyindeliberativeengagement theoreticalstudies
Variable as percentage of total studies Sub-variable as percentage of studies citing related variable
1. DeliberationMethods(83%) 1. Transmissionofcitizengeneratedrecommendationstoformal
decision-makingbodies(40%)
2. Diverseviewpoints(56%) 2. Pluralityofbeliefs,convictions,idealsandviewpoints(80%)
3. Political/Socialcontext
(50%)
3. Representationnecessarytocounterimpossibilityofinvolvingall
affectedpeopleinlargeandcomplexsocieties(33%)
4. Autonomyofparticipantstocounteradaptivepreferences(33%)
Note: Variable percentages refer to the proportion of all studies citing each variable. Sub-variable percentages refer to the proportion of those studies that also cited the sub-variable.
Key finding two: quality methods, context and diversity matter
Stagetwoidentifiedthreesharedvariablesthat
thestudiessuggestinfluenceoutcomesinboth
co-designanddeliberativeengagements:
1. qualitymethods
2. political/socialcontext,and
3. diversityofviewpoints.
Inaddition,eachvariablehasanumberof
sub-variablesthatappeartoincreasethechances
ofpositiveoutcomes,someofwhichrelateto
representation,andsomethatdon’t(as
demonstratedinBox2).
Whilethefocusofthecasewashighlycitedin
co-designtheory,thiswasnotthecasewith
deliberativeengagement.Hencethisvariablehas
beenexcludedforthepurposesofidentifying
sharedvariablesofinterest.
Box 2: Variables and sub-variables influencing positive outcomes in co-design and deliberative engagement
RepresentationRelatedVariables Non-RepresentationRelatedVariables
Quality Methods Political/Social Context Diversity of Viewpoints
Qualityofprocessdesign andfacilitation
Inclusive representation
Involvementofaffectedpeopleandprofessionals
Transmissionofcitizenengagementoutcomestoformaldecision-making
authorities
Autonomyofrepresentatives Equalityofparticipants
Citizenparticipationasdemocraticvalue
Pluralityofbeliefs,convictions,idealsandviewpoints
17
DEM
OC
RAC
Y 20
25STAGE THREE: VALIDATION
Validation involved identifying direct and indirect
references to each of the identified sub-variables,
and measuring their achievement in 20 additional
case studies, once more using secondary sources.
Relevance
The validation process showed that with the
exception of autonomy of representatives,
all of the identified sub-variables were referenced
in the majority of co-design and deliberative
engagement cases, suggesting a high level
of relevance.
Autonomy of representatives was also identified
in 70 per cent of the deliberative engagements,
suggesting that this variable may warrant further
investigation in co-design as well (see Graph 2).
In addition to the representation related
sub-variables, all three non-representation related
sub-variables were identified in the majority of
co-design and deliberative engagements,
suggesting high levels of relevance (see Graph 3).
Graph 2: Proportion of cases citing representation related sub-variables by co-design or deliberative engagement
18
DEM
OC
RAC
Y 20
25
Graph3:Proportionofcasescitingnon-representationrelatedsub-variablesbyco-designor deliberativeengagement
0
20
40
60
80
100
Deliberative engagementsCo-design
Transmission of citizen generated recommendations to formal
decision-making bodies
Quality process design and facilitation
Citizen engagement as an accepted democratic value
Measuring sub-variables in co-design cases
Simpledescriptorsbasedonthetheoretical
reviewwereusedtoassesstheachievement
ofeachofthesub-variables.Inaddition,each
casewasassessedaseitherachieving
consequentialoutcomesornot,whichwas
definedastheacceptanceofcitizengenerated
recommendationsbyformaldecision-making
bodies.
Eightofthe10co-designcaseswereassessed
asachievingqualityprocessdesignand
facilitation36whichwasnotsurprisinggiventhat
studiesweredrawnfromorganisationswith
significantexpertiseindesigningandfacilitating
co-designprocesses.
36 See:Hampsonet.al,2013;DesignCouncil,2016;Woodroffe,2018;ThinkPlace,2017(a);Thinkplace,2017(b);AucklandDesignLab,2015;AucklandDesignLab,2017;andIDEO,2015.
Whatwasmoresurprisingwasthattwoofthe
eightstudiesthatachievedqualityprocessdesign
andfacilitationdidnotachieveconsequential
outcomesduringthetimeframereported37.
Theothertwonon-representationrelatedsub-
variableswerealsoachievedinthemajorityof
cases,includingsevencasesthatachievedthe
transmissionofparticipantrecommendations
directlytoformaldecision-makingbodies38,and
sixcasesachievingcitizenparticipationasan
accepteddemocraticvalue39.
Intermsofrepresentationrelatedsub-variables,
theinvolvementofaffectedpeopleand
professionalswasachievedinthemajorityof
37 See:AucklandDesignLab,2015;andAucklandDesignLab,2017.
38 See:Hampsonet.al,2013;DesignCouncil,2016;Woodroffe,2018;ThinkPlace,2017(a);Thinkplace,2017(b);Budds,2016;andIDEO,2015.
39 See:DesignCouncil,2016;Woodroffe,2018;ACTGovernment,2018;ThinkPlace,2017(a)lThinkplace,2017(b)landBudds,2016.
19
DEM
OC
RAC
Y 20
25cases40,withpluralityofviewpointsachievedin
fourcases41,andequalityofparticipantsachieved
inthreecases42.
Inclusiverepresentationhoweverwasonly
achievedintwocases43andnocasesreferredto
theachievementofautonomyamong
representatives,makingthissub-variablenot
applicabletotheanalysis.
Table5outlinesthenumberofcasesassessed
asachievingeachsub-variableandtheproportion
ofthosecasesthatachievedconsequential
outcomes.
40 See:Hampsonet.al,2013;Woodroffe,2018;ThinkPlace,2017(a);Thinkplace,2017(b);AucklandDesignLab,2017;andIDEO,2015.
41 See:Hampson,et.al2013;DesignCouncil,2016;Woodroffe,2018;andThinkPlace,2017(a).
42 See:Hampson,et.al,2013;DesignCouncil,2016;andAucklandDesignLab,2017.
43 See:Hampsonet.al,2013;andBudds,2016.
Table5:Achievementofsub-variablesandconsequentialityinco-designcases
Sub-variables Consequentiality
Number Per cent Number Per cent
Political/Social Context
Citizenparticipationasdemocraticvalue 6 75% 6 100%
Inclusiverepresentation 2 40% 2 100%
Autonomyofrepresentatives NA* NA NA NA
Diversity of Viewpoints
Involvementofaffectedpeopleandprofessionals 6 60% 5 84%
Equalityofparticipants 3 60% 2 67%
Pluralityofbeliefs,convictions,idealsandviewpoints 4 68% 4 100%
Quality Approach
Qualityofprocessdesignandfacilitation 8 89% 5 63%
Transmissionofcitizenengagementoutcomestoformaldecision-makingauthorities 7 88% 7 100%
Note: Sub-variable percentages refer to the proportion of total valid cases (those referring to the sub-variable) achieving the sub-variable.
*No co-design cases referred to autonomy of participants, making this sub-variable not applicable.
Consequentiality percentages refer to the proportion of cases achieving the sub-variable that also achieved consequentiality, defined as acceptance of recommendations by decision making bodies.
Measuring sub-variables in deliberative engagements
Thesamemeasurementprocesswasfollowedto
assessvariableachievementandconsequentiality
inthedeliberativeengagementcases.
Ofthe10casesreviewed,qualityprocessdesign
andfacilitation44,andthetransmissionofcitizen
engagementoutcomestoformaldecision-making
bodieswerebothachievedinfivecaseseach45.
Inaddition,citizenparticipationasanaccepted
democraticvaluewasachievedinfourcases.46
44 See:Um,2013;Hargrave,2018;Weymouth,2016;Parry,2016;andHosmer,2017.
45 See:Fung,2009andFletcher,2018;Faubion,2012;Weymouth,2016;Hosmer,2017;andParry,2016.
46 See:Fung,2009andFletcher,2018;Faubion,2012;Weymouth,2016;andParry,2016.
20
DEM
OC
RAC
Y 20
25Intermsofrepresentationrelatedsub-variables,
inclusiverepresentation47andpluralityof
viewpoints48wereachievedinfivecaseseach.
Theinvolvementofaffectedpeopleand
professionals49andequalityofallparticipantswere
achievedinthreecaseseach50.Inaddition,
autonomyofrepresentativeswasachievedin
twocases51.
47 See:Um,2013;Hargrave,2018;Faubion,2012;Weymouth,2016;andParry2016.
48 See:Um,2013;Hargrave,2018;Weymouth,2016;Parry,2016;andShen,2012andFletcher,2016.
49 See:Fung,2009andFletcher,2018;Hargrave,2018;andParry,2016.
50 See:Fung,2009andFletcher,2018;Hargrave,2018;andWeymouth,2016.
51 See:Um,2013;andShen,2012andFletcher,2016.
Table6outlinesthenumberofcasestudies
assessedasachievingeachsub-variableandthe
proportionofthosecasesthatachieved
consequentialoutcomes.
Table6:Achievementofsub-variablesandconsequentialityindeliberativeengagementcases
Sub-variables Consequentiality
Number Per cent Number Per cent
Political/Social Context
Citizenparticipationasdemocraticvalue 4 40% 2 50%
Inclusiverepresentation 5 50% 2 40%
Autonomyofrepresentatives 2 29% 1 50%
Diversity of Viewpoints
Involvementofaffectedpeopleandprofessionals 3 30% 0 0%
Equalityofparticipants 3 50% 1 33%
Pluralityofbeliefs,convictions,idealsandviewpoints 5 71% 2 40%
Approach
Qualityofprocessdesignandfacilitation 5 56% 2 40%
Transmissionofcitizenengagementoutcomesto
formaldecision-makingauthorities5 50% 2 40%
Note: Sub-variable percentages refer to the proportion of total valid cases (those referring to the sub-variable).
Consequentiality percentages refer to the proportion of cases achieving the sub-variable that also achieved consequentiality, defined as acceptance of recommendations by decision making authorities.
21
DEM
OC
RAC
Y 20
25Validation Outcomes
Thevalidationofsub-variablerelevanceand
measurabilityhighlightedanumberoffindingsof
criticalimportancetothedevelopmentofthe
conceptualmodelforassessingthequalityand
impactsofcitizenengagementprocesses.
Inparticular,thevalidationprocessshowedthat
whilethesub-variablescouldbemeasured,
degreesofachievementwouldallowformore
accurateassessment.Forexample,affected
peopleandprofessionalsmaybeinvolved
throughouttheparticipationprocess,atparticular
pointsintheprocess,ornotatall.
Havingameasurementscalecapableof
accountingfordegreesofachievementwould
enhancethereliabilityofaconceptualmodelfor
assessingqualityandimpacts.
Inaddition,whilesecondarysourceswereusedto
testthesub-variables,theuseofprimarydatato
applytheconceptualmodelwouldenable
consistencyofdatacollectionandincreasethe
accuracyoftheresults.
Key finding three: interactions matter
Furthertothevalidationoutcomesalready
discussed,allvariableswerepresentbothin
consequentialandnon-consequentialcases,
suggestingthatwhiletheymayormaynotbe
necessary,individualvariablesareunlikelytobe
sufficientinandofthemselves.
Thisfindingsupportsasettheoryapproachto
analysingboththeimpactsandinteractions
betweenrepresentationandnon-representation
relatedvariablesincitizenengagementprocesses.
22
DEM
OC
RAC
Y 20
25CONCEPTUAL MODEL FOR ASSESSING THE QUALITY OF ENGAGEMENT WITH CITIZENS
Theaimoftheconceptualmodelistoprovidea
toolforpublicsectororganisationstoevaluatethe
qualityofengagementwithcitizenstoimprove
boththeoryandpracticeinthisspace.The
conceptualmodelalsosupportsstrengthened
accountabilityofpublicsectororganisationsby
ensuringcitizenengagementadequately
representsthecommunitiesofpeoplemost
affectedbypublicsectordecisions.
Theconceptualmodelincludesthree
representationrelatedvariables,whichhavebeen
developedbymergingthefollowingsub-variables:
inclusiverepresentationandinvolvementof
affectedpeopleandprofessionals,andautonomy
ofrepresentativesandequalityofallparticipants,
inordertoreducecomplexityandenablemore
holisticvariabledescriptors;andbyreframing
pluralityofviewpoints,beliefs,idealsand
convictionstoalsorecognisethatapluralityof
engagementmethodsarerequiredtoensure
diverseviewpointscanparticipateincitizen
engagementprocesses.
Representationrelatedvariablesthereforeinclude:
1. Inclusiverepresentationofaffectedpeople
andprofessionals;
2. Pluralityofviewpointsandengagement
methods;and
3. Autonomyandequalityofparticipants.
Theconceptualmodelalsoincludesthree
non-representationrelatedvariables:
1. Citizenengagementasanaccepted
democraticvalue;
2. Qualityprocessdesignandfacilitation;and
3. Transmissionofcitizengenerated
recommendationstoformaldecision-making
bodies.
Inadditiontotherepresentationandnon-
representationrelatedvariables,theconceptual
modelincludesthreesuggestedoutcomerelated
measurestoassesstheimpactofvariables
achievedthroughthecitizenengagementprocess.
Outcomesmeasuresinclude:
1. Citizensagreeingonthesolutionor
recommendationsarisingthroughthecitizen
engagementprocess;
2. Citizenstrustinginthelegitimacyofthe
citizenengagementprocesstoinfluence
decision-making;and
3. Consequentialitydefinedasdecision-making
bodiesacceptingcitizengenerated
recommendations.
Theseoutcomesmeasuresspeaktotheoverall
legitimacyoftheprocessinengagingcitizensin
decisionmakingandcanbesubstitutedforcase
specificoutcomeswheredeemedsuitable.The
measurementscalesforeachofthevariablesand
outcomesdiscussedaboveareoutlinedatBox3.
23
DEM
OC
RAC
Y 20
25Box 3: Variable and Outcomes Measurement Scale
Variables Not achieved Partially Achieved Fully Achieved
Representation Related Variables
Inclusiverepresentation(affectedpeopleandprofessionals)
Nostrategiesinplacetoensurerepresentativesareinclusivedemonstratedbylackofrepresentationofaffectedpeopleandrelevantprofessionalsatanystageinthedesignanddeliberationprocess
Somestrategiesinplacetoensurerepresentativesareinclusivedemonstratedbyrepresentationofaffectedpeopleandrelevantprofessionalsatvariousstagesthroughoutthedesignanddeliberationprocess
Effectivestrategiesinplacetoensurerepresentativesareinclusivedemonstratedbyrepresentationofaffectedpeopleandrelevantprofessionalsthroughoutallstagesofthedesignanddeliberationprocess
Autonomyandequalityofallparticipants
Noevidenceofpowersharingandfreedeliberationamongparticipantsdemonstratedthroughlowlevelsofcontributionbyallparticipantsandlowpreferencetransformation
Someevidenceofpowersharingandfreedeliberationamongparticipantsdemonstratedthroughmediumlevelsofcontributionbyallparticipantsandmediumpreferencetransformation
Evidenceofpowersharingandfreedeliberationamongparticipantsdemonstratedthroughhighlevelsofcontributionbyallparticipantsandhighpreferencetransformation
Pluralityofviewpoints(andengagementmethods)
Nostrategiesinplacetoengagediverseviewpointsdemonstratedthroughlowlevelsofdiversityinoriginalpositions
Somestrategiesinplacetoengagediverseviewpointsdemonstratedthroughmediumlevelsofdiversityinoriginalpositions
Effectivestrategiesinplacetoengagediverseviewpointsdemonstratedthroughhighlevelsofdiversityinoriginalpositions
Non-Representation Related Variables
Qualityofprocessdesignandfacilitation
Noexpertfacilitationandlackofmethodstosupporteffectiveengagementdemonstratedthroughlowparticipantsatisfactionwiththeprocess
Somefacilitationexpertiseandmethodstosupporteffectiveengagementdemonstratedthroughmediumparticipantsatisfactionwiththeprocess
Expertfacilitationandmethodstosupporteffectiveengagementdemonstratedthroughhighparticipantsatisfactionwiththeprocess
Transmissionofcitizenengagementoutcomestoformaldecision-makingbodies
Notransferofcitizengeneratedinputsorrecommendationstorelevantpoliticalactorsorpubicserviceorganisations
Indirecttransferofcitizengeneratedinput(viastakeholderdevelopedrecommendations)torelevantpoliticalactorsorpublicserviceorganisations
Directtransferofcitizengeneratedrecommendationstorelevantpoliticalactorsorpublicserviceorganisations
24
DEM
OC
RAC
Y 20
25Variables Not achieved Partially Achieved Fully Achieved
Citizenparticipationasanaccepteddemocraticvalue
Nopolitical/publicsectorsupportforcitizenparticipationintheengagementprocessdemonstratedbylackofauthorisationbypoliticalactorsorpublicserviceorganisations
Somepolitical/publicsectorsupportforcitizenparticipationintheengagementprocessdemonstratedthroughupfrontauthorisationbypoliticalactorsorpublicserviceorganisations
Fullpolitical/publicsectorsupportforcitizenparticipationintheengagementprocessdemonstratedthroughupfrontauthorisationbypoliticalactorsorpublicserviceorganisationsandin-principlecommitmenttotheacceptanceofcitizengeneratedrecommendations
Outcomes Measures
AgreedSolutionorRecommendations
Noevidenceofagreementdemonstratedbylowlevelsofparticipantsupportforthefinalsolutionorrecommendations.
Someevidenceofagreementdemonstratedbymediumlevelsofparticipantsupportforthefinalsolutionorrecommendations.
Evidenceofagreementdemonstratedbyhighlevelsofparticipantsupportforthefinalsolutionorrecommendations.
Legitimacy Noevidenceoflegitimacydemonstratedthroughlowlevelsoftrustheldbyparticipantsinthepoweroftheengagementprocesstoinfluencethedecisionsmadebygovernment.
Someevidenceoflegitimacydemonstratedthroughmediumlevelsoftrustheldbyparticipantsinthepoweroftheengagementprocesstoinfluencethedecisionsmadebygovernment.
Evidenceoflegitimacydemonstratedthroughhighlevelsoftrustheldbyparticipantsinthepoweroftheengagementprocesstoinfluencethedecisionsmadebygovernment.
Consequentiality Noevidenceofconsequentialitydemonstratedbynoorlimitedacceptanceofparticipantrecommendationsbydecisionmakingauthorities.
Someevidenceofconsequentialitydemonstratedbypartialacceptanceofparticipantrecommendationsbydecisionmakingauthorities.
Evidenceofconsequentialitydemonstratedbyfullacceptanceofparticipantrecommendationsbydecisionmakingauthorities.
25
DEM
OC
RAC
Y 20
25Theconceptualmodelalsorecommendsarange
ofmethodologiesformeasuringtheidentified
variablesandoutcomes,including:
1. Pre-engagement surveysinlivecasesto
determineparticipantcharacteristicsandthe
diversityofviewpointsrepresentedineach
process(note:thesemeasurescanbe
collectedthroughpost-engagementsurveys
forcompletedcases);
2. Post-engagement surveystoassess
participantexperiences,perceptionsand
agreementontherecommendedsolutions;
3. Targeted interviewswithasmallnumberof
participantsorkeystakeholdersinorderto
unpacksurveyfindingsusingreflective
questioning;
4. Interviews with process organiserstoassess
levelsofinvolvementandequalcontributionby
participants,andtoassesstheauthorisation,
transferofrecommendations,andacceptance
ofrecommendationsbydecisionmaking-
bodies;and
5. Observationsinlivecasestoassesstheequal
contributionofparticipants,qualityofprocess
designandfacilitation,andapparent
agreementwithrecommendedsolutions.
Figure2providesavisualrepresentationofthe
variablesandtheirrecommendedassessment
methods.
26
DEMOCRACY 2025
Figure2:Conceptualmodelforassessingthequalityofengagementswithcitizens
VARIABLES MEASURES ASSESSMENT METHODSInclusiveRepresentation
(affectedpeopleandprofessionals)
QualityofProcessDesign andFacilitation
AutonomyandEquality ofallParticipants
TransmissionofCitizenGeneratedRecommendations
PluralityofViewpoints (andengagementmethods)
CitizenParticipationasa DemocraticValue
Repr
esen
tati
on R
elat
ed
Varia
bles
Non
-Rep
rese
ntat
ion
Rela
ted
Va
riabl
esN
on-R
epre
sent
atio
n Re
late
d
Varia
bles
InvolvementThroughout EngagementProcess
PreferenceTransformation andEqualContribution
DiversityofOriginalPositions
ParticipationSatisfaction
TransferofRecommendations
Authorisation
ParticipantSupportforFinal SolutionorRecommendations
ParticipantPerceptionsofTrust
AcceptanceofRecommendations
Post Engagement Survey (+ pre-engagement survey questions for completed cases) and Observations (live cases)
Pre-engagement survey (Live cases only- collect through post engagement survey
for completed cases)
PreferenceTransformation (comparedtooriginalposition)
ParticipantSatisfaction,includingequalcontribution (Process)
ParticipantPerceptionsofTrust (Legitimacy)
ParticipantSupportforRecommendedSolutions (Agreement)
ParticipantCharacteristics OriginalPositionsofParticipants
Targeted Participants Interviews
Organiser Interviews
ReflectiveQuestioningtoUnpackSurveyFindings
IdentificationofAffectedPeoplesandProfessionals (Characteristics)andtheirinvolvementthroughout
theengagementprocess
AuthorisationofProcess,transferofrecommendationsandacceptanceofrecommendationsbydecisionmakingBodies
AgreedSolutionsorRecommendations
Legitimacy
Consequentiality
27
DEM
OC
RAC
Y 20
25CONCLUSION
Co-designanddeliberativeengagementboth
haveimportantandcomplementaryrolestoplay
inmoderndemocracies,andinparticularinthe
waypublicserviceagenciesengagecitizensin
designinganddeliberatingonissuesofcommon
good.Inaddition,theybothprovideimportant
insightstoguidethesuccessfulengagementof
citizensinpublicpolicyandservicedesign.
Thissystematicreviewlookedspecificallyatthe
variablesthatinfluenceco-designanddeliberative
engagementpracticetodeterminewhatrole
representationplaysinenhancingthelegitimacy
ofpublicdecisions.
Thesystematicreviewidentifiedthreekeyfindings
thatcollectivelyinformedthedevelopmentofthe
conceptualmodelforassessingthequalityof
engagementwithcitizensoutlinedinthisreport.
1. RepresentationandNon-Representation
RelatedVariablesMatter:
Representationrelatedvariables,suchas
inclusivity,autonomy,equalityandplurality,
influenceoutcomesinbothco-designand
deliberativeengagements;however,sotoodo
non-representationrelatedvariables,suchas
processquality,politicalauthorisation,and
connectionsbetweencitizenengagements
andformaldecision-makingbodies.
2. QualityMethods,ContextandDiversityMatter:
Thepolitical/socialcontextanddiversityof
peopleaffectedbythetopicoffocus,will
influencetheselectionofcitizenengagement
methods.Qualityprocessdesignand
facilitationplaysamoresignificantrolein
accommodatingdiversecontextsand
participants,thananyparticulardesignor
deliberationmethod.
3. InteractionsMatter:
Noonevariableislikelytobesufficientinandof
itself.Measuringboththeachievementand
interactionsbetweenvariablesislikelyto
enhanceunderstandingofthedynamicnature
ofcitizenengagementinanever-changing
socialcontext.
Aspublicserviceorganisationsincreasinglystrive
toenhancepublictrustandthelegitimacyof
decisionsmade,effectivecitizenengagementthat
adequatelyrepresentsthosemostaffectedby
thosedecisionsiscrucial.Itisnotsufficientthat
publicsectororganisationsseektheviewsof
thosemostvocalintheircommunities.
Representationmustbeinclusive,equal,and
diverse.Representativesmustbeautonomous
andsupportedbyqualityprocessesthatallow
themtobeactivecontributors.Andboth
governmentsandpublicsectororganisations
mustvaluetheinputofcitizensasdemocratic
agents,committingtotakingseriouslytheirviews
andrecommendations.
Theconceptualmodeloutlinedinthisreport,
providesaframeworkforpublicsector
organisationstomeasurethequalityoftheirown
citizenengagementprocessesinordertoidentify
areasforimprovement.
Themodelalsoprovidesaframeworkforholding
politicalsystemstoaccountoverthelegitimacyof
publicdecisionsthroughthegenuineandeffective
involvementofcitizensindecisionmaking
processes.
28
DEM
OC
RAC
Y 20
25REFERENCES
ACTGOVERNMENT2018.Family Safety Hub: Co-creating pathways to sustainable safety [Online].Available:http://www/communityservices.act.gov.au/_data/assets/pdf_fiile/0006/1198824?Family-Safety-Hub-Design-Report.pdf[accessed21September2018].
ALFRED,J.2009.Engaging Public Sector Clients: From Service-Delivery to Co-Production,UnitedKingdom,PalgraveMacmillan.
ANDERSON,L.B.,DANHOLT,P.,HALSKOV,K.,BRODERSENHANSON,N.&LAURITSEN,P.2015.ParticipationasaMatterofConcerninParticipatoryDesign.Co-design,11(3-4),pp250-261.
AUCKLANDDESIGNLAB2015.Exploring Security of Tenure through Co-design[Online]. Available:https://staticsquare.com/static/55ac5ee5e4b08d4c25220f4b/t/5b7602f82b6a281de0bad415/1534460707769/Rental+Tenure+-+Talkbook+15.9.2015.pdf[accessed21September2018].
AUCKLANDDESIGNLAB2017.Early Years Challenge: Supporting Parents to Give Tamariki a Great Start in Life[Online].Available:https://static1.squarespace.com/static/55ac5ee5e4b08d4c25220f4b/t/5b64ef6103ce647540da30c1/1533341548279/Early+Years+Challenge+report+FINAL+%28web%29.pdf[accessed21September2018].
BAIOCCHI,G.&GANUZA,E.2017. Popular Democracy: The Paradox of Participation,UnitedStatesofAmerica,StanfordUniversityPress.
BASON,C.2010.Leading Public Sector Innovation: Co-creating for a Better Society,UnitedKingdom,PolicyPress.
BINDER,T.,BRANDT,E.&GREGORY,J.2008.DesignParticipation(-s)-ACreativeCommonsforOngoingChange.CoDesign,4(2),pp79-83.
BOHMAN,J.1998.SurveyArticle:TheComingofAgeofDeliberativeDemocracy.The Journal of Political Philosophy,6(4),pp400-425.
BOWEN,S.,MCSEVENY,K.,LOCKLEY,E.,WOLSTENHOLM,D.,COBB,M.&DEARDEN,A.2013.Howwasitforyou?ExperiencesofparticipatorydesignintheUKhealthservice.CoDesign,9(4),pp230-246.
BRANDT,E.,BINDER,T.,MALMBORG,L.&SOKOLER,T.2010.CommunitiesofEverydayPracticeandSituatedElderlinessasanApproachtoCo-designforSeniorInteraction.22nd Conference of the Computer-Human Interaction Special Interest Group of Australia on Computer-Human Interaction.Australia.
BUCHANAN,R.1992.WickedProblemsinDesignThinking.Design Issues,8(2),pp5-21.
BUDDS,D.2016.How One Florida City is Reinventing Itself with UX Design[Online].Available: https://www.fastcompany.com/3065107/how-one-florida-city-is-reinventing-itself-with-ux-design [accessed21September2018].
BURRAL,S.2015.Room for a View: Democracy as a Deliberative System.UnitedInvolve,Kingdom,pp1-68.
BUUR,J.,LARSEN,H.2010.TheQualityofConversationsinParticipatoryInnovations,Codesign, 6(3),pp121-138.
COHEN,J.1997.DeliberationandDemocraticLegitimacy.In:BOHMAN,J.®H,W.(eds.)Essays on Reason and Politics.UnitedStatesofAmerica:TheMITPress.
COLE,B.,PINFOLD,J.,HO,H.&ANDA,M.,2013.ExaminingthemethodologyofparticipatorydesigntocreateinnovativesanitationtechnologiesinruralMalawi.Delivering water, sanitation and hygiene services in an uncertain environment, 36th WEDC International Conference.Kenya.
COREY,Z.2009. Deliberative Democracy and the Emerging Jury System in Japan: A Natural Combination or Two Ships Passing in the Night. CornellUniversity.
29
DEM
OC
RAC
Y 20
25CURATO,N.,DRYZEK,J.S.,ERCAN,S.A.,HENDRIKS,C.M.&NIEMEYER,S.2017.TwelveKeyFindingsinDeliberativeDemocracyResearch.Daedalus- the Journal of the American Academy of Arts and Sciences,146(3),pp28-38.
CURATO,N.&ONG,J.C.2015.InclusionasDeliberativeAgency:TheSelectiveRepresentationofPoorWomeninDebatesandDocumentariesaboutReproductiveHealth.Television and New Media,16(6),pp576-594.
DESIGNCOUNCIL2016.Adur and Worthing: Using co-design to create a lasting legacy [Online].Available:https://www.designcouncil.org.uk/resources/case-study/adur-worthing-using-co-design-create-lasting-legacy[accessed21September2018].
DORST,K.2011.TheCoreof‘designthinking’anditsapplication.Design Studies,32(6),pp521-532.
DRYZEK,J.2016.DeliberativePolicyAnalysis.In:STOKER,G.&EVANS,M.(eds.)Evidence-Based Policymaking in the Social Sciences: Methods that Matter. UnitedKingdom,PolicyPress,pp286-301.
DRYZEK,J.&NIEMEYER,S.2008.DiscursiveRepresentation.American Political Science Review,102(4),pp481-493.
DRYZEK,J.S.&(WITH)NIEMEYER,S.J.2010.Foundations and Frontiers of Deliberative Governance, UnitedKingdom,OxfordUniversityPress.
ELSTUB,S.,ERCAN,S.&MENDONCA,R.2016.TheFourthGenerationofDeliberativeDemocracy. Critical Policy Studies, 10(2),pp139-151.
ELSTUB,S.&MCLAVERTY,P.2014.Deliberative Democracy: Issues and Cases, UnitedKingdom,EdinburghUniversityPress.
ENSERINK,B.&MONNIKOF,R.A.2003.InformationManagementforPublicParticipationinCo-design:EvaluationofaDutchExample.Journal of Environmental Planning and Management, 46(3),pp315-344.
ERCAN,S.A.,HENDRIKS,C.M.&DRYZEK,J.S.2018.PublicDeliberationinanEraofCommunicativePlenty.CrawfordSchool,AustralianNationalUniversity(ForthcominginPolicy and Politics).
EVANS,M.&TERREY,N.2016.Co-designwithCitizensandStakeholders.In:STOKER,G.&EVANS,M.(eds.)Evidence-based Policy Making in the Social Sciences: Methods that Matter. UnitedKingdom,PolicyPress.
FAUBION,S.2012.City of Canada Bay Council Citizens’ Panel. [Online].Available:https://participedia.net/en/cases/city-canada-bay-council-citizens-panel-0[accessed11June2018]
FELICETTI,A.,GASTIL,J.,HARTZ-KARP,J.&CARSON,L.2012.CollectiveIdentifyandVoiceattheAustralianCitizens’Parliament.Journal of Public Deliberation,8(1),pp1-27.
FELICETTI,A.,NIEMEYER,S.&CURATO,N.2015.ImprovingDeliberativeParticipation:ConnectingMini-PublicstoDeliberativeSystems.European Political Science Review,8(3),pp427-448.
FLORIDA,A.2013.ParticipatoryDemocracyversusDeliberativeDemocracy:ElementsforaPossibleTheoreticalGeneology,TwoHistories,SomeIntersections.7th ECPR General Conference Sciences Po.France.
FUNG,A.2009&FLETCHER,S.2018.2005BC Citizens’ Assembly on Electoral Reform[Online].Available:https://participedia.net/en/cases/2005-bc-citizens-assembly-electoral-reform[accessed 11June2018].
GREENBAUM,J.&LOI,D.2012.Participation,thecamelandtheelephantofdesign:anintroduction.Codesign,8(2-3),pp81-85.
HABIBIS,D.&WALTER,M.2015.Social Inequality in Australia: Discourses, Realities and Futures,Australia,OxfordUniversityPress.
30
DEM
OC
RAC
Y 20
25HAMPSON,M.,BAECK,P.&LANGFORD,K.2013.By Us, For Us: The Power of Co-design and Co-delivery [Online].Available:https://media.nesta.org.uk/documents/the_power_of_co-design_and_co-delivery.pdf[accessed21September2018]
HARDER,M.K.,BURFORD,G.&HOOVER,E.2013.WhatisParticipation?DesignLeadstoWaytoaCrossDisciplinaryFramework.Design Issues,29(4),pp41-57.
HARGRAVE,L.2018.Citizens Assembly on Brexit[Online].Available:https://participedia.et/en/cases/citizens-assembly-brexit[accessed11June2018].
HENDRIKS,C.M.2015.CouplingCitizensandElitesinDeliberativeSystems:theroleofinstitutionaldesign. European Journal of Political Research,55(1),pp43-60.
HOLMES,B.2011.Citizens’ engagement in policymaking and the design of public services. DepartmentofParliamentaryServices,Australia.
HOSMER,C.2017.Cormorant Island Economic Development Plan (Canada)[Online].Available: https://participedia.net/en.cases.cormorant-island-economic-development-plan-canada[accessed11June2018].
HUYBRECHTS,L.,BENESCH,H.&GEIB,J.2017.Co-designandthePublicRealm.CoDesign,13(3),pp145-147.
IDEO2015.Designing a school system from the ground up[Online].Available:https://www.ideo.com/case-study/designing-a-school-system-from-the-ground-up[accessed21September2018].
JENNSTÁL,J.2018.DeliberativeParticipationandPersonality:Theeffectoftraits,situationsandmotivation.European Political Science Review, pp1-24.
LEADBEATER,C.2004.Personalisation through Participation: a new script for public services, UnitedKingdom,Demos.
LEE,Y.2007.DesignParticipationTactics:thechallengesandnewrolesfordesignersintheco-designprocess.CoDesign,4(1),pp31-50.
LOWNDES,V.2016.NarrativeandStorytelling.In:STOKER,G.&EVANS,E.(eds.)Evidence-based Policymaking in the Social Sciences: Methods that Matter. UnitedKingdom,PolicyPress,pp138-161.
MCWHIRTER,R.E.,CRITCHLEY,C.R.,NICOLE,D.,CHALMERS,D.,WHITTON,T.,OTLOWSKI,M.,BURGESS,M.&DICKINSON,J.L.2014.CommunityEngagementforBigEpidemiology:DeliberativeDemocracyasaTool.Journal of Personalised Medicine,4,pp459-474.
MICHELMAN,F.I.1997.HowCanPeopleEverMaketheLaws:Acritiqueofdeliberativedemocracy.In:BOHMAN,J.®H,W.(eds.)Essays on Reason and Politics.UnitedStatesofAmerica,TheMITPress,pp145-171.
NABATCHI,T.&LEIGHNINGER,M.2015. Public Participation for 21st Century Democracy,UnitedStatesofAmerica,JohnWileyandSonsInc.
NIEMEYER,S.,ERCAN,S.A.&HARTZ-KARP,J.2013.UnderstandingDeliberativeCitizens:TheApplicationofQMethodologytoDeliberationonPolicyIssues.Operant Subjectivity: The International Journal on Q Methodology,36(2),pp114-134.
PARKER,S.&HEAPY,J.2006.The Journey to the Interface: How Public Service Design Can Connect Users to Reform,UnitedKingdom,Demos.
PARKINSON,J.2003.LegitimacyProblemsinDeliberativeDemocracy.Political Studies, 51,pp180-196.
PARRY,L.J.2016.DogsonInverlochBeachCommunityEngagement[Online].Available: https://participedia.net/en/cases/dogs-inverloch-beach-commuity-engagement[accessed 11June2018].
31
DEM
OC
RAC
Y 20
25PENUEL,W.R.,ROSCHELLE,J.&SHECHTMAN,N.2007.DesigningFormativeAssessmentSoftwarewithTeachers:AnAnalysisoftheCo-designProcess.Research and Practice in Technology Enhanced Learning,2(1),pp51-74.
RYAN,M.&SMITH,G.2014.DefiningMini-Publics.In:GRONLUND,K.,BACHTIGER,A.&SETALA,M.(eds.)Deliberative Mini-Publics.UnitedKingdom,ECPRPress.
SAAD-SULONEN,J.,ERIKSSON,E.,HALSKOV,K.,KARASTI,H.&VINES,J.2018.Unfoldingparticipationovertime:temporallensesinparticipatorydesign.CoDesign,1(14),pp4-16.
SAWER,M.,ABJORENSEN,N.&LARKIN,P.2009.Australia: The State of Democracy,Australia,TheFederationPress.
SHEN,S.2012andFLETCHER,S.2016.October 15 Movement[Online].https://participedia.net/en/cases/october-15-movement[accessed21October2018].
SMITH,G.&WALES,C.2000.Citizens’JuriesandDeliberativeDemocracy.Political Studies,48(1), pp51-65.
SZEBEKO,D.&TAN,L.2010.Co-designingforSociety.Australasian Medical Journal,3(9),pp580-590.
THINKPLACE2017a.Improving Literacy Outcomes for Children. United States of America[Online]. Available:https://www/thinkplaceglobal.com/work/improving-literacy-outcomes-children-1 [accessed21September2018].
THINKPLACE2017b.Those Most Affected Have a Say in Improving the National Disability Insurance Scheme [Online].Available:https://www.thinkplaceglobal.com/work/those-most-affected-have-say-improving-national-disability-insurance-scheme[accessed21September2018].
TRISCHLER,J.,PERVAN,S.J.,KELLY,S.J.&SCOTT,D.R.2018.TheValueofCo-design:TheEffectofCustomerInvolvementinServiceDesignTeams.Journal of Service Research,21(1),pp75-100.
UHR,J.&UHR,G.1998.Deliberative Democracy in Australia: The Changing Place of Parliament, UnitedKingdom,CambridgeUniversityPress.
UM,K.2013.Canada’s World Dialogue Series[Online].Available:https://participedia.net/en/cases/canadas-world-dialogue-series[accessed11June2018].
WEISS,H.A.2013.How Students Display Dialogue, Deliberation and Civic Mindedness: An Analysis of Democracy Plaza.UnitedStatesofAmerica,IndianaUniversity.
WEYMOUTH,R.2016.City of Greater Geraldton Deliberative Participatory Budget[Online].Available:https://participedia.net/en.cases/city-greater-geraldton-deliberative-participatory-budget [accessed11June2018].
WOODROFFE,L.2018.Brent Council: Keeping Vulnerable Young People Safely in the Community [Online].Available:https://www/designcouncil.org.uk/resources/case-study/brent-council-keeping-vulnerable-young-people-safely-community[accessed21September2018].
WRIGHT,S.&STREET,J.2007.Democracy,DeliberationandDesign:thecaseofonlinediscussionforums.New Media and Society,9(5),pp849-869.
ZIMMERMAN,J.,TOMASIC,A.,GARROD,C.,YOO,D.,HIRUNCHAROENVATE,C.,AZIZ,R.,RAVITHIRUVENGADAM,N.,HUANG,Y.&STEINFELD,A.2011.FieldTrialofTiramisu:CrowdSourcingBusArrivalTimestoSpurCo-design.SIGHI Conference on Human Factors in Computing Systems, pp1677-1686.
32
DEM
OC
RAC
Y 20
25ABOUT THE AUTHOR
NicoleMooreisanHonoraryFellowandPhD
CandidateattheInstituteforGovernanceand
PolicyAnalysis,UniversityofCanberra.Nicoleis
alsothe2019(inaugural)ACTLegislative
AssemblyLibraryFellowandhasheldnumerous
publicsectorrolesinalmostadecadeofservice.
In2013,NicolewasawardedaPublicService
ExcellenceAwardforthepromotion,co-design
andco-productionofserviceswithvulnerable
familiesintheACT.Nicole’sresearchisfocusedon
improvingpublicsectorcapabilityinengaging
citizensindecisionmaking,withaparticularfocus
onco-designanddeliberativeengagements.
Thisreportisthethirdoutputfromtheinitiative Democracy 2025—bridging the trust divide.
Forotherreportsinthisseriesvisitourwebsiteat:www.democracy2025.gov.au