examining participation
TRANSCRIPT
1
Examining Participation Victoria Gerrard
Opportunity Lab
Singapore University of Technology and
Design
Ricardo Sosa
Opportunity Lab
Singapore University of Technology and
Design
ABSTRACT
Participatory Design (PD) seeks to promote and regulate
the negotiation of social change. Although many methods
claim to be participatory, empirical evidence to support
them is lacking. Few comprehensive criteria exist to
describe and evaluate participation as experienced by
stakeholders. There is a need for rigorous research tools
to study, validate and improve PD practice. This paper
presents the development and initial testing of PartE
(Participation Evaluation), an interdisciplinary and
intercommunity approach to studying and supporting
participation in PD. Semi-structured interviews based on
the framework showed it to be useful in: a) revealing
differences in how stakeholders view participation and
design, b) developing a personal frame of participation c)
exploration of the future of participatory practices; and d)
suggesting actions to resolve specific challenges or
contradictions in participation at a broader level.
Additionally, these early results of PartE support the need
to move away from considering PD as a practice claimed
by design professionals towards a more open dialogue
between all stakeholders to redefine a collective meaning
of “Participation and Design” for social change.
Author Keywords
participation; participatory design; evaluation
ACM Classification Keywords
H.5.m: Participatory design, participation, evaluation,
framework. H.5.2: User-centered design.
INTRODUCTION
Design can be viewed as a negotiated process of social
change between the individual, the social context and the
designed artefact (Highmore, 2008; Latour, 2005).
Motivated by a general belief that more desirable design
outcomes will occur by engaging people in decision
making, Participatory Design (PD) attempts to promote
and coordinate this negotiation. However, the actual
experiences of participation by those taking part in PD
remain largely unknown. Despite or beyond the good
intentions of designers, there is a lack of evidence from
systematic studies to understand what makes a design
process participatory – the types or levels; the methods
and practices that best suit specific communities, projects
or stages of a social contract; the actual advantages and
limits of its realisation; how it is expressed; and the
strategies or heuristics to identify, promote and study PD.
This paper presents the development and initial results of
PartE (Participation Evaluation), an inter-stakeholder
approach to studying and supporting participation in PD.
PartE aims to reveal what characterises participation in
design and the fundamental principles that can be used to
examine and guide participatory design practice by all
actors involved in the process.
The paper begins by presenting the initial development of
the PartE framework, built upon the participatory
literature across disciplines and our own field experiences
of PD in Mexico, UK, India and Southeast Asia. We then
go on to illustrate how this initial framework was used to
examine participation across PD experiences. PartE is
applied in the form of a visualization map (Participation
Orbit Workbook, POW) to guide semi-structured
interviews with partners of Opportunity Lab (O-Lab) - an
interdisciplinary design lab at Singapore University of
Technology and Design (SUTD) which seeks to
understand and enhance the design role that people play
in social change, of which the authors are both members.
The paper concludes with an invitation to the PD
community to build on this work by contributing,
alongside all PD stakeholders, to the refinement and
deployment of PartE as an open platform to support the
collective exploration of the relationship between
participation and design.
BACKGROUND
Problems of Claiming Participation
Claims to participatory design abound (Cooke et al.,
2001; Hickey et al., 2004; Weber, 2010; Brett, 2003;
Vines et al., 2013; Halkola et al, 2013). Such claims are
often made through the application of specific design
processes supporting democratic ideologies (Balka, 2010,
Kyng 2010). However, the literature describing these
participatory experiences are largely anecdotal in nature
and lack systematic approaches and clear evidence as to
how, when, who and why certain practices work best.
Participation makes PD distinct from general design
practice and therefore any claims to PD must be
substantiated by a rigorous examination of participation.
This is problematic since the meaning of ‘participatory’
in PD varies widely among stakeholders (Vines et al.,
2013). This lack of agreed criteria against which
participation can be evaluated means that the
distinguishing practice of PD – that of participation –
remains largely invalidated. Thus, any claim to impact
achieved through PD - as opposed to other practices of
design – is difficult to justify (Beck, 2002; Michener,
1998; Balka, 2010).
Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are
not made or distributed for profit or commercial advantage and that
copies bear this notice and the full citation on the first page. To copy otherwise, or republish, to post on servers or to redistribute to lists,
requires prior specific permission and/or a fee. PDC’14, 06-OCT-2014, Windhoek, Namibia. Copyright 2014 ACM ISBN 978-1-4503-2256-0…$10.00.
2
Multiple Claims to Participation
Traditionally, claims to PD are made by the formal agents
-from academia, government, NGOs, industry and
international agencies – who have applied methods they
describe as participatory. There are few agreed criteria to
validate the actual type and level of participation as
experienced by all participating stakeholders over time
and in relation to the outcomes of these projects (Vines et
al., 2013). Bergvall-Kåreborn, (2008) captures this idea
when she says “PD is often treated as a unitary concept
despite the many forms of participation, involvement, and
influence of stakeholders”.
As widely documented, with growing technical literacy,
the way people participate in design has, and continues to,
quickly evolve beyond the traditional scope of PD (Star
and Ruhleder. 1996; Clement et al., 2012; Halkola et al.,
2013; Vines et al., 2013). These new forms of
participation occur within evolving social structures
where individuals operate more as autonomous agents
across many interest groups rather than a single
organisation or community (Dewey, 1927; Hardt et al.,
2004). Interactions are negotiated between agents -
including the mechanisms of participation. Rather than
predefined by a PD practitioner, participation is claimed
by individual agents and groups as they together
determine the rules governing their practices of
participation within their shared interests (Ehn, 2008).
Conflicting Claims to Participation
By definition, PD supports the idea of equal partnerships
in knowledge creation. One approach to look at
partnerships is through social interactions in a design
process. Boltanski and Thevenot (2006) examine the
interactive processes that change social coordination.
They explain that societies are collections of co-existing,
interdependent orders each with a unique value-specific
justification. These interacting orders generate not only
conflicts, critiques and disputes but also guide the
distribution of resources and the coordination of social
actors. According to this pragmatic perspective, design
and especially design done across contexts, involves
justification and critique of its value. PD practice attempts
to intentionally moderate the negotiation of how design is
valued. Vines et al. (2013) present this negotiation as
configuring participation, with the trustees of the process
being PD practitioners (Cowen and Shenton, 1996).
However, with no agreed value of design or participation
the criteria to which these practitioners can be held to
account remain limited.
Development is defined in multiple ways as a function of
the multiplicity of ‘developers’ entrusted with the task of
‘development’ (Cowen and Shenton, 1996). Likewise, a
growing diversity of definitions describing PD brings
greater specialisation of process and a rationality of
design intent – according to Freire (1970) the former
creates an ‘inability to think’ and the later degenerates
into ‘myth making’. The transmission of this design
intent tends to lead to ideological myths. PD practitioners
are therefore accountable to the myth constructed around
their specialisation and expect non-specialists to
participate within the same ideological frame. However,
as Balka (2010) and Kyng (2010) eloquently note, the
‘ideal’ of PD as a democratic approach to design is not, in
itself, enough to ensure impactful engagement in design.
By definition, non-design participants and specialists are
not constrained by the same myths as practitioners. They
either have the ability to choose to participate – exit or
voice – or they are excluded from it (Hirschman, 1970).
By acting as trustees of the process, PD practitioners
attempt to invite other actors to participate in their myths.
As a result, this overlooks the unintentional interactions
that occur between a design, an individual and society
(Cleaver, 2002; Mrazek, 2002; Latour 2005). To
summarise Star and Bowker’s (2002) work on
infrastructur-ing – neutral spaces are needed where the
myths currently supporting PD as a practice can be openly
re-negotiated within changing socio-technical structures.
Evaluation to Overcome Conflict
Participation in design can be understood and experienced
on a wide spectrum from; the use of a product (Redstrom
2008); attending a meeting; making a donation (Kuo
2012); protesting (Hirsch, 2009b; Clement et al., 2012);
contributing online (Hess et al. 2008); making and
customising products (Ehn 2008, Bjorgvinsson et al.
2010); contributing information (Simonsen et al. 1997);
inventing and doing DIY; whistle-blowing etc. What is
lacking is an agreed approach to study, adapt, and
improve such diverse participatory design practices.
Mertens (2007) describes how a transformative paradigm
of evaluation is essential to achieving social justice. The
transition to this transformative paradigm requires
reframing of myth brought about by specialisation and
rationality; including power structures, domains and
locations of knowledge, and processes of change. Tools
to communicate values across paradigms are required
which assist in the building of a shared understanding of
participation in design before evaluation of their
processes can begin. As Kyng (2010) notes, PD is
becoming more professionalised and subsequently its
methods are increasingly standardised. However, this
standardisation has more often than not been rationalised
through professional myth rather than empirical evidence
(Vines et al. 2013). As noted by Ehn (2008), platforms
which support the negotiation between traditional and
emergent claims to participation in design are needed.
Some recent notable attempts have been made to reflect
on the meaning of participation in PD. For example
Bratteteig et al. (2012) studied the power between levels
of decision makers in their participatory urban design
work. Through semi-structured interviews Halkola et al.
(2013) evaluated the extent that ‘genuine’ participation
was achieved in a project seeking to engage children in
the design of information infrastructures. Vines et al
(2013) looked more broadly at the way participation in
design is understood and shared lessons from other
domains of knowledge including participatory arts, social
media and crowd-sourcing. Ehn (2008) goes further by
discussing the future of PD as a practice in a landscape of
increasing public engagement in design.
These studies begin to discuss meanings of participation
from the perspective of specific projects. This valuable
3
reflective practice by individual PD practitioners has not
fostered a collective reflection of the participatory nature
of PD as a practice. In this paper we present PartE, a
framework which aims to contribute to the organisation
of the study of PD across experiences, projects and
settings in order to develop a common reference frame
through which to collectively and openly understand,
evaluate, communicate and improve participation across
disciplines and types of PD participants. As an open
discussion platform for all design stakeholders, PartE
ultimately seeks to provide a space where participation in
design can be collectively renegotiated, reclaimed and
repositioned as a validated design practice which creates
impactful community driven social change.
PART-E FRAMEWORK
The goal of PartE is to offer a comprehensive platform to
share experiences, encourage reflection and enable
learning about participation in design by revealing
commonalities and differences between PD initiatives,
and the reasoning behind them. Ultimately the authors
hope to continuously develop the framework among all
design stakeholders as an evolving and constantly
negotiated project. Here we present a first iteration of the
framework alongside an explanation of its development
and some results from preliminary testing. Through these
initial steps we expect to draw critique, and encourage
contributions, from the PD community to further develop
the framework for collective reflection and development
of participatory practice in PD.
The PartE framework was developed by combining cross-
disciplinary scholarship with our own field experiences in
Mexico, UK, India and Southeast Asia over the last
fifteen years, in an attempt to formulate a comprehensive
and universal framework which captures the most
distinctive aspects of PD practice. We took great care to
develop a framework which avoids intrinsic value
assessment, is succinct, avoids a singular disciplinary
approach and language, and is able to capture past,
present and future views of PD. PartE was initially
developed around five dimensions, with a sixth added
after early testing.
The following dimensions are included in the current
version: Objective, Practice, Interaction, Barriers,
Representation and Impact. Each dimension contains
various possible attributes extracted from the PD
literature and related research on social change,
teamwork, etc. These attributes are not presented as a
linear scale, and they are not intended to be mutually
exclusive but rather to capture the features of each
dimension that can occur in PD practice. The attributes
may be experienced simultaneously by a single individual
or group, by different stakeholders at different times, or
longitudinally in the course of a PD initiative. Each
dimension and their attributes are defined, explained and
illustrated below.
Framework Dimensions
Dimension 1: Objective refers to the ultimate goal or
intention to initiate and maintain PD practice. The
categories in this dimension draw from the notion of
“leverage points” developed by Meadow (1998) to
describe the mechanisms required to change a system.
Initially all twelve “leverage points” were included in the
framework before simplifying to the three main
categories as a result of initial testing: a) Material Things,
b) Organisation, Rules & Information Flows, c) Mind-
sets & Paradigms.
Examples of ‘Material Things’ as an objective of PD
include changes to income, savings or schedules, access
to technology and the implementation or deployment of
designed artefacts such as roads, devices, and vaccines.
Most PD projects have an objective of developing a
‘Material Thing’ whether it be a prototype (Ehn, 1993;
Bodker et al. 1991), a new workplace technology (Bodker
et al., 1991; Bloomberg et al, 1993), a product (Hess et
al., 2008), an urban community space (Bratteteig et al.,
2012; Sanoff 2010) etc. Sometimes the delivery of a
‘Material Thing’ can be the main focus of the initiative
and sometimes, as with many prototyping activities, they
can be used as boundary objects to achieve less tangible
objectives such as changes to ‘Organisation, Rules and
Information Flows’ or ‘Paradigms’ (Weber, 2010).
Examples of changing ‘Organisation, Rules and
Information Flows’ as the objective of PD include the
adoption of preventive health practices, access to
education and training, access to new communication
systems, access to information, and the establishment of
‘key performance indicators’. Again, these are a common
feature in many PD projects which attempt to redefine the
relationship between actors by providing a space for
people traditionally marginalized from the design process
to have a say (Greenbaum and Kyng, 1991). For some
projects, the objective to change the way people interact
and share information might be the main purpose of the
project (DiSalvo et al., 2013) while others might see it as
a way to achieve the development of a ‘Material Thing’
or a step towards changing ‘Mind-sets’.
Finally, examples of changing ‘Mind-sets & Paradigms’
as an objective of PD include novel governance
structures, adoption of new goals (for example moving
from not-for-profit to for-profit, or from systems that
promote inequality to those that support equality) and the
emergence of game-changing indicators and social
perceptions (such as in understandings of disability,
marriage and drugs). Hirsch’s (2009b) work with activists
might provide examples of this attribute of objective
within the PD literature, as might the work of artists such
as Robert Ransick who has attempted to reconceptualise
the idea of citizenship in his work on the US-Mexico
border (Weber, 2010).
Dimension 2: Practice refers to the tools and techniques
used in PD. Approaches to PD are varied and diverse,
originating from specific contexts and for different
purposes (Sanders et al., 2010; Brandt et al, 2013).
Previous work to organise these approaches has been
concerned with developing “toolboxes with the ambition
to show that there is a richness of tools and techniques
available that may be combined, adapted and extended to
form the basis for yet newer PD practices”. An initial set
of five attributes were proposed as – ‘accepted’,
‘adapted’, ‘responsive’, ‘evolving’ and ‘open’ methods.
4
These were later reduced after initial trials for enhanced
clarity and consistency to: a) Well-known Formats, b)
Emergent Practices, and c) Novel/Unpredictable
Approaches.
‘Well Known Formats’ captures the use of standard PD
tools and techniques such as those comprehensively
described by Brandt et al (2013). These include textbook
methods such as storytelling and visualisation games
(Ehn and Sjogren, 1991; Johansson 2005), ethnography
(Kensing and Bloomberg, 1998), prototyping (Ehn, 1993;
Sanoff, 2010; Suchman, 2000), the use of probes (Gaver
et al., 1999) and generative tools (Sanders, 2000),
improvisational theatre (Brandt and Grunnet, 2000) etc.
‘Emergent Practices’ refer to tools and techniques
adapted from the existing library of techniques or from
other fields. Practitioners often ‘localise’ methods in
response to available time and resources, numeracy and
literacy, beliefs or other local conditions. Through various
dissemination channels, these adaptations and other novel
practices ultimately become common practice. Many now
‘Well-Known Formats’ of PD were adopted by
practitioners in this way such as ethnography from
anthropology (Orr, 1986). Recent, less interventionist,
examples of these emergent formats include Kiva, a tool
for online peer-to-peer funding for development, and
adaptive re-uses of online social networks to coordinate
citizen movements (Shirky 2008).
Lastly, PD may also be conducted through ‘Novel and
Unpredictable Approaches’, often disruptive to traditional
practice and co-created by the grassroots. Examples of
this include a recent case, in early 2014, in Mexico City
where spontaneous contemporary dance performances at
major subway hubs were organised to support opposition
against hikes in public transport fares (#PosMeSalto).
Such methods are not part of PD toolkits, but they can be
effective in furthering social causes and may mature into
art movements or genres such as the music trova de
protesta in Latin America in the 1960s (Moore, 2001).
Other examples of these novel approaches to PD are
documented by Cynthia Weber in her work on the role
design plays in redefining the relationship between
migrants, citizens and the US State (Weber, 2012).
Through these three attributes, the Practice dimension
thus explores the extent to which PD is approached from
disciplinary training or cultural tradition, or how
creatively they are adapted, created and disseminated as a
result of participation.
Dimension 3: Interaction targets the modes of action and
types of contributions between PD participants. Initially
labelled as ‘Medium’, it was renamed after initial trials.
Three main attributes reflect the level of collaboration
between participants, drawing from models of cross-
disciplinary teamwork by Fruchter (2001), as a)
Contribution of Resources & Information, b) Exchange
and Awareness of Contributions, c) Collaborative
Contributions.
‘Contribution of Resources & Information’ includes
widespread one-way interactions such as extractive data
gathering, introduction of ‘appropriate technology’,
distribution of monetary resources or emergency supplies,
or cases where standard solutions are imported for
dissemination or deployment, such as vaccines, medical
devices, latrines, irrigation pumps, etc. Arnstein (1969)
captures this attribute in the third rung of her ladder of
participation and describes it as a first step to legitimate
citizen participation. This is certainly the case if the use
of information and resources is dictated by the provider
however, there are examples in PD where tools might be
provided by one stakeholder which facilitate the building
of ‘Exchange and Awareness of Contributions’ of other
stakeholder such as the contribution of the MR-Tent by
Bratteteig and Wagner (2012) to support the collaborative
design of a psychiatry clinic in Paris.
‘Exchange and Awareness of Contributions’ range from
cases where participants contribute their specialised
knowledge or skills to cases where they become aware of
each other’s strengths, goals and limitations. As a result,
team effort creates synergies between participants, who
are empathic of each other. In teamwork models (Fruchter
2001) this is captured by the growth of ‘awareness’ and
‘appreciation’ between collaborators of each other’s
goals, constraints and language and a genuine intention to
complement each other.
Lastly, in ‘Collaborative Contributions’, decision-making
occurs as the result of deep understanding between
participants. They contribute to proactive discussions,
provide input before it is requested, and begin to use,
adapt and shape the vocabulary and methods of other
participants. An equivalent process of “mutual learning”
is identified in PD by Robertson and Simonsen (2012) as
providing all participants with increased knowledge and
understandings. Relevant to this dimension is the
distinction made between public communication, public
consultation, and public participation based on “the
nature and flow of information between participants”
(Rowe and Frewer, 2005). The main issues to be
discussed here are related to the levels of interaction in
PD, and whilst in principle deeper understanding levels
may be preferable, the reality is that the appropriateness
of the medium of interaction and the scaffolding process
to develop it, depend on the needs, conditions and
requirements of each PD experience.
Dimension 4: Barriers refer to the type of limitations and
challenges that are more prevalent and particular to PD.
Based on studies where challenges are typified across
projects (Stringer et al 2007), five attributes are included
in this dimension including, a) Economic, b)
Environmental, c) Political, d) Social and e) Individual.
Economic refers to limited participation due to lack of
financial or material assets. Environmental limits are due
to high vulnerability to natural disasters or geographical
barriers. Political limitations include restrictions of
human rights, lack of freedoms, authoritarian leadership,
power structures, etc. Social limits include exposure to
violence, social exclusion, cultural values, stigmas and
taboos, etc. Lastly, individual limitations are related to
cognitive biases, beliefs and values, lack of skills and
self-reliance. The questions that this dimension supports
relate to the potential predominance of barriers across PD
projects, approaches, or the connection between types of
5
barriers and types of objective, practice or types of
interaction.
Dimension 5: Representation captures involvement in
decision-making. PD is differentiated from general design
practice by the inclusion of people directly in the process
(Sanders and Stappers 2012), but the literature suggests
that people may be represented very differently across PD
experiences (Vines et al., 2013; Bergvall-Kåreborn, 2008;
Beck, 2002). In addition to this, the roles of PD agents are
changing (Kyng, 2010) and there is an increase in
autonomous individuals interacting in complex networks
of shared interests (Star and Bowker, 2012). In an attempt
to capture this changing agency within PD we include
three attributes; a) Appropriated or Self-appointed, b)
Entrusted or Delegated c) Direct Autonomous.
‘Appropriated or Self-appointed’ representation refers to
cases where a stakeholder represents others by appointing
themselves in a position of authority or leadership.
Examples of this include journalists, artists, academics,
entrepreneurs or social, religious and political leaders
appropriating the role of representing or voicing the
concerns of a community. These individuals may seek to
establish governance structures where people ultimately
decide whether their interests are represented or not.
Where participatory structures are established, it would
be hard to define the point at which the change agent or a
small initial minority gains legitimacy as participatory
(Barnaud and Van Paassen 2013).
The opposite extreme is, of course, Direct Autonomous
representation, where individuals represent themselves
directly. Examples of this include online problem solving
platforms such as FoldIt, a puzzle game to solve protein
folding challenges set by scientists, or Change.org where
individuals can start a petition to initiate change. For both
examples people represent themselves rather than affiliate
to a particular group. This attribute would describe the
type of representation experienced in larger infrastructur-
ing PD projects described by Star and Bowker (2002)
where individuals belong to complex networks which
span a number of design initiatives and interact and
inform each other in unpredictable and organic ways.
Finally, ‘Entrusted or Delegated’ representation includes
cases where participants explicitly entrust a representative
to be their trustee and make informed decisions in their
collective best interest, or at least to be the spokesperson
for their views. Historically this has often been the case in
many PD projects where the pioneers of the practice
worked with trade unions representing workers in
negotiating the introduction of technology to the
workplace in Scandinavia (Robertson and Simonson,
2012). As PD has shifted from a focus on the workplace
to the community, designers have continued to work in
partnerships with existing local organisations such as
community groups, micro-finance cooperatives, NGOs or
religious groups (DiSalvo, 2012).
The ‘Entrusted or Delegated’ attribute also captures the
idea of enforced delegation as opposed to voluntary
participation (Byrne and Alexander, 2006) where
stakeholders are requested to participate through
professional duty, imposed donor requirement or by
necessity, in exchange for compensation. Additionally,
PD practitioners themselves are often required to balance
how they represent others in their practice. The politics in
PD and its effects on decision-making are documented by
Robertson and Wagner (2013) in their work on the ethics
of PD and by others who reflect on their own experiences
of representation in specific PD projects (Pederson, 2007;
Bratteteig and Wagner, 2012; Bjerknes and Bratteteig,
1988; Bowers & Pycock, 1994).
Dimension 6: The Impact dimension emerged in early
trials as interviewees reflected on the five dimensions by
evaluating projects in terms of longevity and outcomes.
The need for the framework to guide the evaluation of
impact is echoed by Balka (2010) in her call for reflection
on impact of PD in order to “expand [the community’s]
scope of influence”. In response to this, the three
attributes nominated to help stakeholders reflect on how
the impact of PD can be evaluated include: a) Short-term
or Small-scale Changes, b) Long-term or Large-scale
Changes, c) Unintended or Indirect Changes. Examples of
impact being evaluated in terms of ‘Short-term or Small-
scale Changes’ include projects where organisations with
limited resources and a lack of time are dependent on
external funding and where outcomes are pre-defined by
a donor (Karl, 2000).
‘Long-term or Large-scale Changes’ include initiatives
where impact is evaluated in terms of the potential long
term domino effect of social, environmental and
economic impacts of discrete interventions as they
interact and build on each other such as PIPA
(Participatory Impact Pathway Analysis) (Douthwaite et
al., 2008). With the increased normalisation and
professionalization of PD, larger projects with significant
financial backing are beginning to emerge (Kyng 2010).
Approaches to evaluating the impact of these types of
project are described by this attribute.
‘Unintended or Indirect Changes’ refers to the way that
unknown outcomes of PD are evaluated. McAllister
(1999) offers useful advice to include unintended
consequences in impact evaluation of participatory
research. Examples where this is required in PD might
include the evaluation of the impact of infrastructure-ing
initiatives where interactions between agents and
initiatives are complex and indirect by nature (Balka,
2010). Additionally as PD expands from the workplace
towards working within the more unpredictable context of
communities approaches to impact evaluation which are
able to capture the unintended consequences of PD
initiatives are useful.
Table 1 presents all the PartE dimensions along with their
attributes. There is room to improve, extend or refine the
framework, however our aim here is to present and study
this initial version to see how well it captures different
interpretations and experiences of PD from practitioners
from different disciplines and spheres of action.
6
Table 1 Final Iteration of PartE Framework dimensions and attributes
Dimensions Objective Practice Interaction Barriers Representation Impact
Attributes a) Material
Things
b) Organisation,
Rules,
Information
c) Mind-sets,
Paradigms
a) Well-known
Formats
b) Emergent
Practices
c) Novel,
Unpredictable
Approaches
a) Contribution
of resources &
information
b) Exchange &
Awareness of
Contributions
c) Collaborative
Contributions
a) Economic
b) Environmental
c) Political
d) Social
e) Individual
a) Direct,
Autonomous
b) Indirect,
Delegated
c) Appropriated,
Self-appointed
a) Short-term,
Small scale
b) Long-term,
Large scale
c) Indirect,
Unintended
TESTING THE FRAMEWORK
Method
The PartE framework was used to explore its
appropriateness to support reflection and sharing of PD
experiences by individuals with widely different levels of
experience, backgrounds and disciplines. We invited
seven O-Lab partners to have an open dialogue about PD
practices. Their backgrounds range from architecture to
sociology, urban planning, history, and engineering. Their
experience extends from second-year undergraduate
students with initial exposure to fieldwork, to researchers
and practitioners with over a decade of PD experience.
A visual format of the PartE framework called
Participatory Orbit Workbook (POW) was developed to
guide the semi-structured interviews. POW consisted of:
- a hard copy of a circular layout of the six framework
dimensions with an additional axial dimension to
capture changes over time (past, present, future).
- a set of coloured stickers of various sizes to
represent types of participant and importance of
attribute to that participant.
- an instruction booklet containing a glossary,
definitions of dimensions and their attributes, as
well as diverse examples illustrating the attributes
for guidance as interviewees completed their orbit.
Interviewees were asked to first share their personal
definition of, and what they believe to be important in
relation to, PD practice. Then, using stickers of different
sizes and colours they were asked to visually record the
attributes of PD that best describe their experiences on the
POW as they verbally shared with us their PD stories.
Seven O-Lab partners were interviewed in five sessions
(two in pairs – S2 with S3, and S4 with S5). The sessions
ranged from 45 to 90 minutes and took place in various
locations on the SUTD campus. During the interviews
both authors facilitated the discussion and took notes.
After the interviews, with the support of the completed
POW, the authors first reviewed their notes independently
before comparing and analysing their main observations.
Results
Results are organised in three sections. Firstly, PartE is
analysed as a framework to identify trends and patterns in
the perception of participation across PD experiences of
individuals with different levels of experience,
backgrounds and disciplines. Secondly, PartE is analysed
as a framework to support a deep and meaningful
dialogue about the meaning of participation in PD.
Thirdly, the suitability of POW is reviewed as a format
for dialogue based on the PartE framework.
Identifying trends and patterns in perception
Three main patterns were identified by the authors in the
perception of participation emerging during the
interviews, these are represented in Figure 1a and include:
1. A universal recognition of diversity in objectives
across PD practitioners and projects
2. The generalised expectation that objectives of all
stakeholders are bound to change over time
3. A trend to perceive design objectives as higher-level
than those of communities, project partners and
government officials with the future in this
dimension rendered as an ‘enlightenment’ where all
participants are expected to move to higher-level
objectives following the designers’ lead.
Three main patterns were extracted by the authors in
relation to the perceptions of the practice of participation,
as presented in Figure 1b. These were:
1. Formal training and background directly influences
the choice of methods and approaches.
2. Recognition of future opportunities for the design of
new methods and tools, adapted or re-purposed.
3. A lack of reference to the “open” category of
methods and practices, which may suggest a lack of
observation or acknowledgement of non-traditional
PD methods. This relates to a perception of design
leadership in PD practice.
Perceptions of the way participants interact mirror some
of the patterns observed in the objectives dimension, as
shown in Figure 1c. In summary:
1. Designers tend to build collaboration from the early
stages, while they perceive other participants as
content with contributing and receiving resources
and information.
2. A general view of design as spearheading the
change into a future where participants grow more
aware, appreciative and understanding of each other.
Perceived barriers had a marked convergence in all
interviews, as presented in Figure 1d, namely that:
1. Participation is often limited by the mental model of
participants, the social structures and the political
agendas around these projects.
2. Barriers to participation are unlikely to change over
time.
7
Upon reflection, the usefulness of the Barriers dimension
can be questioned as PD participants generally revealed
more interesting limitations to participation across
dimensions, in relation to specific issues rather than when
confined to discussing limitations in general. This
dimension may be removed if future iterations of PartE.
Two patterns of perception of representation in PD were
identified, as presented in Figure 1e. These were that:
1. Similar to the objectives dimension, representation
varies considerable across PD actors and projects.
2. Participants in PD initiatives often start with
delegated or entrusted types of representation but
perceive that the desired future state should change.
Two seemingly opposite variants in how the future
is envisioned were recorded: some interviewees
expressed their desire to see entire groups or
communities adopt higher levels of autonomy via
personal and direct decision-making, while others
prefer to see some individuals appropriate leadership
roles for instance as entrepreneurial change agents
who would lead the initiatives in their communities.
Figure 1. Trends and patterns of perceptions of
participation identified through each dimension of the PartE
framework (a. Objective, b. Practice, c. Interactions,
d. Barriers, e. Representation)
PartE as a support for dialogue
PartE was evaluated against four functions of supporting
dialogue about participation. The first function refers to
the ability of PartE to reveal an acute awareness that
different stakeholders have different understandings about
the roles, responsibilities and desires of other
stakeholders. PartE supported the surfacing of perceptions
about different stakeholders through the combination of
characteristics in different dimensions. Examples of this
include the following quotes from interviewee S6
regarding the perception of the role of grassroots activists,
the government and civil society:
Grassroots Activists
Objectives: “Activists are interested to change mind-sets
however there is little evidence that this has been successful.”
Representation: “They have always and will continue to
appropriate representation.”
Government Agencies
Objective: “Government sees it as a way to share information”
Representation: “It is now delegated by people”
The People
Objective: “They are interested in the more tangible
infrastructure but also concerned by the way it is provided… often through the participation of the middle classes who
participate to prevent infrastructure development which damage
them in some way”
Representation: “There is movement towards autonomous
decision making”
In addition to this, as mentioned above, the tool enabled
interviewees to articulate assumptions about the role of
designers (Figure 1a) as supported by the following
statements:
Objective of Designer: “Practicing to change mind-sets”
Objective of People: “Mainly expect changes in material things
and they want it done quickly.” -S4.
PD Impact: “PD is useful for designers as it provides
information about people, and it is beneficial for users who
indirectly benefit from the results”-S7.
Although POW asked for personal experiences of PD, all
except one interviewee shared instances of planning and
conducting participatory exercises rather than taking part
in one. This suggests an assumption by practitioners that
PD is by definition a guided, established and intentional
design process rather than a collective effort perceived
differently by those involved. Attached to this idea seems
to be a sense of professional duty to perform PD most
notably captured in the Representation and Practice
dimensions:
Representation: “either delegated by someone else or is an
autonomous act” -S4
Practice: “participants generally use accepted or adapted
processes and to a lesser extent evolving processes” -S4.
Representation: “have always and will continue to be delegated
as a representative” –S6
Practice: “used accepted techniques that were taught in
school”-S6
The second intended function of PartE is to assist
interviewees to explore and develop their personal frame
of participation including what they understood by each
characteristic, the relative importance of them to PD and
the value they assigned to them. By contrasting the
opening statements and concluding reflections shared by
our interviewees, we can conclude that in some cases (S2,
S3, S4), PartE served to reveal a number of issues that
were not articulated in the initial opening remarks. For
the rest, PartE seemed to be more useful to organise the
ideas that they did not allude to in the beginning.
PartE was able to clearly reveal the value that different
interviewees have about certain PD characteristics. An
example of this can be seen in one non-designer
8
interviewee’s evaluation of the Interaction dimension
where the process of design moved from an
understanding of people towards contribution of
resources, and that their role diminished over time as the
project goals became more tangible:
Interaction: “We aim to start with understanding, and then aim
to move to contribution of resources, since PD projects are
always about designing stuff” –S5.
In another example the interviewees expressed insightful
views about the social barriers of participatory processes
re-enforcing social norms rather than challenging them:
Barrier: “We saw an important social barrier because
participants often fell in excessive agreement, which was
useless” –S2 and S3
In contrast, other interviewees saw social and political
factors as the biggest barriers to participation but for
different reasons:
Barrier: “Openness of society to participation has a big
impact” –S4
Barrier: “Political and Social barriers have and will continue
to exist for those with less power in the system” –S6
Barrier: “The main challenge here is attitude, also language
problems between ethnic and age groups” –S1
Other instances of this occur in terms of representation
where two interviewees expressed a need for people to
achieve more autonomy through appropriation, whereas
another considered representation was related to a sense
of apathy or an inability to achieve true collaboration:
Representation: “The community doesn't have their own voice.
First they need to take it for themselves and then be autonomous
to make the changes they want for their community.” –S2 and
S3
Representation: “There is … a certain amount of appropriated
representation where people may not intentionally take the lead
but other people were just not paying attention when decisions
were made… If people were better collaborators things might
improve.”-S7
The third function of the framework facilitated
interviewees to reflect on their personal understanding of
participation through the articulation of ideas previously
not made explicit and supporting exploration of the future
of participation. Three interviewees proposed either a
necessary or imminent shift towards the role of a designer
as the provider of tangible outputs to one of facilitator of
social change, which is in clear contrast to S5 views as
cited above:
Objective: “Our first approach was more superficial, had to do
with visible things, we are now bridging to longer-term change
with higher purposes” –S2 and S3
Objective: “Personally began in trying to affect more material
things… hope to challenge boundaries through participation” –
S4
Others questioned the professional roles of PD
practitioners and suggested that PD as a process had more
to offer than the facilitator themselves:
Practice: “Anybody using the space is already adapting it… PD
is not so much to make people design, as it is to make designers
understand that people already design” -S1
While others suggested that the challenges to achieving
PD might be more related to disciplinary and professional
constraints:
Barrier: Civil service people do what they are meant to do, it's
not their job to question” –S6
Barrier: “We are all constrained by where we come from” -S5
These reflections provide an indication of where the
perceived limits of PD practices exist. This helps to
identify PD practices in contexts that need not be
conventionally seen as related to either participation or
design.
A further function of PartE analysed here is its ability to
begin to diagnose PD across dimensions and suggest
action to resolve specific challenges. Included here is the
identification of commonality in specific dimensions such
as an imagined future governance structure for PD:
Representation: “In the future there should be more autonomy
of representation but there still needs to be a political system
which is delegated or entrusted to do something.” –S4
Representation: “In the future representation will be more
autonomous but government will always be delegated to play a
role” –S6
And any contradictions or challenges related to achieving
these commonly accepted futures can be highlighted such
as a desire for PD to achieve changes in worldviews:
Practice: “Hope to challenge boundaries through
participation…there is a shift occurring from accepted to open
processes.” –S4
Practice: “In the future these methods must become more open
as this will probably lead to more success” –S6
Interaction: “There is still opportunity to make better design
through a paradigm shift through exchange of views i.e. design
creates a convenient artefact for conversation” –S7
Many of these comments highlight the inherent politics
and complexity of a PD process such as the following
statement made early during one interview:
Objective: “PD is often eyewash, claiming to challenge
paradigms but mostly dishonest; a deception driven by non-
designers but also by some designers who embrace it without
realising the true purposes of those who promote it” –S7
These statements can be used as a basis for further
exploration, using the experiences of others to begin
overcoming these challenges. Such as the following
statements about the trends experienced in the practice of
PD:
Practice: “We tend to use all types of methods, but we tend to
start with adapted versions and then as the project unfolds
either let go to more open formats if people take the lead, or go
to accepted methods if required” -S4
Practice: “All practice dimensions exist for everyone but
importance changes within the group. This is related to group
dynamics” –S3
Combining these comments with earlier ones related to
the perception of professional duty we begin to gather
evidence to build a picture of a collective understanding
of PD, the designer’s perceived role and their ability to
shape the process. The desire for change was revealed as
interviewees forecast the future of PD however there was
9
also a general scepticism that appropriate structures were
in place to facilitate this shift.
POW to help visualise PartE
The overall feedback from the interviewees was that
POW is useful for visualising the main issues in this
complex topic, as well as for education purposes to help
young or novice practitioners to identify and select the
most adequate PD approaches to suit the needs and
requirements of a given project. This is done without
making any value judgements but allowing personal
reflection on individual value structures. Based on this
initial feedback, we view POW as useful to thread ideas
across disciplines.
As part of this initial trial, various aspects of POW were
refined after each interview based on shortcomings in
communication or as direct suggestions by the
interviewees. These changes included formatting the
framework in a circular rather than an original matrix to
reduce any perception of importance or hierarchy of
attributes within dimensions. The new layout also gave us
the option of using radial distance to represent past,
present and future (desired) states of participation. The
sixth dimension was added to reflect an interest by
interviewees to review the perceived impact of
participation upon completion of the other five
dimensions. After initially allowing interviewees the
freedom to choose the meaning of colour and size we
began to constrain the meaning to assist reflection with
colour denoting different stakeholders, and size the
relative importance of the choice. An additional ‘wild
card’ sticker was provided for the interviewee to use for
dimensions they felt were missing from the framework.
The range of colours and sizes of the dots provided was
expanded following feedback from the interviewees who
wished to have more ways of capturing types of
participants or desired versus expected future directions,
etc. For each iteration the language was simplified, the
number of attributes reduced or grouped, more examples
were added, and one dimension was relabelled.
As the tool was simplified, reflection and conversation
became easier to facilitate. The format and terminology
used however may still be too complex for wider
dissemination and use and will need significant revision
to make it widely accessible. Simple changes such as
renaming the attributes in the Representation dimension
to ‘Represent Themselves’, ‘Represent Others:
Delegated’, Represent Others: Appropriated’ or removing
some terms for simplicity such as changing ‘Mind-Sets,
Paradigms’ to just ‘Mind-Sets’ could be made quite
simple. However it is the authors’ intention to also make
and test more radical changes to the format of PartE such
as developing a completely graphical, 3D, and interactive
online tool. Further testing will target a wider group of
PD stakeholders adapting the language and examples to
various cultures. This is a recognised limitation of the
current study, where the tool was tested on a small group
of individuals in an academic context. The lessons
learned from this small study are valuable to develop
further iterations of the framework and the tools to apply
it across a more diverse cross-section of PD stakeholders.
DISCUSSION
The work presented in this paper stems from a perceived
need for systematic studies of participation in design.
PartE, the framework proposed here starts from a multi-
disciplinary analysis of the literature and is developed in
relation to our own PD experiences. The criteria used to
develop PartE are framed, and the dimensions developed
to capture the framework are introduced. A semi-
structured interview called Participatory Orbit Workbook
(POW) developed from PartE is used to initially evaluate
the capacity of the framework to help PD stakeholders
reflect upon their experiences and draw a shared frame of
reference to talk about participation. PartE is assessed in
four aspects through interviews with PD students,
practitioners and researchers across disciplines. The
results show that PartE is useful to: a) reveal differences
in how stakeholders view participation and design, and
how they may understand their roles, responsibilities and
those of other stakeholders; b) explore and develop a
personal frame of participation including an individual
interpretation of each characteristic, their relative
importance and the value assigned to them; c) articulate
ideas previously not made explicit and support
exploration of the future of participatory practices; and d)
begin to diagnose participation across experiences and
suggest actions to resolve specific challenges or
contradictions at a broader level.
Additionally, these early results of PartE hint at
underlying challenges for PD in relation to its implicit
intentionality. PD currently asserts rather than proves its
existence. The term is used by our interviewees to refer to
a broad range of activities from gathering feedback in
shopping malls (S6) to community design workshops
(S4) to art installations (S7). Rather than reflecting the
application of an agreed domain of knowledge, the term
PD is often used as an admission by design practitioners
of their intent to involve people in their processes.
Claiming PD is common without any assessment of the
level or type of participation achieved. The implications
of this is that PD is evolving without a common
understanding of effective practice.
Entrusting designers with PD has facilitated the
development of practices which align to their traditional
professional paradigms. As recognised by some of our
interviewees, for many designers participation directly
contradicts their training which generally encourages
making decisions on behalf of others rather than in
collaboration with them (S6, S7). This is not intended as a
criticism but rather as a recognition of the role society has
traditionally assigned to designers. From our interviews
there is evidence that the contradiction between these two
social expectations - of autonomous decision maker and
trustee of PD - have led to conflicts between designers
and design stakeholders.
Our interviewees generally considered that attempts at
participation were hampered by the inadequate response
of the people designers sought to engage. Society’s
readiness for PD was expressed as a barrier by all those
interviewed. Perhaps this lack of readiness reflects
limitations of PD processes to articulate the purpose of
engagement. By professionalizing PD people are either
10
justified for not taking responsibility for decision making,
or excluded from doing so due to misalignment of power,
expectations, values and traditions of engagement
between designers and various non-designers.
At least two interviewees mentioned that they had been
taught about the existence, but not the practice, of PD
during their professional training. Another described their
entry to PD through self-study and personal experience.
These experiences suggest that although design
professionals are expected to take the lead in PD activities
there are few outlets for them to prepare for the role.
To overcome these existing social expectations of design
as a non-participatory mechanism of social change, PD
should no longer be asserted as a validated design
practice under the responsibility of designers. Instead, PD
needs to be renegotiated in a wider interdisciplinary and
intercommunity space; becoming the responsibility of
society rather than ‘experts’.
To assist with this negotiation we propose a departure
from the problematic domain specific term PD towards
the more explorative term ‘Participation and Design’.
We consider this will better facilitate dialogue between
multiple stakeholders by:
- Removing the assertion of participation when
describing any attempt to engage people in a design
process
- Allowing all stakeholders to reflect on experiences of
engaging in design and exploring what participation
in design actually means in terms of creating social
change – both for good and bad
- Allowing for greater exploration of the meaning
behind, and relationship between, Participation and
Design including identifying the opportunities and
limitations in this relationship.
- Allowing designers to explore and reframe their role
in developing participatory approaches in design
practice and providing the freedom to overcome
existing challenges and evolve traditional paradigms.
- Providing opportunities in design training to discuss
ideas of participation in general practice, rather than
relating it to specific and generalized processes.
We hope that PartE can contribute to supporting this
dialogue around Participation and Design by providing
an organised structure to dissect ambiguous concepts -
such as ‘context’ ‘power’, ‘ethics’, ‘creativity’,
‘responsibility’ and ‘development’ - in the way people
engage in design. The motivation is not to develop new
‘Participatory Design’ practices but to develop
community accepted modes of design decision making
which stem from a dialogue about participation, design
and social change.
This paper presents the preliminary development and
testing of PartE – a framework to collectively reflect on,
identify, build and validate a common approach to
understanding participation and design. As mentioned,
there is significant work to be done to further iterate the
framework and develop alternative formats which are
more accessible to the diverse stakeholders involved in
PD. This requires a collaborative effort across contexts,
disciplines and backgrounds which we hope will garner
the support of the PD community among many others
working in the field of participation and design.
REFERENCES Arnstein, S. (2011). A ladder of citizen participation. LeGates,
Richard T.; Stout, Frederic Frederic, Stout (Hrsg.): The City
Reader. 5th Edition. New York: Routledge, 238-250.
Balka, Ellen "Broadening Discussion About Participatory
Design: A reply to Kyng," Scandinavian Journal of
Information Systems: 22:1, (2010), 7
Barnaud, C., & Van Paassen, A. (2013). Equity, Power Games,
and Legitimacy: Dilemmas of Participatory Natural
Resource Management. Ecology and Society, 18(2), 21.
Bødker, Susanne, and Kaj Grønbæk. "Cooperative prototyping:
users and designers in mutual activity." International Journal
of Man-Machine Studies 34.3 (1991): 453-478.
Bødker, K., Kensing, F. and Simonsen, J. Participatory IT
design: designing for business and workplace realities. MIT
press, USA. 2004
Boltanski, L, and Thévenot L. On justification: Economies of
worth. Princeton University Press, 2006.
Bowers, John, and James Pycock. "Talking through design:
requirements and resistance in cooperative prototyping."
Proceedings of the SIGCHI Conference on Human Factors
in Computing Systems. ACM, 1994.
Bjerknes, Gro, and Tone Bratteteig. "The memoirs of two
survivors: or the evaluation of a computer system for
cooperative work." Proceedings of the 1988 ACM
conference on Computer-supported cooperative work.
ACM, 1988.
Björgvinsson, Erling, Pelle Ehn, and Per-Anders Hillgren.
"Participatory design and democratizing innovation."
Proceedings of the 11th Biennial Participatory Design
Conference. ACM, 2010.
Bloomberg, J., and J. Giacomi. "A. Mosher, and P. Swenton-
Wall (1993)“Ethnographic Field Methods and their Relation
to Design”." Schuler and Namoida (eds.) Participatory
Design: Perspectives on System Design. Lawrence Erlbaum:
Hillsdale, NJ: 123-155.
Brandt, Eva, Thomas Binder, and Elizabeth B-N. Sanders.
"Ways to Engage Telling, Making and Enacting." Routledge
International Handbook of Participatory Design (2012): 145.
Brandt, Eva, and Camilla Grunnet. "Evoking the future: Drama
and props in user centered design." PDC. 2000.
Bratteteig, Tone, and Ina Wagner. "Disentangling power and
decision-making in participatory design." Proceedings of the
12th Participatory Design Conference: Research Papers-
Volume 1. ACM, 2012.
Brett, E A. Participation and accountability in development
management. The Journal of Development Studies 40.2
(2003): 1-29.
Byrne, E., and P. M. Alexander. "Questions of ethics:
Participatory information systems research in community
settings." Proceedings of the 2006 annual research
conference of the South African institute of computer
scientists and information technologists on IT research in
developing countries. South African Institute for Computer
Scientists and Information Technologists, 2006.
Cleaver, F. Reinventing institutions: Bricolage and the social
embeddedness of natural resource management. The
11
European Journal of Development Research 14.2 (2002):
11-30.
Clement, Andrew, et al. "Probing, mocking and prototyping:
participatory approaches to identity infrastructuring."
Proceedings of the 12th Participatory Design Conference:
Research Papers-Volume 1. ACM, 2012.
Cooke, B, and Uma K, eds. Participation: The new tyranny?.
Zed Books, 2001.
Cowen, M, and Shenton, R W. Doctrines of development.
Taylor & Francis, 1996.
Dewey, John. "The Public and Its Problems." (1927).
DiSalvo, Carl, Andrew Clement, and Volkmar Pipek.
"Participatory design for, with, and by communities."
International Handbook of Participatory Design. Simonsen,
Jesper and Toni Robertson (Eds). Oxford: Routledge.(2012)
(2012): 182-209.
Douthwaite, B, et al. "Participatory Impact Pathways Analysis:
A practical method for project planning and evaluation."
ILAC Brief 17 (2008).
Ehn, Pelle. "Participation in design things." Proceedings of the
Tenth Anniversary Conference on Participatory Design
2008. Indiana University, 2008.
Ehn, Pelle, Bengt Mölleryd, and Dan Sjögren. "Playing in
reality." Scandinavian Journal of Information Systems 2
(1991): 101-120.
Freire, P. Cultural action and conscientization. Harvard
Educational Review 40.3 (1970): 452-477.
Fruchter, R. (2001). Dimensions of teamwork education.
International Journal of Engineering Education, 17(4/5),
426-430.
Gaver, Bill, Tony Dunne, and Elena Pacenti. "Design: cultural
probes." interactions 6.1 (1999): 21-29.
Greenbaum, Joan M., and Morten Kyng. Design at work:
Cooperative design of computer systems. L. Erlbaum
Associates Inc., 1991.
Halkola, Eija, et al. "Searching for ‘Genuine’Participation in
Information Infrastructure Building." IRIS. No. 36.
Akademika forlag, 2013.
Hess, Jan, Sinja Offenberg, and Volkmar Pipek. "Community
driven development as participation?: involving user
communities in a software design process." Proceedings of
the Tenth Anniversary Conference on Participatory Design
2008. Indiana University, 2008.
Hickey, S, and Giles M, eds. Participation--from tyranny to
transformation?: Exploring new approaches to participation
in development. Zed books, 2004.
Highmore, B. The Design Culture Reader. Routledge, 2008.
Hirsch, Tad. "FEATURE Learning from activists: lessons for
designers." interactions 16.3 (2009): 31-33.
Hirschman, A O. Exit, voice, and loyalty: Responses to decline
in firms, organizations, and states. Vol. 25. Harvard
university press, 1970.
Johansson, Martin, and Per Linde. "Playful collaborative
exploration: New research practice in participatory design."
Journal of Research Practice 1.1 (2005): Article-M5.
Karl, Marilee. Monitoring and evaluating stakeholder
participation in agriculture and rural development projects:
a literature review. Sustainable Development Department,
Food and Agriculture Organization of the United Nations
(FAO). Downloaded 5 (2000): 2010.
Kensing, Finn, and Jeanette Blomberg. "Participatory design:
Issues and concerns." Computer Supported Cooperative
Work (CSCW) 7.3-4 (1998): 167-185.
Kolko, J. (2011) Wicked Problems: Problems Worth Solving.
Available online: https://www.wickedproblems.com
Kuo, Pei-Yi, and Elizabeth Gerber. "Design principles:
crowdfunding as a creativity support tool." CHI'12 Extended
Abstracts on Human Factors in Computing Systems. ACM,
2012.
Kyng, Morten. "Bridging the Gap Between Politics and
Techniques." Scandinavian Journal of Information Systems
22.1 (2010): 49-68.
Latour, B. From realpolitik to dingpolitik. Making Things
Public-Atmospheres of Democracy catalogue of the show at
ZKM (2005).
Latour, B. Where are the missing masses? The sociology of a
few mundane artifacts. (1992): 225-258.
Leal, A P. (2007). Participation: the ascendancy of a buzzword
in the neo-liberal era. Development in Practice, 17(4-5),
539-548.
McAllister, Karen (1999) Understanding Participation:
Monitoring and Evaluating Process, Outputs and Outcomes,
Ottawa: IDRC
Meadows, D. (1999). Leverage points. Places to Intervene in a
System. Hartland, Vermont, USA: The Sustainability
Institute.
Mertens, D M. Transformative Paradigm Mixed Methods and
Social Justice. Journal of mixed methods research 1.3
(2007): 212-225.
Michener, V J. The participatory approach: contradiction and
co-option in Burkina Faso. World development 26.12
(1998): 2105-2118.
Moore, Robin. "From the cancion de protesta to the nueva trova,
1965-85." International Journal of Qualitative Studies in
Education 14.2 (2001): 177-200.
Orr, Julian E. "Narratives at work: Story telling as cooperative
diagnostic activity." Proceedings of the 1986 ACM
conference on Computer-supported cooperative work.
ACM, 1986.
Pedersen, Jens. Protocols of Research and Design: Reflections
on a Participatory Design Project (sort Of): PhD Thesis. IT
University of Copenhagen, Innovative Communication,
2007.
Redström, Johan. "RE: Definitions of use." Design Studies 29.4
(2008): 410-423.
Robertson, Toni and Jesper Simonsen. "Participatory Design: an
introduction." The Handbook of Participatory Design, edited
by J. Simonsen and T. Robertson (2012):1-17.
Robertson, Toni, and Ina Wagner. "Engagement, Representation
and Politics-in-Action." The Handbook of Participatory
Design, edited by J. Simonsen and T. Robertson (2012): 64-
85.
Rowe, G., & Frewer, L. J. (2005). A typology of public
engagement mechanisms. Science, technology & human
values, 30(2), 251-290.
Sanders, Elizabeth B-N., Eva Brandt, and Thomas Binder. "A
framework for organizing the tools and techniques of
participatory design." Proceedings of the 11th biennial
participatory design conference. ACM, 2010.
12
Sanders, Liz, and Pieter Jan Stappers. Convivial Design
Toolbox: Generative Research for the Front End of Design.
BIS, 2012.
Sanoff, Henry. Democratic design: Participation case studies in
urban and small town environments. VDM Publishing,
2010.
Saxena, N. C. (1998). What is meant by People's Participation?
Journal of rural development-Hyderabad-, 17, 111-114.
Simonsen, Jesper, and Finn Kensing. "Using ethnography in
contextural design." Communications of the ACM 40.7
(1997): 82-88.
Star, Susan Leigh, and Karen Ruhleder. "Steps toward an
ecology of infrastructure: Design and access for large
information spaces." Information systems research 7.1
(1996): 111-134.
Star, S. L., & Bowker, G. C. (2002). How to infrastructure. In L.
A. Lievrouw & S. L. Livingstone (Eds.), The handbook of
new media (pp. 151–162). London: SAGE.
Stringer, L. C., Reed, M. S., Dougill, A. J., Seely, M. K., &
Rokitzki, M. (2007). Implementing the UNCCD:
Participatory challenges. In Natural Resources Forum (Vol.
31, No. 3, pp. 198-211). Blackwell Publishing Ltd.
Suchman, Lucy. "Organizing alignment: a case of bridge-
building." Organization 7.2 (2000): 311-327.
Vines, John, et al. "Configuring participation: on how we
involve people in design." Proceedings of the SIGCHI
Conference on Human Factors in Computing Systems.
ACM, 2013.
Weber, C. Introduction: design and citizenship. Citizenship
studies 14.1 (2010): 1-16.
Weber, Cynthia. "Design, translation, citizenship: reflections on
the virtual (de) territorialization of the US–Mexico border."
Environment and Planning D: Society and Space 30.3
(2012).
The columns on the last page should be of approximately equal length.