evaluating outcomes of arts engagement: a multi-dimensional approach- a pictorial model
DESCRIPTION
Evaluating outcomes of arts engagement: A multi-dimensional approach- a pictorial model. www2.uiah.fi/projects/metodi/15b.htm. Kim Dunphy, Program Manager, Cultural Development Network PhD researcher, Deakin University. Overview. The problem: - PowerPoint PPT PresentationTRANSCRIPT
Evaluating outcomes of arts engagement: A multi-dimensional approach- a pictorial model
Kim Dunphy, Program Manager, Cultural Development NetworkPhD researcher, Deakin University
www2.uiah.fi/projects/metodi/15b.htm
Overview
The problem: •Reframing the intrinsic/instrumental dichotomy•How can qualitative research about outcomes of arts engagement be presented simply?
The solution:•multi-dimensional framework of outcomes•a pictorial model
Case study: youth theatre project, Scared Cool, Ba Futuru, Timor-Leste
The problem
…..the arts sector’s
failure to find a framework that articulates its value in a cohesive and meaningful way, as well as by its neglect of the compelling need to establish a system for collecting evidence around a set of agreed indicators that substantiate value claims (Scott and Soren 2009)
Intrinsic- instrumental value conundrum
A pictorial model of evaluation for arts engagement
Finding a solution
A pictorial model of evaluation for arts engagement
To every public intervention, (at least) four lenses should be applied.
What is impact in terms of
- economic viability
- social inclusion
-environmental sustainability
- cultural vitality
A pictorial model of evaluation for arts engagement
Taking a multi-dimensional approach: Four Pillars of Sustainability (Hawkes 2001)
Image: Marla Guppy
A pictorial model of evaluation for arts engagement
A pictorial model of evaluation for arts engagement
But, ……………..
an emerging theme of enjoyment, pleasure,
where does it fit?
Themes of pleasure, fun, enjoyment
I loved working with Kallista, because she has something new for us (participant, 24 years)
(In attending) I was thinking about myself too. I wanted …. to have a good time- the show was funny
(foreign audience member)
My family loved this drama, they came to watch and they said, its really really good (participant 21 years)
A pictorial model for arts engagement
A pictorial model of evaluation for arts engagement
Role of pleasure in arts engagement
Cook Islanders view the main, and very important purpose of dancing, as creating pleasure or happiness, relating to a world view that the purpose of life is to have fun, anga’anga tamataora, literally ‘to work pleasure’ (anthropologist Alexeyeff, 2009)
‘Fun’ is a major motivator and outcome of participation in dance programs at school and in community settings for young American and Australian children (educational researchers Bond and Stinson, 2001, 2007)
Maori migrants In Melbourne indicated that ‘fun’ was a major motivator for participation in cultural performing groups
(community cultural development researcher Dunphy, 1993) Young people in northern Ireland were motivated by enjoyment of community music making, to engage in programs about peace-building, through which peace-building objectives of host organisation were achieved
(peace-building researcher Pruitt, 2011)
Most commonly mentioned (36%) personal benefits of attending or participating in arts activities include ‘entertainment or fun’ (Ontario Arts Council, 2010)
Taking a multi-dimensional approach: CIV’s five domains
• Healthy Safe and Inclusive Communities• Dynamic Resilient Local Economies• Sustainable Built and Natural Environments• Culturally Rich and Vibrant Communities• Democratic and Engaged Communities
A pictorial model of evaluation for arts engagement
Economic dimension
Civic engagementdimension
Cultural dimensionSocial
dimension
Environmental dimension
PersonalWellbeing
Personal/spiritual
wellbeingdimension
• spiritual • Historic (Throsby 2001, Holden 2006)• expression of communal meanings.
Reclassifying all outcomes of arts engagement into these dimensions
A pictorial model of evaluation for arts engagement
Taking a multi-dimensional approach: Six dimensions of integrated community development (Ife 2002)
• capacity for empathy (McCarthy 2004)• friendship• belonging/ relatedness/ sense of community• safety• social capital- bonding and bridging
Social
dimension
Economic dimension
•skill development •employment•business development•increased wealth
A pictorial model of evaluation for arts engagement
•sense of place•neighbourhood character•connection to the natural world• air quality•water quality•bio-diversity social
•capacity for empathy (McCarthy 2004)•friendship•belonging/ relatedness/ sense of community•safety•social capital- bonding and bridging
environmental
A pictorial model of evaluation for arts engagement
Reframing much of the ‘intrinsic’•aesthetic•symbolic•authenticity •cognitive growth•historic (Throsby 2001, Holden 2006)•expression of communal meanings.•cultural capital- education (Bourdieu) (McCarthy et al 2004)•identity affirmation•respect for diversity
•sense of place•neighbourhood character•connection to the natural world• air quality•water quality•bio-diversity
environmental
cultural
A pictorial model of evaluation for arts engagement
Reframing much of the ‘intrinsic’•cognitive growth•Aesthetic•symbolic•authenticity•spiritual •Historic (Throsby 2001, Holden 2006)•expression of communal meanings.•cultural capital- education (Bourdieu) (McCarthy et al 2004)•identity affirmation•respect for diversity
•active citizenship•membership of local organisations and decision-making bodies•having a say on important issues•engagement in political process
cultural
civic engagement
A pictorial model of evaluation for arts engagement
Personal/spiritualWellbeing
• pleasure, fun• subjective well-being• self-reported health• psychological distress
Civic engagement • pleasure, fun
• subjective well-being• self-reported health• psychological distress
A pictorial model of evaluation for arts engagement
EconomicViability
CivicEngagement
CulturalVitality
SocialInclusion
EnvironmentalResponsibility
PersonalWellbeing
autonomy
autonomyRelatedness
Relatedness
autonomy
autonomyautonom
yautonom
y
Pleasure
autonomyautonomy
auto
nom
yau
tono
my
Taking a multi-dimensional approach: Six dimensions of integrated community development (Ife 2002)
A pictorial model of evaluation for arts engagement
Increased access to economic resources•skill development •employment•business development•increased wealth
EconomicViability
CivicEngagement
CulturalVitality
SocialInclusion
EnvironmentalResponsibility
PersonalWellbeing
autonomy
autonomyRelatedness
Relatedness
autonomy
autonomyautonom
yautonom
y
Pleasure
autonomyautonomy
auto
nom
yau
tono
my
A pictorial model of evaluation for arts engagement
Considering different perspectives
Different stakeholders
•Beneficiaries•Fieldworkers•Project co-ordinators•Manager•Host organisations•Funders•Government
Most Significant Change(Davies and Dart 2005)
Olaf Breuning (2009), Different Perspectives
Considering outcomes for all stakeholders in arts engagement.(Most Significant Change, Davies and Dart, 2005)
ParticipantsParticipants
Audience MembersAudience Members
Artistic Leader/sArtistic Leader/s
Host OrganisationHost Organisation
FundersFunders
Wider CommunityWider Community
A pictorial model of evaluation for arts engagement
ParticipantsParticipants
Audience MembersAudience Members
Artistic Leader/sArtistic Leader/s
Host OrganisationHost Organisation
Funders
Wider CommunityWider CommunityFamilies and friends not attending
Unfunded activity
Peace-building organisation Ba Futuru
Visiting theatre director, youth co-ordinator, young menteesAudiences of local people; audiences of foreigners
Young performers …
Pride in young person’s achievements
the event was ‘a brilliant PR tool’
that night was a highlight of my life- to see the audience’s response.
I think this show was very important, [....] for human beings to express their feelings
We can [....] change our bad thinking [....] and experience from the past’.
Considering outcomes for all stakeholders in Scared Cool youth theatre project
A pictorial model of evaluation for arts engagement
A pictorial model of evaluation for arts engagement
Intended and unintended outcomes
A pictorial model of evaluation for arts engagement
Intended outcomesBa Futuru’s peace-building mission:
- recovery of children and youth from the distress caused by civil strife in order to reduce levels of violence.
- arts included because of the connection between participatory arts experiences and positive development.
A pictorial model of evaluation for arts engagement
Intended outcomesBa Futuru’s peace-building mission:
- recovery of children and youth from the distress caused by civil strife in order to reduce levels of violence.
- arts included because of the connection between participatory arts experiences and positive development.
A pictorial model of evaluation for arts engagement
Unintended outcomes
Participant Felis:Learning English is important for my future
A pictorial model of evaluation for arts engagement
Positive/ negative outcomes
A pictorial model of evaluation for arts engagement
‘Because the foreigners teach us, that's why I wanted to attend. If the teacher was Timorese, I would not have done this program’. (Participant, 18 years)
‘I am very comfortable when I am with malae (foreigners), because they can understand me. When I ask something, they always answer nicely. When I ask for some help, they will always help me. But East Timor people [….] they never try to analyse what is going on, they just stop in the short term’. (Participant, 19 years)
Positive/ negative outcomes
A pictorial model of evaluation for arts engagement
Positive/ negative outcomes
A pictorial model of evaluation for arts engagement
Unintended
Intended
A pictorial model of evaluation for arts engagement
Depicting intended/unintended; positive/negative outcomes
Expected and unexpected outcomes:
• Comparing between what org expected would happen and what actually occurred
A pictorial model of evaluation for arts engagement
Length of time of change
• Considering short, medium and long term change
A pictorial model of evaluation for arts engagement
Return on Investment
• Comparing what was invested- ‘cost’:• Culturally, economically, socially, personal
wellbeing, civically, environmental
• With ‘benefit’- outcomes in all dimensions
A pictorial model of evaluation for arts engagement
Unintended
Intended
A pictorial model of evaluation for arts engagement
Scared Cool theatre project: outcomes for participants
EconomicViability
CivicEngagement
CulturalVitality
SocialInclusion
EnvironmentalResponsibility
PersonalWellbeing
A pictorial model of evaluation for arts engagement
Unintended
Intended
A pictorial model of evaluation for arts engagement
Depicting intended/unintended; positive/negative outcomes
Considering outcomes for all stakeholders in arts engagement.(Most Significant Change, Davies and Dart, 2005)
ParticipantsParticipants
Audience MembersAudience Members
Artistic Leader/sArtistic Leader/s
Host OrganisationHost Organisation
FundersFunders
Wider CommunityWider Community
A pictorial model of evaluation for arts engagement
Challenges
• Timeframe for evaluation- does the engagement have an impact (short-term) or outcome (longer term)?
• Does reported change lead to actual change?
• Attribution of outcomes, given lack of ‘control’?
A pictorial model of evaluation for arts engagement
In concluding
Thinking beyond intrinsic-instrumentalto
a multi-dimensional, multi-perspective framework
for understanding outcomes of arts engagement
A pictorial model of evaluation for arts engagement