journal of community engagement and scholarship | vol. 9, no. 1

Volume 9, Number 1Spring 2016

Advancing Innovation in Newfoundland and Labrador: Insights for Knowledge Mobilization and University-Community Engagement 9

The Co-Produced Pathway to Impact Describes Knowledge Mobilization Processes Page 31

Volunteers Needed: Bridging Latino Immigrants and Local Communities Through Service Learning and Critical Analytic Practice Ethnography Page 41

Building Civic Capacity for College Students: Flexible Thinking and Communicating as Puppeteers, Community Partners, and Citizen-Leaders Page 46

COMMUNITY VOICES Page 76

STUDENT VOICES Page 79

BOOK REVIEWS Page 83

Outreach and Engagement in the Consortium for Institutional Cooperation

Promotion and Tenure Policies pg. 8

SEE YOU ON OUR CAMPUS!(otherwise known as Omaha)

17TH ANNUAL CONFERENCE OF THE

ENGAGEMENT SCHOLARSHIP CONSORTIUM

VISIONING THE FUTURE OF ENGAGED SCHOLARSHIPUNIVERSIT Y OF NEBR A SK A AT OMAHA

PRE-CONFERENCE | OCTOBER 9-10, 2016 CONFERENCE | OCTOBER 11-12, 2016

0576ADUC0316

engagementscholarship.org/conference | @EngageSchlrship

HOSTED BY THE NORTH CENTRAL REGION

Vol. 8, No. 2 —JOURNAL OF COMMUNITY ENGAGEMENT AND SCHOLARSHIP

Copyright © 2016 by The University of Alabama Press. All rights reserved. ISSN 1944-1207. Printed by University Printing at The University of Alabama.

Vol. 9, No. 1 —JOURNAL OF COMMUNITY ENGAGEMENT AND SCHOLARSHIP—Page 2

Marsha H. Adams, Ph.D. Dean, College of Nursing, The University of Alabama-Huntsville

Andrea Adolph, Ph.D.Director of Academic Affairs andAssociate Professor of English andWomen’s Studies, Department ofWomen’s, Gender, and SexualityStudies, Penn State – New Kensington

Katrice A. Albert, Ph.D.Vice President for Equity and Diversity,University of Minnesota

Jessica Averitt-Taylor, Ph.D.Assistant Professor, Department of Counseling, Social Work, and Leadership, Northern Kentucky University

Anna Sims Bartel, Ph.D. Associate Director of Engaged Learning and Research, Cornell University

Alice M. Buchanan, Ph.D. Associate Professor, School of Kinesiology, College of Education, Auburn University

Delicia Carey, Ph.D.Statistician, Centers for Disease Control and Prevention, Atlanta, Georgia

Doyle Carter, Ph.D.Professor and Director of the QualityEnhancement Plan/Center forCommunity Engagement, Department of Kinesiology, Angelo State University

Richard L. Conville, Ph.D. (retired)Professor of Communication Studies, The University of Southern Mississippi

Virginia Cope, Ph.D. Assistant Dean and Associate Professor, Department of English, Ohio State University-Newark

Susan Curtis, Ph.D.Professor, School of Interdisciplinary Studies, Purdue University

Mary Elizabeth Curtner-Smith, Ph.D.Associate Professor, Department of Human Development and Family Studies, College of Human Environmental Sciences,The University of Alabama

Sandra De Urioste-Stone, Ph.D. Assistant Professor, Center for Research on Sustainable Forests, School of Forest Resources, The University of Maine

David J. Edelman, Ph.D.Professor, Master of Community Planning Coordinator,Urban and Regional Planning,University of Cincinnati

Hiram E. Fitzgerald, Ph.D.Associate Provost, University Outreach and Engagement, and Professor, Department of Psychology,Michigan State University

Nancy Franz, Ph.D. Professor Emeritus, School of Education, Iowa State University

Philip A. Greasley, Ph.D.Associate Provost of University Engagement, University of Kentucky

Sulina Green, Ph.D.Professor, Department of Social Work, University of Stellenbosch, South Africa

Valerie Holton, Ph.D.Director of Community-Engaged Research, Division of Community Engagement, Virginia Commonwealth University

Susan S. Jakes, Ph.D.Extension Assistant Professor and Associate State Program Leader for Community Development,North Carolina State University

Phillip W. Johnson, Ph.D.Associate Professor, Center for Sustainable Infrastructure and Department of Civil, Construction, and Environmental Engineering, College of Engineering, The University of Alabama

Kimberly L. King-Jupiter, Ph.D.Dean and Professor, College of Education, Tennessee State University

William Kisaalita, Ph.D.Professor, College of Engineering,University of Georgia

Monica Kowal, Ph.D. Strategic Planner for Community Engagement, Office of the Provost and Vice President for Academic Affairs, and Lecturer, Department of English Literature and Language, University of New Mexico

J. Robert Krueger, Ph.D.Associate Professor and Director of Environmental and Sustainability Studies; Co-director, London Project Center, and Director, Worcester Community Project Center, Worcester Polytechnic Institute

James Leeper, Ph.D.Professor, Community and Rural Medicine, College of Community Health Sciences, The University of Alabama

Robert C. Liebman, Ph.D.Professor, Department of Sociology, Portland State University

PublisherDr. Samory T. PruittVice President for CommunityAffairs, The University of Alabama

EditorDr. Nick SanyalUniversity of Idaho

Associate EditorDr. Marybeth LimaLouisiana State University

Book Review EditorDr. Andrew J. PearlUniversity of North Georgia

Associate Editor,Student and CommunityKatherine BrunaIowa State University

Editorial AssistantDr. Edward MullinsThe University of Alabama

Assistant to the EditorVicky CarterThe University of Alabama

Production Editor/ Web ProducerKaryn BowenThe University of Alabama

EDITORIAL BOARD


Antoinette Lombard, Ph.D.Professor and Head of the Department of Social Work and Criminology, University of Pretoria, South Africa

Robert L. Miller, Jr., Ph.D. Associate Professor, School of Social Welfare, University of Albany-SUNY

Mary Ann Murphy, Ph.D. Professor, Dyson College of Arts and Sciences, Department of Communication Studies, Pace University

dt ogilvie, Ph.D.Professor, Center for Urban Entrepreneurship, Saunders College of Business, Rochester Institute of Technology

Jacob Oludoye Oluwoye, Ph.D.Associate Professor, Community Planning and Urban Studies, Alabama A&M University

Michael E. Orok, Ph.D.Dean, School of Humanities and Social Sciences, Virginia Union University

Josephine Pryce, Ph.D.Associate Professor, School of Social Work, The University of Alabama

Scott Reed, Ph.D.Vice Provost, Division of Outreach and Engagement, Oregon State University

Howard B. Rosing, Ph.D.Executive Director of the Irwin W. Steans Center for Community-Based Service-learning and Community Service Studies, DePaul University

Sunil Saigal, Ph.D.Professor, Department of Civil and Environmental Engineering, New Jersey Institute of Technology

Amilcar Shabazz, Ph.D.Professor, Department of Afro-American Studies, University of Massachusetts-Amherst

Timothy Shaffer, Ph.D. Assistant Professor, Department of Communication Studies, Kansas State University

Cassandra E. Simon, Ph.D.Associate Professor, School of Social Work, The University of Alabama

L. Steven Smutko, Ph.D.Professor and Spicer Chair of Collaborative Practice, Haub School of Environment and Natural Resources, Department of Agricultural and Applied Economics, University of Wyoming

Kim L. Wilson, Ph.D.Professor and Director of the Landscape Architecture Program and Community and Regional Planning, College of Architecture, University of Nebraska-Lincoln

The Journal of Community Engagement and Scholarship is published at The University of Alabama by the Division of Community Affairs to advance the scholarship of engagement worldwide. To reach the editor, send an email to [email protected] or call 205-348-7392. See also www.jces.ua.edu.

EDITORIAL BOARD (CONTINUED)


Contents

Message from the EditorNick Sanyal Page 5

Message from the Associate EditorMarybeth Lima Page 7

Are We There Yet? Outreach and Engagement in the Consortium for Institutional Cooperation Promotion and Tenure Policies Page 8

Diane M. Doberneck

Advancing Innovation in Newfoundland and Labrador: Insights for Knowledge Mobilization and University-Community Engagement Page 19

Heather M. Hall, Jacqueline Walsh, Rob Greenwood, and Kelly Vodden

The Co-Produced Pathway to Impact Describes Knowledge Mobilization Processes Page 31

David Phipps, Joanne Cummings, Debra Pepler,Wendy Craig,

and Shelley Cardinal

Volunteers Needed: Bridging Latino Immigrants and Local Communities Through Service-learning and Critical Analytic Practice Ethnography Page 41

Diana M. Ruggiero

Building Civic Capacity for College Students: Flexible Thinking and Communicating as Puppeteers, Community Partners, and Citizen-Leaders Page 46

Deborah Thomson, Rebecca Dumlao, and John Howard

Cultural Adaptation of a Substance Abuse Prevention Program as a Catalyst for Community Change Page 57

Melissa Tremblay, Lola Baydala, Natasha Rabbit, Jennilee Louis, and Kisikaw Ksay-yin

A Community-University Approach to Substance Abuse Prevention 7

Lola Baydala, Fay Fletcher, Melissa Tremblay, Natasha Rabbit, Jennilee Louis,

Kisikaw Ksay-yin, and Caitlin Sinclair

COMMUNITY PERSPECTIVES Page 77

STUDENT VOICES Page 80

BOOK REVIEWS Page 84

INSTRUCTIONS TO AUTHORS/MANUSCRIPT PREPARATION Page 93


Nick Sanyal, Ph.D.From the Editor

In less than a decade JCES has moved into a position of excellence and now serves as one of the flagship publications in the field of community engagement scholarship. We have extended our reach beyond North America to include international contributors and readers, and our articles represent diverse disciplines—from social work to nursing, from urban design to civil engineering, and from natural resources to medicine.

I have been asked to step up and continue the exemplary lead of Dr. Cassie Simon, under whom I have been privileged to serve as JCES associate editor. This not only allows me to further serve a field that I am passionate about, but will also indeed be a special privilege to serve with and learn from the best—the JCES editorial team is second to none! JCES was founded as a “new kind of research journal.” We will continue to evolve, mature, and transform as we explore the challenges and opportunities of the day.

While we must continue to innovate as we reward good scholarship, champion bold applications of knowledge, and lend our support as credible experts as we focus on understanding the myriad of challenges that face society at home and around the globe, we must also remain rooted to our primary cause: “disseminating the scholarly works on community engagement that would promote the integration of teaching, research, and outreach.”

As editor, any change I foster will be incremental, spawned as a consequence of the interaction between our contributors, editors, advisory board, and staff. Community engagement is all about becoming better citizens. I see JCES as a way to transform that experiential learning and to create great scholarship and contribute to the scientific, artistic, and humanistic basis of our engagement endeavors.

An early goal I have is to see more first-time authors, students, and community partners published in JCES. These are the people who will carry JCES and its message into the future, and these are the people who are often disenfranchised by

traditional refereed scholarly publications. As an associate editor, board member, and reviewer I read many compelling manuscripts that simply needed enhancements to structure and organization to better convey their story to make a significant contribution to the scholarship of engagement. We must be the journal that provides such space and encouragement. By enhancing the connections between existing knowledge and practice, we can expand the transference of their knowledge and become a more inclusive platform for a larger community of scholars.

As a member of a community of scholars working in a world of practitioners, I look forward to a rewarding, challenging, and productive tenure as editor, and I am thrilled to be working with exceptionally talented and always helpful staff at the University of Alabama and elsewhere in the JCES world.

Some other changes are also taking place at JCES: Dr. Marybeth Lima, professor of Biological and Agricultural Engineering at Louisiana State University, where she serves as director of the Center for Community Engagement, Learning, and Leadership, has agreed to become our new associate editor. She was the winner of the 2007

An Early Goal: Finding Ways to Include More First-Time, Student, and Community Authors in JCES

Dr. Nick Sanyal, Professor of Natural Resources and Society, University of Idaho


Dr. Cassandra E. Simon accepts an award for her work as founding editor of the Journal of Community Engagement and Scholarship at Penn State in September 2015 during the annual Engagement Scholarship Consortium (ESC) Conference. Simon was JCES editor from 2008 until 2015. ESC is a sponsor of the journal.

Thomas Ehrlich Faculty Award for service-learning at LSU, has an extensive background in community engagement, and has been a most effective and active board member. Welcome!

At the University of Alabama, Dr. Ed Mullins has stepped down as our production editor, and Ms. Karyn Bowen has agreed to step in and fill his big shoes. Dr. Mullins will remain on staff as an editorial assistant so we will continue to benefit from his sage and practical advice. I’d also like to thank Dr. Samory Pruitt for his faith and persuasion; Assistant to the Editor Vicky Carter, a UA doctoral student in Social Work, for her unparalleled ability to “herd cats”; and to our new and continuing Editorial Board members and the many other reviewers, who share so freely of their time.

Finally, thanks once again, Cassie, for the leadership, guidance, and friendship that you have offered me. I hope I can continue to earn your trust and deliver your vision.


I still remember the time I held in my hands the first journal article that had I co-authored. It was 1988 and I was a senior in college—the paper was based on work I had completed during a six-month, full-time research internship at

Oak Ridge National Laboratory the previous year. I remember feeling very proud because the paper represented the culmination of a ton of effort, starting with executing a set of somewhat tedious experiments (they didn’t start out being tedious, but between making mistakes and verification experiments, they got tedious), followed by thoughtful analysis, and then writing up the results in the form of a manuscript. My research paper went through at least ten drafts—the back and forth writing and critiquing process between me and my research mentor was my first exposure to writing toward a standard of excellence instead of “the good old college try,” which had been sufficient to that point in my career. The paper had to be strong enough for submission to the Biochemical Journal, which my mentor informed me was “a good journal,” whatever that meant—I had no scale against which to judge, so I took his word for it.

As I reflect on that moment 26 years and many publications later, I now have a strong grasp of what makes a journal article good. A good journal article tells a detailed story that no one has told before—it must be unique to be published. A good article builds on existing literature in some way. It provides context so that a reader can situate the work within a broad framework. And it provides perspective, and a jumping off point for future thought.

I also have a strong grasp of what makes a journal good. There are many tools today to judge the quality of journals that were not available in 1988. Generally, the lower the acceptance rate of a journal, the better quality it is. Another metric often used in STEM1 fields is the impact factor of a journal, the average number of citations per year for the articles published in that particular journal2—so a journal with an impact factor of less than one is considered lower quality than a journal like Science, which has an impact factor of 33.6. As an engineer, I place great stock in numbers. However, I also heed Einstein’s observation, “Not everything that counts can be counted, and not everything that can be counted counts.” As applied to the quality of journals, I believe that acceptance rates and impact factors can give you some information, but not even close to all of it. A good journal has a clear mission, a committed editorial staff, and a reasonable time from submission to publication. It fills a niche by adding critical literature to the field. When I read articles in a good journal, I feel informed, inspired, and reflective.

JCES is a good journal. Its mission is to provide readers with perspectives that integrate community engagement, teaching and learning, and research. Its editorial staff, current and former, is committed to producing a high-quality journal with a quick turnaround time for authors. JCES fills a niche by “lifting all voices” into the literature, with special attention paid to community partner and student voices that tend to be underrepresented in the broad field of engagement. As a long-time reader and reviewer of manuscripts for JCES, I have been privileged to learn many new things and to think about new perspectives. I was honored to be asked to become an associate editor and I look forward to working with the editorial staff, reviewers, and readers of JCES as it continues to grow, inform, and inspire.

Marybeth Lima, Ph.D.From the Associate Editor

1 Science, technology, engineering, and mathematics. 2 These are calculated on a two-year average, so they change periodically for the same journal.

New Associate Editor Joins JCES Team, Looks Forward to Helping Journal Continue “to Grow, Inform, and Inspire”


Are We There Yet?: Outreach and Engagement in the Consortium for Institutional Cooperation Promotion and Tenure Policies

Diane M. Doberneck

More than 20 years since Boyer’s Scholarship Reconsidered and 15 years since Diamond and Adam’s The Disciplines Speak I and II raised awareness about the many ways of defining, conducting, and rewarding scholarship, faculty members continue to cite institutional policies as a significant barrier to scholarly outreach and engagement (Wenger, Hawkins, & Seifer, 2012). Institutional alignment—that is, ensuring an institution’s espoused values are reinforced by its faculty roles and rewards system—continues to be a significant challenge for many institutions of higher education. When institutions of higher education initiate alignment processes on their campuses, reappointment, promotion, and tenure policies for faculty are often a main focus of attention and in some cases a point of contention. In Becoming an Engaged Campus: A Practical Guide for Institutionalizing Public Engagement, Beere, Votruba, and Wells (2011) note:

There is probably no issue as fundamental to institutionalizing public engagement as reappointment, promotion, and tenure (RPT)…. Faculty are unequivocal in their views: RPT policies must support public engagement in order for them and their colleagues—especially junior faculty—to invest significant time and energy in the work (p. 124).

Some higher education institutions have worked to address the institutional alignment

challenge by impaneling faculty and administrator committees to revise their RPT polices to better accommodate scholarly outreach and engagement. Little, however, is known about the nature and ex-tent of these RPT policies: What has been changed? Do the revised policies reflect the scholarship of engagement? How much progress has been made? In other words, are we there yet?

Approach to the StudyResearch Purpose and Questions

The purpose of this study was to understand how institutions recognize and encourage the reporting of scholarly outreach and engagement in their promotion and tenure policies. The researcher pursued the following research questions:

1. What language is used to describe faculty roles and responsibilities in general and outreach and engagement specifically?

2. What role, if any, may outreach and engagement play in faculty members’ promotion and tenure materials?

3. What types of scholarly activities are included in the description of faculty work to encourage the reporting of outreach and engagement?

4. What criteria, if any, are included in promotion and tenure policies to set expectations for quality and encourage excellence in outreach and engagement?

According to the CIC, engagement refers to the “partnership of university knowledge and resources with those of the public and private sectors to

AbstractMore than 20 years since Scholarship Reconsidered and 15 years since The Disciplines Speak raised

awareness about multiple ways of defining, conducting, and rewarding engaged scholarship, faculty members continue to cite institutional barriers to outreach and engagement scholarship. This qualitative study analyzed promotion and tenure policies from 15 Consortium for Institutional Cooperation (CIC) institutions. Thematic and content analysis focused on documents, including policies, instructions, forms, and templates, and followed a two-stage coding process guided by both extant theory and emergent discovery. The study revealed unexpectedly wide variations in language used to describe faculty work; types of examples included in the documents; the role of outreach and engagement in the promotion and tenure process; and criteria for assessing quality and excellence. No policy stood out as an exemplar, though many incorporated exemplary elements. Implications for policy and practice and directions for future research are included in the conclusion.


enrich scholarship, research, and creative activity; enhance curriculum, teaching, and learning; prepare educated, engaged citizens; strengthen democratic values and civic responsibility; address critical societal issues; and contribute to the public good” (CIC Council on Engagement, 2005, p. 2).

In this study, the phrase outreach and engage-ment was used because it acknowledges a broader range of collaborative arrangements with commu-nity partners than the term engagement (Dober-neck, Glass, & Schweitzer, 2012) and because it more accurately reflects the language used in the CIC’s institutional policies.

Purposive SampleThe CIC was selected as a purposive sample

for this qualitative, exploratory study (Kezel, 1999). Established in 1958 as the academic counterpart to the Big Ten Athletic Conference, the CIC focuses on collaborative research, teaching, purchasing, and technology agreements to amplify the impact of investments and ideas across the Consortium’s membership. Since 1958, the CIC has expanded from the original 10 to the 15 member institutions included in this study (Consortium on Institution-al Cooperation, 2015).

The CIC institutions were purposefully cho-sen for this exploratory study because the CIC Engagement Council was an early innovator in the engaged scholarship movement and author of Engaged Scholarship: A Resource Guide (2005), which compared institutional definitions; provid-ed examples of community-engaged scholarship in research, teaching, and service; and promoted the adoption of institutional benchmarks and outcome indicators. The CIC’s institutional characteristics and engagement commitments are summarized in Table 1.

Institutional characteristics. All CIC insti-tutions are classified as research universities/very high (RU/VH) by the Carnegie Foundation for the Advancement of Teaching; 87% are public univer-sities; 13% are private, not-for-profit universities; none are for-profit institutions; 60% have student enrollments over 40,000; 97% are members of the Association of American Universities (AAU); and 87% are members of the Association of Public and Land Grant Universities (APLU).

Institutional engagement commitments. Eighty percent of the CIC institutions are institu-tional members of Campus Compact; 67% have received the elective Carnegie Engagement Classi-fication; 53% are institutional members of Imag-ining America; 40% belong to The Research Uni-

versity Civic Engagement Network (TRUCEN); and 40% are Engagement Scholarship Consortium members.

Sources of DataThe researcher accessed promotion and tenure

documents, including policies, instructions, forms, templates, guiding documents, and frequently asked questions from the CIC institutions’ main human resources websites during the summer of 2013. Documents from these websites represent the institution’s official policy and are, therefore, deemed to be credible and trustworthy sources of data for this study (Whitt, 2001). Documents post-ed on division, college, departmental, or outreach and engagement websites were excluded from this study unless they appeared as links on the main hu-man resources webpage.

Data Coding and AnalysisBecause little is known about the presence

and role of outreach and engagement in the CIC’s promotion and tenure policies, the researcher employed an interdisciplinary bricolage approach to the inquiry instead of using an established conceptual framework in the analysis (Denzin & Lincoln, 2000). Bricolage is concerned with “diverse theoretical and philosophical understandings of various elements encountered during the research” (Kincheloe, 2001, p. 679). This approach allowed the researcher to explore the data thoroughly, drawing upon multiple and sometimes competing concepts from the engagement literature without limiting the exploratory analysis to a single scholarly perspective.

The researcher qualitatively analyzed the doc-uments using a two-stage process: (1) thematic analysis focused on identifying variations and pat-terns (Boyatzis, 1998), and (2) interpretive content analysis focused on determining frequencies in the identified patterns (Krippendorf, 2004). For each research question, a code sheet was developed that summarized significant concepts from the engage-ment literature related to the particular research question. Both nominal codes and absence/pres-ence codes were used in the data analysis (Boyatzis, 1998). Open-ended codes were also used to note emerging concepts relevant to the research ques-tions. If an emerging concept appeared more than one time, the researcher returned to the coded data and re-coded it in light of the newly discovered concept. Through this constant comparative ap-proach, the coding and analysis process incorpo-rated key concepts from the engagement literature


and remained open to the discovery of new ideas emerging from the data (Glaser, 1965; Glaser & Strauss, 1967). The researcher used multi-ple strategies to ensure quality and rigor in this study, including purposive sampling, the creation of an audit trail, code and re-code strategies, and researcher reflexivity (Anafara, Brown, & Mangi-one, 2002; Creswell & Miller, 2000).

ResultsQ1: What language is used to describe

faculty roles and responsibilities in general and for outreach and engagement specifically? For this research question, the researcher examined the promotion and tenure documents for the incorporation of Boyer’s language, including the scholarship of discovery, teaching, application, and integration (1990) and the scholarship of engagement

Table 1. CIC Institutional Characteristics and Engagement Commitments

University of Chicago

University of Illinois

Indiana University

University of Iowa

University of Maryland

University of Michigan

Michigan State University

University of Minnesota

University of Nebraska-Lincoln

Northwestern University

Ohio State University

Pennsylvania State University

Purdue University

Rutgers University

University of Wisconsin-Madison

CIC Institutions

Institutional

Characteristics Engagement Commitments

RU

/VH

Publ

ic

AAU

mem

ber

APL

U m

embe

r

Cam

pus

Con

tact

TRU

CEN

Imag

inin

g Am

eric

a

Enga

gem

ent

Sch

olar

ship

Con

sort

ium

Car

negi

e El

ectiv

e En

gage

men

t Cla

ssifi

catio

n

Enro

llmen

t ov

er40

,000

stu

dent

s


(1996); the Carnegie Community Engagement Classification definition (2015); and Imagining America’s continuum of scholarship (Ellison & Eatman, 2008). Analysis used absence/presence codes. Some institutions used more than one word to describe faculty responsibilities, so the reported numbers and percentages in this section exceed 15 or 100%. These findings are summarized in Table 2.

Analysis revealed that the majority of the institutions (73%) continued to use the word service in their policies. Of the 11 CIC institutions that used service, only four distinguished among service to the university, discipline, and community. Three of the 15 institutions used the word outreach in their policies. Three of the 15 institutions used the word engagement in their policies. Two institutions did not refer to service, outreach, engagement, or Extension as faculty responsibilities. One institution used the Kellogg Commission’s terms of discovery, learning, and engagement to describe faculty roles and responsibilities (Kellogg Commission on the Future of State and Land-Grant Universities, 2000). None of the promotion and tenure policies incorporated Boyer’s language, referenced the Carnegie Foundation’s definition, or mentioned Imagining America’s continuum of scholarship.

Q2: What role, if any, may outreach and engagement play in faculty members’ promotion and tenure materials? For this research question, the researcher coded the documents inductively and examined the policies for key ideas including outreach and engagement as responsibilities that cut across research, teaching, and service (CIC Council on Engagement, 2005).

Emerging from the analysis were four mutually exclusive nominal codes that characterized the role of outreach and engagement in the promotion and tenure process. In University of Chicago and Northwestern University policies, outreach and engagement was not recognized or encouraged during promotion and tenure. In University of Iowa and University of Minnesota policies, outreach and engagement was recognized as a form of faculty work but explicitly described as one subsidiary to the main faculty functions of research and teaching. This excerpt from University of Iowa’s policy exemplifies the recognized but subsidiary code:

Continued teaching and research excel-lence, and to a lesser extent the quality of other major professional contributions to the University or to society in general,

form the basis for salary increases as they do for promotion and tenure (Universi-ty of Iowa, 2004, p. 4)…. The criteria for promotion and tenure include teaching and research, and other forms of profes-sional contributions. Since teaching and research are the central focus of faculty, other professional contributions are con-sidered subsidiary to these fundamental tasks (University of Iowa, 2004, p. 12).

For the majority of institutions (53%), outreach and engagement were considered to be neither subsidiary nor privileged; faculty members could report scholarly outreach and engagement activities in the same way they would report traditional scholarship.

At the University of Illinois, Indiana University, and the University of Wisconsin-Madison, faculty members had the option of reporting outreach and engagement as their primary form of faculty work. This excerpt from University of Illinois policy typifies this code:

There are certain faculty roles for which the weighting of criteria for measuring excellence in research, teaching, and ser-vice may be appropriately different, such as in some forms of outreach and public engagement (University of Illinois at Ur-bana-Champaign, 2012, p. 6)…. Faculty members who are in positions that are primarily public engagement oriented should be evaluated with heavy weight on the quality of performance in the activi-ties provided. Activities should share the following characteristics: 1. They contribute to the public welfare

or the public good.2. They call upon the faculty member’s

academic or professional expertise.3. They directly address or respond to

societal problems, issues, interests or concerns (University of Illinois at Ur-bana-Champaign, 2012, p. 7).

Finally, analysis showed almost half of the CIC’s institutional policies (46%) framed outreach and engagement as a scholarly endeavor that cuts across faculty responsibilities in teaching, research, and service. This excerpt from Pennsylvania State University policies exemplifies this cross-cutting characterization of outreach and engagement:


Outreach activities should be properly documented and considered in the promotion and tenure process: Under service when they are mostly service, under teaching when they involve teaching, and under research and scholarship when they result in publication or activity that can be valued in those terms (Pennsylvania State University, 2012, p. 8).

Q3: What types of scholarly activities are included in the description of faculty work to encourage the reporting of outreach and engagement? For this research question, the researcher analyzed the documents using the Typology of Publicly Engaged Scholarship (Doberneck, Glass, & Schweitzer, 2010); Bringle and Hatcher’s (1996) definition and description of service-learning; and Imagining America’s Figure



Indiana University

University of Iowa




Universityof Minnesota

Universityof Nebraska-Lincoln




Purdue University

Rutgers University


Language Describing Faculty Roles and Responsibilities

Research

Research

Teaching

Teaching

Teaching

Instruction

CIC Institutions

Contribution to the Intellectual Community

Service and Scholarship of Service to the University,

Society, and the Profession

Engagement—Engagement, including Outreach and Extension

Service (Public Engagement, Profession/Disciplinary, University)

Resident Instruction

Service/Engagement

Service within Academic and Broader Community

Outreach, Service, or Administration

Scholarship of Teaching and Learning

Publications and Creative Work

Research/Creative Activity

Research andCreative Activity

Research/Creative Activity

Research/Creative Work

Research, Scholarship, or Creative Activity

Scholarship of Research and Creative

Accomplishments

Discovery—Creative Endeavor, Research,

Scholarship

Learning—Teaching and Learning

Teaching, Advising, and Mentoring

Scholarship

Scholarship

Service

Service

Teaching Service

Scholarship Teaching Service

Research Teaching

Teaching

Teaching

Service

Research Teaching Service, Outreach

Service

Table 2. Language Describing Faculty Roles and Responsibilities


Eight concept describing academic and public products (Ellison & Eatman, 2008). Analysis used absence/presence codes followed by the grouping of institutions by the number of types of examples of outreach and engagement mentioned in the policies.

Analysis revealed that the University of Chicago, the University of Nebraska-Lincoln, and Northwestern University did not mention any types of outreach and engagement in their policies. Indiana University, the University of Iowa, the University of Minnesota, and Rutgers University included between one and four examples. The University of Maryland, the University of Michigan, and The Ohio State University included between five and eight examples. The University of Illinois, the Michigan State University, Pennsylvania State University, Purdue University and the University of Wisconsin-Madison included nine or more examples of outreach and engagement in their policies. The outreach and engagement activities most often mentioned were non-credit conferences, seminars, or workshops; professional short courses, certificates, or continuing education; educational materials to promote public understanding; and clinical, diagnostic, and patient care.

Only Pennsylvania State University and the University of Michigan mentioned service-learning specifically. Indiana University and the University of Michigan included special instructions to administrators and promotion and tenure committee members to consider collaborative scholarship, new scholarly communications, impact on diverse communities, and interdisciplinary or entrepreneurial forms of scholarship that may differ from traditional scholarship. This excerpt from the University of Michigan’s memo exemplifies these special instructions:

I encourage you to give full recognition, both in evaluating tenure and promotion cases, and in considering faculty annual activities reports, to the broad range of entrepreneurial, outreach, and creative activities in which faculty engage. These activities may enhance any of the criteria on which faculty are measured—teaching, research, and service. They may include involvement with other sectors of a sort that has not traditionally been considered in faculty evaluations, or they may include creative activity that does not take the form of traditional scholarship. Examples include:

1. Creating service-learning and action-based learning opportunities for students.

2. Creating new instructional methods.3. Engaging in community-based re-

search.4. Engaging in research funded by

industrial, non-profit, or other non-federal or foundation sources.

5. Creating a start-up company that en-hances the broader scholarly, public service, or health care missions of the university.

6. Engaging in creative performance.7. Creating new or enhanced practices,

products, or services.8. Working with the Office of Technol-

ogy Transfer to patent or license an invention.

9. Encouraging and instructing stu-dents in entrepreneurial and public service activities.

10. Developing collaborative approaches to solving complex world problems (Hanlon, 2012, p. 1).

Q4: What criteria, if any, are included in pro-motion and tenure policies to set expectations for quality and encourage excellence in outreach and engagement? For this research question, the researcher examined the promotion and tenure documents for the inclusion of quality and excel-lence criteria from Lynton (1995), Michigan State University’s Points of Distinction (1996); Glassick, Huber, & Maeroff (1997), and the Communi-ty-Campus Partnerships for Health (CCPH) ex-panded Glassick list of criteria (Jordan, 2007). The documents were also analyzed for references to the National Review Board for the Scholarship of En-gagement (a national body that peer reviews fac-ulty promotion and tenure dossiers), CES4Health (a national body that peer reviews scholarly com-munity-engaged products created for public audi-ences), and the inclusion of community partners as peers in the promotion and tenure review process (Freeman, Gust, & Alsohen, 2009; Gelmon, Jordan, & Seifer, 2013). These findings are summarized in Table 3.

Analysis revealed that two-thirds of the CIC institutions made no reference to criteria for tra-ditional scholarship or outreach and engagement in their policies. The University of Illinois, Indiana University, the University of Iowa, Michigan State University, and the University of Nebraska-Lincoln


included criteria for quality and excellence for all types of faculty scholarship. The University of Illinois and Indiana University includ-ed additional evaluation criteria for outreach and engagement. This excerpt from Indiana University’s policy exemplifies these additional criteria:

Evaluations of research can never be reduced to a simple metric: judgments about the quality of work, and its influence, impact, util-ity, and creativity can-not be fully captured by the count of publi-cations and citations or by a journal impact factor (Indiana Uni-versity, 2013, p. 6)….Excellence in Service/Engagement: Candi-dates seeking tenure and/or promotion on the basis of Excellence in Service/Engagement must provide evidence for national/interna-tional visibility and stature resulting from service activities (even abundant local com-mittee work is insuf-ficient). The key is to demonstrate that the candidate’s efforts have been sustained and transformative, for a professional association, government agency, or non-academic community (Indiana University, 2013, p. 8).

None of the promotion and tenure policies refer to criteria promoted by Lynton (1995), Glassick, Huber, and Maeroff (1997), or the CCPH’s expanded Glassick list of criteria (Jordan, 2007). Only Michigan State University included all four criteria from Points of Distinction (e.g., significance, impact, scholarship, and context), though three additional

institutions included one or more of those criteria in their policies. None of the policies mentioned the National Review Board for the Scholarship of Engagement or CES4Health as options for external review of the dossier or products, respectively. Even though the University of Illinois policy includes this statement about the review of public engagement, “There are some public engagement activities…[that] should be evaluated thoroughly by both inside and outside evaluators” (University of Illinois at Urbana-Champaign, 2012, p. 7), none of the CIC’s policies advocated for the inclusion of



Indiana University

University of Iowa




Universityof Minnesota

Universityof Nebraska-Lincoln




Purdue University

Rutgers University


Evaluation Criteria in RPT PoliciesCIC

Institutions

Quality, Quantity, Effectiveness

Scholarship, Significance, Impact, Context

None None

None None

None None

None None

None None

None None

None None

None None

None None

None None

Quality, Impact, Significance None

None

None

Excellence, Innovation, Impact, Scholarship

Quality, Influence, Impact, Utility, Creativity

Quality; Impact; Dissemination or Adoption; If appropriate, integration with research and teaching

National and International Visibility (abundant local activity is insufficient);

Sustained and transformative activities.

For all scholarship

For engaged scholarship

Table 3. Evaluation Criteria for All Scholarship and For Outreach and Engagement


community partners as peers in the promotion and tenure review process. Discussion

This study revealed unexpectedly wide variations in how CIC institutions recognize and encourage outreach and engagement in their promotion and tenure policies. There was little congruence in language used to describe faculty roles and responsibilities or in the role of outreach and engagement in promotion and tenure review processes. In addition, types of outreach and engagement included as examples in the policies varied. Very few policies included standards for quality and excellence.

Seventy-three percent of the CIC institutions permitted the reporting of outreach and engagement as either a primary focus or a part of a faculty member’s promotion and tenure materials. This is in keeping with recent observations that the engagement movement has made advancements—namely, there is more of it and structures are in place to support it (O’Meara, 2011). Despite this recognition, the majority of the CIC insti-tutions continue to use the word service instead of engagement in their policies, and of those using service, only a few make the critical distinction among service to the university, the discipline, and the community.

Are we there yet? Not quite. While the CIC institutions have made some progress in revising their policies, there is still a long way to go to fully align promotion and tenure policies to encourage and support scholarly outreach and engagement. No institutional policy clearly stands out as an exemplar; however, many policies have strong elements that are noteworthy, including:

• Pennsylvania State University policy defines all forms of faculty work as scholarly endeavors and sets forth faculty expectations in these terms: the scholar-ship of teaching/learning; the scholarship of research and creative accomplishment; and the scholarship of service to univer-sity, society, and profession (Pennsylvania State University, 2012, p. 3).

• Indiana University and University of Michigan policies acknowledge newer forms of scholarship and encourage reviewers to consider them. Purdue University recognizes a variety of achieve-ment and the diversity of academic enterprise at a land grant institution (Purdue University, 2012, p. 1).

• University of Illinois, Indiana University, and University of Wisconsin-Madison policies allow faculty to report outreach and engagement as their primary form of faculty work.

• Michigan State University’s policy en-courages faculty members to indicate peer reviewed publishing (p. 11) and con-tracts and grant awards (p. 15) that have outreach and engagement components with asterisks (Michigan State University 2012a, 2012b, 2012c).

• Rutgers University’s policy makes distinc-tions between expectations for county agents and Extension specialists at differ-ent levels of review (Rutgers University, 2013a, 2013b, Form 1-C, Form 1-D, Form II-3, Form II-4) and promotes detailed re-porting of non-credit instructional activi-ties by requiring descriptions of program, title, duration, audience, enrollment, eval-uation/teaching effectiveness, and evalua-tion/program content (Rutgers Universi-ty, 2013b, Form 1-C, page 3).

• University of Illinois and Indiana Univer-sity policies include specific evaluation criteria for outreach and engagement (University of Illinois at Urbana-Cham-paign, 2012, p. 12; Indiana University, 2013, p. 8).

• University of Iowa’s policy encourages post-tenure allocation of effort to focus on learning, quality, and responsibility, with an expanded view of service, administration, and outreach (University of Iowa, 1994, p. 15).

Limitations and Future Research DirectionsThis study was exploratory, with a small,

purposive sample that provided limited analysis of outreach and engagement in institutional policies. Because institutional characteristics influence faculty members’ participation in outreach and engagement (Colbeck & Wharton-Michael, 2006; O’Meara & Rice, 2005; Thornton & Jaeger, 2008; Wade & Demb, 2009), future research could build upon this study by analyzing policies from different institutional groupings or by expanding the sample size. Second, there is growing evidence that disciplines influence how faculty members frame, collaborate, and execute outreach and engagement (Buzinsk, Dean, Donofrio, Berger, Heighton, Selvi, & Stoecker, 2013; Doberneck, Glass, & Schweitzer, 2012; Kreber, 2009,;


Volgelgesang, Denson, & Jayakumar (2010), Wade & Demb (2009)). Future studies could analyze division, college, or departmental level policies to better understand how disciplinary expressions of outreach and engagement are manifest in mid-level institutional promotion and tenure policies. Third, the analysis of institutional policy is by definition a study of formal structures within institutions. In contrast with official written policies, there is some evidence to suggest faculty, especially junior ones, are informally counseled to under-report their outreach and engagement during promotion and tenure reviews (Ellison & Eatman, 2008). Future research could focus on better understanding of the informal structures and messages (e.g., hallway conversations, advice from mentors, feedback on draft dossiers) faculty receive about whether and how to include outreach and engagement in their promotion and tenure materials. Related research may also explore the gap between stated institutional policy and actual implementation by RPT committees and identify successful strategies for closing the policy-implementation gap. This is vitally important if future faculty from Generation X, Generation Y, and Millennials are to be recruited, retained, promoted, and tenured at institutions of higher education (Janke, Medlin, & Holland, 2013). Finally, institutional policies supporting outreach and engagement are necessary but not sufficient for changing institutional cultures. Future studies could determine effective ways to build capacity for mentors, promotion and tenure committee members, and external reviewers to provide critical and constructive feedback to faculty about their outreach and engagement across all phases of their careers—from hiring, annual reviews, promotion and tenure, and post-tenure review (Beere, Votruba, & Wells, 2011). National organizations, including the National Review Board for the Scholarship of Engagement and CES4Health.info are national mechanisms for peer review of reappointment, promotion, and tenure documents and scholarly products for public audiences, respectively. These organizations are important, but additional capacity building efforts to strengthen peer review of community engaged scholarship on college and university campuses are needed as well.

Implications for Policy and PracticeThis study’s findings point to a significant gap

between the language promoted by community engagement leaders and those making institutional policies. Closing this rhetoric-policy gap should be a priority for leaders of the community engagement

movement. Second, key concepts from the scholarship of engagement, such as Imagining America’s continuum of scholarship or Figure Eight concept (2008) and CCPH’s expanded Glassick criteria (Jordan, 2007) are absent from all CIC’s promotion and tenure policies. National level organizations need to make more effort to include key concepts from these nationally recognized, evidence-based studies about outreach and engagement in institutional promotion and tenure policies. Third, outstanding examples of scholarly outreach and engagement in reappointment, promotion, and tenure should be identified, celebrated, and shared broadly, so that they may serve as exemplars and counteract the shared narrative about the difficulty of outreach and engagement in promotion and tenure. In addition, more effort needs to be made to identify successful faculty scholars and to encourage them to serve as mentors for pre-tenure faculty. Fourth, many of the CIC’s promotion and tenure policies have strong elements supportive of outreach and engagement as valued faculty work. Because cooperation is one of the CIC’s strengths, fostering cross-institutional dialogue about outreach and engagement in promotion and tenure policies could significantly strengthen each institution’s policies and establish a more supportive culture for outreach and engagement throughout its member institutions.

ReferencesAnfara, V.A., Jr., Brown, K.M., & Mangione,

T.L. (2002). Qualitative analysis on stage: Making the research process more public. Educational Researcher 31(7), 28–38.

Boyatzis, R.E. (1998). Transforming qualitative information: Thematic analysis and code development. Thousand Oaks, CA: Sage Publications.

Boyer, E. (1990). Scholarship reconsidered. Princeton, NJ: Carnegie Foundation for the Advancement of Teaching.

Boyer, E. (1996). The scholarship of engagement. Journal of Public Outreach, 1, 11–20.

Bringle, R.G., & Hatcher, J.A. (1996). Implementing service-learning in higher education. The Journal of Higher Education 67(2), 221–239.

Buzinski, S.G., Dean, P., Donofrio, A.F., Berger, A.T., Heighton, L.P., Selvi, A.F., & Stoecker, L.H. (2013). Faculty and administrative partnerships: Disciplinary differences in perceptions of civic engagement and service-learning at a large, research-extensive university. Partnerships: A Journal of Service-learning and Civic Engagement 4(1), 45–75.

CES4Health Review. (2015). http://www.ces4health.info/.


Consortium on Institutional Cooperation. (2015). https://www.cic.net/about-cic.

CIC Council on Engagement. (2005). Engaged scholarship: A resource guide. Champaign, IL: Committee on Institutional Cooperation.

Carnegie Community Engagement Classification, (2015). Http://nerche.org/index.php?option=com_content&view=article&id=341&Itemid=92#CE%20def.

Colbeck, C.L., & Wharton-Michael, P. (2006). Individual and organizational influences on faculty member’s engagement in public scholarship. New Direc-tions for Teaching and Learning, No. 205. New York, NY: Wiley.

Creswell, J.W., & Miller, D.L. (2000). Determin-ing validity in qualitative inquiry. Theory into Practice 39(3), 124–130.

Denzin, N., & Lincoln, Y. (2000). Handbook of qualitative research (2nd ed.). Thousand Oaks, CA: Sage.

Diamond, R.M., & Adam, B.E. (1995). The disci-plines speak: Rewarding the scholarly, professional, and creative work of faculty. Washington, DC: American Association for Higher Education.

Diamond, R.M., & Adam, B.E. (2000). The dis-ciplines speak II: More statements on rewarding the scholarly, professional, and creative work of faculty. Washington, DC: American Association for Higher Education.

Doberneck, D.M., Glass, C.R., & Schweitzer, J.H. (2010). From rhetoric to reality: A typology of publicly engaged scholarship. Journal of Higher Education Out-reach and Engagement 14(4), 5–35.

Doberneck, D.M., Glass, C.R., & Schweitzer, J.H. (2012). Beyond activity, place, and partner: How pub-licly engaged scholarship varies by intensity of activity and degree of engagement. Journal of Community En-gagement and Scholarship 4(2), 18–28.

Ellison, J., & Eatman, T.K. (2008). Scholarship in public: Knowledge creation and tenure policy in the engaged university. Syracuse, NY: Imagining America.

Freeman, E., Gust, S., & Aloshen, D. (2009). Why faculty promotion and tenure matters to community partners. Metropolitan Universities 20(2), 87–103.

Gelmon, S.B., Jordan, C.M., & Seifer, S.D. (2013). Rethinking peer review: Expanding the boundaries for community-engaged scholarship. International Journal of Research on Service-learn-ing and Community Engagement 1(1), 1–10.

Glaser, B.G. (1965). The constant comparative method of qualitative analysis. Social Problems 12, 436–445.

Glaser, B.G., & Strauss, A.L. (1967). The dis-covery of grounded theory: Strategies for qualita-

tive research. Chicago, IL: Aldine.Glassick, C.E., Huber, M.T., & Maeroff, G.I.

(1997). Scholarship assessed: Evaluation of the pro-fessoriate. San Francisco, CA: Jossey-Bass.

Hanlon, P. (2012, February). Recognition of entrepreneurial, outreach, and creative activities (memo to deans, directors, and department chairs). Ann Arbor, MI: Provost and Vice President for Academic Affairs, University of Michigan. Avail-able at: http://www.provost.umich.edu/facul-ty/promotion_guidelines/Entrepreneurial%20memo.pdf.

Indiana University. (2013). Guidelines for tenure and promotion reviews at Indiana Univer-sity Bloomington. Bloomington, IN: Office of the Vice President for Faculty and Academic Affairs, Indiana University.

Janke, E., Medlin, K., & Holland, B. (2013, November). Cultivating and rewarding the mosaic of faculty scholarly talents and contributions. Pa-per presentation at the 13th Annual International Association for Research on Service-learning and Community Engagement Conference, Omaha, NE.

Jordan, C. (Ed.) (2007). Community-engaged scholarship review, promotion, & tenure package. Peer Review Workgroup, Community-Engaged Scholarship for Health Collaborative, Communi-ty-Campus Partnerships for Health.

Kellogg Commission on the Future of State and Land-Grant Universities. (2000). Renewing the covenant: Learning, discovery, and engagement in a new age and a different world. Washington, DC: National Association of State Universities and Land-Grant Colleges.

Kezel, A.J. (1999). Sampling in qualitative inquiry. In B.F. Crabtree and W.L. Miller (Eds.) Doing qualitative research (2nd ed.), pp. 33–45. Thousand Oaks, CA: Sage Publications.

Kincheloe, J.L. (2001). Describing the brico-lage: Conceptualizing a new rigor in qualitative research. Qualitative Inquiry 7(6), 679–692.

Kreber, C. (Ed.). (2009). The university and its disciplines: Teaching and learning within and beyond disciplinary boundaries. London and New York: Routledge.

Krippendorf, K. (2004). Content analysis: An introduction to its methodology (2nd ed.). Thou-sand Oaks, CA: Sage Publications.

Lynton, E.A. (1995). Making the case for pro-fessional service. Sterling, VA: Stylus Publishing.

Michigan State University. (1996). Points of distinction: A guidebook for planning and evaluat-ing quality outreach. East Lansing, MI: Michigan


State University. Michigan State University. (2012a). Faculty

guide for reappointment, promotion, and tenure review at Michigan State University. East Lansing, MI: Human Resources, Michigan State Universi-ty.

Michigan State University. (2012b). Recom-mendation for reappointment, promotion, or ten-ure action form (Form D). East Lansing, MI: Of-fice of the Provost, Michigan State University.

Michigan State University. (2012c). Recommen-dation for reappointment, promotion, or tenure action (instructions). East Lansing, MI: Office of the Provost, Michigan State University.

National Review Board for Scholarship of Engage-ment. (2015). http://www.scholarshipofengagement.org/index.html.

O’Meara, K. (2011). Faculty civic engagement: New training, assumptions, and markets needed for the engaged American scholar. In J. Saltmarsh & M. Hartley (Eds.), “To serve a larger purpose”: Engagement for democracy and the transformation of higher educa-tion, pp. 177–198. Philadelphia, PA: Temple Univer-sity Press.

O’Meara, K., & Rice, E.R. (2009). Faculty priori-ties reconsidered: Rewarding multiple forms of scholar-ship. San Francisco, CA: Jossey Bass.

Pennsylvania State University. (2012). Adminis-trative guidelines for HR-23 promotion and tenure pro-cedures and regulations, 2012-2013. University Park, PA: Pennsylvania State University.

Purdue University. (2012). West Lafayette Cam-pus promotion and tenure policy. West Lafayette, IN: Office of the Provost, Purdue University.

Rutgers University. (2013a). Academic reappoint-ment/promotion instructions. New Brunswick, NJ: Rutgers University.

Rutgers University. (2013b). Academic reappoint-ment/promotion forms. New Brunswick, NJ: Rutgers University.

Thornton, M.L., & Jaeger, A.J. (2008). The role of culture in institutional and individual approaches to civic responsibility at research universities. Journal of Higher Education 79(2), 160–181.

University of Illinois at Urbana-Champaign. (2012). Promotion and tenure. Office of the Provost Communication No. 9. Urbana-Champaign, IL: Uni-versity of Illinois. Available at: http://provost.illinois.edu/communication/09/2013/Communication_9.pdf.

University of Iowa. (1994). 10.6 Post-tenure ef-fort allocation. Operations Manual. Human Resources Division I Employment, General. Chapter 10. Facul-ty. Ames, IA: University of Iowa. Available at: http://

www.uiowa.edu/~our/opmanual/iii/10.htm#104.University of Iowa. (2004). Procedures for tenure

and promotion decision-making at the University of Iowa. Ames, IA: University of Iowa.

Vogelgesang, L.J., Denson, N., & Jayakumar, U.M. (2010). What determines faculty-engaged scholar-ship? The Review of Higher Education 33(4), 437–472.

Wade, A., & Demb, A. (2009). A conceptual model to explore faculty community engagement. Michigan Journal of Community Service-learning 15(2), 5–16.

Wenger, L., Hawkins, L., & Seifer, S.D. (2012). Community-engaged scholarship: Critical junctures in research, practice, and policy. Journal of Higher Ed-ucation Outreach and Engagement 16(1), 171–181.

Whitt, E.J. (2001). Document analysis. In C. F. Conrad, J.G. Haworth, & L.R. Lattuca (Eds.), Quali-tative research in higher education: Expanding perspec-tives (2nd edition), pp. 447–454. Boston, MA: Pearson Custom Publishing.

AcknowledgmentsThe author would like to thank Michigan State

University’s National Collaborative for the Study of University Engagement for its support of this project; Berlinda Tolsma for assistance with data entry and analysis; and John Schweitzer for review of presentation and manuscript drafts. The author would also like to thank conference participants at the 2013 Engagement Scholarship Conference in Lubbock, TX, and 2014 International Association for Research on Service-Learning and Community Engagement in New Orleans, LA, for their thoughtful comments on preliminary presentations of this data.

About the AuthorDiane M. Doberneck is the assistant director

of the National Collaborative for the Study of University Engagement and coordinator of the Graduate Certification in Community Engagement at Michigan State University.


Advancing Innovation in Newfoundland and Labrador: Insights for Knowledge Mobilization and University-Community Engagement

Heather M. Hall, Jacqueline Walsh, Rob Greenwood, and Kelly Vodden

AbstractIn this paper, we provide insights for knowledge mobilization and university-community

engagement based on the lessons learned from the Advancing Innovation in Newfoundland and Labrador Project. Out hope is to provide a window into the experiences of academics as they navigate the complexities and politics of mobilizing research and engaging with diverse stakeholders. Despite the challenges of this work, presented by factors inside and outside the academy, it is crucial to enhance our capabilities if we are to maximize the impact of universities in linking theory, research, and expertise with critical social and economic needs, such as enhancing innovation.

IntroductionIn January 2013, the Leslie Harris Centre of

Regional Policy and Development (Harris Centre) at Memorial University, in partnership with the Navigate Entrepreneurship Centre (Grenfell Cam-pus), and the Canadian Regional Development: A Critical Review of Theory, Practice, and Potentials project team launched the Advancing Innovation in Newfoundland and Labrador project to synthe-size and share knowledge related to innovation and ways it can be fostered with key innovation stake-holders in Newfoundland and Labrador. The project was inspired by the Contextualized Health Research Synthesis Program (CHRSP) approach created by Stephen Bornstein in the Newfoundland and Labrador Centre for Applied Health Research at Memorial University. This approach aims to synthesize and contextualize research for Newfoundland and Labrador versus conducting new research on a particular topic. The Innovation Project included a team of researchers from Memo-rial University and an advisory committee made up of key representatives from industry associations, the provincial government, the federal government, the university, college, and labour. The project deliv-erables included a series of reports, innovation case studies, innovation workshops, a website, and an innovation summit.

In this paper, we provide insights for knowl-edge mobilization and university-community engagement based on the lessons learned from the project. We begin with a brief overview of some of the key challenges and opportunities identified in the knowledge mobilization and community-en-gagement literatures. In the next section we intro-duce some of the key concepts in the innovation

literature that highlight the importance of learning and collaboration between industry, government, postsecondary institutions, and communities. We then provide an overview of the Innovation Project and approach, which is followed by a discussion on the main challenges and opportunities that we en-countered during the project. Our hope is to provide a window into the experiences of academics as they navigate the complexities and politics of mobilizing research and engaging with diverse stakeholders. Despite the challenges of this work, presented by factors inside and outside the academy, it is crucial to enhance our capabilities if we are to maximize the impact of universities in linking theory, research, and expertise with critical social and economic needs, such as enhancing innovation.

Knowledge Mobilization and University-Community Engagement

Postsecondary institutions across Canada, and internationally, are increasingly embracing knowledge mobilization and university-community engagement through a variety of mechanisms (Hall, 2009; Levin, 2011; Heisler, Beckie, & Markey, 2012). The Carnegie Foundation for the Advancement of Teaching describes community engagement within a post-secondary context as “collaboration between institutions of higher education and their larger communities (local, regional/state, national, global) for the mutually beneficial exchange of knowledge and resources in a context of partnership and reciprocity” (New England Resource for Higher Education, 2016). This can often include service-learning, community-based experiential learning, community-based participatory research, and community-based research (Hall, 2009; Heisler,


Beckie, & Markey, 2012; Castledon, Sloan Morgan, & Lamb, 2012). While community engagement, defined in this way, is focused on knowledge exchange, knowledge mobilization, on the other hand, includes “public participation, translating ideas into accessible language, working with me-dia, social networking strategies, [and] podcasting” (Hall, 2009, p. 19) among other means to bring “knowledge, people and action together” (Bennet & Bennet, 2007, p. 17).

We share Bud Hall’s (2009) argument that the collective resources of universities and colleges represent the “largest accessible, available, and underutilized resource for community change and sustainability” (p. 13). Likewise, Barbara Holland and Judith Ramaley (2008) highlight “the urgent need to summon our collective wisdom to address critical social, economic, cultural, and environ-mental threats” (p. 334) by bringing together aca-demic institutions and communities. Despite this, there are still a number of challenges confronting academics within postsecondary institutions when they focus their efforts on community engagement and knowledge mobilization. In relation to the tra-ditional trifecta of research, teaching, and service, in 1996 Boyer argued: “At tenure and promotion time, the harsh truth is that service is hardly mentioned. And even more disturbing, faculty who do spend time with so-called applied projects frequently jeopardize their careers” (p. 13). More than a decade later, this is still the case in a number of postsec-ondary institutions (Jackson, Schwartz, & Andree, 2008; Moore & Ward, 2010; Jaeger, Katz, Jameson, & Clayton, 2012). While we recognize that this var-ies among and within institutions, it still poses a significant challenge where it does exist, especially for emerging scholars, as further discussed below in relation to our experience with the Innovation Project. Other challenges include time, financial support, and building and sustaining relationships for engagement (Moore & Ward, 2010; Heisler et al., 2012; Castledon et al., 2012).

We turn now to a discussion of some of the key arguments emerging from the innovation literature that support and necessitate university-community engagement and knowledge mobilization.

Learning and Interaction to Promote InnovationOne of the major arguments emerging from

the innovation literature in the last decade is the importance of interaction and learning between a wide variety of actors including individuals, firms, industry associations, and support institutions like government, universities, colleges, and innova-

tion centres (Hall, Walsh, Vodden, & Greenwood, 2014; Asheim, Boschma, & Cooke, 2011; Tödtling & Tripple, 2011; Nauwelaers, 2011; Rodrí-guez-Pose, 2013). This supports the argument that “innovation is increasingly recognized as a social process” (Wolfe, 2009, p. 15) versus a linear process including the phases of invention, production, mar-keting, and diffusion (Sternberg, 2009). Simply put “firms do not innovate in isolation” (Nauwelaers, 2011, p. 468).

The term “quadruple helix” (Carayannis & Campbell, 2009; Leydesdorff, 2012) is often used to describe the various innovation stakeholders including business, community, government, and postsecondary institutions. Related to this is the importance of innovation support systems often called “regional innovation systems” (Cooke, 1992; Cooke & Morgan, 1998) or “innovation ecosystem.” For example, the Canadian Independent Panel on Federal Support to Capital Research and Develop-ment (2012, pp. 2–15) explains how the “innovation ecosystem” includes,

not only firms, universities, colleges and polytechnics, but also a spectrum of intermediary players [technology transfer offices, college applied research offices, public research institutes and pro-grams, incubators, angels and venture capitalists]…characterized by effective synergies, connections, and flows of knowledge and ideas.

Given this emphasis on interaction and learn-ing between and among innovation stakeholders, university-community engagement and knowledge mobilization can play an important role in support-ing business innovation.

The Advancing Innovation in Newfoundland and Labrador Project

As noted earlier, in January 2013 the Harris Centre at Memorial University—in partnership with the Navigate Entrepreneurship Centre (Grenfell Campus) and the Canadian Regional Development: A Critical Review of Theory, Practice and Potentials project team—launched the Innovation Project to synthesize and share knowledge related to innovation and the ways it can be fostered with key innovation stakeholders in Newfoundland and Labrador. In the following sections, we provide a brief overview of the Harris Centre and the CHRSP approach. We then turn to a discussion on the Advancing Innovation in Newfoundland and


Labrador approach including a description of the advisory committee, the innovation workshops, and the innovation summit.

A Brief Overview of the Harris Centre and the CHRSP Approach

The Harris Centre was launched in October 2004, with a mandate to facilitate and coordinate Memorial University’s activities in regional devel-opment and public policy. It developed a series of programs and supports to connect Memorial facul-ty, staff, and students with the needs of the province. These include organizing regional workshops in partnership with community-based organizations, holding public policy forums, and establishing ap-plied research funds in partnership with govern-ment and private sector partners. The Harris Centre also developed the online public engagement tool called Yaffle. As the Harris Centre has built its brand based on values of independence, integrity, and practical application, it has established a reputation within the university, the province, and internation-ally as a trusted knowledge broker and mobilizer.

Given the Harris Centre’s focus on knowledge mobilization, we were inspired to try the CHR-SP approach, created by Stephen Bornstein in the Newfoundland and Lab-rador Centre for Applied Health Research at Me-morial University. CHR-SP provides systematic reviews of topics identi-fied in partnerships with key decision-makers in the health sector. More importantly, this infor-mation is contextualized to take into account the unique issues, chal-lenges, and capacities in Newfoundland and Labrador (Newfound-land & Labrador Cen-tre for Applied Health Research [NLCAHR], 2013; Memorial Uni-versity Faculty of Med-icine [MUNMED], 2013; Barrett, Bornstein, Kean, & Navarro, 2011). In terms of process, the CHRSP approach in-cludes several stages: (1) identify pressing issues

of concern in partnership with health system deci-sion-makers; (2) use research expertise to develop research questions based on these concerns; (3) synthesize international research literature on the subject and contextualize it to Newfoundland and Labrador—this includes taking into account the unique provincial challenges and capacities; and (4) quickly produce research results that are easily ac-cessible and in usable formats.

In recent years, the Harris Centre has support-ed a number of innovation-related research initia-tives (Table 1). The focus on innovation results from the widespread understanding that innovation is critical for economic growth and the recognition that Newfoundland and Labrador businesses have the potential to be far more innovative than current evidence suggests (Greenwood, Pike, & Kearley, 2011). Given the widely recognized importance of innovation for economic development but also re-gional development more generally, the emphasis on partnerships in fostering innovation in a region, and the abundance of existing literature on this topic, innovation was selected as the theme for the Advancing Innovation in Newfoundland and Lab-rador Project, the Harris Centre’s first CHRSP-like initiative.

Description Title

The Social Dynamics of Economic Performance in City-Regions

• National project was led by David Wolfe and Meric Gertler at the University of To-ronto

• NL component was led by Rob Greenwood• Focused on three themes: the social

dynamics of innovation, talent attraction and retention, and civic governance and inclusion

Networks for Business Innovation: Building Social Capital in Corner Brook, NL

• Led by Jose Lam (Memorial University – Grenfell Campus) and included a team of individuals from government, post-second-ary, business and community organizations

• Focused on investigating who people talk to and work with to map out these con-nections and networks and their roles in business innovation

Canadian Region-al Development: A Critical Review Of Theory, Practice And Potentials

• Led by Kelly Vodden (Memorial University – Grenfell Campus), with Co-Investigators Bill Reimer (Concordia University – Que-bec), David Douglas (University of Guelph – Ontario), and Sean Markey (Simon Fraser University – British Columbia)

• Focused on the following themes: place-based development, collabora-tive, multi-level governance, rural-urban interactions, integrated development, and innovation and learning in four Canadian provinces

Table 1. Innovation-Related Research Initiatives Supported by the Harris Centre


The Advancing Innovation in Newfoundland and Labrador Approach

The team included Rob Greenwood (execu-tive director – the Harris Centre), Heather Hall (postdoctoral fellow – the Harris Centre and Department of Geography project coordinator), Kelly Vodden, (associate professor research – En-vironmental Policy Institute), and Jacqueline Walsh (assistant professor – business), together with an honours undergraduate student (Kyle White) and Ph.D. student (Ken Carter), both focusing on inno-vation in Newfoundland for their thesis research in the Department of Geography. The composition of the team, with backgrounds in business, geography, and political science, reflected the interdisciplinary approach to the complex issue of business innova-tion. The team also included members at varying stages of their academic career. This proved to be a very useful method of introducing and embedding new researchers into existing relationships with community members.

The project team prepared a four-page back-ground document outlining the key objectives and three-phase approach, including a knowledge synthesis, a series of innovation workshops, and an innovation summit. Like the CHRSP approach, the knowledge synthesis summarized in a succinct fashion the latest research on innovation with in-sights for advancing innovation strategies in the context of Newfoundland and Labrador. The in-novation workshops, on the other hand, ground truthed these insights and reported on how the re-search findings from the knowledge synthesis could help foster innovation in Newfoundland and Lab-rador. As well, the workshops were used to report back to community partners on related research findings that had been previously ex-plored in that particular re-gion in the province. The in-novation summit then distilled lessons for policy and practice (Table 2). The team was careful to include both urban and rural parts of the province in all aspects of the project to counteract the urban bias in the innovation literature and because Newfoundland and Labrador is one of Can-ada’s most rural provinces, with more than half of its population residing in rural and small town communities

as of 2011 (Vodden, Gibson, & Porter, 2014; Statistics Canada, 2012).

The Innovation Project Advisory CommitteeThe Innovation Project team invited key

innovation stakeholders in the “quadruple helix” (Carayannis & Campbell, 2009; Leydesdorff, 2012) to become members of an advisory committee. The advisory committee included 15 representatives from industry associations, the provincial govern-ment, the federal government, the university, the college, and labour. The roles of the advisory com-mittee, which were outlined in a terms-of-reference document, were to: provide feedback on proposed workshop locations; provide advice and comments on the workshop reports; identify existing relevant data and resources; identify key local contacts in each of the workshop locations; highlight import-ant local or stakeholder specific issues for consider-ation; review emerging themes and lessons and pro-vide advice to the project and research teams on the final report; and assist with publicity for all events and reports. From the start, it was emphasized that the final project report would reflect the independence of the research team and that the final content for the report would be the responsibility of the project team.

We held five advisory committee meetings in March, April, and September 2013 and January and March 2014. The March 2013 meeting provided an introduction to the project as well as an over-view of the advisory committee terms of reference. In the April 2013 meeting we discussed workshop locations, times, possible local stakeholders and research for the knowledge synthesis. The September

• A Knowledge Synthesis on innovation, summarizing the latest research on innovation and insights for advancing innovation strategies in Newfoundland and Labrador

• An Innovation Summit

• A final report based on the key findings from the the AINL project that provides recommendations for policy and practice

• The http://innovational.ca website to host innovation-related research studies in Newfoundland and Labrador

• Five Innovation Workshops in Kittiwake, Labrador Straits, Northern Peninsula, St. John’s, and Corner Brook as well as a series of Innovation Workshop Reports

• A series of Innovation Case Studies on in Newfoundland and Labrador in partnership with the Canadian Regional Development project

Table 2. Advancing Innovation in Newfoundland and Labrador Project Deliverables


2013 meeting was focused on preparing for the innovation summit with the advisory committee providing feedback on the knowledge synthesis and findings from the workshops as well as recom-mending participants for the innovation summit. At the January 2014 meeting we discussed the final report and insights for policy and practice while the March 2014 meeting was focused on next steps for the advisory committee and ideas for disseminating the Innovation Project materials.

The Innovation WorkshopsThroughout May and June 2013, we held five

Innovation Workshops in Kittiwake, Labrador Straits, the Northern Peninsula, St. John’s and Cor-ner Brook (see Figure 1). These locations reflected urban, rural, and remote regions from across the province, which was essential for understanding place-based challenges and opportunities as well as combating one-size-fits-all policy approaches. These locations also reflected places where previous community-based research had been undertaken on related issues. The workshops provided an ex-cellent opportunity to report back to the stakehold-ers in each region. We used a variety of methods to try and encourage participation in the workshops. For example, prior to the workshops we traveled to some of the locations where we had limited research connections to meet with local stakeholders to dis-cuss the project, select dates for the workshops, and tour innovative companies identified by the adviso-ry committee, local stakeholders, and previous re-search. By visiting the regions in advance we were ultimately trying to show our interest in building relationships within the communities as well as our willingness to be engaged at a very practical and meaningful level. Another recruitment strategy in-cluded contacting individuals who had previously participated in one or more of the research projects highlighted in Table 1.

We decided on a half-day format to encourage more businesses attendance. We know it is difficult for small business owners, in particular, to be away from their businesses for long periods of time. We also tried to hold the workshops in conjunction with other meetings. For example, in Kittiwake we held our workshop in conjunction with a Canadi-an Manufacturers and Exporters (CME) business network meeting. The format for these workshops included:

• A brief overview of the Innovation Project by the project coordinator

• A presentation based on prior research undertaken in the region and on themes

related to innovation in the regional economy by a project team member

• Question and answers• A presentation on firm-level innovation in

Newfoundland and Labrador by a project team member

• Questions and answers• A panel discussion with regional repre-

sentatives from business, the community, government, and/or postsecondary to respond to earlier presentations and speak about what strategies were needed to enhance innovation in their region

• A breakout discussion on challenges, opportunities, and strategies, and

• A survey using TurningPoint technology (voter keypads) to select the top challenges, opportunities, and strategies according to participants

Seventy-six people attended the workshops in-cluding 16 representatives from business and social enterprises and the balance from community-based organizations, industry associations, postsecond-ary institutions and all levels of government. The workshop attendance breakdown was as follows: St. John’s, 23; Corner Brook, 17; Kittiwake, 16; North-ern Peninsula, 11; and Labrador Straits, 9.

The Innovation SummitIn October 2013, we held a full-day Innovation

Summit in St. John’s (the provincial capital). We invited innovation stakeholders from each of the

Figure 1. Workshop Sites


workshop locations and from across the province. In total, 46 participants attended from all three lev-els of government, business and labour, Memorial University and the College of the North Atlantic, and community organizations. The summit start-ed with a brief overview of the Innovation Project, which was followed by presentations on the key findings from the knowledge synthesis and key les-sons from the innovation workshops. The morning also included a panel discussion with representa-tives from business, the community, government, and postsecondary who responded to the key find-ings from the knowledge synthesis and innovation workshops. The afternoon consisted of facilitated breakouts on the critical gaps that needed to be considered for advancing innovation and how these critical gaps could best be addressed. Each group reported back with their top three gaps, which were identified through a dotmocracy1 exercise. These critical gaps were then entered into the Turning-Point technology to select the top gaps that need-ed to be addressed. A closing panel followed this with representatives from business, the communi-ty, government, and postsecondary, responding to these critical gaps and how the various stakeholders could address them.

Insights for Knowledge Mobilization and Com-munity-University Engagement

During the Innovation Project we encountered a number of expected and unexpected challeng-es and opportunities. The challenges included: the demise of the regional matchmaker; the politics of timing; working in the business and not on the business; academic independence versus co-produc-tion; and the academic publish or perish mentality. Opportunities, on the other hand, included: hope, optimism, and networking; reporting back, validating research findings and building rela-tionships; exploring new research topics; student engagement; and informing policy.

We turn now to a discussion of each while highlighting how they offer insights for knowledge mobilization and community-engaged research.

The Demise of the Regional MatchmakerJust as regional economic development agen-

cies were being abolished in the UK (Kitagawa, 2013) and across Canada (Hall & Greenwood, 2013; Gibson, 2013), in May and June 2012, the federal

and provincial governments announced that they were discontinuing the funding for the Regional Economic Development Boards (REDBs) in New-foundland and Labrador. The REDBs were created in 1995 in response to growing economic chal-lenges impacting communities across the province and were designed to be the ‘facilitators of regional economic development’ (Report of the Ministerial Committee, 2005). As we have noted elsewhere, “the REDBs acted as a ‘matchmaker’ between di-verse regional interests and provided a point of con-tact for information about government programs and policies in many rural regions” (Hall, Vodden, & Greenwood, forthcoming). The demise of the REDBs impacted the Innovation Project in several ways, including: the loss of a key partner and the in-troduction of a contentious policy issue into project design and stakeholder dialogue and relationships.

The structure of the REDBs included profes-sional economic development staff and a volunteer board of directors made up of representatives from municipalities, business, community development, education and training, labour, and other organi-zations (Hall et al., forthcoming). As a result, they were well connected to many of the key innovation stakeholders within their respective regions. More importantly, the REDBs had provided a quick and efficient “one-stop-shop” to disseminate informa-tion and gather contacts. Team members had ben-efited from this function played by the REDBs in previous related research initiatives. However with their demise, the Innovation Project lost this point of contact in the region. We also lost a key region-al development partner that would have played an integral role in advancing a number of the recom-mendations from the Innovation Project. The deci-sion to close the REDBs was done with little con-sultation and took many organizations by surprise. As result, it became a fairly contentious policy issue especially in a number of rural regions across the province in the period leading up to and during the Innovation Project. In many of the innovation workshops, participants were keen to discuss the REDBs and what regional development could look like after their closure.

The Politics of TimingRelated to this, several members of the

Innovation Project team were labeled “political” by certain government stakeholders because of our critical discussion of the decision to close the REDBs in other research reports and for discussing the REDBs at the innovation workshops. This issue was exacerbated by deep provincial budget cuts and

1 Participants were provided with 10 dot stickers and voted (by placing one or more stickers) on a flip chart listing the critical gaps. The three with the most stickers went forward from the breakout groups to the larger discussion.


layoffs in early 2013 that resulted in further cuts to regional development organizations along with dramatic reductions in government staff and pro-vincial spending, which precluded some govern-ment officials from attending the innovation ses-sions. Also complicating (and politicizing) matters, the governing provincial political party was losing support in public opinion polls (CBC, 2014). Thus, the demise of the REDBs, the deep cuts and lay-offs and this weakening in public support, created a perfect storm of political sensitivity that presented a number of unexpected challenges for the Inno-vation Project, including the loss of financial sup-port and participation from some key provincial government actors.

Ward and Jones (1999) refer to this issue as the mode of entry, which is shaped by the politi-cal-temporal contingency of research. Simply put, they suggest that the political timing of research has significant implications for the research project. In their paper, they discuss the secretive nature and political sensitivities with researching training and enterprise councils in the United Kingdom when they were in the political limelight. As a result, re-searchers experienced issues with access and polit-ical sensitivity (see also Hall, 2012). Likewise, Des-mond (2004) discusses the politics of time and the impacts on quality and access to information. She argues, “as any stand up comedian knows, timing is everything, and it is particularly relevant when interviewing elites during moments of political sen-sitivity” (p. 266). In the Innovation Project case, it impacted collaboration and stifled critical and in-formed discussion on pressing policy concerns fac-ing rural areas across the province. It also highlights the importance of recognizing and responding to political sensitivities when trying to inform policy and practice through research.

Working in the Business Versus Working on the Business

We also experienced challenges with getting business owners or managers to attend the innova-tion events. This is largely because many small- and medium-sized business owners are often too busy “working in the business” and they lack the time to step back and attend events or what we call “work-ing on the business” (see McGoff, 2012). Members of the Advisory Committee also brought this issue to our attention. To contend with this challenge we used the innovation case studies as a way to gath-er feedback and information from businesses. We also sought to partner with existing industry events. In particular, we had excellent business turnout at

the Kittiwake innovation workshop where we part-nered with a CME’s Central Continuous Improve-ment Network (CCIN). This business network formed three years ago and includes seven manu-facturing firms that meet regularly to share business advice and ideas. The CCIN network also receives one-on-one coaching/mentoring from CME. For the innovation workshop, the CCIN held their own meeting in the morning and participating business-es were encouraged to stay for the workshop, while workshop participants were encouraged to arrive early and join the CCIN and Innovation Project teams for innovation tours of several local firms. The Innovation Project team then provided lunch and we continued with the innovation workshop throughout the afternoon.

Academic Independence Versus Co-productionThe Harris Centre brand of integrity and inde-

pendence has provided a means to ensure scholars that the projects and funds brokered with com-munity, industry and government partners will not compromise their findings, conclusions, and recommendations. The Harris Centre has a policy of not responding to Requests for Proposals, as it will not compete with the private sector, and clients paying for consulting reports usually own the intel-lectual property. If an external partner comes to the Harris Centre with funding or to broker a project, it is with the explicit understanding that there will be consultation and engagement during the research process, which is often driven by a need identified by the partner, but the university researcher(s) re-tains independence in what is in the final report. For most stakeholders, this has value, as they often are conflicted within their own organization to ex-amine difficult issues. The relative independence of university researchers provides the means to access research and expertise that may pose difficult an-swers. The partner may wish to distance themselves from the conclusions, in whole or in part, but they now have research to inform their decisions.

The Innovation Project Advisory Committee understood this. As the research progressed, howev-er, and the ground truthing workshops took place, some partners heard negative comments about their programs or policies. In some cases they welcomed this information as a way to improve, but in others they were defensive or failed to appear at the sum-mit or some of the final committee meetings. As long as the integrity of the research was maintained, and the workshops and summit offered means for clear and balanced input from stakeholders (such as dotmocracy and voting keypads), the project team


was comfortable with the process results. Signifi-cant revisions were made to the final document and its recommendations based on the Advisory Com-mittee’s feedback—their expertise and perspectives made for a better result. When the project team failed to respond to their suggestions, it was based on an informed dialogue, building on the research and process. Advisory Committee members were not always happy, but most respected the integrity of the process and of the project team.

Publish or Perish

One final challenge is the “publish or perish” mentality that confronts many individuals within academia, which can act as a deterrent to commu-nity-engaged research and knowledge mobilization. Similar to our discussion earlier in this paper, the “publish or perish” mentality often refers to how ac-ademic hiring, tenure, and promotion committees only recognize (or place more value on) peer-re-viewed publications. Jaeger et al. (2012) suggest, “community-engaged work is still perceived as an ‘add-on’, rather than integrated into faculty roles” (p. 160). In a study of faculty engagement, Moore and Ward (2010) explain how participants in their study were labeled as outliers within their depart-ments and academic institutions. They also felt the pressure to accumulate the so-called “‘coin of the realm’: peer-reviewed publications and grant fund-ing” (p. 52). Similarly Jackson et al. (2008) argue that in Canada, “One of the major challenges to the growing movement for community-university en-gagement is the nature of traditional academic ten-ure and promotion (T&P) procedures, which tends to reward disengagement” (p. 133).

Publish or perish is increasingly playing a strong role in grant applications and university rankings (van Dalen & Henkens, 2012). While we recognize that this varies between institutions and within in-stitutions, it still poses a significant challenge where it does exist, especially for emerging scholars. This pressure to publish leads to the mentality that “it no longer matters what you write, but only how often, where and with whom you write” (p. 1283). While business leaders and government officials have re-viewed our knowledge synthesis and final report (including recommendations to enhance innova-tion), these manuscripts are not traditional peer-re-viewed academic outputs and may or may not be “counted” on our academic CVs. While Memorial University’s senior administration has expressed a commitment to engaged scholarship, most recently through the establishment of a Public Engagement Framework, the extent to which this has transferred

to department P&T committees has been inconsis-tent, with peer reviewed publications and funding remaining as the dominant criteria. Considering three of the project team members were emerging scholars (one postdoctoral fellow, one recent facul-ty hire, and one faculty member undergoing tenure review), this posed some challenges.

Community-university engagement and knowledge mobilization efforts also take time (see also Castledon et al., 2012). In the Innovation Project Team, time was required for booking the venue and catering, sending out invites, organizing panels and supplies, and making travel arrangements in the lead up to the innovation workshops and summit. Because our chosen communities included both rural and urban regions spanning the entire province, the team traveled in excess of 3,900 kilometres over the course of six weeks in May and June 2013. After the workshops and summit, our priority was getting the reports out to the public while the momentum was there and the discussion was fresh. Our next priority was then spending time on producing peer-reviewed publications. However, in the “publish or perish” environment time spent on community engagement and knowledge mobilization is often viewed as secondary to peer-reviewed publications (and in some cases even wasted time that could have been better spent on the latter). Interestingly, innovation stakeholders at the summit identified this mentality as one of the critical gaps impacting innovation in the province (Hall et al. 2014). Despite these challenges, we experienced several positive outcomes in using this approach. We turn now to a discussion of these opportunities.

Hope, Optimism and Networking: “It’s Like Having a Wedding after a Funeral”

As noted earlier, rural regions across the prov-ince were significantly impacted by the closure of the REDBs and the deep provincial budget cuts and layoffs. In the Northern Peninsula, one participant commented, “There’s only so many bullets a man can take before he dies,” while another in the Lab-rador Straits described how the last year was one of the most depressing times she had ever worked in (see also Hall et al., forthcoming). The innovation workshops were seen by many regional develop-ment stakeholders as an opportunity to come to-gether and discuss the impacts of these cuts and new strategies for the future. One participant even ar-gued: “It’s like having a wedding after a funeral.” The innovation events also brought together a diverse array of stakeholders from business, postsecondary institutions, government, and the community. This


provided networking opportunities, some of which have continued beyond the innovation project. For example, participants in several regions have held their own follow-up meetings to discuss the research findings and next steps. This also empha-sizes the need for findings and recommendations to be disseminated in a manner that allows commu-nity partners to gain maximum follow-up benefits from their participation in the project in the spirit of knowledge mobilization as a process of “moving new ideas and shared understanding into the hands of the people at the point of action” (Bennet & Ben-net, 2007, p. XIII).

Reporting Back, Validating Research Findings and Building Sustainable Relationships

Project team members were each involved in at least one of the innovation-related research ini-tiatives outlined in Table 1. Most of the innovation workshop locations were also case study regions in one or more of these research projects. The innova-tion events, therefore, provided a platform for the researchers to report back and in some instances validate initial research findings. These repeat en-counters with the same community members raise some key issues for success in community-based re-search. The importance of reciprocity and partner-ships when building sustainable relationships were highlighted in the introduction to this article as part of the framework for meaningful engagement ini-tiatives. The necessity of collaborative arrangements is also often highlighted in the academic literature. For example, Fisher et al. (2004, p. 29–30) report that university researchers have historically creat-ed a negative impression by using their perceived dominance to take advantage of external stakehold-ers without giving them back something in return. Establishing partnerships built on trust and integ-rity become even more integral when the research team wishes to continue to engage with the same stakeholders on multiple levels for various research projects over time. The research team has a com-mon interest in economic development, particularly in rural areas. There is no quick fix and short-term relationships would not be beneficial to either party.

For example, the Canadian Regional Devel-opment project included two case study regions in Newfoundland and Labrador: Kittiwake and the Northern Peninsula. In the Northern Peninsula the research team had placed particular focus on the project themes of innovation and governance. Both primarily rural regions were sites for Innovation Project workshops. This provided previous connec-tions as contacts for the team as well as an opportu-

nity for the research team to meet a commitment to report back to each of the regions on project results, with the valuable assistance of the project coordi-nator and other project resources. Further, through a combined effort between the research project and team, case studies of innovation within small and medium sized firms and social enterprises in these regions were completed, providing addition-al insights for both groups. Finally, the Canadian Regional Development project received provincial level exposure, increasing the project’s knowledge mobilization impact.

Exposure to New Research TopicsCommunity engagement provides an opportu-

nity to interact with a variety of stakeholders and to build a researcher’s capacity and reputation in spe-cific areas. It also exposes the researcher to a variety of issues and challenges that are outside the scope of the project being undertaken. Research ideas arise organically and can easily be validated as important to community stakeholders. From the Innovation Project findings, one team member developed a Social Sciences and Humanities Research Council of Canada proposal and was able to use the final report as evidence to support the research ques-tion being addressed in that application. As well, through the project a member of the team learned about a mining firm with innovations in both hu-man resources management and mineral explora-tion and processing technology, forming the basis of subsequent case study research. Finally, knowl-edge and relationships built during the Labrador Straits workshop helped to advance a subsequent federally-funded research initiative to identify de-velopment assets in that region.

Student EngagementTwo students—Kyle White (Geography un-

dergraduate student) and Ken Carter (Geography Ph.D. student)—were also engaged in the project. Kyle was the note-taker at all five innovation work-shops. He was also a co-author on each of the work-shop reports and lead author on the innovation case studies. Ryser, Markey, & Halseth (2013) cite a number of benefits to introducing undergraduate students to community-based or community-en-gaged research. For example, it can “expose them to the complexity of community development issues, build support and career networks and foster stu-dent interest in graduate studies or a research ca-reer” (p. 13). With the project, both students were just starting innovation-related research of their own. For Kyle, his participation on the Canadi-


an Regional Development and Innovation Project influenced his desire to focus on sustainable innovation for his undergraduate honours thesis. It also inspired him to pursue graduate studies in public policy. As members of the project team, both students benefited from opportunities to pres-ent their work, gain knowledge, and strengthen relationships in their study areas. We strongly believe that undergraduate and graduate students build research and networking skills, gain valuable research experience, and develop confidence through knowledge mobilization and community-university engagement.

Informing PolicyOne of the major benefits of knowledge

mobilization and community-university engage-ment is the opportunity to inform pressing policy concerns. As Boyer (1996) argued: “The academy must become a more vigorous partner in the search for answers to our most pressing social, civic, eco-nomic and moral problems” (p. 13). From the onset we were committed to offering insights for poli-cy and practice, which were provided in our final report. As noted earlier, our advisory committee included government policymakers and the work-shops engaged with representatives from all levels of government. It became clear at our final advisory committee meeting in March that the committee wanted to continue beyond the project. The final report was well received with plenty of discussion about where to go from here, which is one of the major goals for this type of community-engaged project. Many of the stakeholders also acknowl-edged their role in advancing innovation, which they did not previously seem to accept.

In May 2014 we publicly released the report through a media campaign organized by the Har-ris Centre. Copies of the report were also mailed to every participant who was engaged throughout the workshops and/or summit. One outcome thus far was Innovation Week, organized by a number of innovation-support organizations involved in the project. We were invited to present our major findings for policy and practice at two events during Innovation Week. We also organized a live webcast of the presentation made at an Innovation Out-look event through the Harris Centre to make the presentation accessible to all project participants. The final report and findings were referenced sev-eral times during Innovation Week by senior pol-icy-makers, leaving us optimistic that some of the recommendations will translate into new policies and approaches. We also submitted our final report

to a federal consultation on science, technology, and innovation. Several members of the project team are planning follow-up sessions in some regions and with key government innovation departments and other innovation organizations to discuss the major findings for innovation policy and practice.

ConclusionsThe goal of this paper was to provide a window

into the experiences of academics as they navigate the complexities and politics of mobilizing research and engaging with diverse stakeholders. We presented the Innovation Project as a practical example of university-community engaged research and knowledge dissemination as complimentary techniques for addressing economic challenges (in this case advancing innovation) in Newfoundland and Labrador. This recount of this project, including the methodology and the researchers’ perceptions, adds to the growing body of literature on good practices and challenges in this area. As we noted in the introduction, understanding both the benefits and the challenges of knowledge mobilization and community-engaged research is crucial to maxi-mizing the impact of universities in linking theory, research and expertise with critical social and economic needs, such as enhancing innovation.

The success of projects like ours should not be measured solely based on the number of peer-reviewed articles published. Success for this project must be measured by its overall impact on the communities involved; the mobilization of key stakeholders to achieve a common goal; the validation of methods used in community-engaged research; the capacity building opportunities for the researchers and students; the exposure to new networks and new research ideas; the dissemination of collective knowledge and reports containing the voices of community participants to influential stakeholders and policy-makers; and the strength and longevity of the relationships being nurtured with every interaction. Every research project involves challenges, but few provide the opportunities and rewards found in community-engaged research.

ReferencesAsheim, A., Boschma, R., & Cooke, P. (2011).

Constructing regional advantage: Platform policies based on related variety and differentiated knowl-edge bases. Regional Studies, 45(7), 893–904.

Bennet, A., & Bennet, D. K. Farfard, M. Fon-da, T. Lomond, L. Messier, & N. Vaugeois. (2007). Knowledge mobilization in the social sciences and humanities: Moving from research to action. Frost,


West Virginia: MQI Press, Knowledge Series.Barrett, B., Bornstein, S., Butler, J., Kean, R.,

& Navarro, P. (2011). Supporting health deci-sion-makers with contextualized research evidence. In J. Bascu and F. Macqueen Smith (Eds.), Inno-vations in knowledge translation: The SPHERU KT casebook. Saskatchewan: SPHERU.

Boyer, E.L. (1996). The scholarships of engage-ment. Journal of Public Service & Outreach, 1(1), 11–20.

Carayannis, E.G., & Campbell, D.F.J. (2009). “Mode 3” and “Quadruple Helix”: Toward a 21st century fractal innovation ecosystem. International Journal of Technology Management, 46(3), 201–234.

Castledon, H., Sloan Morgan, V., & Lamb, C. (2012). “I spent the first year drinking tea”: Explor-ing Canadian university researchers’ perspectives on community-based participatory research involv-ing Indigenous peoples. The Canadian Geographer, 56(2), 160–179.

CBCNews. (2014). Kathy Dunderdale resigna-tion: ‘It is time to step back’. Retrieved from http://www.cbc.ca/news/canada/newfoundland-labrador/kathy-dunderdale-resignation-it-is-time-to-step-back-1.2506233.

Cooke, P. (1992). Regional innovation systems: Competitive regulation in the new Europe. Geofo-rum, 23(3), 365–382.

Cooke, P., & Morgan, K. (1998). The association-al economy: Firms, regions, and innovation. Oxford: Oxford University Press.

Desmond, M. (2004). Methodological challeng-es posed in studying an elite in the field. Area, 26(3), 262–269.

Independent Panel on Federal Support to Re-search and Development. 2012. Innovation Canada: A call to action. Review of federal support to research and development—Expert Panel report. Ottawa: Gov-ernment of Canada.

Fisher R., Fabricant, M., & Simmons, L. (2004) Understanding contemporary university-commu-nity connections. Journal of Community Practice, 12(3–4), 13–34.

Gibson, R. (2013). Collaborative governance in rural regions: An examination of Ireland and New-foundland and Labrador. Doctoral dissertation sub-mitted to the Department of Geography, Memorial University of Newfoundland and Labrador.

Greenwood, G., Pike, C., & Kearley, W. (2011). A commitment to place: The social foundations of innovation in Newfoundland and Labrador. Re-treived from https://www.mun.ca/harriscentre/reports/research/2011/CommitmenttoPlace_Har-ris_Sept2011Web.pdf.

Hall, B.L. (2009). Higher education, community engagement, and the public good: Building the fu-ture of continuing education in Canada. Canadian Journal of University Continuing Education, 25(1), 11–23.

Hall, H.M. (2012). Stuck between a rock and a hard place: The politics of regional development ini-tiatives in Northern Ontario. Doctoral dissertation submitted to the Department of Geography, Queen’s University.

Hall, H.M., & Greenwood R. (2013). The state of regional development in Canada: What the hell is go-ing on? Paper presented at the Annual Conference of the Canadian Association of Geographer’s, St. John’s, August 13.

Hall, H.M, Vodden, K., & Greenwood, R. (forthcoming). From dysfunctional to destitute: The governance of regional economic development in Newfoundland and Labrador. International Planning Studies.

Hall, H.M., Walsh, J., Vodden, K., & Green-wood, R. (2014). Challenges, opportunities, and strat-egies for advancing innovation in Newfoundland and Labrador. St. John’s: Harris Centre.

Heisler, K., Beckie, M., & Markey, S. (2012). Expectations and realities of engaged scholarship: Evaluating a social economy collaborative research Partnership. Journal of Community Engagement and Scholarship, 4(1), 25–35.

Holland, B., & Ramaley, J. (2008). Creating a supportive environment for community-university engagement: Conceptual frameworks. Conference Proceedings, HERDSA Annual Conference, pp. 33–47.

Jackson, E.T., Schwartz, K., & Andree, P. (2008). Aligning tenure and promotion procedures for com-munity-university engagement: Dialogue for action. In D.E. Clover and C. McGregor (Eds.), Proceedings of the Third International Community-University Exposition (CUexpo 2008), (p. 133–134). Victoria, University of Victoria.

Jaeger, A.J., Katz Jameson, J., & Clayton, P. (2012). Institutionalization of community-engaged scholarship at institutions that are both land-grant and research universities. Journal of Higher Educa-tion Outreach and Engagement, 16(1), 149–167.

Kitagawa, F. (2013). City-regions, innovation and universities. In P. Cooke (Ed.), Reframing region-al development (pp., 341–358). New York: Routledge.

Leydesdorff, L. (2012). The Triple Helix, Qua-druple Helix, ..., and an n-tuple of helices: Explanato-ry models for analyzing the knowledge-based econo-my? Journal of the Knowledge Economy, 3(1), 25–35.

Levin, B. (2011). Making research matter. Edu-cation Policy Analysis Archives, 12(56), 1–20.


McGoff, C. (2012). The primes: How any group can solve any problem. Hoboken NJ: John Wiley & Sons, Inc.

Moore, T.L., & Ward, K. (2010). Institutionaliz-ing faculty engagement through research, teaching, and service at research universities. Michigan Jour-nal of Community Service-learning, 17(1), 44–58.

MUNMED. (2013). A unique approach to ev-idence-informed health care decisions. MUNMED, Faculty of Medicine, 25(1), 10.

Nauwelaers, C. (2011). Intermediaries in re-gional innovation systems: role and challenges for policy. In P. Cooke, B. Asheim, R. Boschma, R. Mar-tin, D. Schwartz, and F. Tödtling (Eds.), Handbook of regional innovation and growth (pp. 467–481). Cheltenham: Edward Elgar.

New England Resource Center for Higher Education. (2016). Carnegie Community Engage-ment Classification. Retrieved from http://nerche.org/index.php?option=com_content&view=arti-cle&id=341&Itemid=618#CEdef.

Newfoundland & Labrador Centre for Applied Health Research (NLCAHR). (2013). CHRSP: Evidence in context: Contextualized syntheses to support decision making — an innovative New-foundland and Labrador approach(presentation). St. John’s: Newfoundland and Labrador Centre for Applied Health Research.

Report of the Ministerial Committee on the Process to Renew Regional Economic Develop-ment. 2005. Final Report. Government of New-foundland and Labrador.

Rodríguez-Pose, A. (2013). Do institutions matter for regional development? Regional Studies, 47(7), 1034–1047.

Ryser, L., Markey, S., & Halseth, G. (2013). De-veloping the next generation of community-based researchers: Tips for undergraduate students. Jour-nal of Geography in Higher Education, 37(1), 11–27.

Statistics Canada. (2012). Census Profile. 2011 Census. Statistics Canada Catalogue No. 98-316-XWE. Ottawa: Statistics Canada.

Sternberg, R. (2009). Innovation. In R. Kitchin and N. Thrift (Eds.), International encyclopedia of human geography, Vol. 1 (pp. 481–490). Oxford: Elsevier.

Tödtling, F., & Tripple, F. (2005). One size fits all? Towards a differentiated regional innovation policy approach. Research Policy, 34, 1203-1219.

Vodden, K., Gibson, R., & Porter, M. (2014). A brighter future for whom? Rural and regional de-velopment in the Williams era. In A. Marland, and M. Kerby (Eds.), First among unequals: The premier,

politics and policy in Newfoundland and Labrador (pp. 213–230). McGill-Queens University Press.

Van Dalen, H.P., & Henkens, K. (2012). In-tended and unintended consequences of a pub-lish-or-perish culture: A worldwide survey. Journal of the American Society for Information Science and Technology, 63(7), 1282–1293.

Ward, K.G., & Jones, M. (1999). Researching local elites: Reflexivity, ‘situatedness’ and politi-cal-temporal contingency. Geoforum, 30, 301–312.

Wolfe, D.A. (2009). 21st century cities in Can-ada: The geography of innovation. The 2009 CIBC Scholar-in-Residence Lecture. Ottawa: Conference Board of Canada.

AcknowledgmentsWe would like to thank the research teams,

advisory committees, and funders from the NL ISRN project (Social Sciences and Humanities Research Council of Canada), the Rural-Urban Project (CRRF, MNL, the Canada-Newfoundland and Labrador and Labour Market Development Agreement), the Canadian Regional Development project (Social Sciences and Humanities Research Council of Canada Grand No. 410-2010-2273), the Navigate Entrepreneurship Centre, and Industry Canada. We would also like to thank Kyle White and Ken Carter for their research assistance on this project.

About the AuthorsHeather M. Hall is a postdoctoral fellow in the

International Centre of Northern Governance and Development at the University of Saskatchewan. Jacqueline Walsh is an assistant professor of Busi-ness at the Grenfell Campus of Memorial Universi-ty. Rob Greenwood is executive director of the Leslie Harris Centre of Regional Policy and Development at Memorial University (St. John’s Newfoundland). Kelly Vodden is an associate professor in the Envi-ronmental Policy Institute at the Grenfell Campus of Memorial University. The Grenfell campus is in Corner Brook, Newfoundland.


The Co-produced Pathway to Impact Describes Knowledge Mobilization Processes

David Phipps, Joanne Cummings, Debra Pepler, Wendy Craig, and Shelley Cardinal

AbstractKnowledge mobilization supports research collaborations between university and community part-

ners which can maximize the impacts of research beyond the academy; however, models of knowledge mobilization are complex and create challenges for monitoring research impacts. This inability to suffi-ciently evaluate is particularly problematic for large collaborative research networks involving multiple partners and research institutions. The Co-produced Pathway to Impact simplifies many of the complex models of knowledge mobilization. It is a logic model based framework for mapping the progress of research dissemination uptake implementation impact. This framework is illustrated using collaborative research projects from Promoting Relationships and Eliminating Violence Network (PREVNet), a pan-Canadian community-university network engaging in knowledge mobilization to pro-mote healthy relationships among children and youth and prevent bullying. The Co-produced Pathway to Impact illustrates that research impact occurs when university researchers collaborate with non-ac-ademic partners who produce the products, policies, and services that have impacts on the lives of end beneficiaries. Research impact is therefore measured at the level of non-academic partners and identified by surveying research partners to create narrative case studies of research impact.

Knowledge mobilization helps make academic research accessible to non-academic audiences and supports collaborations between academic researchers and non-academic partners such as community-based organizations. Knowledge mo-bilization is a process that supports action oriented research and finds novel approaches to persistent social, economic and environmental challenges. Knowledge mobilization has elements of: 1) uni-versity “push” of research beyond the academy; (2) community “pull” of research from the academy; 3 “knowledge exchange” between community and the academy; but extends those to include 4) the co-production of research that has academic merit and also has relevance for community action (Phipps & Shapson, 2009). Knowledge mobilization can thus support community engaged scholarship and community-based research as well as service-learning when the learning opportunity is meeting the needs of a community derived re-search question. There has been increasing atten-tion paid to knowledge mobilization and related activities as the academic research community seeks to articulate and maximize the various im-pacts of university research beyond the academy (Donovan, 2011; Grant, 2015).

Despite this increasing attention to articulat-ing the impacts of research there is little evidence that research is creating extra academic impacts

(Bhattacharyya & Zwarenstein, 2009). Sandra Nutley and colleagues point out that “a central irony is the only limited extent to which evidence advocates can themselves draw on a robust ev-idence base to support their convictions that greater evidence use will ultimately be beneficial to public services” (Nutley, Walter, Davies, 2007, p. 271). Although it is feasible to measure the impact of a single knowledge mobilization inter-vention by testing indicators pre- and post-inter-vention, it is challenging to evaluate a complex sys-tem of knowledge mobilization where there may be multiple research collaborators practicing a diversity of knowledge mobilization methods with diverse end users.

In a recent review of leading models for knowl-edge mobilization such as the circular Knowledge to Action Cycle (Graham, Logan, Harrison, Straus, Tetroe, Caswell, & Robinson, 2006) and the mod-els of Bennet and Bennet (2008), Phipps, Jensen and Myers (2012) concluded that many models of knowledge mobilization are highly complex. This conclusion is not surprising because knowl-edge mobilization is a complex process described by Bennet and Bennet (2008) as collaborative en-tanglement: “Collaborative entanglement means to purposely and consistently develop and sup-port approaches and processes that combine the sources of knowledge and the beneficiaries of that


knowledge to interactively move toward a common direction such as meeting an identified community need” (p. 48).

Knowledge Mobilization Pathway to ImpactIn an effort to simplify a system of knowl-

edge mobilization that reflects movement toward a common direction of impacts we turned to a logic model (Frechtling, 2007) where activities produce outputs that in turn produce outcomes that then produce impacts (Figure 1a). By mapping such a logic model onto knowledge mobilization process-es, it is possible to draw a sequence of stages that lead from research to impacts (Figure 1b). In ad-dition, it allows for the development of metrics at each stage of the logic model.

Dissemination. Knowledge mobilization sup-ports dissemination beyond traditional academic publishing and conference presentations. This dissemination can include publishing activities such as press releases, clear language research sum-maries, as well as more iterative tools such as social media. It also involves active, in person methods such as research events where researchers engage actively with organizations seeking to engage with research and research expertise (Phipps, 2011). The goal of dissemination is to move research out of the academic setting and into practice and pol-icy settings where it can progress towards impact.

Uptake. Once an organization has received research information from a dissemination activity it takes that research into the organization with a goal of determining whether the research is useful for informing decisions about policy, professional practice, and/or social services. Uptake can include presentations at staff meetings (that may or may not include the original researcher), internal eval-uation, as well as comparisons to the literature and existing practice.

Implementation. Once the research has been taken up and passed through internal assessment, the organization may choose to use the research when developing new or improved products, pol-icies, and services. Implementation in the knowl-edge mobilization context is an activity internal to the non-academic partner that uses research evi-dence to inform organizational decisions.

Impact. Impact is the effect the research-in-formed products, policies, and services have on end users as measured by the non-academic orga-nization. It is measured not only in metrics of uti-lization but also by changes in the lives of citizens, the health of the environment, or animal welfare, depending on the ultimate end user the organiza-tion is seeking to address.

This model as illustrated in Figure 1b creates a pathway to impact that enables the monitoring of progress. By understanding the goals of each stage of the pathway to impact, it is possible to assess the benefits accruing along the pathway; however, the linearity of this model may be a limitation. Linear models of research use have long been abandoned in favour of more iterative models of research use that show sustained engagement between research-ers and non-academic partner organizations (Nut-ley, Walter, & Davies, 2007; Greenhalgh & Wierin-ga, 2011). Linear models create risks that research evidence merely “transferred” to end user organi-zations may be misinterpreted or misused. Linear models create challenges of attribution which is the extent to which impacts can be attributed to the use of specific research outputs (Boaz, Fitzpatrick, & Shaw, 2008). By requiring a moment of transfer from the academic to the non-academic setting, linear models also reinforce academic and non-ac-ademic silos.

Figure 1a. Traditional Knowledge Mobilization Logic Model

Figure 1b. Knowledge and Mobilization Logic Model

Traditional Knowledge mobilization logic model

Activity Output Outcome Impact

Knowledge mobilization logic model

Research Dissemination ImplementationUptake Impact


Knowledge Mobilization Co-produced Pathway to Impact

Academic research networks are expected to collaborate with non-academic partner organiza-tions to make new discoveries and transform those discoveries into impacts. This requires a more it-erative version of the pathway to impact than is shown in Figure 1b because this process requires collaboration and co-production at each stage of the pathway. A circular or iterative logic model has previously been recommended for evaluat-ing knowledge translation (Davison, 2009) such as the Knowledge to Action Cycle (Graham et al., 2006) and a cyclical model proposed for education research (Amo, 2007). The iterative aspects of circular and cyclical models can be embedded into the knowledge mobilization logic model of Figure 1b to produce a Co-produced Pathway to Impact, as illustrated in Figure 2. The Co-produced Path-way to Impact maintains collaboration throughout the process and creates an iterative relationship between the non-academic partners and academic researchers, while maintaining an overall progres-sion from research development to ultimate im-pact. As illustrated in Figure 2 there are domains where academic research and policy/practice ac-tivities remain distinct; however, the central over-lapping space is a shared space of collaboration where co-production occurs at each stage of the pathway.

Co-production occurs at each stage of the path-way and accelerates the impact of research. For ex-ample, co-production at the research stage ensures partners’ readiness to take up findings because of their input on the nature of the research questions, methods, and interpretations. Co-production in the research stage enhances partners’ motivation and engagement with research content—the new knowledge will be relevant to them. At the dis-semination stage, the research findings are tailored to meet the partners’ needs from knowledge mo-bilization products. These products are produced in an accessible format for the partners. Different partners can then tailor the same research findings into their own relevant and actionable knowledge mobilization products that further heighten net-work engagement and increases dissemination. Partners enhance dissemination through their organizational channels with a breadth and depth that researchers cannot achieve. The ongoing mu-tual and reciprocal support and collaboration be-tween the researchers and partners in the uptake and implementation stages enables organizational transformation in response to the new research

findings. Traditionally, as research moves to im-pact, there is a decrease in engagement across the four stages of the pathway and engagement of the academic partner is lowest in the ultimate impact stage. Unlike the traditional process of research dissemination with research “handed” to partners, our framework supports an ongoing relationship through the knowledge mobilization processes. As illustrated in Figure 2 each stage of the pathway confers benefits for both researchers and partners, leading to new research questions, knowledge, and potential knowledge mobilization products.

This Co-produced Pathway to Impact is illustrated with examples from PREVNet (www.prevnet.ca). PREVNet is a multi-disciplinary and multi-sectorial network founded in 2006 on the premise that to prevent bullying strategies are re-quired in every setting where Canadian children and youth live, learn, work, and play. PREVNet includes 121 researchers from 21 disciplines (e.g., psychology, education, social work, law, business, criminology, policy, psychiatry, nursing) collaborating with 63 national public and community sector organizations.

PREVNet addresses the increasingly recognized and urgent need to provide all adults responsible for socializing children and youth with knowledge and tools to choose, implement, eval-uate, and sustain effective bullying and violence prevention initiatives. Although many bullying prevention programs are available, they often lack empirical evaluation, and have the potential to be ineffective or, in some cases, harmful (Dodge, Dishion, & Langsford, 2006; Farrington & Ttofi, 2011). Programs based on science with evidence of effectiveness are not well disseminated, particularly to isolated and vulnerable communi-ties; moreover educators are most likely to choose programs and resources on the basis of word-of-mouth, rather than evidence (Cunningham, Vail-lancourt, Rimas, Deal, Cunningham, Short, & Chen 2009). PREVNet promotes engaged schol-arship by collaborating with its member organiza-tions to develop evidence-based initiatives that rest on four pillars: education/training, assessment/evaluation, prevention/intervention, and policy/advocacy (Pepler & Craig, 2011). PREVNet has a focus on participation by non-academic partners and a target of action oriented impacts which are hallmarks of authentic community engagement (Stoecker, 2009).

PREVNet’s research, training and knowledge mobilization projects are at various stages of development from research to impact. The projects


below are presented as a snapshot in time to illustrate the different stages of the Co-produced Pathway to Impact. Each of the projects is a collaboration between academic and non-academic partners. The ongoing, sustained collaboration of each project described below creates the criti-cal feedback loops illustrated at each stage of the pathway (Figure 2). In this manner the academic and non-academic partners not only contribute their complementary expertise to the project, but the collaboration enables critical reflection on the creation of new knowledge and its application to the prevention of bullying.

An example of partner-led co-produced research. The Quazar Positive Behaviour Recogni-tion Program: Wynford Motivation Works is col-laborating with PREVNet and the Toronto District School Board’s Build Character Build Success ini-tiative to produce animated videos and lesson plans to build elementary students’ motivation to behave in ways that exemplify each of this initiative’s 10

positive character traits shown to be important for healthy relationships: respect, responsibility, empathy, kindness and caring, teamwork, fairness, honesty, co-operation, integrity, and perseverance.

PREVNet academic researchers and Wynford entered into an intense collaborative co-pro-duction process for program development and evaluation research. The first draft of program con-tent was collectively reviewed and subsequently revised and enhanced to reflect current scientific evidence about character development and violence prevention. A manual for school implementation was produced to ensure program fidelity. Now named the Quazar Positive Behaviour Recognition Program, it is ready for dissemination to end users, with an ongoing evaluation component. There is a website that introduces and enables schools to reg-ister for the program. It is currently launched and being evaluated in four Toronto and five Kingston Ontario elementary schools.

Research benefits. New knowledge about

PREVNet’s Co-produced Pathway to Impact

Academic Researcher

Policy/Practice Partner

Research Benefits

DisseminationBenefits

UptakeBenefits

ImplementationBenefits Impact

• New knowledge• Deeper/new partnerships• Academic trainees • New methods• New tools• New research questions

• Publications• Conferences, workshops• Social media, videos • Media and public awareness• IP including patents

• Validation of research• Policy/practice trainees• New research questions • Contextualization of research• Technology license• Best practices established

• Research informed policy, practice, service• New research questions • Policy/practice trainees• New program funding• New product developed and brought to market• Changes in programs

• Citizens served• Social, economic, environmental, health benefits • Media and public awareness• Vulnerabilities addressed• New research questions

ImplementationUptakeDisseminationResearch

Figure 2. Co-produced Pathway to Impact


the positive characteristics important for healthy relationships; new collaborative activities between researchers and partners, such as the Toronto District School Board; engaged graduate student experiences.

An example of dissemination—Public Health Agency of Canada (PHAC) Canadian Best Practices Portal (CBPP). CBPP is an authori-tative repository for annotated and evidence-based health promotion practices. PREVNet research-ers and graduate students have collaborated with PHAC since 2009 to create and populate the Violence Prevention Stream for the CBPP as a tool for disseminating evidence-based violence preven-tion practices, tools, and interventions. Each year, violence prevention programs developed in Cana-da and internationally are reviewed by PREVNet academic researchers and PHAC and those meeting the stringent inclusion criteria are included on the portal. The Violence Prevention Stream currently hosts 80 programs on the site, and there are 3,000 unique visitors annually.

After conducting six focus groups with educators and community organizations to explore the usability of the portal, feedback has resulted in improvements to the site. PREVNet and PHAC have developed a Needs Assessment Toolkit, to enable stakeholders to select programs that will be effective, relevant and appropriate for their specific populations and local needs, further enhancing the utility of the CBPP as a dissemination tool. PREVNet researchers have actively promoted the CBPP violence prevention portal through public presentations and professional conferences.

Dissemination benefits. These provide im-proved functionality using the Needs Assessment Toolkit; web based and social media promotion; improved accessibility of the 80 evidence-based programs, as well as four academic presentations at conferences; and increased decision maker aware-ness regarding the importance of evidence-based violence and bullying prevention programs.

An example of Uptake —Family Channel StandUP! Campaign. Family Channel is a com-mercial-free network offering family television entertainment in 5.8 million homes across Canada. Its target audience is children aged 9–12. Family Channel has been involved in Bullying Awareness Week every November for the past for 11 years, and approached PREVNet to be its official research partner in 2006. In 2012, a comprehensive Bul-lying Awareness Teacher’s Guide for Grades 4–6 was written by a team of graduate students from across Canada under the leadership of a PREVNet

researcher. The 80-page guide brought together current evidence-based information about bully-ing, cyberbullying, bullying and LGBTQ students, bullying and students with exceptionalities, build-ing a respectful classroom climate, and a plan for lead-up activities before and daily activities during Bullying Awareness Week. After reviewing and evaluating this resource and the accompanying tip sheets and associated activities, Family Chan-nel contributed professional graphics and design. The 2012 Teacher’s Guide was downloaded 2,250 times. In February 2012, Dr. Wendy Josephson, professor, Department of Psychology, University of Winnipeg, and three students held a series of focus groups with 41 elementary and high school teachers from Winnipeg and the surrounding area to review the 2012 Teacher’s Guide. Based on this input, the 2013 guide was revised and refined.

Uptake benefits. Family Channel validated the academic research in a real world setting; graduate students gained skills working with non-academic audiences (Family Channel and teachers); user audience input was used to refine the resource; resource made available to end users.

An example of implementation—Girls United Training, Girl Guides of Canada (GGC). Beginning in 2006, consultations with GGC leadership revealed that the training provided to Girl Guide Leaders, known as “Guiders,” did not specifically address bullying and relational aggression, nor was bullying addressed in the GGC Code of Conduct even though Girl Guide leadership identified bullying and relational aggression as needing to be addressed. A working group with leading researchers on girls’ aggression was convened, and then a PREVNet researcher and graduate student worked with senior GGC training developers to co-create the Girls United Training Module for adult leaders and Girls United Badge for Girl Guides. The initial iteration of the training was presented to the PREVNet Social Aggression Working Group, attended by leading Canadian researchers working in the field of social aggression and by staff from several youth-serving community organizations. A training module was developed based on the feedback from the work-ing group. GGC training developers simplified the language, sharpened the messaging, and supplied graphic design. Between 2006 and 2008, PREVNet delivered the training to over 75 Senior Trainers (who in turn trained other local trainers, who then trained Guiders) in British Colombia, Nova Sco-tia, and Ontario. PREVNet collected participant evaluations of these training sessions (N = 129)


and found high levels of satisfaction, with a mean rating of 4.7 on a 5-point scale assessing percep-tions of value and relevance, understanding of topic, and increased confidence about addressing social aggression among girls. Similar ratings were found by PREVNet from 27 participants who took the training from a Senior Girl Guide Trainer (mean = 4.6), providing evidence that the “Train the Trainer” model was effective (Daniels & Quigley, 2009).

From 2007 until August 2013, 1,445 Guiders completed on line training. Between October 2007 and August 2013, 18,873 Girl Guides achieved the Girls United Badge, indicating they had fulfilled the required activities designed to develop their understanding of healthy relationships with their peers. This example illustrates how co-produced evidence informed training (the Girls United Training Module for adult leaders, and Girls United Badge for Girl Guides) was disseminated to the PREVNet Social Aggression Working Group, was taken up by GGC training developers and imple-mented in a national train-the-trainer campaign.

Uptake benefits. These included a research informed training program; graduate student experience working in a practice setting; expansion of program to an online version; and Girl Guides developing an enhanced understand-ing of healthy relationships

An example of impact—the Healthy Rela-tionship Training Module (HRTM). The HRTM was developed through a Community of Practice that included PREVNet academic researchers, students and three youth-serving non-profit organizations: Big Brothers Big Sisters Canada, Canadian Red Cross, and Scouts Canada. By shar-ing resources and exchanging knowledge, the goal was to enhance each organization’s capacity to foster respectful, safe, caring, and inclusive environments for children and youth.

Adult leaders play a critical role socializing children and youth: they serve as role models, mentors, guides, supports, and teachers. To be effective in their work with children and youth, they need explicit training about how healthy development depends on healthy relationships, and how to identify and address bullying and other unhealthy relationship dynamics. There was an assumption that professionals and volun-teers who work with children and youth have the knowledge, confidence, and skills they need to create healthy social climates and prevent bullying, yet explicit, comprehensive, and evidence-based training was missing. Working collaboratively, the

Community of Practice co-created the HRTM to address the gap in relationship training.

The module consists of a comprehensive Facilitator’s Guide, a slide presentation deck, and a Participant’s Handbook. Following a multi-step process in which PREVNet and Community of Practice members move from visioning to design to evaluation and training, the HRTM was co-cre-ated in stages, with a graduate student preparing a first draft that was extensively presented and critiqued through multiple Community of Prac-tice meetings. Based on participant feedback and questionnaire results, an extensive revision of the HRTM Facilitator Guide was completed.

Within the three partner organizations, the HRTM was integrated into existing training resources and procedures. Pre- and post-training pilot data were collected using the “Knowledge Confidence Skills: Healthy Relationship Question-naire,” with a pilot data set of 505 participants from the partners. Analysis of these data revealed sig-nificant increases in participants’ confidence and commitment to fostering healthy relationships.

The following comments from partner organization leaders speak to the rapid uptake and implementation of HRTM that occurred by the end of 2012:

…if you look at knowledge mobilization, that knowledge that was presented, all the research and best practices made its way down to the field, which I think was a huge benefit. Across Canada, we incorporated portions of the Healthy Relationships training into all our prevention education materials. For example, in our training for teachers in bullying prevention we have integrated a module on healthy relationships. These teachers train Youth Facilitators and share information on healthy relationships. Our Youth facilitators deliver workshops to younger students and talk about healthy schools and healthy relationships. We have 3,500 Youth facilitators across Can-ada and reached over 260,000 youth with information on bullying prevention and healthy relationships last year. We also recently updated our Be Safe! Program for children ages 5 to 9 (formerly known as c.a.r.e.). Our 8th edition contains a section on healthy relationships. We hope to reach over 30,000 children, parents and teachers with the new kit over


the next year. — Lisa Evanoff, National Training Manager Canadian Red Cross

This year we have the potential of reach-ing more than twenty-four thousand youth, right from Vancouver Island to Newfoundland. So that’s a goodly num-ber of youth and as far as adults, poten-tially more than twenty-seven thousand volunteer leaders. If you include all of our paid staff as well as our volunteers, we’re looking at over one hundred and two thousand individuals. — DeEtte Bryce, past Training Representative for Fraser Valley Council, B.C. Scouts Canada

The impact has been very exciting given our magnitude across the country – we work in every province and now have some relationships and programs in each of the territories as well. We’re able to bring these new resources to children and youth, parents/guardians, volunteers/mentors, service delivery staff, and exec-utive staff and boards across the country. In 2012 Big Brothers Big Sisters agencies across Canada served over 40,000 children and youth—every child and youth, along with their volunteer men-tors and parents/guardians, benefits from the Healthy Relationship training. Susan Climie, director of training, Big Brothers Big Sisters of Canada,

Impact benefits. Gaps in training iden-tified and addressed; training developed and provided to make safer spaces for children and youth across Canada. Training contributes to economic and social benefits.

These examples illustrate how sustained engagement between academic researchers, stu-dents, and non-academic partners enables: the co-production of research (Wynford); the dissem-ination of research (PHAC); the uptake of research evidence into non-academic programs (Family Channel); the implementation of research evidence into products and services (Girl Guides); and, the eventual impact of evidence informed training on the lives of end beneficiaries (HRTM). The use of the Co-produced Pathway to Impact has a number of implications for the practice of knowledge mobilization as described below.

DiscussionReflecting on the PREVNet experiences of

collaborations between academic researchers and students with non-academic partners including (but not limited to) Wynford, Toronto District School Board, PHAC, Family Channel, Girl Guides, Red Cross, Scouts Canada and Big Brothers Big Sisters we have not only developed and imple-mented the Co-produced Pathway to Impact but can draw conclusions on its utility as a framework describing knowledge mobilization processes.

It is clear from the HRTM collaboration that impact is measured at the level of the non-academ-ic partner. Academic impacts arise from research and dissemination, but impacts on the lives of end beneficiaries are mediated through the products, policies and services of non-academic partner organizations. The HRTM was a training program co-produced with academic researchers but delivered nationally though Community of Practice partners such as Red Cross, Scouts Canada, and Big Brothers Big Sisters. That impact is a function of non-academic partners has also been demonstrated by Sarah Morton, who has shown the critical role of research users in mediating impacts of research beyond the academy (Morton, 2014).

As illustrated by the HRTM example, in a co-production process, research can skip dissemination and uptake and move directly to implementation, which then has an impact. There was no need for dissemination and uptake because the end users of the HRTM were involved in its creation. This outcome is unique to co-production where the process of undertaking the research can have an impact (i.e., influence decision making) even before the research has been disseminated. Co-production can therefore help to address issues of attribution (Boaz, Fitzpatrick & Shaw, 2008).

When these impacts are measured by partners evaluating the effects of their efforts on their stakeholders and end beneficiaries, the stories of impacts can be told through narratives and case studies. Structured impact case studies were the unit of assessment for the UK Research Excellence Framework (REF; www.ref.ac.uk) and research on the REF confirmed this method as the best meth-od for articulating impacts of research beyond the academy (Grant, 2015).

Because the pathway from research to im-pact may be measured over years, researchers and academic institutions need to remain in contact with non-academic partners to be able to capture the narrative case studies of impact. Without this


active follow up or continual engagement with partner organizations, academic researchers may have little appreciation of the impacts of their research. This has been confirmed by an evalua-tion of knowledge mobilization programs by the Social Sciences and Humanities Research Council of Canada (SSHRC, 2013).

If funders, such as SSHRC, want to generate impacts from their investments in research, then they need to fund uptake and implementation activities within partner organizations. These activities can be supported by funding graduate student internships and post-doctoral fellowships in partner organizations to support uptake and implementation. This strategy will provide the non-academic organization with ready access to academic research expertise and will provide the student/fellow with experience in non-academic professional environments.

Finally, knowledge mobilization is often described using the metaphor of “bridging the gap” between the silos of research and policy/practice; however, this metaphor maintains the academic and non-academic silos. In co-production there is no gap to bridge. Academic researchers and non-academic partners come together in a shared space of collaboration (see Figure 2). They main-tain their own independent spaces but research, dissemination, uptake, and implementation occur in a collaborative environment. In contrast impact beyond the academy is expressed in the non-aca-demic environment only.

Future Work/Issues ArisingThis theoretical framework is a snapshot in

time of a number of research collaborations at various stages along the Co-produced Pathway to Impact. Some of the observations are retrospective and are not intended to make predictions of future benefits arising from the research. To establish how the Co-produced Pathway to Impact works for a single collaborative research project, one would follow a co-produced research project such as the Quazar Positive Behaviour Recognition Program as it progresses through dissemination to uptake to implementation and eventually to impact. Howev-er, a number of potential challenges arise. It may take years for impact to be realized. Many research projects will not proceed all the way to impact, as other factors such as availability of resources and competing products may prevent good research from proceeding to impact. Therefore, the ques-tion arises about the unit of measurement and evaluation: should the Co-produced Pathway to

Impact be applied at the project, program/unit, institutional or network level?

Additionally, the linearity of the logic model underpinning the Co-produced Pathway to Impact may not be an issue. A number of linear logic model based frameworks describing the flow of research to impact have been described by Alberta Innovates Health Solutions (Graham, Chorzempa, Valentine, & Magnan, 2012), and by the Commonwealth Scientific and Research Organisation in Australia (Morgan, 2014) and is linearly referred to as research uptake, use and impact at the Centre for Research on Families and Relationships, University of Edinburgh (Morton, 2014). What is interesting about this convergent thinking is that knowledge mobilization profes-sionals seem to be getting comfortable with the linearity of these pathways. Linear models for a single knowledge mobilization project have been abandoned in favour of iterative models such as the Knowledge to Action Cycle (Graham et al., 2006). When working in a system of knowledge mobilization, however, a portfolio of projects, such as described for PREVNet, does move towards im-pact. And this movement is linear from research to impact. Different projects at different stages in the in the linear Co-Produced Pathway to Impact create the opportunity to further examine projects and develop indicators describing each to the stage of the pathway.

ConclusionsThe Co-produced Pathway to Impact requires

that researchers and research partners engage in ongoing collaboration throughout the process from research to impact. PREVNet’s deep and sustained collaborations may not be feasible or desirable for some community organizations or university researchers; however, in collaborative networks that have a mandate to not only create new knowledge but also to translate that knowl-edge into improved economic, social, health, cultural or environmental impacts the Co-pro-duced Pathway to Impact creates a framework that describes the progress of collaborative research as it develops from research into new products, pol-icies and services. It also illustrates that getting to impact is a shared enterprise and activities in both academic and non-academic partner sites need to be eligible expenses in research funding programs. A number of recommendations arise for those wishing to use the Co-produced Pathway to Impact to describe knowledge mobilization processes.

For academic researchers: Since impacts of


research beyond the academy are mediated by non-academic partners it is important to stay in touch with non-academic partners who may be using academic or co-produced research to inform new products, policies, and services. Only by working with partners to tell those stories will academic researchers be able to articulate the impacts of research.

For non-academic partners: The role of the non-academic partner in community-campus collaborations is more than a co-creator of research knowledge or passive recipient of academic research. It is the community partner, not the academic researcher, who will implement research evidence into products, policies and services to benefit stakeholders. The Co-produced Pathway to Impact highlights the critical role of non-academic partners in mediating research impact.

For research institutions: Public policy drivers such as the UK’s Research Excellence Framework (REF) are driving UK academic institutions to articulate the impacts of university research beyond the academy (Grant, 2015). It can take many years for research to be taken up by part-ners and implemented into the products, policies and services that will then have an impact on end beneficiaries. Without an institutional office like a Knowledge Mobilization Unit (Phipps & Shap-son, 2009), research institutions have little ability to identify these impacts. Sustainable research networks such as PREVNet maintain relation-ships with non-academic partners and over time collaborate with them to articulate the benefits of research projects at each stage of the Co-produced Pathway to Impact.

ReferencesAmo, C. (2007). Conceptualizing research

impact: the case of education research. Canadian Journal of Program Evaluation, 22(1), 75–98.

Boaz, A., Fitzpatrick, S., and Shaw, B. (2008) Assessing the impact of research on policy: A review of the literature for a project on bridging research and policy through outcome evaluation. Retrieved from http://www.psi.org.uk/pdf/2008/bridging-project_report_with_appendices.pdf

Bennet, A. & Bennet, D. (2008). Knowledge mobilization in the social sciences and humanities: moving from research to action. Frost, West Virgin-ia, USA: MQI Press.

Bhattacharyya, O., & Zwarenstein, M. 2009. Methodologies to evaluate effectiveness of knowl-edge translation interventions. In S. Strauss, J. Te-troe, and I. Graham (Eds.), Knowledge translation

in health care: Moving from evidence to practice (pp. 249–260). Oxford, UK: Blackwell Publishing 2009.

Cunningham, C.E., Vaillancourt, T., Rimas, H., Deal, K., Cunningham, L., Short, K., & Chen, Y. (2009). Modeling the bullying prevention pro-gram preferences of educators: A discrete choice conjoint experiment. Journal of Abnormal Child Psychology, 37(7), 929–943.

Daniels, T., & Quigley, D. (2009). Girls unit-ed: Creating a nationwide intervention program to address the use of social aggression within girls’ groups. In W. Craig, D. Pepler, & J. Cummings (Eds.) Rise up for respectful relationships: Prevent bullying (pp. 145–168.) PREVNet Series, Volume 2. Kingston, Canada: PREVNet Inc.

Davison, C.M. (2009). Knowledge translation: implications for evaluation. New Direction for Eval-uation, 2009(124), 75–87.

Dodge, K.A., Dishion, T.J., & Langsford, J.E. (2006). Deviant peer influences in intervention and public policy for youth. Social Policy Report, 20(1), 3–19.

Donovan, C. (2011) State of the art in assess-ing research impact: introduction to a special issue. Research Evaluation, 20(3), 175-179.

Farrington, D., & Ttofi, M. (2011). Bullying as a predictor of offending, violence and later life outcomes. Criminal Behavior and Mental Health, 31, 90–98.

Frechtling, J.A. (2007). Logic modeling meth-ods in program evaluation. San Francisco, CA: Jossey-Bass.

Graham, I.D., Logan, J., Harrison, M.B., Straus, S.E., Tetroe, J., Caswell, W., & Robinson, N. (2006). Lost in knowledge translation: time for a map?. Journal of Continuing Education in the Health Professions, 26(1), 13–24.

Graham, K.E.R., Chorzempa, H.L., Valentine, P.A., & Magnan, J. (2012). Evaluating health re-search impact: Development and implementation of the Alberta Innovates – Health Solutions im-pact framework. Research Evaluation, 1–14. doi: 10.1093/reseval/rvs027.

Grant, J. (2015) The nature, scale and ben-eficiaries of research impact: An initial analysis of Research Excellence Framework (REF) 2014 impact case studies. Higher Education Fund-ing Council of England. http://www.hefce.ac.uk/media/HEFCE,2014/Content/Pubs/Independen-tresearch/2015/Analysis,of,REF,impact/Analysis_of_REF_impact.pdf Accessed June 20, 2015.

Greenhalgh, T., and Wieringa, S. (2011). Is it time to drop the ‘knowledge translation’ meta-phor? A critical literature review. Journal of the


Royal Society of Medicine, 104(12), 501–509. Morgan, B. (2014) Income for outcome: Aus-

tralia and New Zealand are experimenting with ways of assessing the impact of publicly funded research. Nature, 511, S72–S75. Retrieved from: http://www.nature.com/nature/journal/v511/n7510_supp/pdf/511S72a.pdf.

Morton, S. (2014) Creating research impact: The roles of research users in interactive research mobilisation. Evidence & Policy: A Journal of Re-search, Debate and Practice, 11(1), 35–55. Re-trieved from http://dx.doi.org/10.1332/174426514X13976529631798.

Nutley, S., Walter, I., & Davies, H. (2007). Us-ing evidence: How research can inform public ser-vices. Bristol, UK: The Policy Press.

Pepler, D., & Craig, W. (2011). Promoting re-lationships and eliminating violence in Canada. International Journal of Behavioral Development. 35(5), 389–397.

Phipps, D., & Shapson, S. (2009). Knowledge mobilisation builds local research collaborations for social innovation. Evidence & Policy, 5(3), 211–27.

Phipps, D. (2011). A report detailing the de-velopment of a university-based knowledge mobi-lization unit that enhances research outreach and engagement. Scholarly and Research Communica-tion, 2(2), 1–13.

Phipps, D.J., Jensen, K.E., & Myers, J.G. (2012). Applying social sciences research for public benefit using knowledge mobilization and social media. In Asuncion Lopez-Varela (Ed.), Theoretical and methodological approaches to social sciences and knowledge management. Rijeka, Croatia: InTech.

SSHRC. (2013). Evaluation of SSHRC’s knowledge mobilization funding opportunities. Retrieved from www.sshrc-crsh.gc.ca/about-au_sujet/publications/KMb_evaluation_2013_e.pdf.

Stoecker, R. (2009). Are we talking the walk of community-based research? Action Research, 7(4), 385–404.

About the AuthorsDavid Phipps is executive director, Research

and Innovation Services in the Office of Research Services at York University in Toronto, Canada. Joanne Cummings is knowledge mobilization director, PREVNet. Debra Pepler is a professor in the Department of Psychology at York University. Wendy Craig is a professor in the Department of Psychology, Queen’s University, Kingston, Ontario,

Canada. Shelley Cardinal is National Aboriginal consultant to the Canadian Red Cross RespectED: Violence and Abuse Prevention program (Vancouver).


Volunteers Needed: Bridging Latino Immigrants and Local Communities Through Service Learning and Critical Analytic Practice Ethnography

Diana M. Ruggiero

AbstractThis essay presents a brief ethnography of a small Latino community in Tennessee and their

interaction with local volunteers following a disastrous flood that occurred in July 2014. The ethnography, in this case in the form of a screenplay, depicts the overall intercultural sensitivity of the volunteers, the affected, and the interpreters. In the process, this essay also considers such creative analytic practice (CAP) ethnographies may help students involved in Spanish and community service-learning courses as well as communities bridge the “self ”/“other” gap that so often distances Latino immigrants and locals.

It is well known that migrant populations contend with numerous challenges in their adopted society, from secondary citizenship and discrimina-tion to cultural differences. This exclusion is exac-erbated, if not caused, by language barriers. Indeed, as scholars in the social sciences have long noted (e.g., Hill, 1995, 2008), it is often language that demarcates the difference between “us” and “other.” Unfortunately, this polarizing form of thinking can be extremely detrimental to the progress of both marginalized and dominant populations within a city. When we fail to realize that we do indeed have new residents who are monolingual in Spanish and then ignore the needs that this population has, we are not only doing a disservice to them but to the population in general in that we reify and maintain the very structural differences that keep Latinos in an undesirable and marginal position in relation to the dominant society (see, for example, Gomberg-Muñoz, 2011).

As a language professor with community service-learning courses, I cannot help but attend to the needs of Latino immigrants and connect students with this reality. While grammar instruc-tion and cultural activities such as dancing salsa and eating at a Mexican restaurant are valid forms of teaching and learning Spanish, learning a for-eign language also entails applying the language in the context of daily life. For me, this means visit-ing a local community and using the language as much as possible in a positive and beneficial man-ner. Though situating language use within a lived context can be achieved through study abroad, an immersive language experience can likewise often be accomplished within one’s own local community. The benefits in this case, however, extend well be-

yond the individual student and the development of language abilities.

This essay chronicles an actual event that took place during the week of July 1, 2014 in Memphis, Tennessee, a city of more than 650,000 residents, following flooding in June. Demographically, the city is majority African American (63.3%) and Caucasian (29.4%), with a growing Latino popula-tion (6.5%) for which there are few resources in this heavily monolingual city (http://quickfacts.census.gov/qfd/states/47/4748000.html).

The events deal with the aftermath of a period of unseasonably heavy rainfall during the month of June. There was major flooding and aid was needed throughout the city. Many agencies, such as the Red Cross, were there to aid those who needed it, but as the Red Cross is staffed locally, they reflected the demographic majority of the city; there were very few, if any, Spanish speakers available. This created a huge need for third party interpreters who could speak both English and Spanish and help the affect-ed families receive aid. This third party was vital to the success of both parties by being a bridge across the language barrier.

Language differences provided enough of a barrier for the disaster relief team that the process to receive aid was stalled to the point of barely func-tioning. The need for bilingual volunteers was vital, but the availability and turnout was abysmally low. Agencies that exist to bridge the gap between the two communities were working overtime to get the word out and to find those willing to volunteer their services. Emails were sent and social media was alerted, but despite this there were very few people willing to help. Some of the responses were that the holiday was approaching and there was no time;


others were that they had no confidence in their Spanish speaking ability. Both responses, while seemingly reasonable, were in fact forms of further exclusion of the communities. Such responses are understandable given that being a formal interpret-er takes several years of training; however, for this cause, all that was required of the volunteers was assistance with a form that was already bilingual (Spanish-English). Issues that arise commonly in this situation are the ability to read and write on the part of the Spanish speaking residents.

During the week of July 1, 2014, I was able to help with interpreting and also to pass out cleaning kits for homes damaged by the flood. From obser-vations and personal experience during the event, the jobs were very simple and straightforward; they were about helping to ease the fear for those affected and to provide help through the aid pro-cess. This required at minimum an intermediate command of the language. For those who passed out the cleaning kits, a beginning Spanish level or less was required. The questions were the same, and with the correct teacher anyone could successfully parrot the sentences. That is to say, anyone with the desire to help would have not only been welcomed, but much needed and of great service. Those who showed up found a lively community that, despite the devastation, continued to remain optimis-tic. The community was an open place, and they continued to do the best they could while support-ing one another. The children were running around and playing and neighbors were sharing what little they had. There was some worry that permeated the air, some unease and fear…worries about where to go, what would come next for them, and how they would be able to pick up and carry on. There was a tangible tension in the air, mostly in the form of questions such as: How are the damages going to be paid for? What aid will the management of the housing community affected by the flood give? But mostly the community was grateful for the help and the assurance that things would get better.

Though the language skills required to as-sist the Red Cross in this particular case were in-deed minimal, it was perhaps the relative cultural awareness and sensitivity of the interpreters that made more of a difference in serving the affected families as shown below. It is perhaps here, at the intersection of community service-learning and language instruction, that language instructors such as myself can make a lasting contribution toward the betterment of society.

Indeed, the service-learning approach to the acquisition of a foreign language is an approach

that I would have previously benefited from before the incident of the week of July 1, 2014. Prior experience with such a disaster relief event within the community would have quelled the butterflies in the pit of my stomach. As it so happened, I had not taken any service-learning courses as a student. It becomes very difficult to ease another person’s fears as your fears and misgivings run rampant. There was not enough preparation in the world for the emotional impact of the job that I was to perform. As previously noted, the translation and transcription of information to the form was not in and of itself difficult. The hardest part was oc-cupying a neutral territory, reminiscent of Homi Bhabha’s (1994) “third space,” as a Latina and nat-uralized U.S. citizen with first-hand knowledge of the trepidation and uncertainty that comes with being an immigrant and an immigrant in need of assistance amid crisis. It was unsettling knowing that, at the end of the day, I could return to my cozy home when the homes of those I was helping, those I identified with in many ways more than with my fellow neighbors, were destroyed by the flood.

Following this experience, I realized that I needed a productive way to help me process what I had encountered and experienced. I also realized that, if this were the case for me as a Latina, than certainly my own students could and would also benefit from similarly creatively processing their service-learning experiences. Rather than journal, I opted to create a creative analytic practice (CAP) ethnography to represent what I had observed and encountered during a total of 12 hours worth of vol-unteer work and observation. My experiences were additionally informed by in-depth interviews with those affected by the flood and with the director of Latino Memphis. The brief screenplay presented below exemplifies the dialogs and interactions that occurred during volunteer hours.

CAP Ethnography: The ScreenplayThe term “creative analytic practice ethnogra-

phies” was coined by Laurel Richardson to refer to a host of interpretive processes informing research and representation in the social sciences and hu-manities following the postmodern turn (Rich-ardson & St. Pierre, 2005). Though hardly new, the various forms of creative ethnographic represen-tations now practiced across different disciplines and fields share a fundamental concern with the nature of social reality, the subjective positioning of the researcher, and the relationship between the researcher and those being researched. What CAP ethnography offers is a way of approaching


and representing social reality that is perhaps most reflective of the way reality is actually experienced in everyday life; as heterogeneous, contested, and contradictory.

I was drawn to CAP ethnography, and specifi-cally screenplay writing (as a form of CAP ethnog-raphy), in part as a result of my experience with the service-learning course, but also because of its potential to draw audiences into the interpretive process. As Nathaniel Kohn (2000) notes in his observations about the collaborative and dialogic nature of screenplay writing, the format of the screenplay encourages academics to treat their writings as an open dialogue. The screenplay requires openness to critique and frankness that traditional academic writing does not often employ (aside from the peer review process). It also invites the audience to share the sights, feelings, and sounds of the characters that it portrays, as well as in the final interpretation. By helping the reader contextualize everyday activities and the emotion behind them, the use of the screenplay as a creative analytic practice allows for the evocation of the human element within research. It also allows for a potential transformative experience for both the writer and the audience in that it necessitates active engagement on both parts in elucidating meaning from the text. This is especially important when considering the politics surrounding the issue of immigration and Latinos in the United States.

The screenplay excerpt (see Appendix A), then, is written with the intention of inviting audiences to reflect on their encounters with Latino immigrants as well as with their experience of natural disasters, loss, and need. They are called to think past their stereotypes, preconceptions, and beliefs concern-ing Latinos and the issue of immigration and to empathize with the struggle and humanity of the people affected by the flood (both victims and aid workers).

(Editor’s Note: At this point in the text, we recommend that you go to the end of the

document and read the screenplay excerpt, .)

Regarding the choice of presenting the screenplay in both Spanish and English, Richardson (2005) explains that “language is a constitutive force, creating a particular view of reality” (p. 960) and that it is tied to the idea that the ethnographer cannot be separated from who he/she is; thus the play was originally written in Spanish (my first language).

The screenplay excerpt, though brief, depicts a telling exchange between a Red Cross volunteer, interpretr, and Latino family, and captures the greater function of intepreter as cultural interpreter. The Red Cross volunteer in this case, though humane in his/her treatment of the flood victims, necessarily acts within strict boundaries and guide-lines dictated by the task assigned—in this case, processing claim forms and handing out cleaning kits. Mr. Lopez, in dire need of assistance extend-ing far beyond what the Red Cross could offer, is understandably nervous and wary of approaching and divulging personal information to the Red Cross. Indeed, though the status of Mr. Lopez is left unspoken, it is presumed on the part of the Red Cross volunteer that he is undocumented. The reasons for Mr. Lopez’ reticence, however, may be a result of numerous factors, an element of the screen-play left open to interpretation and debate. The end result, however, is that there is a deep mistrust of authority, despite the fact that an organization like the Red Cross transcends national boundaries (as the volunteer worker alludes to in his/her comment regarding his/her unconcern with the place of ori-gin of Mr. Lopez’s document). Into this scene where much is left unspoken the interperter enters.

Though speaking little in the script itself, the interpreter plays a crucial role in mediating and bridging the divide between the two distinct social realities represented here by the Red Cross volun-teer and Mr. Lopez. Though there to help, the Red Cross volunteer inadvertently alienates Mr. Lopez in two interrelated ways: by failing to acknowledge and respond to his particular story and concern for his daughter as well as by denying him agency in assigning him to a preconceived and generic category (or categories) distinct and opposite from that occupied by the Red Cross volunteer (i.e., victim, Latino, foreigner/immigrant, undocument-ed/illegal, and “other”). Mr. Lopez’s alienation is further exacerbated by yet another major factor; the language barrier. The interpreter in this case becomes the voice and therefore means of agency for Latinos such as Mr. Lopez. Thus even though the task of the interpreter in this scenario is rather banal (communicating the content and meaning of a form already written in Spanish), the significance of the interpreter’s role as a mediator navigating the power dynamics inherent in the space between the Red Cross volunteer and Mr. Lopez is greatly heightened and cannot be understated.

The impact of the gulf between the Red Cross and the Latinos affected by the flood and of the relative weight of the responsibility we (mean-


ing myself and my students) had assumed during this service-learning project was overwhelming. Horror, sorrow, compassion, guilt, shame, and anger coupled with an intense sense of urgency in my desire and determination to help were but a few of the emotions that overtook me. I vividly recall a woman with five children pleading with me to take her youngest, an infant, home with me for the evening, so damaged was her home. With tears in my eyes, I returned to her the child she had placed in my arms as the Red Cross volunteers adamantly shook their heads and sternly warned us: “Unless you have room for them all, do not take anyone home with you.” We were pained by the limitations of our role. Yet, we also knew that we were provid-ing something far greater than interpreting skills and cleaning kits: We provided a human element to an otherwise cold and alienating process. We lis-tened when the Red Cross could not. We navigated cultural differences that could have potentially led to misunderstanding, anger, and further mistrust. And in so doing, we advocated and provided a voice for an otherwise voiceless “other” in a poten-tially dehumanizing moment of crisis and need.

Perhaps it was the overwhelming sense of voicelessness and the desire to give voice to the families affected by the flood that led me to process my experience through the form of a screenplay. I now realize that the reality or truth of this experience lies in neither the voice of the Red Cross volunteers, the Latino families, nor the inter-preter alone, but somewhere between them as well as between what transpired during the event (my memory creatively captured in the screenplay) and my reflection and reading of the event now several months later. The process has been at once illumi-nating, therapeutic, and cathartic on an intellectual and personal level.

Though journals are indeed valuable tools for processing such experiences, I contend that CAP ethnographies like the screenplay offer teachers and students undertaking similar service-learn-ing projects a unique opportunity to unpack, reflect on, and critically assess their experiences in a personally meaningful way. Within my own class, the screenplay and its actual performance by the students (using different readings or interpretations of the roles) provided a rich space for exploring the role, responsibilities, and significance of language interpreters. It also served as a launching point for deep and meaningful conversations about the so-cial, cultural, and political conditions and dynam-ics surrounding Latino immigrant communities, as well as about disaster relief and the nature of the

relationship between the Latino community and local, national, and transnational service providers. It also reminded us of our individual responsibili-ties within our own communities to look after and care for one another in times of distress and need, regardless of language, culture, or any other social barriers that may exist. As my class concluded by the end of the semester, we are all human beings, not categories and labels to be checked off on a form; and just as we are all liable to be in a posi-tion of need, we are all also capable of lending assis-tance. Following this service-learning experience, I am confident that my students will not hesitate to respond to the call: “Volunteers needed.”

ReferencesBhabha, H.K. (1994). The location of culture.

New York, NY: Routledge.Bolaños, A.F., & Verdesio, G. (Eds). (2002).

Colonialism past and present: Reading and writing about colonial Latin America today. Albany, NY: State University of New York Press.

Hill, J. (1995). Mock Spanish: A site for the indexical reproduction of racism in American English. Language and Culture, 2. Retrieved from http://language-culture.binghamton.edu/sympo-sia/2/part1/.

Hill, J. (2008). The everyday language of white racism. Chichester, U.K.: Wiley-Blackwell.

Kohn, N. (2000). The screenplay as postmod-ern literary exemplar: Authorial distraction, dis-appearance, dissolution. Qualitative Inquiry, 6(4), 489–510.

Gomberg-Muñoz, R. (2011). Labor and legal-ity: An ethnography of a Mexican immigrant net-work. New York, NY: Oxford University Press.

Richardson, L., & St. Pierre, E.A. (2005). Writ-ing: a method of inquiry. In N.K. Denzin & Y.S. Lincoln (Eds.), The Sage handbook of qualitative re-search, 3rd edition (pp. 959–978). Thousand Oaks, Ca: Sage Publications.

About the AuthorDiana M. Ruggiero is an assistant profes-

sor in the Department of Foreign Languages and Literature at the University of Memphis.


Appendix A. Screenplay Excerpt

SCREENPLAY IN SPANISHUn carro nos lleva por una avenida y vemos de paso a muchas casas a la derecha que están muy abandonadas, viejas y caídas. Entramos en un gate y ahí nos piden una identificación. Estacionamos frente a un edificio que está al lado de la piscina. En frente, muchas casas rodantes están mojadas, con olor a humedad. Mucha ropa está fuera secándose con lo que queda del sol de la tarde. Entramos al edificio.

VOLUNTARIA DE LA CRUZ ROJA¿Cómo está? Gracias por venir a ayudarnos.

INTÉRPRETESi, claro, está bien.

VOLUNTARIA DE LA CRUZ ROJABueno, vamos a comenzar, estamos llamando por nombre en orden de llegada.

SEÑOR LOPEZHola, gracias por la ayuda. Vengo porque mi casa se llenó de agua, casi no salimos, estamos muy mal. Mi hija está enferma y el olor a humedad ya no se aguanta.

VOLUNTARIA DE LA CRUZ ROJA¿Qué está diciendo?

INTÉRPRETEEstá diciendo que hubo agua dentro de su casa y que su hija no está bien…

VOLUNTARIA DE LA CRUZ ROJA(interrumpiendo)Bueno pero tenemos que llenar el formulario para que pueda cobrar el dinero del daño.

INTÉRPRETEEstá bien.

VOLUNTARIA DE LA CRUZ ROJAVamos a comenzar, ¿Me puede dar una identi-ficación?

El Señor Lopez no está feliz, no sabe que hacer. El necesita calmarse, sabe que no tiene una identificación y piensa que alguien lo puede de-portar. La voluntaria se da cuenta de su miedo.

VOLUNTARIA DE LA CRUZ ROJA No se preocupe, no somos el gobierno, solo so-mos una organización de voluntarios, queremos ayudarlo.

INTÉRPRETESeñor Lopez, muestre lo que tenga, hasta un pasaporte de su país de origen. Todo va a estar bien, solo queremos ayudarle. Estamos para servirle.

SCREENPLAY IN ENGLISH A car takes us on a main road. We pass many houses on the right. They are very abandoned, old and falling apart. We enter a gate and they ask for an ID. We park in front of a building that is next to a swimming pool. Across the street there are many trailers that are wet and musty. Many clothes are drying outside with what is left of the afternoon sun. We enter the building.

RED CROSS VOLUNTEERHow are you? Thanks for coming to help.

INTERPRETERYeah, sure, no problem!

RED CROSS VOLUNTEERWell, let’s start, we are calling them by order of arrival.

Mr. Lopez Hi, thanks for the help. I’m here because my house was filled with water. We almost did not make it out of the house before the water started to get in, we are not ok. My daughter is sick and we can’t stand the musty smell any longer.

RED CROSS VOLUNTEERWhat is he saying?

INTERPRETERHe is saying that there was water inside his house and that his daughter is not well...

RED CROSS VOLUNTEER(Interrupting)Good, but we need to fill in the form so you can collect money for the damage.

INTERPRETERIt’s OK.

RED CROSS VOLUNTEERLet’s start, Can you show me an ID?

Mr. Lopez is not happy. He does not know what to do. He needs to calm down, he knows he has no identification, and he is afraid someone might be deported. The Volunteer realizes his fear.

RED CROSS VOLUNTEER Do not worry. We are not the government. We are just an organization of volunteers, we want to help.

INTERPRETERMr. Lopez, show what you have, even a passport from your country of origin. Everything will be fine. They just want to help you. We are here to serve you.


Building Civic Capacity for College Students: Flexible Thinking and Communicating as Puppeteers, Community Partners, and Citizen-Leaders

Deborah Thomson, Rebecca Dumlao, and John Howard

AbstractCollege students face a complex world filled with pervasive social problems that require strong

knowledge bases, critical thinking abilities, and sustained engagement in civic life. This article details key pedagogical practices for our innovative health puppetry program, in which undergraduate honors students use puppets to share information about healthy eating, diabetes prevention, and active lifestyles with children and their families in community settings. We articulate a notion of “flexible thinking” as the ability to take on and perform new roles within the public/civic arena by seeing complex social problems from multiple perspectives and responding with creative solutions and engaged action. We look to the written reflections of our student puppeteers to share, in their own words, multiple ways their thinking and communication changed as they grew as puppeteers, community partners, and citizen-leaders. We also offer insights about promoting flexible thinking through in-depth service-learning.

In the spring of 2011, 13 undergraduate honors students from a variety of majors were part of the inaugural “Puppet Shows that Make a Difference!” class, a service-learning honors seminar team-taught by two of us with another of us serving as guest lecturer and project advisor. Our goal was to give our students the experience of using health puppetry to speak with children in our community about ongoing childhood overweight and diabetes issues. We spent the first half of the semester train-ing our students as puppeteers using large colorful puppets and scripts purchased from the longstand-ing educational puppetry organization The Kids on the Block (n.d.). We also taught our students an interdisciplinary course curriculum focused on interpersonal, intercultural, and small group communication, with guest speakers on topics like healthy eating, childhood obesity and diabetes, child development and family relations, and educa-tional principles for children. In the last month of our class, we visited nine different low-cost (or no-cost) after-school programs that partner with our university’s service-learning center. Our students performed the puppet shows to nearly 300 chil-dren. Each show was approximately 30 minutes long, with 20 minutes of scripted performance and 10 minutes of interactive dialogue.

But the puppet shows were more than just a set of community-based experiences. We recog-nized that students engaged in rich, well-designed, service-learning projects learn not only through hands-on experiences in the community; they also

learn through sustained self-reflection. In-depth written reflections can help teach students to con-sider their experiences thoughtfully to “generate, deepen and document” their learning (Ash & Clay-ton, 2009; Rama & Battistoni, 2001). Throughout our course, we asked our students to write weekly reflections, called articulated learnings (ALs) in response to prompts about key course topics. We also required a final essay that took the form of a longer AL that integrated and highlighted learn-ing from across the semester. The ALs were based on a model for critical reflection (Ash & Clayton, 2009). This model—known as the DEAL model—provided a clear framework for students to orga-nize and understand their own experiences via (D)escription, (E)xamination, and (A)rticulation of (L)earning in the journals so that they would glean new meaning from their community inter-actions, rather than just to have an experience out-side the classroom. This model not only encourages students to think in-depth and critically about topics being explored, but also easily lends itself to scholarly analysis.

Below we will argue that our health puppetry project led students to develop and engage in flexi-ble thinking and communication with members of their community. In our thematic analysis of stu-dent journals kept over the course of the semester, we find flexible thinking in a) perspective-taking shifts experienced by students as they enact the roles of puppeteer, teacher, leader, and citizen and b) increasing awareness of students’ sense of civic


responsibility and agency in “making a difference” as citizen-leaders. We end our analysis with sug-gestions for scholars and practitioners to promote flexible thinking through in-depth service-learn-ing courses. But first, we provide a brief literature review and a detailed explanation of our class and how it worked.

Literature Review and Background InformationToday’s college students face an increas-

ingly complex world filled with pervasive social problems that require the knowledge, skills, and informed agency to put learning into action in their communities. The National Task Force on Civic Learning and Democratic Engagement (2012) points out that “full civic literacies cannot be garnered only by studying books; democratic knowledge and capabilities also are honed through hands-on, face-to-face, active engagement in the midst of differing perspectives about how to ad-dress common problems that affect the wellbeing of the nation and the world” (p. 3). For students to truly care about problems, they need to experience them outside of what they may read as an example or statistic in a textbook. Or, as Battistoni (2013) states, a broad understanding of civic knowledge (and subsequent action) “includes a deeper knowl-edge of public issues, including their underlying causes as well as of how different community stake-holders understand issues” (p. 115–116). Students studying a health issue might, for instance, learn about the multiple facets of community and family life that could contribute to health issues as well as to become more sensitive to the cultural traditions surrounding food and health.

To be well-prepared to work in communities, then, students must be able to think about different perspectives on a problem, consider varied courses of action, and determine ways to collaborate effec-tively to create solutions that serve a larger purpose (Saltmarsh & Hartley, 2012). Indeed, students must be empowered to embrace an ethical civic identity (Knefelkamp, 2008). For example, students could learn that there are diverse professionals who could address complex health issues from varied perspectives and areas of expertise. Students could also be given opportunities to actively experience new ethical civic identities by playing unique roles in the community.

Further, as Minnich (2012) points out, “edu-cation and democracy both thrive on inquiry, on experimentation that may enable discovery” (p. 25). Effective civic action requires what we are calling “flexible thinking.” Flexible thinking is nec-

essary to take on and perform new roles within the public/civic arena. It requires the ability to see the “big picture” of a complex social problem, along with the ability to look at that problem from multiple perspectives, responding with creative solutions and engaged action. It requires the type of “fluid intelligence” Cattell (1963) described as necessary for “adaptation to new situations” (p. 3). Fluid intelligence is “the ability to be creative, make leaps of insight, and perceive things in a fresh and novel manner” (Potter, 2013, p. 78). Community-based performance helps students to develop this kind of flexible thinking, as the per-former must venture into unfamiliar territories of self and other, both as they take on the role of a character in a puppet show and as they take on the role of citizen-leader within their community. Structured reflection about community experienc-es can also help students develop flexible thinking as they must look back at what worked (or didn’t) in one situation and then be creative in considering new alternatives to use in the future.

Using Service-learning and Performance To Address Complex Social Problems

Recognizing the importance of students’ civ-ic learning as well as building their capacity for flexible thinking and action, we created a ser-vice-learning course to take on the problem of childhood overweight and diabetes, both of which are severe problems in our community. In North Carolina two thirds of all adults (67.5%) are over-weight or obese. The state also ranks 50th in the United States for rates of childhood obesity (Pitt County, 2008). In our county, it is estimated that 40% of elementary-age children and adolescents are overweight or at risk of becoming overweight Pitt County, 2008). Given the pressing social issues facing us, these interactive puppet shows were not just “fun” (although they were that); they were a way to share accurate information on healthy eating and active lifestyles with children at-risk of developing obesity and eating-related health problems such as diabetes.

These puppet shows were also a way for our students to learn from the children in their com-munity about what kids face when it comes to eating well, using fluid intelligence to gain new perspectives on these health problems. All of our puppet shows contained both a scripted scene and a time to interact, so children in the audience could speak directly with the puppets. Our puppets, for instance, asked the children questions like “What could you eat for a healthy snack?” And our pup-


peteers were frequently surprised by the children’s answers. In this sense, our project created a unique learning partnership between college student learners and elementary school learners, with pup-pets in the middle. As Bringle and Hatcher (2002) point out, campus-community partnerships are developed through person-to-person interactions that are dynamic. Moreover, partnerships between campus members, such as our university students, and community members, such as the children, are recognized to be both complex and challeng-ing (Jacoby, 2003), since the partners come to the interactions “from different worlds” (Sandy & Holland, 2006, p. 30). So, as a primary learning goal, we hoped our students would “learn to put themselves back into the mix of humanity…work-ing ‘with’ people (in the community) rather than ‘on’ them” (Boyte, 2009, p. 15), being able to shift their thinking accordingly. We also hoped that our students would see themselves increasingly as citizen-leaders “making a difference” in the very communities where they were performing.

Block-Schulman and Jovanovic (2010) state that “service-learning programs work because they engage students wholly—involving the intel-lect, the body, and the emotions in a social arena to assert an ethical posture (as active citizens)” (p. 93). Performance lends itself to this holistic level of engagement, because the student-as-perform-er engages the flesh, the memory, the senses, the emotions, the voice, and even the human spirit in the act of performing. Pineau (2002) describes performance as “a medium for learning,” one that “requires the rigorous, systematic, explora-tion-through-enactment of real and imagined experience in which learning occurs through sensory awareness and kinesthetic engagement” (p. 50). This type of performative learning through embodiment is closely aligned with Minnick’s (2003) notion of deeply democratic thinking as “play” in which students get “caught up in imagi-

native moments, not tied down to or locked within what he or she already knew or what logically fol-lowed” (p. 24). We found that this form of “play” as a learning strategy was quite effective, as we discuss below.

Our Course: Puppet Shows That Make A Difference!

Our “troupe” included 13 undergraduate honors students who were majoring in disciplines as diverse as chemistry, criminal justice, commu-nication, biology, accounting and nursing, among others (see Figure 1). They were primarily soph-omores and juniors, and they were fairly evenly split by gender. Almost half of the class planned to go into a health profession, and most of these in-tended to attend medical school after college. Only 3 of our 13 claimed to have any performance expe-rience; only one had ever worked with puppets. We were also lucky to have three teaching assistants, graduate students in our department’s master’s program in health communication. We had trained these students as puppeteers during the previous semester, both for our own process of learning to work with our new puppets and so that they would, in turn, be able to help our undergraduate students with the difficult skills of puppeteering such as eye focus, puppet gesture, and lip sync.

We had two faculty members team teach-ing this course. One of us has extensive perfor-mance experience, and the other has extensive service-learning experience. Both of us are also graduates of our university’s Engagement and Out-reach Scholar’s Academy, an intensive program to learn about community-engaged scholarship with institutional and financial support to develop new projects. Because both of us study and teach communication, we invited a number of colleagues from areas such as nutrition, pediatrics, and child-hood development to present on these topics to give our students interdisciplinary knowledge.

Figure 1. Puppeteer Class


These areas of understanding were key to our project, not just because we wanted our students to see the “big picture” of childhood overweight and diabetes, but also because they would need to be able to answer children’s questions on these topics in their puppet shows.

To provide a strong sense of the challenge our students faced in learning to be puppeteers, we must first describe these puppets. These were not hand puppets. They were huge, kid-sized puppets with arm rods, requiring both physical strength and substantial performance skills by puppeteers (see Figure 2). This style of puppetry is based on the ancient Japanese Bunraku style in which the performer stands behind the puppet dressed in black, as a kind of shadow to the puppet. The performer also wears a black mesh hood in order to see the audience, allowing for better interaction. Although the shows were fully scripted (and copy-right demanded no deviation from these scripts), each performed puppet script was followed by an interactive question and answer session between the kids in the audience and the puppets. Thus the need for interdisciplinary learning by the students and another need for them to quickly flex their thinking from being a puppet using a script to being a puppet that could accurately and respon-sively answer each child’s questions.

We were able to purchase the puppets and scripts with grant funding from our university. As part of this, we were also able to bring in two train-ers from the head office of The Kids on the Block, who conducted a one-day puppeteering workshop with our students. Consequently, the students were very well prepared as puppeteers and had many chances to practice and develop their puppet skills both in and out of class before actually doing the community puppet shows.

Throughout our course, we asked our students to write weekly reflections in response to prompts about key course topics. Both the prompts and the students’ writing followed the DEAL model for critical reflection (Ash & Clayton, 2009; Ash, Clayton, & Moses, 2009; Ash, Clayton, & Atkin-son, 2005). The DEAL model encourages stu-dents to describe and then explain their learning using the following sentence stems: “I learned that…; I learned this when…; This learning matters because…; In light of this learning….” Our discussion prompts covered topics within three categories: personal growth, civic learning, and ac-ademic enhancement. Topics included community partnerships, leadership, performing with chil-dren, teamwork, exploring cultures, and making

a difference, among others. The ALs also traced students’ experiences as they learned about the academic foundations of their practices and as they learned from their interactions with com-munity members as engaged citizen-performers. The final AL essay called for an integration of the multiple dimensions of their experiences.

Research Questions and MethodOur investigation of service-learning and flex-

ible thinking considers the intersections among role playing/perspective taking, civic engagement, and flexible thinking in students’ writings. That is, we looked carefully at the essays to see whether our primary learning goals for students were being met. To this end, we posed the following research questions:

RQ1: How did playing new roles (puppe-teer, teacher, etc.) contribute to students’ abilities to take the perspective(s) of oth-ers (and enact flexible thinking)?RQ2: What did the service-learning expe-riences mean to students, relative to their citizenry and perceived ability to facilitate change?

Students were asked to participate in the study, but had the option not to include their responses in the research. They were told that their reflections would be used to analyze the content and critical thinking. So, while we recognized that the prompts would influence the content of their responses, we also expected that the structure would give our students some grounding in what to write about (as opposed to open-ended journaling where students may choose to simply recall experiences without delving deeper). The DEAL model encourages students to think deeply about the content of their learning, as well as their related thoughts, feelings, and developing skills. As researchers, our hope

Figure 2. Puppeteers


was to gain some understanding of the meanings and significance our students would make of their service-learning and related classroom experiences. Consequently, our primary method of analysis is focused on generating understanding as it emerges from students’ own voices (i.e., grounded theory).

We chose the “constant comparative method” (Lindlof, 1995, p. 222) so students’ voices could be heard without imposing a rigid theoretical frame. This approach enables researchers to develop com-mon and overarching themes, frames, and principles through a systematic but flexible form of inquiry. Two important aspects of this method are that “it specifies the means by which theory grounded in the relation-ships among data emerges through the management of coding (hence, grounded theory), and it shows explicitly how to code and conceptualize as field data keep flowing in” (Lindlof, 1995, pp. 222–223). In this study, thematic elements were identified in the student reflections and compared across authors. Below we share the themes that emerged from our analysis and the connections among them.

Learning New Roles: Puppeteer, Community Partner, Citizen-Leader

Minnick (1985) lists “play” as a characteristic of the kind of thinking that we should be teaching in the college classroom, noting that “Such play is not always fun: It can take us to scary places. But it also unclenches, releases” (p. 24). Many of our students began the semester feeling “clenched” about what lay ahead. As honors students, they were accustomed to performing well on tests and receiving high marks on essays. But, for most, performing was an entirely new venture, one squarely outside of the proverbial “comfort zone.”

Interestingly, in their first week of journaling about the class, these 13 students collectively used the word “comfort” 21 times, all basically describing how uncomfortable they were about performing. Rep-resentative of this was Mike, a pre-med student, who described the discomfort he experienced when students were asked to sing their names as part of a warm-up activity. Mike wrote that the activity “forced students like me out of their zones of comfort.” He elaborated:

I was slightly taken aback on the first day of this class when asked to sing in front of my classmates for several short periods as a part of warm-up exercises. I do not often sing in front of people, and much less often in front of those I barely know.

Ron, too, described the anxiety he felt upon reading scripts aloud for the first time, writing, “I sink into my chair, excessively, unnecessarily fearful of being selected to assume the vocal identity of a character about whom I know slightly more than nothing at all.” This process of taking on a character would eventually prove liberating for our students, freeing them up for other possibilities of self. But this “liberation” was only after hours of practice and in-creasing self-confidence as performers and educators.

Moving Between Self and Other Richard Schechner (1985) theorizes the restored

behavior of taking on a theatrical role as “me behaving as if I am someone else” (p. 37). Schechner describes the performer’s stance as she inhabits the character as a dance between the “not me” and the “not not me,” explaining:

While performing, a performer experienc-es his own self not directly but through the medium of experiencing the others. While performing, he no longer has a “me” but has a “not not me,” and this double negative rela-tionship also shows how restored behavior is simultaneously private and social. A person performing recovers his own self only by go-ing out of himself and meeting the others—by entering a social field (p. 112).

For our students, this interplay between self and other was not an easy process. The challenges of performance went beyond learning the physical me-chanics of puppetry such as lip sync, eye focus, and gesture. It was entering Schechner’s “social field” in which the self would be doubly displaced, first as our students took on a puppet character and then a second time as they came into dialogue with their young audi-ences, who offered them a child’s eye view of the world.

Interestingly, it was the mental image of this future performance partner, the imagined children, which brought Jada some comfort as she navigated the anxieties of performing before her peers in these early days of our class:

During my reading I focused so much on the part that everyone else in the room seemed to disappear. While I continued to talk as Chris-tine I began to imagine myself in a room full of young children. … I believe that I was able to get into the reading as I began to imagine children because I love working with children. I know that they are not judgmental and that they love it when adults act crazy.


Ron also navigated performance anxieties with an eagerness for learning this new role when he wrote in his first reflection: “I am eager to start practicing with the puppets, and molding my own personality to theirs.” Ron likely intended to com-municate that he would start the process of shaping his character’s personality (rather than his own). But the mistake is revealing of what would happen for many of these students, as the process of taking on a character freed them up for other roles and for seeing the world from other points of view, to think more flexibly.

Teaching and LeadingIn Hersey and Blanchard’s (1977) situational

leadership model, leaders can delegate to their followers (in other words, let their followers lead), when the follower’s readiness level is high. In our case, this happened when the students were ready to go off script in the interactive portion of their shows. Having embodied the knowledge about healthy eating and diabetes through rehearsal and performance, they were ready to teach, and they were ready to lead others. After eight weeks, our students stepped out into their community to perform. As they did this, they took on a new role, that of teacher. Arnold, another pre-med student, spoke to this challenge and the learning opportunities it presented when he wrote:

Questions like “Can diabetes kill you?” were hard to answer when dealing with children, but showed that they were relat-ing to the topic and trying to understand as much about it as possible. Dr. Collier came and instructed us on the medical aspects of diabetes in order to improve our knowledge so we could answer ques-tions with factual information. This was knowledge that we could apply immedi-ately to the performances that were in our immediate future, something that is a rar-ity in college courses.

Arnold imagined this learning continuing into his future role as physician, stating “I will also continue to learn as much as possible about diabe-tes, and continue to educate my community about the disease in some way.”

As our students became more comfortable with their roles as puppeteers, they shifted into new roles as teachers and leaders. This was clearly demonstrated during the one show when students had to think on their feet because they received

no questions from the audience. We had become accustomed to a room full of little hands popping up when the puppets took questions. But in this location the children had been highly disciplined to sit still and say nothing, and this discipline contin-ued into the interactive portion of our show. After a prolonged silence ended the show, our students came out and took their bows. They put their pup-pets down, took their black hoods and gloves off, and began to interact with the children as them-selves. Maggie started off by saying to the group, “You really don’t have any questions about healthy eating or diabetes?” Then a question came, and then another, turning into our longest audience dialogue and the only one that took place between performer and audience with no puppet in between.

Perspective TakingThese sorts of dialogic experiences with chil-

dren in our community challenged our students to see the beyond their own frames of reference. Melanie writes about this learning when she first had to answer the question “What is diabetes?” posed by a young person in her audience. Melanie stated that her “first instinct […] was to begin by explaining the function of glucose in the cells and its conversion to energy in order to be used for carrying out daily activities.” She quickly realized that a highly technical explanation would likely not be met with wide understanding from the K-2 crowd. Melanie continues:

As I gained performance competence, I started to think more about who my audience was, and what kind of answers they would understand. This learning matters because it made me realize who I was answering, and take into account what their level of understanding about biology would be as an elementary school student. In light of this learning, I began to think like a kid when the question and answer portion of the show began so that I would be able to answer questions in a way the child asking the question would comprehend.

Melanie not only had to think like her character to answer the question, she had to think like her audience, displaying the kind of entry into Schech-ner’s (1985) “social field” that enables a performer to escape the perspective of the self, however briefly. Maggie also spoke about this when she was sur-


prised by the willingness of the children in her audience to share their own stories, writing:

Once we began talking about diabetes, I realized through the children’s numerous comments that many of them had first-hand experience with diabetes through family members, but also that they did not know much about the condition.

Our students were now not just learning about the problem of diabetes in our area from the perspec-tive of a physician or a professor, they were learn-ing from the personal stories of children with a diabetic grandmother who had died from the disease or a diabetic uncle who took shots of insulin every day.

Making a DifferenceWe had titled our course “Puppet Shows that

Make a Difference!” hoping that our planned puppet shows would do just that in our community as our students shared information about healthy eating and diabetes with youth at risk of diet-related health problems. Although we did not assess the impact that these puppet shows had on our audiences, what we found was that the perception of making an impact was deeply meaningful to our students. Caitlin’s final essay exemplified this:

I felt at the beginning of this class that it would not be possible for me to change the world, the nation, or even a city on my own. I seemed to block out the “…That Make a Difference” part of the class. I did not see how it was possible for a freshman in college to do anything that would have enough impact to be considered “mak-ing a difference”. I realized that if I could help one child, this class, and ultimately I, would be a success. Yet, the complete breakthrough did not occur during [the lectures]. […]Finally, on the day in which we performed for the children, a complete breakthrough occurred. I saw the children and how receptive they were. I saw their excitement, and the excitement of the adults around me. It finally seemed real. I had made a difference.

Of course, we should be skeptical that these puppet shows could affect the kind of dramatic societal changes that Caitlin articulates. But we can see in her optimism and investment in community that spark of what it feels like to be an agent of change.

Astin and Astin’s (1996) Social Change Model of Leadership Development posits that there are three levels of change for engaged citizenship: individual, group, and community. At the first level, individu-als gain a consciousness of self and a commitment to the value of civic engagement. Next, at the group level, individuals come together for collaboration and common purpose. This sets the stage for the final level: community engagement. Christine Cress (2011) also writes about how service-learn-ing can engage students in these three stages, noting that “students are able to extend their intellectual capacities to include empathy and problem solv-ing that will have a real community impact” (p. 78). She continues: “in this spirit, service-learning offers students the opportunity to become critically conscious citizens with the knowledge and skills for creating more equitable democratic communi-ties” (p. 78). Our student Caitlin echoes this notion as she ties the work of performance to the work of democracy and citizenship, writing:

A belief in the ability to make a difference is even more important to the citizens of this and any other democratic nation. Democracy thrives on the belief of its citi-zens that they can make a difference. This is what drives people to the voting booths, to rallies, and to signing or forming peti-tions. If the belief in a single person’s ability to change the world was lost, I believe that democracy itself would be lost as well.

Caitlin’s conception of perceived civic agency as fundamental to a thriving democracy offers an insight into the kind of flexible thinking that our service-learning puppeteers experienced as they went out to perform “making a difference.”

Embracing CommunityPart of perceived civic agency, for our

students, was embracing the idea that they were part of multiple communities, including the sometimes overlooked ones surrounding their university, and that they needed to be an active participant, a citi-zen, in those communities. Justin, in his final paper, thought about this as he discussed the importance of “teamwork,” writing:

Last, but definitely not least, the most important teamwork was with our class and our partners in the community, both the adults and the children. This was the most delicate and important version of


teamwork we learned about this semester. I learned that we, as East Carolina University students, as current Pitt County residents, as citizens of the United States of America, have a civic responsibility to improve and take part in each and every community we are involved in. Every community from my immediate family to the enormous student body we have at East Carolina flourishes only when we work together to try to improve it.

Justin here identifies several “communities” that he is a part of—from his family, to his univer-sity, to his current county of residence. Justin’s senti-ments speak to the power of service-learning to help invest students with what Justin aptly terms the “civic responsibility to improve and take part in” the com-munities of which we are a part. Another way to think about this is that Justin was moving from the “not me” to the “not not me” on his return from his journey through the “social field.” The larger com-munity of Pitt County was now not-not him. His statements also demonstrate that he was able to think about community from multiple perspectives.

By the end of the semester, many of our students were reconceptualizing themselves as community leaders. Arnold wrote about this new citizen-leader role when he stated:

We are becoming leaders by consciously choosing to symbolically communicate the importance of eating healthily to chil-dren through puppetry. … Leaders are not perfect people. They can be anyone. A leader is simply someone with the right skills who, when the opportunity arises, has the courage to step forward and say, “I can make a difference, starting today.”

Melanie had this same realization, noting the importance of each individual’s contributions as part of the larger collective effort that is community leadership. She wrote:

I had a few epiphanies about making differences in the world around us. I real-ized that you already have what it takes to make the world a better place. Making a difference to the world may seem like an enormous task, but it is in fact the col-lective effort of everyone to make small contributions with a lot of heart. The size of the contribution is not what matters

most. The key here is to have the heart to do it.

In both Arnold’s and Melanie’s sentiments, we see students who are conceiving of themselves as leaders as they rethink what citizen-leadership is, something that does not necessarily come only from “perfect people,” in Arnold’s conception, or from “enormous tasks,” in Melanie’s. It can come from those who choose to step forward, even in small acts.

Building Capacity for Civic EngagementOur inaugural puppetry course demonstrates

what can happen when students are given the opportunity to focus in-depth on a communi-ty-based issue/problem. That is, our students were asked to consider multiple human perspectives, to draw on insights from multiple disciplines as well as across specialties within a single discipline (i.e. small group communication, leadership commu-nication, and interpersonal communication) and to do this in a single semester. This type of inter-disciplinary and cross-disciplinary learning is, unfortunately, a rarity in most college classes that are instead structured to focus on subject matter rather than giving students a “breadth of under-standing” about an issue. Admittedly, there are many structural and functional obstacles in the way. As Fitzgerald, Burack, and Seifer (2010) note:

Pressures to build strong university-com-munity collaborations pose difficult prob-lems for the academy because they demand interdisciplinary cooperation, rejection of provincial disciplinary turfism, changes in the faculty reward system, a re-focusing of unit and institution missions and the breaking down of firmly established and isolated silos (p. x).

Still, we must overcome such challenges, because providing students with a greater breadth of understanding is essential to enabling them to think deeply about the complex issues we face in today’s world. Being a contemporary problem-solv-er frequently requires more than just disciplinary knowledge; civic action necessitates higher ordered flexible thinking that applies, evaluates, and integrates knowledge. Indeed, focusing on a single community-based issue rather than on a subject specialty area is one way to help students capture the complexity of a contemporary problem (in our case, eating-related health problems for chil-


dren). This offers a new possibility for what others have identified as the important process of “doing democracy” and “fostering civic action” (The National Task Force, 2012).

Our course, heard through the student voices in their reflections, reiterated to us how important “big” (pressing, social) issues are in our classes and how students can be empowered to address some aspect of those “big” issues. Just as there may be many partners necessary to solve complex prob-lems, there could be possibilities for students representing many disciplines to work together. Working with different partners and being able to think about an issue from different perspectives is important far beyond any single issue. In a diverse global society, everyone must interact regularly with people representing different experiences and assumptions. So, the abilities to look at a problem from multiple perspectives will be essential to col-lege graduates functioning as active citizens in very dynamic and ever-changing democracies.

Our project had several challenges and limita-tions. One was simply the labor-intensive nature of this type of performance work. Many hours were spent outside of the classroom conducting rehearsals and scheduling performances. There was a very steep learning curve here! For example, one of the lessons we learned was that our child audience members were more attentive audience members when they had their teachers sitting nearby. This was a lesson we learned the hard way at an after-school program where the teachers left and we had to monitor behavior while trying to do our jobs as performers.

Another limitation had to do with the depth of partnering that took place in our project. While individual college courses can be subject to the limitations of a semester-long commitment to a community program (See Stoecker & Tryon, 2009), classes that are issue-focused like ours can be part of a longer-term programmatic effort or what Heath and Frey (2004) call “communi-ty collaboration.” These authors rightly note that “in most communities today, it is a necessity for groups, organizations, and institutions to work together collaboratively to confront complex issues” (p. 189). In subsequent puppetry proj-ects, we have worked to partner more deeply with one agency, and to write our own scripts based on interviews and focus groups with members of our community on the topic of healthy eating. Similarly, scholars from other disciplines could partner with community members to develop interactive puppet shows to address other commu-

nity issues. We see nutrition students developing interactive puppet shows to teach children about healthy eating and to learn about children’s eating habits. We see dentistry students doing the same with dental health or sport science students doing the same with exercise (two topics our continuing puppetry students have been working on).

Although we chose not to conduct pre- and post-tests to attempt to quantify our student’s growth, their reflections showed us what they learned while interacting with the children. These opportunities to learn-while-doing were invalu-able and went far beyond the cognitive content emphasized in many, even most, college courses. They offered positive emotion, excitement, and “in the moment” kinds of thoughtful responding. We saw that allowing our students the opportu-nities to “play” both in and out of the classroom helped them to get more excited about not just the “thinking” part of their work, but also about the community-based “doing.” We hope that future scholars and practitioners will develop new ways to capture the element of this rich “play” that can invigorate and motivate both learning and doing. Staying motivated and having fun is not just nice, it is fundamental to opening up new possibilities and to keeping us enthusiastic enough to overcome problems and dilemmas we face along the way.

This service-learning course has also shown us the value of in-depth written reflections. We agree with Ash and Clayton (2009) that critical thinking through reflection requires careful consideration and planning to specify detailed learning outcomes and to design reflections that help students achieve those specific outcomes. We also recognize the importance of aligning our student outcomes with assessing the overall community impact—some-thing that we admittedly did not address in the single semester allotted for this course. We did not gather formal feedback from the children we per-formed for, but we did learn from the places where children laughed or were especially engaged and from questions and comments they gave. In the future, seeking more formal feedback can help us to improve the quality of our performances and to understand what our audiences are getting out of them. Still, we realize that teaching toward com-munity impact and designing related assessments will also require broad, interdisciplinary efforts over time. Researchers need to work with practi-tioners to determine what will work to demonstrate impact and to show the long-term sustainable commitment to making positive changes. Students can have a part to play in identifying what really


matters. Working as co-educators (Zlotkoswki, Long, & Williams, 2006) and solo educators for others (like the children in our audiences), ser-vice-learning students offer real potential for look-ing at long-standing situations with fresh eyes… and that may be just where innovative responses to “what’s never been done” can begin to take root.

While our study demonstrated a single semester’s course projects, we are well aware that the community partnerships started through these classes require our long-term commitment and efforts, even as we train new puppeteers and work with other community-based programs focused on our chosen issue. As Enos and Morton (2003) aptly note, ideal service-learning is not just about the single transactions between partners, but rather about “the continuing possibility (that partners) will be transformed in large and small ways” (p. 20). We wholeheartedly agree. Teaching for flexible thinking has real possibilities to change students, faculty, universities, and communities as we, together, address the issues that demand our attention. Our students, working collaboratively with others on and off campus, began to see how their own efforts are part of the larger whole, a part of a complex problem’s long-term solution. They saw how they “can make a difference” as players in community collaboration. This is the exciting civic challenge we face; it is also what is most needed for a bright future for our communities, for us all.

ReferencesAsh, S., & Clayton, P. (2009). Generating,

deepening, and documenting learning: The power of critical reflection in applied learning. Journal of Applied Learning in Higher Education, 1(1), 25–28.

Ash, S., Clayton, P., & Atkinson, M. (2005). Integrating reflection and assessment to capture and improve student learning. Michigan Journal of Community Service-learning, 11(2), 49–59.

Ash, S., Clayton, P., & Moses, M. (2009). Learning through critical reflection: A tutorial for service-learning students. Self published.

Astin, A.W., & Astin, H.S. (1996). A social change model of leadership development: Guidebook, Version III. Los Angeles, CA: Higher Education Research Institute.

Battistoni, R. (2013). Civic learning through service-learning: Conceptual frameworks and research. In P. Clayton, R. Bringle, and J. Hatcher (Eds.), Research on Service-learning Conceptual Frameworks and Assessments, Vol. 2A: Students and Faculty, pp. 111–132. Sterling, VA: Stylus.

Bloch-Schulman, S., & Javonovic, S. (2010).

Who’s afraid of politics: On the need to teach political engagement. Journal of Higher Education, Outreach, and Engagement, 14(1), 83–100.

Boyte, H. (2009). Repairing the breech: Cultural organizing and the politics of knowledge. Partnerships: A Journal of Service-learning and Civic Engagement, 1(1), 1–21.

Brammer, L., Dumlao, R., Falk, A., Hollander, E., Knutson, E., Poehnert, J., Politano, A., & Werner, V. (2012). Core competencies in civic engagement. Journal of Service-Learning in Higher Education. University of Louisiana System. Retrieved from https://www.merrimack.edu/live/files/160-core-competencies-in-civic-engagement.

Bringle, R., & Hatcher, J. (2002). Campus-community partnerships: The terms of engagement. Journal of Social Issues, 58(3), 503–516. doi:10.1111/1540-4560.00273.

Cattell, R.B. (1963). Theory of fluid and crystallized intelligence: A critical experiment. Journal of Educational Psychology, 54(1), 1–22.

Pitt County Pediatric Dietitians and Nutrition Educators, (2008). Achieving a healthy weight in children. North Carolina Healthy Weight Initiative.

Cress, C. (2011). Consensus, collaboration and community: Mutually exclusive ideas? In C. M. Cress, D.M. Donahue and Associates (Eds.) Democratic dilemmas of teaching service-learning: Curricular strategies for success, pp. 73–81. Sterling, VA: Stylus.

Enos, S., & Morton, K. (2003). Developing a theory and practice of campus-community partnerships. In B. Jacoby and Associates (Ed.). Building partnerships for service-learning (pp. 20–41). San Francisco, CA: Jossey-Bass.

Fitzgerald, H., Burack, C., & Seifer, S. (2010). Handbook of engaged scholarship: Contemporary landscapes, future directions, Vol. 2: Community-campus artnerships. East Lansing, MI: Michigan State University Press.

Heath, R., & Frey, L. (2004). Ideal collaboration: A conceptual framework of community collaboration. In P. Kalbfleisch (Ed.), Communication yearbook 28 (pp. 189–231). Mahwah, NJ: Erlbaum.

Hersey, P., & Blanchard, K.H. (1977). Management of organizational behavior, (3rd ed.). Englewood Cliffs, NJ: Prentice Hall.

Jacoby, B., and Associates (Ed.). (2003). Building partnerships for service-learning. San Francisco, CA: Jossey-Bass.

The kids on the Block Puppets. (n.d.). Available at: http://kotb.com/.

Knefelkamp, L.L. (2008). Civic identity:


Locating self in community. Diversity and Democracy, 11(2), 1–2.

Lindlof, T.R. (1995). Qualitative communication research methods. Thousand Oaks, CA: Sage.

Minnich, E. (2012). Educating democratically: An interview. Higher Education Exchange, (Kettering Foundation), pp. 24–34. Available at https://www.kettering.org/wp-content/uploads/4_HEX2012_Minnich.pdf.

Minnich, E. (2003). Teaching thinking: Moral and political considerations. Change, 35(5), 18–24.

Pineau, E.L. (2002). Critical performative pedagogy: Fleshing out the politics of liberatory education. In N. Stucky and C. Wimmer (Eds), Teaching performance studies, pp. 41-54. Carbondale, IL: Southern Illinois University Press.

Potter, W.J. (2013). Media literacy, 6th edition. Los Angeles: Sage.

Rama, D., & Battisoni, R. (2001). Service-learning: Using structured reflection to enhance learning from service. Available at http://www.compact.org/disciplines/reflection/index.html.

Saltmarsh, J., & Hartley, M. (2012). “To serve a larger purpose”: Engagement for democracy and the transformation of higher education. Philadelphia, PA: Temple University Press.

Sandy, M., & Holland, B.A. (2006). Different worlds and common ground: Community partner perspectives on campus-community partnerships. Michigan Journal of Community Service-learning, 13(1), 30–43.

Schechner, R. (1985). Between theatre and anthropology. Philadelphia, PA: University of Pennsylvania Press.

Stoecker, R., & Tryon, E. (Eds.). (2009). The unheard voices: Community organizations and service-learning. Philadelphia, PA: Temple University Press.

The National Task Force on Civic Learning and Democratic Engagement. (2012). A crucible moment: College learning and democracy’s future. Washington, DC: Association of American Colleges and Universities.

Zlotkowski, E., Longo, N.V., & Williams, J.R. (2006). Students as colleagues: Expanding the circle of service-learning leadership. Providence, RI: Campus Compact.

Author’s Note

This research project was approved by the University and Medical Center’s Institutional Review Board as research project #11-0147. Following our IRB protocol, we sought written informed consent to access the student journals at the start of the semes-ter. A graduate student collected the consent forms, and they were kept by one of our administrative staff members until after grades were posted at the end of the semester, so as not to create pressure on students to consent. We have changed the names of our students to preserve anonymity.

About the AuthorsDeborah Thomson is an associate professor and

John Howard and Rebecca Dumlao are professors, all in the School of Communication at East Carolina University.


Cultural Adaptation of a Substance Abuse Prevention Program as a Catalyst for Community Change

Melissa Tremblay, Lola Baydala, Natasha Rabbit, Jennilee Louis, and Kisikaw Ksay-yin

AbstractThe aim of the current paper is to discuss the use of Outcome Mapping as a tool for evaluating

community and stakeholder changes that occurred when a prevention program was culturally adapted and delivered through a community-university partnership. To the authors’ knowledge, this paper represents the first account of using Outcome Mapping as an evaluation tool in a Canadian Indigenous context. Changes in the behavior, actions, activities, and relationships of five boundary partners were retrospectively documented using the tool. Data demonstrated positive impact on Elders and students; growing community investment in and support for the Maskwacis Life Skills Training program’s cultural components; progressive increases in community ownership of the program; and growth in the community-university partnership. Overall, Outcome Mapping provided a systematic method for understanding peripheral changes that are often overlooked in conventional research and evaluation, but that nonetheless indicate progress toward community changes and long-term impact.

IntroductionGrowing evidence for the value of com-

munity-based participatory research (CBPR) (Israel, Schulz, Parker, & Becker, 1998; Minkler & Wallerstein, 2003) has resulted in a proliferation of studies that utilize this approach (Jagosh, Macau-lay, Pluye, Salsberg, Bush, Henderson, Sirett, Wong, Cargo, Herbert, Seifer, Green, Greenhalgh, 2012). With an emphasis on translating research findings for use in communities, bidirectional learning and capacity building, equitable involvement, and hon-oring multiple forms of knowledge, CBPR does away with the conventional hierarchy between researchers and those being researched. In this way, CBPR is particularly suitable for work with Indigenous communities (Castleden, Morgan, & Lamb, 2012; Cross, Friesen, Jivanjee, Gowen, Ban-durraga, Matthew, & Maher, 2011; Gauld, Smith, & Kendall, 2011). Indeed, the option of commu-nity participation in research affecting Indigenous peoples is recognized as an ethical imperative (Ca-nadian Institutes of Health Research, 2007). When community control and agency are emphasized and Indigenous community members contrib-ute in meaningful ways to the research process, local capacity is strengthened and community-lev-el changes can occur (Kelly, Saggers, Taylor, Pearce, Massey, Bull, Odo, Thomas, Billycan, Judd, Reilly, & Ahboo, 2012; Salimi, Shahandeh, Malekafzali, Loori, Kheiltash, Jamshidi, Frouzan, & Majdza-deh, 2012). The task of evaluating and measuring community-level change is not simple or straight-

forward. Although the randomized controlled trial has long been recognized as the gold standard for academic research, experimental approaches are often inappropriate for evaluating programs that take place in community settings (Judge & Bauld, 2001; Victora, Habicht, & Bryce, 2004). Kelly (2010) notes that communities are complex and constantly evolving; as such, evaluations of community-based initiatives cannot adhere to a single theory or model, and cannot test direct cause-and-effect relationships as in a closed sys-tem. In response to concerns with utilizing exper-imental designs to evaluate complex community initiatives, innovative evaluation approaches are beginning to emerge (Dart & Davies, 2003). Out-come Mapping (Earl, Carden, & Smutylo, 2001) represents one such approach. The aim of the cur-rent paper is to discuss the use of the tool for eval-uating community and stakeholder changes that occurred when a substance abuse and violence pre-vention program was culturally adapted and deliv-ered through a community-university partnership. To the authors’ knowledge, this paper represents the first study that uses Outcome Mapping as an evaluation tool in a Canadian Indigenous context.

The Community and the ProjectThe Maskwacis community, located in central

Alberta, Canada, is made up of four Cree Nations: Louis Bull, Montana, Samson, and Ermineskin. The four nations have a combined population of approximately 15,000. These neighboring na-


tions function independently, with separate chief and councils, education directors, and schools. Approximately 53% of the Maskwacis popula-tion is under the age of 18 (Grekul & Sanderson, 2011). As a result of colonization and its ongo-ing impact on Indigenous peoples, high rates of substance abuse and acts of violence pose a challenge for many First Nations communities in Canada. In response to these challenges, Mask-wacis community members invited University of Alberta researchers to partner with them to cul-turally, adapt, deliver, and evaluate a substance abuse and violence prevention program to chil-dren and youth in Maskwacis schools. In order to realize this goal, a community-university partner-ship was formed in the spirit of CBPR. The first stage of the project involved culturally adapting the evidence-based Life Skills Training (Botvin, Baker, Renick, Filazzola, & Botvin, 1984; Botvin & Griffin, 2014) program to incorporate Cree cul-ture, language, and values. Elders and community staff led the cultural adaptation process (Baydala, Fletcher, Tremblay, Rabbit, Louis, Ksay-Yin, & Sinclair, 2016). The adapted Maskwacis Life Skills Training (MLST) program was subsequently de-livered for three years in Maskwacis schools by community program facilitators.

Evaluation of the MLST program was carried out over three years of program implementation. The evaluation consisted of (1) focus groups conducted at the end of each program year with school personnel, Elders, program facilitators, parents, and students to obtain feedback on the program, discuss suggestions for improvement, and understand the successes and challenges of the program; and (2) pre- and post- program questionnaires distributed to student partici-pants (National Health Promotion Associates, 2011a, 2011b). By the third year of implementa-tion, it was clear that MLST program effects were extending into the wider community. Partners were aware of a web of individual and commu-nity-level outcomes that could not be compre-hensively captured with questionnaires or focus groups alone. Further, program facilitators often shared meaningful stories about program impact, but did not feel that evaluation processes allowed for documenting this informal data. According-ly, Outcome Mapping (Earl et al., 2001) was uti-lized to supplement the evaluation of the MLST program. This paper will share project findings documented through Outcome Mapping, describe the use of Outcome Mapping as a retro-spective evaluation tool, and discuss the suitability

of this method for use in a CBPR partnership as well as with Indigenous communities.

MethodsOutcome Mapping

Outcome Mapping was developed in Canada by the International Development Research Cen-tre, and was released to researchers and practi-tioners in 2001. Since that time, Outcome Mapping has mainly been used in Africa, Latin America/ Caribbean, and Asia, with only 2% of reported Outcome Mapping use in North America (Smith, Mauremootoo, & Rassmann, 2012). Although the tool was created for use in an international development context, research has documented its potential applicability to more diverse settings, including economically developed countries (Smith et al., 2012).

Outcome Mapping is a participatory approach to planning, monitoring, and evaluation. It rep-resents a shift from defining results in terms of long-term impact to defining results in terms of observable changes in partners’ behaviors, actions, activities, and relationships. Although long-term impact is the ultimate goal of community-based projects, Outcome Mapping recognizes the importance of also tracking smaller-scale, incre-mental changes that indicate meaningful progress. An exclusive focus on impact can preclude a focus on these incremental changes, “severely limiting…potential for understanding how and why impact occurs” (Earl et al., 2001, p. 6). Further, the com-plex nature of community initiatives causes dif-ficulty in linking large-scale impacts to discrete projects or initiatives. For this reason, Outcome Mapping acknowledges that the contributions of multiple projects, programs, organizations, and individuals are necessary to achieve large-scale impact. The work of a single initiative in isolation is not sufficient to achieve social change. Outcome Mapping is therefore useful in focusing on a proj-ect’s contributions to outcomes and impacts rather than attempting to attribute outcomes and impacts to one particular project.

The Outcome Mapping manual (Earl et al., 2001) describes 12 steps divided into three stag-es. The first stage is Intentional Design. This is a planning stage during which a project defines the changes it will aim to bring about as well as the strategies that will be used to contribute to the change process. The second stage is Outcome and Performance Monitoring. During this stage, mon-itoring priorities are identified, and methods are provided for monitoring progress toward desired


outcomes. The third and final stage is Evaluation Planning. Appendix A (Page 67) lists the three stages and the steps required. Moving through the stages requires a series of workshops, with workshop implementation instructions provided in the manual.

Our Approach to Outcome MappingRecent research indicates that the procedure

is especially relevant when adapted to meet the needs of an individual project (Smith et al., 2012). In practice, many researchers have employed the tool as part of a retrospective evaluation (Nyan-gaga, Smutylo, Romney, & Kristjanson, 2010). In this case, only the first stage, Intentional Design, is applicable. Stages 2 (Outcome and Performance Monitoring) and 3 (Evaluation Planning) are applicable when Outcome Mapping is used to prospectively plan, monitor, and evaluate a project.

For the current project, Outcome Mapping was used as a retrospective evaluation tool. From January to December 2013, approximately 150 hours were dedicated to workshops designed to move through the required steps described in stage one. These workshops were facilitated by the team’s program evaluator and always took place in the community setting. Workshop participants consisted of four academic team members, seven community team members, and two communi-ty Elders; this team is referred to as the working group committee (WGC).

When the WGC began the process, the proj-ect’s mission and vision statements had already been established (see Appendix B). Subsequent steps were followed in order, with boundary part-ners, outcome challenges, progress markers, and strategies developed by the WGC. After generat-ing retrospective progress markers, the evalua-tor examined a number of data sources to collect evidence for progress. In particular, the evaluator examined minutes from four years of weekly WGC meetings; bi-annual reports to the project’s funder; transcripts from post-program focus groups con-ducted over three years; facilitator reports complet-ed after each MLST class session over three years; and pre and post program questionnaires complet-ed by MLST students over three years. These data were systematically extracted, coded, and classi-fied according to the identified progress markers. After data were extracted and compiled, the WGC engaged in a series of meetings to discuss findings. These discussions allowed the WGC to reflect on areas where progress had advanced considerably as well as areas where less progress had occurred.

After progress markers were identified, work-shops focused on identifying the strategies used to achieve the outcome challenges. In consultation with a practitioner, the WGC decided not to apply the strategy categorization process described in the Outcome Mapping manual. Instead, the evaluator separated identified strategies into those that the WGC had already successfully employed and those that would be useful in moving forward. In this way, the technique was not only useful as a retrospective evaluation tool, but was informative for ongoing program planning and improvement. The WGC decided not to employ Step Seven: Organizational Practices, as it was determined that the first six steps provided sufficient information and detail for the retrospective evaluation.

FindingsChanges in the behavior, actions, activities,

and relationships of five boundary partners in-cluding Elders, leadership/education directors, schools, community and university partners were documented using the first stage. Changes in the student boundary partner were also document-ed, but were included in a separate paper (Baydala et al., 2016). These changes indicated meaningful progress toward desired outcomes. Consistent with the program’s vision and mission statements, out-come challenges and examples of progress markers are presented below for each of the five boundary partners. Examples of progress markers are present-ed in table form along with a summary of changes to support each progress marker. In keeping with guidelines outlined in the manual, progress markers were divided according to changes that the WGC expected to see (indicating an early response to proj-ect activities), liked to see (indicating more active engagement of boundary partners), and loved to see (indicating profound change in boundary partners). Finally, strategies for achieving progress markers are summarized for each boundary partner.

EldersAs in other Cree communities, Maskwacis

Elders are held in the highest regard and act as community educators, historians, and storytellers. Because Cree culture and language were at the heart of the MLST program, Elders were critically important to the program’s planning and implementation. Following from the program’s vision toward healthy First Nation communities, the outcome challenge identified for Elders was to strengthen relationships between Elders and youth. Table 1 lists progress markers and a summary of changes for this


outcome challenge.Outcome Mapping allowed the WGC to

evaluate a bi-directional learning relationship between the MLST program and community Elders. In particular, not only was the MLST program founded on the knowledge and direction of Elders; the WGC was also promoting behavioral changes in Elders by encouraging the formation of Elder-youth relationships in the community.

The primary strategy toward building relation-ships between Elders and youth was to invite Elders into Maskwacis classrooms as part of the MLST program. When Elders could not be physically pres-ent during MLST classes, digital stories, narrated in Cree by community Elders, were used to reinforce cultural teachings. Also, facilitators deliberately emphasized the importance of respecting commu-nity Elders and created opportunities for Elders and youth to interact in positive ways both in the classroom and through extracurricular events in-cluding a hide tanning cultural camp. In addition to allowing the WGC to identify existing strategies for promoting Elder-youth relationships, Outcome Mapping prompted the WGC to generate ideas for additional relation-ship-building activities. For example, the WGC planned to implement a tea and bannock day where students could invite their grandpar-ents to engage in story-telling at their schools.

Leadership/Education Directors

The Maskwacis four Nations are governed by separate leaders (i.e., chiefs and council members), and have separate education directors. In Maskwacis as in other First Nation communities, formal approval from commu-nity leaders during the project’s initiation was required; this approval

was granted in the form of band council resolutions1 from each Nation. The outcome challenge here was to reawaken the spirit of our leaders’ and education directors’ Nehiyaw mamitonecikan (i.e., Cree spir-it). Progress markers are listed in Table 2.

Toward the outcome challenge of reawakening their Cree spirit, leaders and education directors were strongly supportive of the MLST program’s goal to promote culture in Maskwacis schools. Although communicating with leaders was diffi-cult due to their busy schedules and demanding jobs, the WGC did establish and maintain ongoing relationships with a number of prominent commu-nity members, resulting in one chief taking on an advocacy role in support of the program, and in all leaders and education directors signing letters in support of the program.

In order to facilitate progressive changes in leaders and education directors, community staff employed a number of relationship building strategies. In particular, leaders and education directors were invited to all MLST events, were given MLST newsletters, and were provided with

Progress Marker

Expect to See Summary of ChangesAre present in the school

• teachers requested more Elder involvement.

• were increasingly utilized.

Like to See Summary of ChangesOngoing relationships with theprogram

• Elders attended meetings to discuss sustainaible programfunding and to sign support letters for the program.

• board created by the WGC to sustain the MLST program.

Love to See Summary of ChangesEngage in youth Programs

• Elders and youth were connected through MLST special events.

Table 1. Elders — Outcome Challenge: Strengthen Elder-Youth Relationships

Table 2. Leadership and Education Directors — Outcome Challenge: Re-awaken the Cree Spirit of our Leaders

Progress Marker

Expect to See Summary of ChangesIndicate thatthey want theprogram in theschools

Love to See Summary of ChangesAdvocate forthe program

• Leaders and educaton directors signed letters of supportfor the program.Chief attended a meeting with a government minister to discuss options for sustainable program funding.

• Leaders signed band council resolutions to allow the MLST program in the community, and supported the concentgathering process for students to take part in the MLST evaluation.

Like to See Summary of ChangesHave ongoingrelationshipswith the program

• Staff began to deal more directly with chiefs of educationdirectors rather than being referred t administrative staff.

•

1A band council resolution is a writ-ten decision made by the governing body of a band (usually consisting of a chief and councilors). A band council resolution must have the support of the majority of council members.


a comprehensive business plan. Com-munity staff also reg-ularly attempted to schedule meetings to provide updates. Regarding prospec-tive strategies, the WGC committed to provide leaders and education directors with monthly up-dates, and to invite leaders from all four Nations to meet as a collective.

Evaluating lead-ers and education directors as bound-ary partners allowed the WGC to recog-nize how important they were to program sustainability and to facilitating positive community change. As part of the process, the WGC discussed that it was important to maintain relationships with leaders and education directors beyond obtaining band council resolutions and formal written approval. In order for the program to thrive and have a commu-nity-wide impact, it was necessary for the WGC to provide leaders and education directors with mul-tiple opportunities to become familiar with and involved in the MLST program.

SchoolsWhile the support of community leaders and

education directors was necessary to allow the MLST project in the Maskwacis community, the support of school personnel was necessary to allow for the project to experience success in Maskwacis schools. The outcome challenge for schools was to support and promote the program in the school environment, particularly the cultural aspects of the program. Progress markers and a summary of changes are listed in Table 3.

The above summary highlights that school personnel became increasingly welcoming, sup-portive, and interested in the MLST program and the program’s cultural teachings. During the first year of the program, teachers were hesitant to accept the program, rarely supported facilitators with classroom management, and reported a num-ber of issues with facilitator reliability. By the third

year, teachers provided expressly positive feedback about facilitators and the program, allowed extra MLST class time, began to advocate for the pro-gram, and incorporated cultural teachings into core subjects. Outcome Mapping allowed for the WGC to identify these progressive changes in school personnel.

To promote these changes, a number of strat-egies were utilized. An important strategy at the outset of the project was to invite schools to partic-ipate in program training delivered to facilitators. This allowed for schools to gain an understand-ing of the program before it began. Additionally, schools were given the opportunity to participate in focus groups at the end of each project year in order to provide feedback. Each year, this feed-back led to additional strategies being employed for schools. In particular, the second and third years of the program saw more emphasis placed on facilitators being punctual and reliable, facilita-tors communicating with teachers, and facilitators being present in schools outside of MLST class time. Evaluation updates and promotional ma-terial were also provided to schools as a means of keeping school personnel informed about the program. Through Outcome Mapping, the WGC was able to identify and reflect on each of these strategies and the extent to which they were successful with schools.

Using the technique, a number of prospective strategies were also identified for schools. Primarily,

Progress Marker

• Facilitators and teachers reported that teachers increasinglyassisted with classroom management.

Like to See Summary of Changes

Express interestin cultural aspectsof the program

• By the third year, facilitators reported that teachers wereengaged in culture and language learning; many teachersutilized cultural teachings in core subjects.

Provide classroom management

Expect to See Summary of Changes

Invite facilitatorsto school functions

•

Allow more classtime for programdelivery

•

Facilitators were invited to school feasts, healing circles, professional development days, and parent-teacherinterviews.

By the end of the third year, teachers allowed facilitatorsto stay in their classrooms for an extended time, even when MLST classes rean over into other subjects.

Attend MLSTevents

• School attendance at proram events was consistentlylimited.

Love to See Summary of ChangesRequest additional cultural activities

• By the third year, schools requested facilitators and Elders to act as guest speakers in the school outside of MLST class time.A facilitator was invited to present at a PD day.

Advocate for the program

• School personnel spoke about the program at interagency meetings; one school principal included the MLST program in a school funding proposal.

•

Table 3. Schools-Outcome Challenge: Promote and Support the MLST Program


because all three years of the program saw low attendance from school personnel at MLST events, the WGC determined that they needed to em-ploy more targeted efforts to involve schools in MLST events. Additionally, it was determined that facilitators should spend time in their classrooms before the MLST program began in order to de-velop rapport with students and teachers and to establish mutual expectations. The WGC also committed to sharing a facilitator code of conduct with teachers, providing more regular updates to school principals, and annually revisiting the program’s memorandum of understanding with schools. Finally, unlike schoolteachers, facilitators were not University educated; rather, they were well-informed and knowledgeable regarding their culture and language. It was important for teachers to understand and respect the cultural qualifica-tions of facilitators from the outset of the project. Accordingly, it would be important for the MSLT program to more clearly com-municate with school personnel regarding the qualifications of facilitators at the beginning of the school year. Overall, Outcome Mapping provided a means for the WGC to generate prospective strate-gies in a systematic way that allowed for targeting areas where additional progress could be made.

Community Part-ners

C o m m u n i t y partners (Table 4) included both MLST facilitators and pro-gram administrative staff. The outcome challenge for com-munity staff was to practice, promote, and support a Nehi-yaw (i.e., Cree) worl-dview through the program.

In identifying and evaluating progress markers for community partners, the WGC began to recognize the complexity of community partners’ roles, which extended beyond delivering the MLST program to students. These included developing their leadership capacity, as well as building relationships with Elders, students, schools, other MLST staff, the wider community, and university partners.

It was important to clarify strategies that facilitated the success of community partners in their complex roles. Strategies to this end were focused on promoting culture within the work-place. In particular, the job descriptions of com-munity staff emphasized a significant cultural component, and it was clearly communicated to community staff that they were expected to spend time each day engaging with and learning from Elders. Additionally, community partners were

Progress Marker

• According to teachers, students were relaxed and secure with facilitators; students who were tyically quiet becamefrank and honest during MLST classes.


Build relationshipswith the widercommunity

• A number of organizations, including schools within and outside of Maskwacis, requested that MLST staff deliver professional development workshops.

Build relationships with students


Have strong relationships witheach other

•

Willling to discusschallenges withuniversity partners

•

Facilitators described significant growth in terms of supporting one another, working as a team, and fostering positive relationships among one another.

Love to See Summary of Changes

• During the third year, community partners created a non-profit society to hold MLST funding, and officially transitioned into leadership roles, such as Executive Director of the non-profit. Community partners created a business plan for the non-profit, presented the business plan to potential funders, and planned for sustainability of the program.

•

Build relationships with schools

• Teachers and facilitators reported strong, positive teacher-facilitator relationships.

Build relationships with university partners

• Community partners familiarized university partners with the Maskwacis community and Cree culture.

• Community partners made meaningful contributions to the evaluation, acdemic presentations, and reports.

By the final year of the program, community partners regularly brought up issues of concern with university partners (e.g., questions about copyright and program material ownership).

Take on leadership roles

• Community partners began to coordinate smaller projects and conduct meetings with leadership.

Are accountableto the Elders • Elders were acknowledged in all meetings and presenta-

tions.Space was reserved on meeting agendas for Elders. The non-profit society created by community staff wasstructured to be accountable to community Elders.

•

• Elders were consulted regarding all program decisions.

•

Table 4. Community Partners — Outcome Challenge: Practice, Promote, and Support a Nehiyaw Worldview


permitted to participate in ceremonies and other cultural events during work hours, and took part in multiple culturally relevant professional devel-opment opportunities. To the extent that commu-nity partners were able to strengthen their cultural knowledge and connection with Elders, relation-ship building and accountability were similarly strengthened.

Moving forward, the WGC determined that it would be important to regularly revisit the staff code of conduct and handbook to ensure consis-tent staff expectations and standards. Another pro-spective strategy was to focus more on enhancing Cree language skills, and relatedly, to institute a Cree naming ceremony for all community staff.

University PartnersUniversity partners (Table 5) included the

project’s principal investigator, research coordina-tors, research assistants, and a program evaluator. For university partners, the outcome challenge was to practice authentic CBPR.

The mapping technique highlighted that Uni-versity partners experienced significant growth in their capacity to practice authentic CBPR. This was made possible by community part-ners’ willingness to educate university partners, which in turn facilitated a strong and trusting partnership.

A number of strategies were rel-evant to university partners. Perhaps most importantly, university partners were physically present in the com-munity as often as possible. The WGC held weekly meet-ings in the commu-nity; accordingly, university partners were present in the community at least once per week. Hav-ing regular meetings between partners was essential for relationship build-

ing. University partners also traveled to the com-munity to work with community partners on program adaptations, conference abstracts, presen-tations, and to complete day-to-day project tasks together. Community partners shared that it was also imperative for university partners to attend community events and ceremonies, and to create opportunities for partners to informally socialize. While these strategies were employed through-out the project, the WGC had not evaluated why and how these strategies were effective until they began to use the technique. In this way, the process allowed for partners to discuss and clarify their assumptions regarding CBPR and the strategies necessary for establishing a strong and equitable community-university partnership.

Through these discussions, the WGC identi-fied a number of additional strategies for commu-nity-university partnerships. Primarily, at the out-set of a project, partners should establish a mutual understanding of expectations in order to avoid later confusion and misunderstanding. In this vein, the WGC recommended that community partners be involved in budget discussions from the begin-ning of the project in order to understand how and

Progress Marker

• Community partners reviewed all presentations, reports, publications, and other written material before being shared


• University partners participated inmultiple meetings withgovernmental and other potential fundders to discuss funding

Consensus-based decision-making is practiced.


Community partners havethe opportunityto gain academicknowledge.

•

•

The project’s principal investigator agreed to act as anadvisory board member to the non-profit society created bycommunity partners.Community partners participated in data collection, analysis, and synthesis, and were actively involved in all presentations.

Love to See Summary of ChangesCommunity partners have their own budget.

•

University partners incorpo-rate Indigenous research methods into the project.

Support from all staff was required to make decisions during the WGC meetings; by the final year, community partners were involved in hiring new university staff.

•

University partners are committed to promoting MLSTsustainability.

All WGC meetings were held in the community and began with a prayer. The university consent process was changed to allow for Elders to provide oral rather than written consent. Evaluation tools were adapted to be culturally relevant.

•

•

•

• Community partners attended multiple training workshops including grant writing and project management.

During the final year of the project, community partners were allotted a portion of the overall grant to manage as their own budget.

University partners increase capacity to conduct CBPR with Indigenous communities

Community partners continually educated univesity partners about cultural protocol and expectations.

•

University partners attended ceremonies and other community events, as well as culturally relevant workshops, conferences, and training.

•

Table 5. University Partners — Outcome Challenge: Practice authentic CBPR


why budget decisions are made; this would help community partners to develop a more solid un-derstanding of budget management when projects later transition into sustainable, community-led programs. Additionally, both partners should be aware of the power imbalance that is inherent to grant funds being held at the university rather than in the community. Most, if not all, commu-nity-university partnerships must deal with the reality that large funding agencies award research grants solely to university partners, who are then accountable for managing funds. Although in the current project, a portion of the overall grant was allotted to community partners to manage them-selves, funds were still administratively filtered through the university, which was reportedly frus-trating for community partners. In order to deal with these frustrations and the power imbalance that accompanies this situation, partners must be open and honest with one another, and willing to discuss the uncomfortable circumstances that such a funding arrangement can result in. Again, the overarching strategy for working through these challenges was to develop strong and trusting rela-tionships between partners throughout the project.

DiscussionTo supplement the MLST program evaluation,

Outcome Mapping was employed as a retrospec-tive evaluation tool by community and academic research partners. Through the process, a number of boundary partners were identified, including Elders, community leaders and education direc-tors, schools, as well as community and university partners. Outcome challenges, progress markers, and strategies were identified for each boundary partner through a series of team workshops, and evidence to support progress was systematically collected from meeting minutes, focus groups, reports to funders, and program facilitators’ daily reports.

Overall, partners were in strong agreement that Outcome Mapping provided a meaningful method for evaluating and telling the MLST pro-gram’s story of progress. In this way, the process was congruent with Indigenous worldviews. In particular, Indigenous research methods often emphasize narrative and relationality. Through workshops, team members told stories of their experiences with the program, informing the creation of outcome challenges and progress markers. This also allowed for the project’s milestones to be structured as more of a narrative than conventional evaluation methods,

resulting in a coherent story of progress. Similarly, because the WGC created progress markers, the project was able to capture outcomes that were significant to community partners.

Importantly, the process was extremely bene-ficial for the current project. Of particular impor-tance was the opportunity that Outcome Mapping offered for relationship building, both among community staff members and between commu-nity and academic partners. Bringing team mem-bers together to collaboratively define evaluation outcomes served as a reminder that partners were working toward a common goal. Participating in interactive workshops also allowed for partners to more fully understand multiple perspectives, and to consider ideas that were often innovative and novel. In this way, rich opportunities were provid-ed for partners to learn from one another, and pro-vided a catalyst for community partners to artic-ulate and communicate expectations of university partners. Further, it provided a means for all WGC members to feel a sense of ownership over and investment in the evaluation process. The MLST evaluation was perceived as less of an academic endeavor, and more of a learning opportunity for both community and academic partners. Disen-tangling the complex web of progress also served to enhance staff morale. The WGC described feel-ing a sense of immense accomplishment at the evi-dence for progress elucidated by the process.

Similarly, Outcome Mapping prompted the WGC to recognize outcomes that might otherwise be overlooked by more conventional evaluation tools. Although there is obvious importance in remaining focused on long-term impacts such as reduced substance abuse and healthy First Nation communities, the process allowed for the WGC to take pride in smaller accomplishments indicating progress toward these long-term goals. A systemat-ic examination of smaller-scale changes and strat-egies also allowed for the WGC to examine how changes occurred, and why less progress occurred in certain areas. Although the WGC had used tools such as phase diagrams in the past in order to illustrate progress, Outcome Mapping demon-strated far more complexity as well as peripheral and unanticipated changes. Likewise, it allowed for the WGC to examine outcomes and strategies that reached beyond the end of the evaluation, and to consider strategies and markers of progress mov-ing forward into a phase of program sustainability.

Although the program was highly beneficial for the current project, a number of challenges must be highlighted. Primarily, it required a large


time commitment from both partners. Because Outcome Mapping is a participatory process, it is essential for all team members to be present at workshops. Additionally, one person must be assigned to facilitate workshops and to organize and track data, which in itself represents a signifi-cant time commitment. Further, because Outcome Mapping is a relatively new method, it can be dif-ficult to secure a facilitator who has an in-depth understanding and experience with the process. Investing time and energy into it also requires a flexible project funder. For the current proj-ect, funders did not impose particular evaluation methods, and trusted team members to employ appropriate evaluation tools. This flexibility is essential for the success of Outcome Mapping. Indeed, framing knowledge with its use could be seen as a risky undertaking in the world of academia, where randomized controlled trials and experimental methods are the standard. However, in complex community settings, the technique is highly valuable and arguably necessary for evaluating and understanding community progress and change.

This project represents the first account of Outcome Mapping in a Canadian Indigenous community. Given the need for evaluation meth-ods that align with the non-linear, multi-faceted processes of community change and development, this project makes an important contribution to the evaluation literature. Our project not only demonstrates how Outcome Mapping can accu-rately and comprehensively capture the change process in community projects; it also highlights how the method can be used to bring together and engage community and university partners in the evaluation process.

Finally, this project details how Outcome Mapping can be effectively adapted by a commu-nity-university partnership.

ConclusionWhile conventional research and evalua-

tion methods can provide valuable information regarding the effectiveness of a program, CBPR projects require the additional use of innovative methods that can account for the complex nature of community change. For the current project, Outcome Mapping provided a systematic meth-od for understanding peripheral changes that are often overlooked in conventional research and evaluation, but that nonetheless indicate progress toward community changes and long-term impact. Among other findings, data demonstrated positive

impact on Elders and students; growing com-munity investment in and support for the MLST program’s cultural components; progressive in-creases in community ownership of the program; and growth in the community-university part-nership throughout the project. The method is a highly valuable tool for CBPR projects. It supports growth in community and academic capacity and relationship building between community and ac-ademic partners. Outcome Mapping can enhance research and evaluation quality and contribute to project sustainability by offering a framework for capturing outcomes that are meaningful for community partners.

ReferencesBaydala, L., Fletcher, F., Tremblay, M., Rabbit,

N., Louis, J., Ksay-yin, K., & Sinclair, C. (2016). A community-university approach to substance abuse prevention. Journal of Community Engage-ment and Scholarship, 9(1), 67–75.

Botvin, G.J., Baker, E., Renick, N.L., Filazzola, A.D., & Botvin, E.M. (1984). A cognitive-behav-ioral approach to substance abuse prevention. Ad-dictive Behaviors, 9(2), 137–147.

Botvin, G.J., & Griffin, K.W. (2014). Life skills training: Preventing substance misuse by enhanc-ing individual and social competence. New Direc-tions for Youth Development, 2014(141), 57–65.

Canadian Institutes of Health Research (CIHR). 2007. CIHR guidelines for health research involving Aboriginal people. Ottawa, ON: CIHR.

Castleden, H., Morgan, V.S., & Lamb, C. (2012). “I spent the first year drinking tea”: Explor-ing Canadian university researchers’ perspectives on community-based participatory research in-volving Indigenous peoples. The Canadian Geog-rapher, 56(2), 160–179.

Cross, T.L., Friesen, B.J., Jivanjee, P., Gow-en, L., Bandurraga, A., Matthew, C., & Maher, C. (2011). Defining youth success using culturally ap-propriate community-based participatory research methods. Best Practices in Mental Health, 7(1), 94–114.

Dart, J., & Davies, R. (2003). A dialogical, story-based evaluation tool: The most significant change technique. American Journal of Evaluation, 24(2), 137-155.

Earl, S., Carden, F., & Smutylo, T. (2001). Out-come Mapping: Building learning and reflection into development programs. Ottawa, ON: International Development Research Centre.

Gauld, S., Smith, S., & Kendall, M.B. (2011). Using participatory action research in communi-


ty-based rehabilitation for people with acquired brain injury: From service provision to partner-ship with Aboriginal communities. Disability and Rehabilitation, 33(19-20), 1,901–1,911.

Grekul, J., & Sanderson, K. (2011). ‘I thought people would be mean and shout.’ Introducing the Hobbema Community Cadet Corps: A response to youth gang involvement? Journal of Youth Studies, 14(1), 41–57.

Israel, B.A., Schulz, A.J., Parker, E.A., & Beck-er, A.B. (1998). Community-based research: A partnership approach to improve public health. Annual Review of Public Health, 19(), 173–202.

Jagosh, J., Macaulay, A.C., Pluye, P., Salsberg, J., Bush, P.L., Henderson, J., Sirett, E., Wong, G., Cargo, M., Herbert, C.P., Seifer, S.D., Green, L.W., & Greenhalgh, T. (2012). Uncovering the benefits of participatory research: Implications of a realist review for health research and practice. Milbank Quarterly, 90(2), 311–346.

Judge, K., & Bauld, L. (2001). Strong theory, flexible methods: Evaluating complex communi-ty-based initiatives. Critical Public Health, 11(1), 19–38.

Kelly, T. (2010). Five simple rules for evalu-ating complex community initiatives. Community Investment 22(1), 19–23.

Kelly, J., Saggers, S., Taylor, K., Pearce, G., Massey, P., Bull, J., Odo, T., Thomas, J. Billycan, R. Judd, J., Reilly, S., & Ahboo, S. 2012). “Makes you proud to be black eh?”: Reflections on meaningful Indigenous research participation. International Journal for Equity in Health, 11(40), 1–8.

Minkler, M., & Wallerstein, N. (Eds.). 2003. Community-based participatory research for health. San Francisco, CA: Jossey-Bass.

National Health Promotion Associates, Inc. (2011a). Life skills training questionnaire: Elemen-tary school version. Retrieved from www.lifeskill-staining.com.

National Health Promotion Associates, Inc. (2011b). Life skills training questionnaire: Middle school version. Retrieved from www.lifeskillstain-ing.com

Nyangaga, J., Smutylo, T., Romney, D., & Kristjanson, P. (2010). Research that matters: Outcome Mapping for linking knowledge to pov-erty-reduction actions. Development in Practice, 20(8), 972–984.

Salimi, Y., Shahandeh, K., Malekafzali, H., Loori, N., Kheiltash, A., Jamshidi, E., Frouzan, A. S., & Majdzadeh, R. (2012). Is community-based participatory research (CBPR) useful? A systemat-ic review on papers in a decade. International Jour-

nal of Preventive Medicine, 3(6), 386–393. Smith, R. Mauremootoo, J., & Rassmann, K.

(2012). Ten years of Outcome Mapping adapta-tions and support. Ottawa, ON: Outcome Mapping Learning Community.

Victora, C.G., Habicht, J., & Bryce, J. (2004). Evidence-based public health: Moving beyond randomized trials. American Journal of Public Health, 94(3), 400–405.

About the AuthorsMelissa Tremblay is a program evaluator in

the Department of Pediatrics at the University of Alberta. Lola Baydala is an associate professor in the Department of Pediatrics at the University of Alberta. Natasha Rabbit is the executive director of the Nehiyaw Kakeskewina Learning Society in Maskwacis, Alberta. Jennilee Louis is a research assistant for the Nehiyaw Kakeskewina Learning Society in Maskwacis, Alberta. Kisikaw Ksay-yin is an Elder in the Nehiyaw Kakeskewina Learning Society, Maskwacis, Alberta.


Appendix A. Outcome Mapping Steps (Earl, Carden, & Smutylo, 2001)

Stage 1: Intentional Design

OM Step Description

1. Vision ject will aim to contribute.

2. Mission

3. Boundary Partners with whom the project interacts directly and whom the project has the 4. Outcome Challenges describe how the behavior, relationships, actions, and/or activities of boundary partners will change if the program is maximally successful.5. Progress Progress markers are developed for each outcome challenge. These are more incre- Markers mental changes in the behavior, relationships, actions, and/or activities of boundary partners, separated into changes that the project would expect to see, like to see, and love to see.

6. Strategy Maps Strategy maps are created for each outcome challenge and categorized into a six-cell matrix according to the type of relationship between the project and the boundary partner.

7. Organizational Outcome challenges and strategy maps are reviewed in order to articulate the organi- Practices zational practices that will be most useful for aiding a project in contributing to intended change.

Stage 2: Outcome and Performance Monitoring

8. Monitoring Decisions are made as to what will be monitored on an ongoing basis (organizational Priorities practices, strategies, and/or progress toward outcomes) and what will be evaluated in more detail in the future.

9. Outcome An outcome journal is established for each boundary partner to track progress toward Journals 10. Strategy A strategy journal is created to record data on the strategies being used to promote Journal change in boundary partners.

11. Performance A performance journal is created to record data on how a program is operating and Journal

Stage 3: Evaluation Planning

11. Evaluation Utilizing each of the elements created through the preceding steps, an evaluation plan Plan is created that describes the main elements of the evaluation to be carried out for the program.

Appendix B. Maskwacis Life Skills Training Program Vision and Mission

TakakisimototatTo Pray

To the Father of All CreationMamawihoyotawiymaw

To Bless Us and Grant UsTita Sawemikoyahk

Proper Child Up BringingMiyo Opikinawasowin

Tansih Teysih MacitahkHow To Begin

Tantey ehocicikFrom Where They Originate

TakiskinwahamacikTo Teach Them

OyotawiymawotowiyinkewinaThe Creator’s Laws

OteynikanIn the Future

KosisiminawakOur Grandchildren

KitanskotapanawakOur Great Grandchildren

Mina aniki ekiyceskwaAnd Those Children

TamatinumatoknistotamowinBy Sharing Our Understanding

AkememototamowinBy Perseverance

Tita neyhiwecikTo Speak Cree

KitawasImsInawakOur Children

MamawokamatowinBy Collective Effort

Tansi Tehsih KaskitahyakHow We Will Succeed

Mina KeyteyayaOkakeskewinaAnd the Elder’s Teachings Kanihtawkihkosicik

Who Are Not Yet Born

TanistameytamihkkiskinotahawasowinBy Prioritizing Child Guidance

KiskinotahawasowinGuiding Our Children

Mission Statement


A Community-University Approach to Substance Abuse Prevention

Lola Baydala, Fay Fletcher, Melissa Tremblay, Natasha Rabbit, Jennilee Louis, Kisikaw Ksay-yin, and Caitlin Sinclair

AbstractIn response to high rates of substance abuse in their communities, members of the Maskwacis four

Nations invited university researchers to partner in culturally adapting, implementing, and evaluating an evidence-based substance abuse and violence prevention program, the Life Skills Training program (Botvin & Griffin, 2014). This project used a community-based participatory research (Israel, Schulz, Parker, & Becker, 1998; Minkler & Wallerstein, 2003) approach, and was carried out by university and First Nation community partners. To evaluate the impact of the adapted program, students completed pre and post questionnaires, and community members participated in focus groups. The adapted Maskwacis Life Skills Training program was delivered in schools for three years. Students’ knowledge increased significantly during program delivery, and strong support was documented from community members. This project demonstrates the impact that can be facilitated by culturally adapting and delivering a prevention program, and by forming a community-university partnership.

IntroductionThe Maskwacis First Nations, located in central

Alberta, Canada, include four communities: LouisBull, Montana, Samson, and Ermineskin. These neighboring Plains Cree Nations have a combined population of approximately 15,000 with roughly 53% of the population aged 17 or under (Grekul & Sanderson, 2011). The Nations are governed by independent chief and councils, and have separate education directors and schools.

National media attention has focused on high rates of crime and gang violence in Maskwacis, largely ignoring the rich and vibrant Cree culture that threads the four Nations together. In Maskwa-cis, Cree history, culture, and language are featured prominently in traditional community ceremonies and cultural events that regularly take place. Com-munity Elders view their culture and language as a means to combat the social and public health problems that face Maskwacis community members as a result of the destructive impact of colonization, residential schools, and forced assimilation. Research evidence supports this perspective, demonstrating that a positive cultural identity can be a protective factor against substance abuse and violence for Indigenous children and youth (French, Kim, & Pillado, 2006; Gazis, Connor, & Ho, 2009; Kulis, Napoli, & Marsiglia, 2002).

Project InitiationThe current project stemmed from a previ-

ous pilot project led by a partnership between university researchers and Alexis Nakota Sioux Nation members (Baydala, Fletcher, Worrell, Kajner, Letendre, & Rasmussen, 2014). Alexis community members sought to address the root causes of fetal alcohol spectrum disorder, and invited university researchers to partner with them in working toward substance abuse prevention. This pilot project involved the cultural adaptation, delivery, and evaluation of a substance abuse and violence prevention program in the Alexis com-munity. The Botvin Life Skills Training (LST) pro-gram (Botvin, Baker, Renick, Filazzola, & Botvin, 1984; Botvin & Griffin, 2014) was chosen for the project, based on the results of a literature review that identified the program as having extensive, high-quality evidence to support its effective-ness, including multiple randomized control trials (National Registry of Evidence-based Programs and Practices, 2008). The three-year LST program has three levels, each including between 8 and 14 one-hour modules, and begins in either elementary or junior high school (see Table 1).

In 2010, a Maskwacis community member attended a presentation delivered by university researchers and Alexis community members. Fol-lowing this presentation, Maskwacis community members invited university researchers to partner in introducing and adapting the LST program in their own community.


Project ObjectivesThe partners identified three broad objectives

for the current project: (1) culturally adapt the LST program to reflect the language, culture, and visual images of the Maskwacis community; (2) deliver the adapted program in Maskwacis schools; and (3) evaluate the impact of the adapted program. Given the established effectiveness of the LST pro-gram in reducing substance abuse and violence and increasing skills related to prevention, the aim of the current project was to deliver the culturally adapted program with fidelity to the original curriculum, and to document the impact of the adapted program.

Program AdaptationsResearch has demonstrated the importance

of programming for Indigenous children to reflect Indigenous worldviews and culture (Hare, 2011). Further, the cultural content of programming has been correlated with enhanced learning for Indige-nous children (Tsethlikai & Rogoff, 2013). A recent policy report also declared that the representation of Indigenous cultures, languages, and traditions in school classrooms is essential for promoting the academic success of Canadian Indigenous children and youth (Toulouse, 2008). For minority popula-tions in general, culturally adapted evidence-based programs have been shown to be more effective than standard programs (Kumpfer, Magalhaes, & Xie, 2012).

The LST program is a generic program prov-en highly effective with students from different geographic regions, socioeconomic circum-stances, and racial-ethnic backgrounds (Botvin, Griffin, Diaz, & Ifill-Williams, 2001). However, because the LST program does not incorporate Cree values, language, culture, and identity, Maskwacis community members and Elders decided to culturally adapt the program. In keeping with the principles of knowledge to action research (Gra-ham, Logan, Harrison, Straus, Tetroe, Caswell, & Robinson, 2006), adaptations to the LST program, and the corresponding creation of the Maskwacis

Life Skills Training (MLST) program, began in the first year of the project.

During the first year, a six-member adaptation committee was formed consisting of Elders from each of the Maskwacis four Nations. This commit-tee met weekly to complete adaptations to the first level of manuals. Adaptations to the second and third levels were completed by a rotating group of Elders. During adaptation meetings, over 30 dif-ferent Elders from all four communities reviewed the original LST curriculum and provided recom-mendations for adaptations, consisting of Cree lan-guage and syllabics, Elders’ teachings, and personal life stories. Following committee meetings, com-munity and university partners worked together to adapt the manuals. Additionally, a community member created visual images for the manuals that reflected the Maskwacis culture and community. After adaptations were completed, Elders reviewed and provided approval for the adapted manuals.

Throughout the adaptation process, Elders were instrumental in ensuring that the adapted program accurately reflected the Cree culture. For example, one of the modules in the original LST curriculum focused on the harmful effects of tobacco use. However, Elders shared the impor-tance of distinguishing between “poison tobacco” and “sacred tobacco” (i.e., kistemaw). In Mask-wacis, kistemaw has important spiritual, cultural, and ceremonial purposes, such as being offered to communicate gratitude in advance of a request. Accordingly, the original LST module on smoking was adapted to focus on both the detrimental effects of poison tobacco as well as the healthy traditional use of kistemaw. Additionally, in or-der to reinforce program adaptations, MLST staff created digital stories to accompany each program module; these were guided and narrated by community Elders.

As Figure 1 shows, the adaptation process resulted in a program that incorporated both Western and Indigenous foundations of substance abuse and violence prevention. Rather than infus-ing Indigenous pillars of effective substance abuse prevention into an existing Western framework, the integrity of the lessons from each worldview was maintained.

MethodsCommunity-Based Participatory Research

This project used a community-based partic-ipatory research (CBPR) approach (Israel, Schulz, Parker, & Becker, 1998; Minkler & Wallerstein, 2003). With an emphasis on tangible benefits for

Elementary School Program I 3

Level Grade Modules

8 IIIII

4 8 boosters 5 8 boosters

I 6

8 IIIII

7 8 boosters 8 8 boosters

Junior High School Program

Table 1. Life Skills Training Levels


communities, co-learning, equitable involvement, and respect for multiple forms of knowledge, CBPR can contribute to leveling power imbalances between community and university partners. Because Indigenous peoples are calling for research to be done with and for Indigenous com-munities rather than on Indigenous communities, CBPR has been recognized as particularly suitable for research involving Indigenous peoples (Cas-tleden, Morgan, & Lamb, 2012; Koster, Baccar, & Lemelin, 2012; Smith,1999). The partners in this project applied the principles of CBPR in multiple ways. In particular, academic team members trav-eled to the community each week to engage in team meetings; meetings began with smudging and a prayer conducted in the local Cree language; a consensus-based decision-making model was fol-lowed with contributions from all team members;

program evaluation was participatory, with input and guidance from all partners; and all reports and program materials were created collaboratively and approved by all partners. The project was approved by the research ethics board at the part-ner university and by the Maskwacis First Nations through a Band Council Resolution (the authori-ty mechanism by which elected representatives in First Nation communities support and authorize an action).

Focus GroupsA total of 25 focus groups were held over the

course of the three-year project with 42 school personnel, 102 students, 18 Elders, four parents, and 12 MLST facilitators. With the exception of student focus groups, which were held with classes of approximately 20 students, focus groups includ-

Social Skills

Culture

Language

HistoryIdentity

Knowledge

Self ManagementRefusal Skills

Figure 1. Model Incorporating Abuse and Violence Prevention Ideas from Both Western and Indigenous Traditions


ed between six and eight participants to enable effective information sharing (Kreuger & Casey, 2015). Focus groups took place at the end of each of the three program years, and were held sepa-rately with school personnel from each school, students from each school, Elders, parents, and facilitators. The length of focus groups was between one and two hours.

Using purposive sampling, university partners recruited MLST facilitators, school personnel, and students to participate in focus groups, while community partners recruited Elders and parents. Community and university partners worked together to gather consent for participation. Although university partners facilitated all focus groups, community partners were present during Elder focus groups to translate from Cree to English where necessary. University research team members who conducted focus groups had graduate-level training in research methods and focus group facilitation as well as specific experience working with Indigenous communities.

Different focus group guides were developed for school personnel, students, Elders, parents, and MLST facilitators. Questions focused on partic-ipants’ experiences with the program adaptation and delivery, program impact on students, schools, and the community, unanticipated successes and challenges, and suggestions for improvement. Focus groups were conducted using a conversa-tional style whereby participants were invited to speak to one another rather than directly to the facilitator, disagreements were encouraged, and participants were guided to express their thoughts in their own words and on their own terms (Kitzinger, 2005).

With participants’ permission, focus groups were audio recorded and transcribed verbatim. Data were managed with ATLAS.ti software, and analyses were completed using content analysis. Data were grouped according to common content, and a preliminary coding scheme was developed. The coding scheme was refined during discussions among partners, and themes were subsequently identified. Transcripts and preliminary data anal-yses were presented to participants as a means of member checking.

QuestionnairesTo evaluate learning of program content, ques-

tionnaires were distributed to students before and after each year of program delivery. The LST ques-tionnaire (National Health Promotion Associates, 2011a; 2011b) and Piers-Harris Self-Concept Scale, Second Edition (Piers & Herzberg, 2002) were distributed to students. The wording of LST ques-tionnaires was adjusted to accommodate students’ reading levels. The elementary school version of the LST questionnaire included items regarding knowledge and attitudes toward smoking and drinking, as well as social and personal self-man-agement skills. The junior high version of the LST questionnaire included similar content with an added focus on drug refusal skills. Additionally, LST questionnaires were adapted to include a cul-tural knowledge scale consisting of items created by community members and Elders. From the ini-tial stages of the project, community partners em-phasized the importance of using a strengths-based approach. Accordingly, questionnaire items that directly questioned frequency of substance use were eliminated. Instead, questionnaire items focused on constructs that are highly related to decreased substance use (i.e., knowledge, attitudes, self-management, refusal, and social skills).

Children who provided verbal assent and whose parents provided written consent for partic-ipation completed questionnaires before and after each year of program delivery (see Table 2). Where assent and/ or consent was not provided, children still received the MLST program but did not com-plete questionnaires.

A sequential longitudinal cohort design was used, and a linear mixed model was used to analyze the questionnaire data. The linear mixed model has advantages over repeated measures ANOVA for working with repeated measures data (Kreuger & Tian, 2004). Primarily, a linear mixed model gives structure to the error term by adding additional random-effect terms. This is important to properly account for error that can arise due to correlations between data points within the same subject (intra-subject correlation). Also, the linear mixed effects model can accommodate missing data points, and is less likely to lead to spurious results (both Type I and Type II error) with categorical data.

Year 1 Pre Year 1 Post Year 2 Pre 216 189 103 96

Year 2 Post Year 3 Pre Year 3 Post 79 67

Table 2. Number of Students Who Completed Questionnaires


ResultsFocus Groups

Focus group findings were grouped into three overarching themes, consisting of (i) program impact; (ii) factors that contributed to the program’s success; and (iii) suggestions for improvement.

Program impact. Focus group participants described the impact of the MLST program in a number of areas. School impact was a particular area of focus. Participants described how the program allowed for traditional Elders’ teachings to be brought in to the schools; that the program facilitated both student and teacher learning about Cree culture and the Maskwacis community; and how the program had resulted in positive class-room changes. As one teacher commented, “there’s a more positive environment in the classroom...we’re all communicating and we’re talking and there’s no yelling.”

Student impact was also described. Partici-pants not only discussed how students had demon-strated learning of program content; they also described how the program had contributed to elevating students’ self-esteem, respectful attitudes, and listening skills. According to one student, “I learned to like myself for who I am.” Similarly, as a school staff member stated, “I noticed their self-esteem was brought up...I noticed with their attitudes, they’ve been more respectful...to the class teacher and they’ve learned to respect themselves and their fellow peers...” Additionally, focus group participants reported that the MLST program had contributed to students’ enhanced sense of pride in their identity. One teacher described how the program had “opened up a lot of doors for them to try and realize that being who they are is okay.” Similarly, Elders reported an increase in youth attendance at cultural events in the community.

Focus group participants also described Elder impact. Elders recounted how the program had facilitated their own learning by reminding them about ancestral teachings shared by other Elders. Elders also felt that their participation in the pro-gram allowed them to contribute to their com-munity in meaningful ways, resulting in positive } personal impact. As one Elder described, “it warmed my heart to make me feel wanted.”

Similarly, facilitator impact was described. Facilitators felt that they had benefited from being immersed in cultural teachings, and described bringing traditional teachings into their own homes for their children and grandchildren to learn: “I had pride in what I was teaching. And for my personal life...I’m raising my grandson, and he

gets to have a bulk of what I teach.” Additionally, facilitators were impacted by the opportunity to build strong relationships with Elders, and also described feeling proud to make a difference in their community.

Factors contributing to program success. Among the multiple contributors to the MLST program’s success, participants described the focus on cultural teachings as paramount. Accord-ing to participants, it was critical for facilitators and Elders to teach students about the impor-tance of respect, honoring the Creator, speaking Cree, learning Maskwacis history, and developing cultural pride. These teachings were fundamental to student engagement and overall program acceptability. One teacher described how,

This content would not have worked had it not been presented culturally…we prayed before every class, we burned sweet grass…and the Elder led them in prayer and that just locked them in, they knew they were home once they did that.

In order to enrich the program’s cultural con-tent, Elder involvement was also critical. Facilita-tors felt that Elders’ guidance and knowledge made program adaptations successful, while students, facilitators, and school personnel appreciated Elders’ presence in the classroom. As one facilitator described, “it seems like the more we bring the Elders in, the more the children benefit.”

Elementary Students

Variable Baseline Year 3 Post p valueOverall knowledge 52.9 67.4 <0.001Anti-smoking knowledge 43.4 63.6 <0.001Life skills knowledge 34.8 41.3 <0.001Cultural knowledge 66.5 84.3 <0.001Attitude 61.8 76.4 <0.001Anti-smoking attitudes 60.2 75.6 <0.001Anti-drinking attitudes 63.4 76.9 0.0024Life skills 46.2 63.4 <0.001

Table 3. Mean Pre and Post Scores on LST Questionnaire Expressed as Percentages

Junior High Students

Variable Baseline Year 3 Post p valueOverall knowledge 55 65 <0.001Anti-drinking knowledge 50 57 0.0226Life skills knowledge 54 62 <0.001Cultural knowledge 76 93 <0.001Attitude 75 88 <0.001Anti-smoking attitudes 76 90 <0.001Anti-drinking attitudes 74 86 0.0301Life skills 47 77 <0.001

Drug refusal skills 38 79 <0.001Assertiveness skills 63 74 0.0153Relaxation skills 63 73 0.0989Self-control skills 50 67 0.0015

Table 4. Mean Pre and Post Scores on LST Expressed as Percentages


Additionally, focus group participants com-mended the program’s community relevance. Because the program was adapted specifically for the Maskwacis community by Maskwacis Elders and community members, the program incor-porated community relevant language and visual images as well as local knowledge. Additionally, because MLST facilitators were community members, they were familiar to many students; even where facilitators and students did not have pre-existing relationships, focus group participants reported that students could easily identify with facilitators from their own community.

Facilitator skills were also important for the program’s success. Equipping facilitators with the necessary skills was a challenge in the first year of the program, with school personnel noting steady improvements in the second year of implementa-tion, and providing exclusively positive feedback in the third year. The most important facilitator skills described by focus group participants were sincer-ity, confidence, the ability to engage students, the flexibility to accommodate students’ learning and reading levels, and classroom management compe-tence. One teacher noted that, “it was really great that [the facilitator] was so enthusiastic about the program…it got the kids engaged, and they could sense that she really cared.”

Although it was important for facilitators to possess skills in leading students through program teachings and activities, teacher involvement was also described as essential. Facilitators noted that when teachers supported and contributed to MLST classes, students demonstrated increased engage-ment. It was helpful for teachers to assist with classroom management, to provide feedback to facilitators, and to supplement facilitators’ teachings with their own knowledge and experience. By the third year of implementation, all facilitators and school personnel reported strong relationships.

Finally, it was important for the MLST pro-gram to demonstrate compatibility with the school’s core curriculum. According to school personnel, the MLST program complemented students’ regular social studies curriculum, and was a suitable replacement for regularly scheduled health classes. This simplified the task of fitting MLST classes into busy classroom schedules. Even when classroom schedules were hectic, however, teachers were willing to prioritize the program: “They’re life skills, they’re things that are literally going to get them into their adulthood, so maybe math can wait a little while.”

Suggestions for improvement. Focus group participants made a number of suggestions for program improvement related to program delivery. In the first year of implementation, it was strongly suggested that facilitators take part in additional training to enhance their teaching and classroom management skills. This was addressed during the second and third years of implementation. In addition, focus group participants communicated that the program should be delivered for the full school year rather than for only four months of the year. One teacher felt that, “In order to really get the full benefit of the program, it needs to be reinforced all year long…so they are always reminded and supported.”

Suggestions for improvement were also made related to program content. During the first year of implementation, a recommendation was made to increase the involvement of Elders in program delivery, to incorporate more cultural teachings, and to improve the readability of manuals. Each of these suggestions were addressed by setting a goal to bring Elders into each MLST class, and by adapting manuals to include stronger cultural elements and lower reading levels. Suggestions were also made to add modules related to grieving and gender roles, as well as to implement a par-ent component. Finally, focus group participants strongly suggested that the program include more hands-on activities: “When we would do activities, it was amazing, it was the most interaction I would get out of my kids all year.”

QuestionnairesQuestionnaire results are summarized below,

with scores at baseline and at the final data collection point (i.e., year 3 post) depicted. Tables 3 and 4 show LST questionnaire scores for elementary and junior high students respectively. Analyses demonstrated statistically significant increases in scores for elementary students on all LST questionnaire scales. For elementary students, the Overall Knowledge scale included items rele-vant to anti-smoking knowledge (i.e., the harmful effects of smoking) and life skills knowledge (i.e., communication, decision-making, advertising, self-esteem, dealing with stress, and assertiveness). Items relevant to cultural content (e.g., Cree words

Elementary Students

Variable Baseline Year 3 Post p valueTotal score – elementary 47 50 0.5125Total score – junior high 46 46 1.0000

Table 5. Total Pre and Post Scores on Piers-Harris Chil-dren’s Self-Concept Scale-2 Expressed as Mean T Scores


and cultural protocol) were included in the Cul-tural Knowledge scale. The Attitude scale included items relevant to anti-smoking and anti-drinking attitudes. Finally, the Life Skills scale included the same content as the life skills knowledge items, but inquired about behaviors related to these content areas rather than inquiring about knowledge.

Statistically significant differences between baseline and the end of year 3 were also demon-strated for junior high students on all LST ques-tionnaire scales with the exception of relaxation skills. For junior high students, Overall Knowledge included anti-drinking knowledge and life skills knowledge. The Cultural Knowledge scale includ-ed items with the same content as that of the ele-mentary questionnaire. Also similar to elementary students, the Attitude scale included anti-smoking and anti-drinking attitude content. Finally, the Life Skills scale included items relevant to drug refusal skills, assertiveness skills, relaxation skills, and self-control skills.

Table 5 depicts elementary and junior high students’ scores for the Piers-Harris Children’s Self-Concept Scale-2. There were no statistically significant differences between baseline and the end of year 3 for elementary or junior high students on the Piers-Harris-2.

DiscussionThe current project demonstrates the mean-

ingful impact that can be achieved by culturally adapting and delivering an evidence-based pre-vention program in a First Nations community. Quantitative results obtained from pre and post LST questionnaires showed significant positive increases in knowledge, skills, and attitudes between baseline and the end of year three. Find-ings reflect that students not only retained the knowledge communicated to them through the MLST program, but that students learned progres-sively more from the program each year. Further, because students’ scores increased related to both original program content and additional cultural content, these results demonstrate that students improved their knowledge, skills, and attitudes re-lated to substance abuse from both Euro-Western and Indigenous perspectives.

With regard to the Piers-Harris questionnaire, there were no statistically significant differences between scores at baseline and the end of year three. This may be due in part to students’ scores being average at baseline (i.e., t scores approximated 50). It may not be realistic to expect mean self-concept scores to increase beyond an average range. Further,

like other social-emotional measurement tools, the Piers-Harris-2 was validated using a U.S. sample of primarily Caucasian children (Piers & Herzberg, 2002), and its applicability to Indigenous children has not been established. Moving forward, it will be important to establish and utilize pre and post measurement tools that reflect the unique realities of Indigenous children.

In addition to pre and post questionnaires, focus groups were implemented to provide insight regarding program impact, factors contributing to program success, and suggestions for improve-ment. Conducting focus groups after each year of program delivery allowed for suggestions to be addressed continually, contributing to growing community support. Overall, focus group participants communicated strong support for and investment in the program as a promising means to address substance abuse and violence and to enhance traditional culture in Maskwacis. Focus groups provided information regarding the wide-spread impact of the program on schools, Elders, facilitators, and students.

Key to program impact was the utilization of a community-based participatory approach that incorporated Western and Indigenous pillars of knowledge. In this way, the MLST program hon-ored the concept of “two-eyed seeing” (Figure 2), developed by Mi’kmaw Elders Albert and Mudena Marshall. “Two-eyed seeing” recognizes multiple diverse perspectives as valid without privileging one viewpoint over another. Enacting this principle also means acknowledging that multiple perspec-tives can lead to a richer understanding of health issues than one perspective alone. By acknowledg-ing the value of Western and Indigenous substance abuse prevention models, our community-univer-sity partnership provided an example of how Indig-enous and Western research paradigms can co-ex-ist in a space that honors both worldviews. Focus groups revealed that this was critical to the success

Figure 2. “Two-Eyed Seeing” Perspectives


of the project. In particular, schools and commu-nity members identified the central importance of cultural adaptations to the program’s acceptance by students, schools, and the wider community. Focus group participants also reported that adapta-tions elevated the program’s potential to reach and engage students by presenting content in a way that was relevant to their community and cultural con-text. Similarly, adaptation of MLST questionnaires was important to honor community partners’ per-spectives regarding the importance of capturing the program’s impact on students’ cultural knowledge.

The current project also suggests that CBPR can act as a catalyst for community change. How-ever, although pre and post questionnaires and focus groups provided the opportunity to obtain important community feedback on the MLST program, our research team identified a number of unanticipated and peripheral community impacts that were not accounted for by the use of our conventional evaluation methods. For example, community partners took on leadership roles in the program and formed a non-profit society to sustain the MLST program beyond the terms of our original research grant. In order to more fully understand these additional impacts, we supple-mented the current evaluation methods with Out-come Mapping (Earl, Carden, & Smutylo, 2001; Tremblay, Baydala, Rabbit, Louis, & Ksay-yin, submitted for publication). Outcome Mapping is a tool that is sensitive to community change and development, and that emphasizes the significant process of culturally adapting and implementing programs in the context of a CBPR partnership.

Finally, a potential limitation of this study should be acknowledged; namely, the current study does not include a control group. In order to engage all four Nations in this project, it was necessary to deliver the MLST program in all Maskwacis schools. An intervention group and control group were ini-tially defined; however, because these groups came from the same schools, it was not possible to main-tain the integrity of the control group. In particular, students often switched back and forth between control group and intervention group classrooms, and teachers indicated that students were sharing MLST teachings with students in the control group. Consistent with our experience, recent literature indicates that treating complex community change initiatives as controlled experiments may be inap-propriate, particularly in the context of CBPR (Kel-ly, 2010). In addition, we intentionally selected an evidence-based program with extensive research to support its effectiveness. As a result, it was not

necessary to once again prove the effectiveness of the program; rather the aim of this project was to deliver the culturally adapted program with fidelity to the original curriculum, and to document the impact of the adapted program.

ConclusionMaskwacis community members and Elders

established a partnership with University of Alberta researchers to culturally adapt, implement, and evaluate an evidence-based substance abuse and violence prevention program. Students’ overall knowledge increased significantly during the three years of program implementation, and strong support was documented from schools, Elders, students, and other community members. This project demonstrates the considerable impact that can be facilitated by culturally adapting and delivering a prevention program, and by forming and maintaining a strong community-university partnership.

References

Baydala, L., Fletcher, F., Worrell, S., Kajner, T., Letendre, S., Letendre, L., & Rasmussen, C. (2014). Partnership, knowledge translation, and substance abuse prevention with a First Nations community. Progress in Community Health Partnerships, 8(2), 145–155.

Botvin, G.J., Baker, E., Renick, N.L., Filazzola, A. D., & Botvin, E.M. (1984). A cognitive-behav-ioral approach to substance abuse prevention. Ad-dictive Behaviors, 9(2), 137–147.

Botvin, G.J., Griffin, K.W., Diaz, T., & Ifill-Wil-liams, M. (2001). Drug abuse prevention among minority adolescents: Posttest and one-year fol-low-up of a school-based prevention intervention. Prevention Science, 2(1), 1–13.

Botvin, G.J., & Griffin, K.W. (2014). Life Skills Training: Preventing substance misuse by enhanc-ing individual and social competence. New Direc-tions for Youth Development, 2014(141), 57–65.

Castleden, H., Morgan, V. S., & Lamb, C. (2012). “I spent the first year drinking tea”: Explor-ing Canadian university researchers’ perspectives on community-based participatory research in-volving Indigenous peoples. The Canadian Geog-rapher, 56(2), 160–179.

French, S.E., Kim, T.E., & Pillado, O. (2006). Ethnic identity, social group membership, and youth violence. In N. G. Guerra & E. P. Smith (Eds.), Preventing youth violence in a multicultural society (pp. 47–73). Washington, DC: APA Books.

Gazis, N., Connor, J.P., & Ho, R. (2009). Cul-


tural identity and peer influence as predictors of substance use among culturally diverse Australian adolescents. Journal of Early Adolescence, 30(3), 345–368.

Graham, I.D., Logan, J., Harrison, M.B., Straus, S.E., Tetroe, J., Caswell, W., & Robinson, N. (2006). Lost in translation: Time for a map? Jour-nal of Continuing Education in the Health Profes-sions, 26(1), 13–24.

Grekul, J., & Sanderson, K. (2011). “I thought people would be mean and shout.” Introducing the Hobbema Community Cadet Corps: A response to youth gang involvement? Journal of Youth Studies, 14(1), 41–57.

Hare, J. (2011). “They tell a story and there’s meaning behind that story”: Indigenous knowledge and young Indigenous children’s literacy learning. Journal of Early Childhood Literacy, 12(4), 389–414.

Israel, B.A., Schulz, A.J., Parker, E.A., & Becker, A.B. (1998). Review of community-based research: Assessing partnership approaches to improve public health. Annual Review of Public Health, 19, 173-202.

Kelly, T. (2010). Five simple rules for evaluating complex community initiatives. Community Invest-ments, 22(1), 19-23.

Koster, R., Baccar, K., & Lemelin, R.H. (2012). Moving from research ON to research WITH and FOR Indigenous communities: A critical reflection on community-based participatory research. The Canadian Geographer, 56(2), 195-210.

Kitzinger, J. (2005). Focus group research: Us-ing group dynamics to explore perceptions, expe-riences, and understandings. In I. Hollway (ed.), Qualitative research in health care (pp. 56–70). Maidenhead: Open University Press.

Kreuger, R.A., & Casey, M. (2015). Focus groups: A practical guide for applied research (5th ed.). Thousand Oaks, CA: Sage Publications.

Kreuger, C., & Tian, L. (2004). A comparison of the general linear mixed model and repeated mea-sures ANOVA using a dataset with multiple missing data points. Biological Research for Nursing, 6(2), 151-157.

Kulis, S., Napoli, M., & Marsiglia, F.F. (2002). Ethnic pride, biculturalism, and drug use norms of urban American Indian adolescents. Social Work Research, 26(2), 101–112.

Kumpfer, K.L., Magalhaes, C., & Xie, J. (2012). Cultural adaptations of evidence-based family in-terventions to strengthen families and improve chil-dren’s developmental outcomes. European Journal of Developmental Psychology, 9(1), 104-116.

Minkler, M., & Wallerstein, N. (Eds.). 2003. Community-based participatory research for health.

San Francisco, CA: Jossey-Bass.National Health Promotion Associates, Inc.

(2011a). Life skills training questionnaire: Elementary school version. Retrieved from www.lifeskillstrain-ing.com.

National Health Promotion Associates, Inc. (2011b). Life skills training questionnaire: Middle school version. Retrieved from www.lifeskillstrain-ing.com.

National Registry of Evidence-based Programs and Practices. (2008). Retrieved from http://www.nrepp.samhsa.gov/ViewIntervention.aspx?id=109.

Piers, E.V., & Herzberg, D.S. (2002). Piers-Har-ris Children’s Self-Concept Scale™, Second Edition. Los Angeles, California: Western Psychological Ser-vices.

Smith, L., (1999). Decolonizing methodologies: Research and Indigenous peoples. London, UK: Zed Books.

Toulouse, P.R. (2008). Integrating Aboriginal teaching and values into the classroom. Sudbury, ON: Laurentian University.

Tsethlikai, M., & Rogoff, B. (2013). Involve-ment in traditional cultural practices and American Indian children’s incidental recall of a folktale. De-velopmental Psychology, 49(3), 568–578.

About the AuthorsLola Baydala is an associate professor in the

Department of Pediatrics at the University of Alberta. Fay Fletcher is professor and associate dean academic, Faculty of Extension, the Uni-versity of Alberta. Melissa Tremblay is a program evaluator in the Department of Pediatrics at the University of Alberta. Natasha Rabbit is the exec-utive director of the Nehiyaw Kakeskewina Learn-ing Society in Maskwacis, Alberta. Jennilee Louis is a research assistant for the Nehiyaw Kakeskewina Learning Society in Maskwacis, Alberta. Kisikaw Ksay-yin is an Elder in the Nehiyaw Kakeskewi-na Learning Society, Maskwacis, Alberta. Caitlin Sinclair is research coordinator, Department of Pediatrics, University of Alberta.


Our Journey into a CBPR Project: Health and Nutrition Solutions in the Alabama Black Belt

Project UNITED is a federally funded grant that addresses obesity issues in the Alabama Black Belt. The Black Belt represents some of the poorest counties in the United States and is plagued with chronic health conditions—obesity, diabetes, high blood pressure, cancer—which are predominantly found in African Americans. Community leaders from the Black Belt Community Foundation, a 501(c)3 nonprofit corporation whose purpose is to help improve the quality of life of residents in the Alabama Black Belt, participated in the project as community health scholars. Project UNITED partnered community health scholars and academic scholars to develop proposals that addressed obesity issues in rural communities. A team of three community health scholars and three academic scholars collaborated and formed, Home Sweet Home, a two-year, $50,000, multigenerational obesity intervention in Greene and Sumter counties. This paper provides reflections from the three community health scholars regarding their experience in a community based participatory research (CBPR) project.

The paper is presented as first person testimonials describing the experiences of each participant.

Darlene Robinson, Greene County Being one who is always looking for ways to

improve myself and my community, I took a step of faith in response to the invitation I received to become a community health scholar with Project UNITED. I also believed I would acquire knowledge that would help my community. In the community my mission is to improve everyday life of parents and their children. Even though my plate was already full, I felt it would be an opportunity to find ways to improve myself and my community.

The Project UNITED journey introduced us to individuals from around the state who also wanted to improve themselves and their communities. I could tell the training was well thought out, as we visited a small garden at a rural school where the youth were being introduced to growing their own foods. This was the Project UNITED pilot project. Educating our community about healthy lifestyle changes is very important and we learned ways to take it home. Project UNITED brought professors from the University of Alabama and community people together as a learning tool that showed that we all share the same passion of making Alabama a healthier place to live.

The circle was complete when our team formed. We had three members from the community and three members from academia, with a mission of curtailing obesity by educating ourselves. As a group we merged our thoughts, hashed out our ideas, made a plan, found an audience—the preschoolers and their parents—and formed a roadmap to get us where we wanted to go. We named our program “Home Sweet Home.” Our mission was to introduce and teach a new way of thinking about eating. We sought to change current statistics that says we are an obese society.

If you’re busy, think you have that all you can handle, feel you know all you need to know, then CBPR is probably for you, because it takes all you’re doing and all you can handle and turns it into a mission that you’ve been on all the time but gives you directions to get to a better place. Then it will not just be your idea of what the community needs, but the community will be able to tell you what they want.

Debra Clark, Sumter CountyIn 2013, I was chosen by the Black Belt

Community Foundation to be a community

Yawah Awolowo, Debra Clark, and Darlene Robinson


health scholar. This was because of my health-related awareness work at the Health and Wellness Education Center (HWEC) Disease Management Program. At the first meeting my first thought was “Why am I here?” realizing that I did’t have time to take on an additional project. But my second thought was “how can I not participate?” considering the impact this project could have on my community. Certainty, it is through strong community partnerships with local agencies that has made HWEC successful. The agency has made great strides to implement initiatives to address obesity, diabetes, cardiovascular disease, and other chronic illnesses to improve the health of local residents.

As a result of this project, I1. Was invited to present a paper entitled

“Fighting Obesity for the Health of It” at the Rural Health Conference in Tuscaloosa in 2013.

2. Received the Rural Heroes Award from the UA College of Community Health Sciences.

3. Was able to expand the Jump To It (what is it?) program for additional communities.

4. Produced a simple handout on the traffic light diet to promote healthy eating for distribution at health fairs and community events.

5. Have witnessed the greatest impact within my own family. My 5-year old grandson was never introduced to yogurt, but now eats yogurt and encourages his mom to “add stuff ” to make it good.

Although this fight has been tedious and time consuming, I must admit that rewards have been greater than the struggle. The program has helped build capacity and strengthen my agency by connecting me with a team of experts with genuine concern and interest, which correlates with the agency’s mission of promoting healthier lifestyles through education and prevention.

Yawah Awolowo, Sumter County

As a community health scholar participating in Project UNITED, I have enjoyed adding to what I already knew about health-related issues and unhealthy eating. I nurtured my family in an organic, self-sufficient, natural homebirth/home schooled permaculture environment and have sustained five seriously committed years of organic farming on a 66-acre, family-owned farm. My journey with CBPR and the development of a partnership with academic researchers not

only expands my baseline knowledge, but the collaborative process has also allowed me to continue to put into practice my knowledge in a way to benefit my community. I have learned that while academic researchers are very knowledgeable about health and treatment, I also bring a great deal of knowledge about nutrition and health. Additionally, I have expertise about the rural community where I live, which I know is valuable for eliminating obesity in this community.

My participation in Project UNITED enlightened my interest in heart disease and obesity, major health concerns in my family. My mother, who was 90 years young developed heart disease. She had no hypertension, diabetes, no medicines for her entire life. The doctors informed the family that there was a blockage in one of her heart valves; surgery was necessary to remove the blockage. One of the major reasons I chose to be involved as a community health scholar in Project UNITED was because I watched my 90-year-old mother go through heart surgery and witnessed her spectacular recovery as she completely changed her eating habits. The other reason for my interest is because there are so many people in my family who have serious health conditions. My sister had surgery two weeks prior to my mother and my daughter has also had medical issues. For these reasons, my desire has been fueled to establish alternative eating habits as a means to prevent disease, especially with kids at an early age. I believe this can be done by letting them be involved with the purchase, preparation, cooking, and serving of good healthy foods.

Project UNITED was the right program for my personal and community needs to learn more about prevention of this disease in our community. I realized that the cultural normality of food intake always consists of some type of animal products on a daily basis that cause blockages and slow down the process of the heart pumping blood and oxygen.

INVOLVEMENT IN HOME SWEET HOME PROJECT

Debra Clark As a community health scholar I have been

able to work with a team of people who share my interest of eliminating diseases through education and awareness. We met as a group utilizing a tool called speed dating to develop a professional relationship/marriage that has grown into a big happy family. Consider the three-legged stool;


one cannot stand without the other. Each of us brings so much to table that I wonder how we could have been so close without ever crossing paths. Nevertheless, we have bonded and now look for ways to improve our efforts by enhancing our writing skills and seeking additional resources to promote sustainability to enhance our community for a better future. Community health scholars: What better name can you give to such individuals who are a community’s first choice when it comes to health care. We are the movers and shakers in our community.

Darlene Robinson Home Sweet Home is our project. We

named it, we designed the program, we listened to the community, we respected the community, we educated ourselves, and we educated the community. The program was not handed to us; it was made by us for us with the mission in mind of changing lifestyles by teaching our families how to spend time together, talk to each other, share with others, cook with love, train our children how to prepare a dish and making home truly Home Sweet Home. The mission of introducing new ways of cooking to our parents was well received. The preschoolers enjoyed the experience and the parents appreciated the information because our eating habits have led to many health improvements. The traffic light playing cards (red light = foods to avoid; eat; yellow = sometimes foods; green = go foods) were designed as a way for families to learn what foods to stop eating, for example cautiously eating the yellow foods and eating lots of all the green foods. Each family received a deck of Home Sweet Home cards designed by the team.

The program was introduced in the communi-ties represented by the community health scholars. The team was glued together by a passion to make a difference, and we believe it is the start of great things to come. Our coming together was not by accident; it was time for academia and the community to unite on a mission of bringing the idea of healthy eating to the community.

Yawah Awolowo Project UNITED has been a perfect learning

catalyst. One of the major goals of the project was to partner community leaders with academic researchers to develop and pilot test an obesity intervention program. We spent the year getting to know each other’s interests and personalities with an emphasis on building trust and working

toward sustaining a lasting relationship. This was important to me because I have had several business encounters where I have shared ideas and people have either ignored or taken my ideas. These experiences have made me wary to trust experts sometimes. Therefore, any emphasis on building trust and sustainability are going to be important for me and my community.

I eventually ended up working with a team of six. We talked about potential projects based on our interests where we would be competitive for funding internally. We also evaluated the best approaches to write the grant using all our strengths. As a member of the team, I also earned my IRB certificate, which builds on the training I received in medical ethics and research. We titled our submission: Home Sweet Home, with a focus on home food environment. This project addresses obesity in the age group 2 to 5 years (pre-k). Our primary goal was to get 2-5-year-old children, along with their parents, to learn how to choose good, healthy produce, prepare food in a clean area, practice safety measures, make it a child friendly environment, create their own recipes using the things they like to eat, and spark their interest in healthy eating habits from an early age. Thankfully, our collaborative proposal received funding for one year through a competitive internal grant process.

As I reflect on the past year of my role as a Community health scholar and the value that CBPR has added to my life, I believe that the time invested in the program will save lives in my community. And that is time well spent. As the program began its journey into the community, the best began to happen with children and the parents. We prepared familiar food with new twist, collards greens with no meat; to make home-cooked foods with fresh herbs and olive oil; the children preparing salads, constructing parfaits and serving their parents and grandparents; providing chef hats and aprons and gifts for families to continue their healthy journeys.

About the AuthorsYawah Awolowo is an organic farmer and chef

at Mahalah Farms in Cuba (Sumter County), Alabama. Debra Clark is founder and executive director of the Health and Wellness Education Center in Livingston (Sumter County), Alabama. Darlene Robinson is founder of Imagine Me Youth Development Program in Greene County, Alabama.


Cultivating Equitable Partnerships Through Cultural Synthesis

Eric Conrad, Meghan Shewmake, Courtney Shows, and Jen Nickelson

Sustainable community partnerships require more than good intentions, necessitating the facilitation of true collaboration to be successful (Blouin & Perry, 2009). McLean and Behringer (2008) assert that for a truly equitable partnership to exist, each party involved must both contribute to, and receive benefits from, the relationship. These contributions from students, campus partners, and communities provide meaningful experiences that result in reciprocal benefits among all partners. Practices that enable equitable collaborations include the establishment of personal relationships, reciprocity, flexibility in adapting to unexpected outcomes, and collaboration to overcome barriers (King, Williams, Howard, Proffitt, Belcher, & McLean, 2004). Fostering these practices result in increased sustainability with the potential to expand collaborative efforts.

The Holt Health Fair is a community event with the goal of providing health screenings and information dissemination to the community of Holt, Alabama. The inception of this annual event was developed through the formation of a collaborative between The University of Alabama (UA) students within the Delta Xi Chapter of the Eta Sigma Gamma (ESG) National Health Education Honorary, and members of the Holt community. Through the maturation of this partnership, stakeholder capacity has increased through mutual support and empowerment. This evolution is the direct result of a partnership that strongly values community voice and input. The following is a student perspective illustrating pertinent community engagement principles, as adapted from McLean and Behringer (2008), and their application in the development of the Holt Health Fair and community partnership.

Partnerships Thrive on Personal Connections Two commonly asserted theoretical

approaches to community change are that of

“cultural invasion” and “cultural synthesis.” Cul-tural invasion occurs when external institutions or academics impose values and agendas with brazen disregard for community voice. Cultural synthesis necessitates those same external forces support and supplement community efforts to collabora-tively solve community need (Green & Kreuter, 2005). The primary distinction between these the-oretical approaches is in the formation of personal connections within the community. Development and fostering of these relationships demonstrate an investment in the community’s well-being that extends beyond the cessation of singular research projects and funding.

In 2008, a collective of concerned citizens established the Holt Community Partnership (HCP) in an effort to develop initiatives within Holt that “transform lives through opportunity, education, unity, and safety” (HCP, 2010). The HCP was comprised of Holt community members, local law enforcement, religious leaders, board of education affiliates, as well as faculty and staff from UA. This partnership established a foundation in which personal connections between UA and the Holt community could be fostered through cultural synthesis.

The establishment and cultivation of this relationship led to the inclusion of UA student associations, such as the ESG health organization. Since the integration of ESG and Holt, students have been afforded the opportunity to foster relationships within the community through interaction and participation at HCP meetings as well as through additional community collaboratives and initiatives. This community immersion is a transformative process in which programs and initiatives are no longer simple extracurricular activities, but vested experiences in which tangible change can result within the community where these relationships are formed.


Participation and Communication Requires Reciprocity

A true collaborative is one that includes an empowering and power-sharing dynamic that promotes an equitable partnership (Israel, Schulz, Parker, Becker, Allen, & Guzman, 2008). This dynamic may be challenging for academics due to the relinquishment of power or the perceived threat to validity, but is necessary to ensure cultural synthesis and capitalize on community expertise to enhance initiatives and improve outcomes. Support for this synergistic relationship can be achieved through the mobilization and utilization of university resources to supplement community efforts. The reciprocal nature of an equitable partnership ensures community collaborators have input in the development and implementation of programs that can be tailored to meet the diverse needs of the community (Ahmed & Palermo, 2010).

Prior to the inception of the health fair, the HCP and UA collaborators established a community festival within Holt. Over time, attendance and participation in the festival waned, necessitating a reevaluation of the initiative which ultimately resulted in the conclusion that the festival was no longer feasible and meeting the needs of the community. Collaborators began an iterative process to develop a new initiative within Holt to maximize the collective capacity of UA stakeholders and the community. Consistent with the theoretical approach of cultural synthesis, this cyclical method emphasized concerns created by community members and the capacity of stakeholders to address this need. The outcome of this reciprocal dynamic was a community proposal for the establishment of a Holt Health Fair and the creation of processes to support the initiative. Critical to this process was the explicit attention to the knowledge and expertise of community members and emphasis on empowerment inclusive of information sharing, decision-making power, resources, and support within the partnership.

The importance of reciprocity does not diminish following the approval of the initiative, but must be maintained throughout its entirety. As a health education organization, ESG was enthusiastic regarding the community initiated proposal for a health fair within Holt. While well intentioned, unbridled enthusiasm may result in an overzealousness that neglects or excludes community input and precipitates cultural invasion. Eta Sigma Gamma student and faculty representatives perpetuated an

equitable dynamic through attendance at HCP sessions to ensure continued communication that emphasized community-initiated development, implementation, and evaluation. Respect for the unique perspective and expertise of Holt collaborators produced a reciprocal transfer of knowledge, skills, and resources that enhanced the capacity of all stakeholders.

Communication and Cooperation Overcome Barriers

Partnerships sustained through personal relationships that prioritize reciprocal communication and power facilitate leveraging of collective capacity to overcome project barriers (Green & Kreuter, 2005). A significant challenge following the decision to develop a health fair was creating a suitable event on an abbreviated timeline. Previously established relationships with the HCP facilitated efficient role delineation, allowing for the attainment of resources and services necessary for event implementation. Following the initial implementation of the Holt Health Fair, a stakeholder evaluation identified further barriers that adversely affected project outcomes. Promotion efforts in year one were primarily managed by ESG students which utilized radio and television broadcasts, as well as flyer distribution, yet intended reach was lower than expected. Year two of the initiative improved these processes through increased community involvement in the dissemination, utilizing the cultural influence of local church networks and businesses. Collaborations with Holt High School teachers and coaches resulted in presentations from high school students as well as performances by the dance team and band. The expansion of entertainment activities and the addition of childcare were incorporated to increase event appeal and facilitate easier access to services for community participants with children. Auxiliary barriers that were also addressed included the improvement of issues in regards to venue, transportation, as well as the expansion of vendors and services. Addressing and reducing barriers throughout the evolution of the health fair was expedited through cooperative mechanisms that prioritized continual communication and assessment between ESG, the HCP, as well as UA and community organizations.

Progress Necessitates Increased Capacity The readiness of the partnership to be

adaptable and amenable to increase contributions to the program strengthens the collaborative and


improves program success (Sandy & Holland, 2006). Compared to the health fair, the Holt Festival required less resources and investment from each collaborator in the partnership. During the initial conception of the Holt Health Fair, the partnership determined current stakeholders needed to expand their capacity to accomplish the successful implementation of the initiative. The UA stakeholders increased capacity through incorporation of ESG student members and the expansion of university collaborators able to provide resources and health services. Barriers identified served as the impetus for increased community contributions resulting in systematized processes to increase promotion and engagement. Maintaining the equitable relationship through this expansion necessitated that each collaborator augment their contribution to improve the initiative and preserve reciprocity.

Expanding capacity to increase contributions of the partnership should also result in increased benefits to all stakeholders (King et al., 2004). Through the development and improvement of the Holt Health Fair, offerings were expanded from each collaborator which in turn increased the benefits each party received. University students within ESG and other collaborating organizations were provided the opportunity for experiential learning in their respective disciplines. Through application of classroom content, students benefit from deeper understanding of course material and are afforded the opportunity to strengthen cultural competency, leadership attributes, and self-efficacy (Powell & Conrad, 2015). Community members gained the expertise of UA students and faculty, access to health services through the university and external organizations, as well as media coverage and publicity. The benefits resulting from increased capacity were cultivated through relationships that foster trust and reliance in the collaborative process. The increased contribution and capacity can then be extended and utilized in other community partnerships.

Successful Partnerships Breed Successful Partnerships

The principles and benefits of equitable partnerships are transferable regardless of initiative, event, or program (Rhodes, Malow, & Jolly, 2010). The same practices of fostering and sustaining partnerships of reciprocity through cultural synthesis serve to reinforce relationships facilitating initiative success. Partnerships capable of overcoming project barriers promote capacity

building through multidirectional learning on an individual or community level. Cultivating improved competence and proficiency through existing partnerships result in increased knowledge, skills, and expertise that enhance diverse community initiatives.

In addition to being active partners of the Holt Health Fair, ESG is also involved in additional health related initiatives within the Holt community. One such coalition is the Holt afterschool program in which ESG members educate elementary students regarding health topics in collaboration with the community organization Tuscaloosa’s One Place. An additional initiative is the Holt Health Lab in which UA and Holt collaborators aim to empower the community to meet health needs through screening, education, and policy. Similar to the Holt Health Fair, these initiatives incorporate processes to ensure community voice and reciprocity to enhance program success. Lessons learned in each initiative may serve to build on successful aspects and avoid barriers that other projects may have encountered. In this way, the expertise and capacity of each partnership is extended beyond itself to ultimately benefit the community through pervasive processes.

Partnership Length Increases SuccessSuccessful partnerships that provide

meaningful contributions and benefits to the community and university is a long-term process that requires commitment from both parties (Israel et al., 2008). Cultural synthesis asserts that the establishment and maintenance of trust and rapport among communities necessitate that commitments extend beyond single projects or funding. This prioritization on relationships over outcomes mediate an equitable power-sharing dynamic in which the partnership can more efficiently overcome barriers and increase capacity to achieve goals. These approaches allow for the development of an infrastructure that promotes longevity and sustainability which increases partnership and project success.

The successes of the Holt Health Fair and other community initiatives are direct results of an equitable partnership facilitated through cultural synthesis. Utilizing this approach, UA collaborators prioritized people over processes to develop and strengthen relationships among the Holt community. Through this genuine engagement and interaction within Holt, ESG student perspectives were altered from a mentality that prioritized a tangible product to a mindset


that strongly desired an increase in community wellness, well-being, and empowerment. This vested interest fostered reciprocity among ESG students, UA faculty, and Holt community partners through the establishment of processes that respected community voice and perspective. Utilizing the collective strengths of the partnership, event deficiencies were identified and barriers were surmounted as each stakeholder evaluated and subsequently expanded contributions to the initiative, increasing student and community capacity. Lessons learned from the development, implementation, and evaluation of the Holt Health Fair can be applied in diverse community initiatives to further transform lives through opportunity, education, unity, and safety. Establishing an equitable partnership within Holt did not deny differences among multiple perspectives, but embraced diversity by affirming undeniable support through cultural synthesis to create improved outcomes in community engagement.

ReferencesAhmed, S.M., & Palermo, A.S. (2010). Com-

munity engagement in research: Frameworks for education and peer review. American Journal of Public Health, 100(8), 1,380–1,387.

Blouin, D.D., & Perry, E.M. (2009). Whom does service learning really serve? Communi-ty-based organizations’ perspectives on service learning. Teaching Sociology, 37(2), 120–135.

Green, L.W., & Kreuter, M.W. (2005). Health program planning: An educational and ecological approach. New York: McGraw-Hill.

Holt Community Partnership. (2010). How the HCP developed. Retrieved from https://holtcom-munitypartnership.wordpress.com/about/.

Israel, B.A., Schulz, A.J., Parker, E.A., Becker, A.B., Allen, A.J., & Guzman, J.R. (2008). Critical is-sues in developing and following community based participatory research principles. In M. Minkler & N. Wallerstein (Eds.), Community-based participa-tory research for health: From process to outcomes (pp. 53–76). San Francisco: John Wiley & Sons.

Jacob, W.J., Sutin, S.E., Weidman, J.C., & Yea-ger, J.I. (2015). Community engagement in higher education. In Community Engagement in Higher Education (pp. 1–28). SensePublishers.

King, B., Williams, W., Howard, S., Proffitt, F., Belcher, K., & McLean, J.E. (2004). Creating the bridge: The community’s view of the expanding community partnerships. In B.A. Behringer, B.C. Bach, H. Daudistel, J.W. Fraser, J. Kriesky, & G.E.

Lang, (Eds.), Pursuing opportunities through part-nerships: Higher education and communities (pp. 75–85). Morgantown, WV: West Virginia Univer-sity Press.

Minkler, M., & Wallerstein, N. (Eds.). (2011). Community-based participatory research for health: From process to outcomes (2nd ed.). San Francisco, CA: Jossey-Bass.

McLean, J.E., & Behringer, B.A. (2008). Estab-lishing and evaluating equitable partnerships. Jour-nal of Community Engagement and Scholarship,1(1): 66-71.

Powell, B., & Conrad, E. (2015). Utilizing the CIPP model as a means to develop an integrated service-learning component in a university health course. Journal of Health Education Teaching, 6(1), 21-32.

Rhodes, S. D., Malow, R. M., & Jolly, C. (2010). Community-based participatory research (CBPR): A new and not-so-new approach to HIV/AIDS pre-vention, care, and treatment. AIDS Education and Prevention, 22(3), 173-183.

Sandy, M., & Holland, B. A. (2006). Different worlds and common ground: Community partner perspectives on campus-community partnerships. Michigan Journal of Community Service Learning, 13(1), 30-43.

About the AuthorsEric Conrad is a doctoral student, Meghan

Shewmake is working on a master’s degree, Courtney Shows is an undergraduate student, and Dr. Jen Nickelson is an associate professor, all in the Department of Health Science at the University of Alabama.


I am honored to have been asked to serve as the new Book Review Editor for the Journal of Community Engagement and Scholarship. Not only do I view this as a great opportunity to serve in the advancement of higher

education community-engaged scholarship, but also as an opportunity for me to learn from authors, scholars, and community partners from across the country and around the globe.

I am currently director of Academic Engage-ment at the University of North Georgia, where my overarching responsibility is to institutionalize community-engaged scholarship across our five campuses. Among the ways that we accomplish this is to engage with and develop faculty members, work with students and let them know about the educational opportunities available to them, and develop mutually beneficial communi-ty partnerships.

I am a relatively recent graduate of the University of Georgia’s Institute of Higher Edu-cation, and my scholarly agenda includes faculty member motivations related to the scholarship of engagement, student motivations and outcomes related to service-learning experiences, and insti-tutional and public policies that affect community engagement in higher education. In my research, I utilize the complementary strengths of both quantitative and qualitative analytic methods to fully explore scholarly questions, often through interdisciplinary collaborations. I am also interested in exploring how graduate students

are socialized for and participate in community engagement, as well as strategies for integrating engagement into the STEM fields.

My goal with the book reviews in JCES will be to introduce readers to literature that advances knowledge, provides practical advice, dissemi-nates best practices, and encourages conversa-tion and dialogue. Book reviewers will not only provide summaries of recently published titles relevant to our field, but will also critically analyze and examine the works through the lens of community engagement.

I invite faculty members, administrators, staff members, students, and community partners to offer their interpretations of the literature. From its beginning, JCES has always intentionally been a “different kind of journal,” which helps to estab-lish a niche among scholarly publication outlets. In addition to providing established scholars and researchers with a platform to disseminate their work, we also seek to hear from all of the people who work to make these multi-faceted communi-ty-campus partnerships work. In the true spirit of community engagement, we encourage author-ship from first-time authors, students at all levels, and community partners.

To paraphrase Ernest Boyer, higher educa-tion has the potential, and I might even say the responsibility, to advance both intellectual and civic life, and I believe that JCES plays an import-ant role in this process. I gratefully look forward to the opportunity to work with the rest of the JCES editorial and publishing team as we continue to build on the wonderful foundation already in place.

Dr. Andrew J. PearlUniversity of North Georgia

Book Review Editor

JCES invites submission of book reviews that speak to a wide range of issues relevant to the scholarship of engagement. Reviews of books within the social sciences, natural sciences and math, medicine and health, the environment, law, business, philosophy, religion, communication, and the arts and humanities are encour-aged. Although reviews of individual books are the most common, JCES also invites submission of several reviews that speak to a common topic area, to be published as a group. All book reviews submitted to JCES should provide readers with a broad overview of the book, but should go beyond this description to discuss central issues raised, strengths and limitations of the text, and current issues of theory and practice raised by the book that are germane to the subject matter and engaged scholarship.

Follow the format style for Book Reviews found on our website (http://jces.ua.edu/submit-to-jces/) and on Page 83 of this issue of JCES. Email your review as a Microsoft Word attachment to [email protected].

Instructions to Book Reviewers


Comparing, Contrasting, and Combining Civic Engagement Education Through the Lenses of Social Entrepreneurship and Service-learning

Review by Paul H. Matthews University of Georgia

Sandra L. Enos, Service-Learning and Social Entrepreneurship in Higher Education: A Pedagogy of Social Change. Palgrave Macmillan: New York, NY, 2015, 101 pages. DOI: 10.1057/9781137554444.0001

Is social entrepreneurship education just a kind of service-learning found in business schools? Or is it grounded in such different history, assump-tions, and practice that its commonalities with community-based pedagogy are only superficial at best? The field of social entrepreneurship—an “innovative, social value-creating activity” (Wei-Skillern, Austin, Leonard, & Stevenson, 2007, p. 4)—is sometimes construed as analogous to service-learning, but on many campuses the two fields seem to occupy disparate and non-overlap-ping spaces. Sandra Enos’s brief volume (84 pages in four chapters, plus prefix, one appendix, and refer-ences), published in 2015 as part of the Community Engagement in Higher Education series (Palgrave Macmillan), seeks to change that, and to illustrate some of the ways in which these pedagogies of service-learning and social entrepreneurship can, do, and do not intersect on college campuses.

In the first chapter (“The Landscape of Social Change Education”), Enos—a sociology professor with experience in both service-learning and social entrepreneurship—sets the context with a sketch of some recent discussions of university engage-ment’s place in higher education, and contends that both fields focused on in this volume should be considered a subset of “social change education.” She provides evidence for the growing institution-alization of each, for instance through journals and professional associations, and draws parallels specifically between the existence of overarching organizations that encourage campus involvement in both movements (e.g., between Campus Com-pact and Ashoka U). Next, she summarizes prior work presenting these two fields’ commonalities (e.g., these are social-change oriented movements seeking greater campus reach; entail communi-ty-based partnerships; and provide opportunities for real-world applications of knowledge) as well

as differences (e.g., a stronger focus on student learning in service-learning, versus on commu-nity/societal change in social entrepreneurship; a stronger orientation toward “disruption” of existing structures and practices in the latter; and differing language as well as desired learning outcomes). While Enos also contends that “[S]ocial entrepre-neurship prizes the founding of new organizations” (p. 18), other researchers have noted the growth and value of social “intrepreneurship”—i.e., working to implement change within existing structures and organizations (Bornstein, 2007; Bornstein & Davis, 2010)—a construct not included in Enos’s overview.

Her second chapter (“Organizing for Engage-ment”) comprises the bulk of the book’s research contribution, an attempt to determine how these two pedagogies are organized and interrelate on campuses that offer both. Enos reports on website analyses and interviews she conducted with staff at the 10 campuses currently recognized with both Carnegie’s community engagement classification and the “Changemaker Campus” designation by Ashoka U. She contends that these “exemplars of service-learning and of social entrepreneurship education” (p. 23) should offer a view into the current status of institutionalization and potential relationships between these subfields of commu-nity engagement on U.S. campuses. As such, she investigates whether there are “patterns in the organization of these programs, whether there are typical disciplinary homes for service-learning and social entrepreneurship courses, …and how pro-grams that offer [them] are related to each other” (p. 27). However, Enos reports that little consisten-cy was found; instead, “what we see are individual profiles, tied to institutional size, history, culture and leadership” (pp. 32–33). While she summa-rizes information about each campus in a table, its content reads more like field notes than an


analysis. As might be expected from a range of institutional sizes, types, and locations, she found that the colleges and universities studied had varying amounts of coursework, centralization, and support structures, and used different terminology in talking about their initiatives.

In chapter three, “Challenges for Ser-vice-Learning and Social Entrepreneurship,” Enos iterates several of the criticisms that have been extended toward these two sets of practice, includ-ing (for service-learning) a lack of “deliberate and intentional incorporation of civic skills” (p. 45), as well as issues of privilege, social justice, and the problematization of “service” (pp. 47–48). For social entrepreneurship, Enos notes that this field sometimes uses different terms (e.g. “social enter-prise”) implying different foci; that innovations may struggle in being implemented at scale; and that the “celebratory nature of innovations in the social entrepreneurship space makes it difficult to mount a critical review of these practices” (p. 55). She also summarizes concerns that social entre-preneurship may tend to “paint the state, unions, public employees and other existing nonprofits as the enemy” (pp. 56–57), undermining rather than enhancing existing efforts, which may contrast with service-learning’s focus. In terms of social en-trepreneurship education, Enos notes that existing courses have been criticized for a lack of emphasis on complex problem-solving, and for mismatches between desired competencies identified in course syllabi and those recommended by practitioners in the field. While interesting, and a good overview of some of the critiques offered for each field, this chapter does not contribute much new to the discussion beyond summarizing.

In the book’s final chapter, “Educating for Engagement: A Turning Point,” Enos renews her recommendations that, as two subsets of campus/community engagement, service-learning and social entrepreneurship should become better in-tegrated institutionally. She provides examples from her own and other universities as demonstra-tions of how these two fields may fit into a broader conceptualization of civic engagement, as well as how particular disciplines might implement both. After repeating her calls for more cross-fertiliza-tion, Enos suggests the need “to challenge each oth-er on what works—in teaching and research—and certainly on what strategies get us closer to work-ing with community partners in problem-solving reciprocal ways” (p. 83). Finally, the book’s ap-pendix provides a list of campus representatives for Chapter 3’s interviews.

Several aspects of this book raise issues worthy of contemplation for those in the community engagement field. The explicit comparison and contrasting between the two sets of philosophies and practices illuminates several interesting points. For instance, Enos contends:

Among the most important differenc-es between service-learning and social entrepreneurship include the former’s focus on groups and collaboration vs. the latter’s on individualism, and vocabulary borrowed from the nonprofit sector vs. vocabulary taken from the business world. Other points that differentiate these fields are skepticism about market-based approaches to social problems on part of the service-learning community com-pared to an embrace of market-based solutions in the social entrepreneurship field. The service-learning field aims to educate students for civic engagement and democracy while some versions of social entrepreneurship work at solving problems outside of or independent of politics (pp. 16–17).

Likewise, a careful consideration of what each field might offer the other is a valuable exercise. Could some of the tools and concepts used in social entrepreneurship—e.g., the double/triple bottom line for determining return on invest-ment—be helpful for assessing service-learn-ing programs’ impacts? Could service-learning’s established expertise in reflection and in highlight-ing the importance of reciprocity with community partner organizations bolster the success of social entrepreneurship education and application?

In several areas, the book could be stronger. For instance, while couched at times in the lan-guage of research, this volume’s findings from campus interviews do not seem to follow many standards of qualitative methodology (e.g., thick description, coding, etc.), and as such fall some-what short. Likewise, while the author includes a solid listing of prior articles that relate to this work’s guiding questions, Enos’s style of reviewing literature often seems more focused on enumerat-ing a list of each author’s points, rather than syn-thesizing them as a whole. While in two separate chapters Enos offers other authors’ definitions for “social entrepreneurship,” she does not ever clarify how, specifically, she is using the term; and, while she notes that “the practice of social entrepre-


neurship is distinct from social entrepreneurship education” (p. 11), that distinction is not consis-tently apparent in the volume.

Several chapters enumerate details about certain journals, organizations and initiatives, yet omit others of similar potential interest and merit without explanation. For instance, for service-learning and community engagement, she lists several journals (JCES; the Michigan Journal of Community Service Learning) and conferences and organizations (IARSLCE; Campus Compact), while other equally relevant resources (e.g., the Journal of Higher Education Outreach and Engage-ment; the Engagement Scholarship Consortium) are omitted. On the social entrepreneurship side, Enos provides a strong focus on both the Skoll Foundation and Ashoka, but does not acknowl-edge other important resources and players such as the Acumen Fund, the Schwab Foundation for Social Entrepreneurship, or Echoing Green (Born-stein & Davis, 2010). Likewise, her “short history of social entrepreneurship” seems to lay the found-ing of this field at the feet of Jeffrey Skoll and Bill Drayton, with no mention of key international “pioneers” (Bornstein & Davis, 2010, p. 13) such as Muhammad Yunus (Grameen Bank) and Fazle Abed (Bangladesh Rural Advancement Commit-tee). Finally, frequent typos and minor errors (e.g., in names of national organizations, journals, and funders, or in referring to the “2012 [sic] Carnegie round of applications” [p. 29]) also distract some-what from the book’s content.

In the end, Enos defends and extends her thesis statement: “Some may believe that service-learning and social entrepreneurship models of educating students for community engagement are incompatible but I am going to argue that they are not” (p. 70); however, as her own research shows, even on campuses consid-ered exemplary in each area, at present these two fields tend not to converge in practice. For readers interested in how these two branches of engaged pedagogy might interweave more productively, Service-Learning and Social Entrepreneurship in Higher Education may be a good starting point, from either side of the equation.

AcknowledgmentThe JCES editorial team would like to acknowl-

edge and thank Palgrave Macmillan for providing an electronic copy of the book for this review.

ReferencesBornstein, D. (2007). How to change the world:

Social entrepreneurs and the power of new ideas (updated edition). New York, NY: Oxford Univer-sity Press.

Bornstein, D., & Davis, S. (2010). Social en-trepreneurship: What everyone needs to know. New York, NY: Oxford University Press.

Wei-Skillern, J., Austin, J.E., Leonard, H., & Stevenson, H. (2007). Entrepreneurship in the social sector. Thousand Oaks, CA: Sage Publishing.

About the ReviewerPaul H. Matthews is associate director of the

University of Georgia Office of Service-Learning.


Book on eService-Learning Posits Goals of Service-Learning Pedagogy and Technology’s Role in Achieving Them

Review by Sarah Stanlick Lehigh University

Jean Strait and Katherine J. Nordyke (Eds.), eService-Learning: Creating Experiential Learning and Civic Engagement Through Online and Hybrid Courses. Sterling, VA: Stylus Publishing, 2015, 182 pages. ISBN 978-1-62036-064-4.

Higher education has become increasingly technology-centric with massive open online courses (MOOCs), hybrid courses, and class-room technologies pushing us to reconsider how knowledge is shared, how identities develop, and how learners are reached. With this shift, there are both challenges and opportunities for the field of service-learning and civic engagement to consider. At the recent Pathways to Achieving Civic Engagement Conference hosted by North Carolina Campus Compact, a clear theme emerged with regard to technology’s impact on the future of service-learning and civic engagement. Some spoke with trepidation, and others with bold hope for innovations that could make service-learning even more accessible in the online format.

As educators, if we seek to develop the civic knowledge, skills, and values/attitudes of our stu-dents, that practice can take place in many differ-ent forms and formats. Harnessing technology’s power to support that diverse sharing of ideas, connections to modes of communication across boundaries, and scaffolding of critical reflection and growth are all domains in which the experi-ence of service-learning and civic engagement can be deepened. However, not all are hopeful for tech-nology’s role in education or society. Psychologist and MIT Professor Sherry Turkle (2012) has done extensive work on the divisive nature of technol-ogy and its capacity to push people further apart and encourage less “human” interactions. Clearly, there is much to be learned and new horizons to explore if we are to meaningfully and effectively implement technology in the service-learning and civic engagement field.

Editors Jean Strait and Katherine Nordyke are ahead of the curve in this respect, having pub-lished a volume that delves into the many expres-sions, benefits, and potential pitfalls of engaging in the online world. In their work eService-Learn-

ing: Creating Experiential Learning and Civic Engagement Through Online and Hybrid Courses, the editors bring together a diverse collection of author experience to speak to the future of ser-vice-learning in an online capacity. Divided into three main sections, the book provides a scaffold for readers to think through the ways in which experiential learning and civic engagement can take place in online and hybrid courses.

Moving from essentials to models to future directions, the book clearly scaffolds the knowl-edge about e-service-learning for the reader to both learn new frameworks and imagine the possibili-ties for their own practice. In the opening chapter, the editors—joined by Jane Turk—provide a com-prehensive review that sets up many of the issues and implications of service-learning, both digitally and analog. They identify civic knowledge, skills, and values/attitudes as the ultimate goal of the pedagogy that is service-learning, and thus posit the question of how technology helps us to realize those goals. Leora Waldner provides a critical un-derstanding of the components of e-service-learn-ing and, further, what the values of a high-quality experience should encompass. In chapter 3, Nor-dyke explores the ways in which service-learning at large can be realized in a course, and then asks us to think about the appropriate usage when tech-nology is involved. Whether it be direct or indirect service, integrated or standalone, there are many options for what service-learning can look like in the digital realm, and it is incumbent upon the curriculum designer to make intentional choices about that delivery. Finally, Strait rounds out this section of chapters by exploring the role of tech-nology as a support system for service-learning, and specifically e-service-learning models.

In the next set of chapters, models for e-service-learning across disciplines and geographic areas are explored. Each provides a


specific context for e-service-learning, from uni-versity adoption at-large to an online business course to the assessment of experiences in an established service-learning office. One of the chapters that focuses on exemplars is a case study on service-learning after Hurricane Katrina. Strik-ing a parallel between today’s political landscape and the intense focus on the Syrian refugee crisis, one can look at this chapter as especially salient. A spike in attention and awareness can lead to a situation that susceptible to largely technocratic or unsustainable engagement. In the chapter on post-Katrina efforts, the example of the Each One, Teach One program provides evidence of a successful model that integrated hybrid online, in-person experiences for sustainable engagement and community partner relationship develop-ment. This section reminds us that we must think critically and carefully about the way in which service-learning and civic engagement is being implemented in truly meaningful, high-quality ways regardless of the realm—online or in-person.

Moving into the final section of the book, the focus shifts to the future. The penultimate chapter is a call from John Hamerlinck to leverage technol-ogy and community engagement to make higher education more purposeful and impactful. Trends in neoliberalism and internationalization in high-er education—coupled with an increased focus on providing asynchronous experiences—have driven the growth and development of MOOCs and hybrid courses. Hamerlinck cites a number of examples from domains of cost, student success, and debt that point toward the need to rethink-ing the purpose and usefulness of higher educa-tion. Within the space constraints of a chapter, it is impossible to exhaust all of the issues facing the wicked problem of higher education delivery and transformation. However, there is one notable absence: the issue of access. I can envision an opportunity for a critical examination of these issues of access and equity in future works, espe-cially as the opportunities for e-service-learning continues to grow and concern about the digital divide remains very real. That being said, the exciting opportunity that he highlights is the ability re-imagine higher education, leverage technology to create community, and highlight one’s work in service-learning and civic engagement to shift the expectations and models of higher education. In his vision, social media can be a tool to promote civic ideals, the openness of online courses can dis-seminate knowledge across stakeholders, and the creative, collaborative capacity of joint projects can

be ever-expanded across geographies. Communi-ty-engaged teaching with technology, connectiv-ism, and the decentralization of knowledge bases can have implications for the field, while strength-ening values of community and public purpose for all of higher education.

Strait and Nordyke have put together a timely and useful resource for imagining and reconsider-ing the role of technology as a support, facilitator, and space for service-learning and civic engage-ment to take place. As a scholar-practitioner with significant experience in using technology in the classroom, I found this book to be an immense-ly helpful tool for those at any level of technolo-gy adoption. The contributors give us a catalyst to spark creative thinking about the delivery of ser-vice-learning education while retaining a focus on the values and quality of community engagement. Whether you look toward this brave new world of technology to expand your practice or deepen its impact, this volume is an essential resource.

ReferencesTurkle, S. (2012). Alone together: Why we

expect more from technology and less from each other. New York: Basic Books.

AcknowledgmentThe editorial team of the Journal of Commu-

nity Engagement and Scholarship would like to ac-knowledge and thank Stylus Publishing for provid-ing copies of the book for this review.

About the ReviewerSarah Stanlick is director of the Center

for Community Engagement and professor of practice in sociology and anthropology at Lehigh University.


Book Advances Theory and Evidence-Driven Design and Redesign of Community Leadership Development Programs

Review by Jennifer W. PurcellKennesaw State University

K. Pigg, S. Gasteyer, G. Martin, G. Apaliya, and K Keatings, Community Effects of Leadership Development Education: Citizen Empowerment for Civic Engagement. Rural Studies Series, Vol. 3. Morgantown: West Virginia University Press, 2015, 240 pages. ISBN-10:1940425581.

This volume is the product of a five-year collaborative study on the impact of community leadership development programs spanning six states. The book is the third volume of the Rural Studies Series sponsored by the Rural Sociological Society, and the research presented was funded in part by the U.S. Department of Agriculture’s National Institute of Food and Agriculture. The purpose of the book is to engender thoughtful theory and evidence-driven design and redesign of community leadership development programs.

Organization and ContentThe book’s nine chapters are organized into

two parts. In Part I, the authors dedicate six chapters to the presentation of their analysis of data collected on community leadership development. In Chapter One, the authors provide a historical context of community leadership development (CLD). They define community-based programs as those that are (1) focused on community issues and needs; (2) organized and managed locally or by a community group; and (3) sponsored by local resources. Although CLD programs are diverse in their target audience, curriculum, delivery, and specific community-level outcomes, the authors contend the common element across programs is their goal to “increase local leaders’ capacity to improve the community’s well-being” (p. 21). The authors suggest investment in leadership development and capacity-building responds to observed deficits in the community. They contend current and future community leaders in smaller, rural communities have inadequate preparation to address the complexity of social factors impacting their communities. In closing Chapter One, the authors propose two significant reconsiderations for CLD. First, they suggest CLD program sponsors and practitioners examine evaluation

strategies for how program outcomes for individual participants, organizations, and communities are achieved. Identifying how CLD achieves outcomes at various levels is critical to evaluating and improving programs. Second, the authors call for a conceptualization of leadership that moves beyond individual roles and talents to one that recognizes the collaboration and interaction through which leadership and leaders manifest. This conceptual view of leadership as a social process informs the learning intervention that is CLD and is the premise on which the study is built.

Chapter Two details the impact of CLD on individual participants. The key finding related to individual participants informed by data collected in 24 communities with CLD programs, or “treatment sites,” was respondents who participated in CLD programs had significant learning when compared to respondents who lived in counties without programs. CLD participation yields learning that is associated with measureable increased levels of civic engagement and enhanced social capital within communities. Chapter Three expands upon the impact of CLD on social capital by illustrating how involvement in community organizations is broadened and intensified among CLD participants. In Chapter Four, the authors utilize a modified application of Flora, Flora, and Fey’s (2003) community capitals framework to demonstrate the positive impacts of CLD on community development.

Chapter Five addresses participant diversity, curriculum design, and community effects. The authors present data related to the six individual-level outcomes of CLD identified in the study (Personal Growth and Efficacy, Community Commitment, Shared Future and Purpose, Community Knowledge, Civic Engagement, and Social Cohesion). In Chapter Six, the authors press


the core idea of the book: community leadership is about community change (p. 119). Their intent is to reveal strategies for CLD program improvement through empirical evidence. Chapter Six concludes Part I with a series of questions for practitioners and sponsors to consider as they design or redesign CLD offerings.

Part II includes three chapters that address the implications of the research for practitioners and future research. In this section, the authors detail a conceptual framework for community leadership development that integrates existing literature and findings from their research. The authors first emphasize the individual motivations for developing community leadership capacities and the myriad expressions of civic engagement (Chapter Seven). The authors suggest civic engagement leverages individual capacities for leadership and builds upon networked, relational leadership within a community in order to influence positive change (Chapter Eight). Hence, developing individual leadership capacity is elemental to promoting civic engagement; however, such individual-level development is merely a first step in CLD.

As demonstrated through the study, effective CLD interventions include curriculum that addresses individual, organizational, and community-level outcomes. Absence of any one of the three curricular outcomes is a missed opportunity to influence change and impact the community. The authors conclude with a discussion of a model of CLD informed by their research through which participants first attain and/or demonstrate (1) improved human capital self-efficacy and governance skills, (2) increased community commitment, (3) shared future and purpose, and (4) increased political knowledge (political capital). Based on the model, CLD participants then demonstrate increased social cohesion and ability to bridge networks and increased civic engagement. These CLD participant behaviors then result in enhanced civic infrastructure and community capacity that yield community-level awareness of responsiveness to social needs and the inevitable changing conditions experienced in communities.

AudienceThe book is tailored for CLD practitioners,

sponsors, and evaluators through its accessible theoretical literature, study design, and data analysis. The volume is valuable for researchers who will be impressed with the breadth of relevant

citations and integration of multiple disciplinary literature bases. Additionally, the authors provide five sets of study questions throughout the text that would be helpful for structuring independent or group learning exercises for graduate students or as a means of professional development for community leadership development practitioners.

ConclusionCommunity effects of leadership development

education: Citizen empowerment for civic engagement is an essential read for anyone responsible for the design and evaluation of community leadership development programs. Although targeted program evaluations and assessment of participant learning exist, there is a dearth of empirical research on CLD effects on communities and the interplay between individual, organizational, and community-level outcomes. This much anticipated volume heralds a new era of scholarship and practice that engages interdisciplinary literature and empirical research to more critically and precisely explore the benefits of the vast investment in CLD. The authors are frank in their discussion of the research’s limitations; the factors influencing CLD outcomes are numerable and controls are challenging. Despite the acknowledged limitations of the research, this volume is a significant contribution to CLD. The research presented provides an unprecedented synthesis of literature and empirically based insights into the recruitment practices, curriculum design, and evaluation strategies of CLD. Furthermore, the volume is a welcomed summons for scholars and practitioners to build upon the recommendations contained therein and advance the design of community leadership development programs and the ways in which such programs’ outcomes and impacts are measured.

The volume will also appeal to community engagement practitioners and community-engaged scholars who support and participate in university-community partnerships. University-sponsored community engagement activities, including community-engaged scholarship, are championed for their positive impact on communities; however, comprehensive monitoring and measuring the varied impacts of these activities remains a challenge. Pigg and his colleagues challenge us to reconsider the design of programs intended to support leadership and community development. While the emphasis of their research addresses community leadership development specifically, there are implications for the diverse


array of university-community partnerships. For example, the authors address multiple program design elements: primary and secondary outcomes, sponsorship mobilization, collaborative learning, the political and value-laden nature of community-engaged efforts, and diversity among participants among others.

The text may also encourage university leaders to reflect on the ways in which academic, co-curricular, and outreach programs meet their communities’ leadership development education needs. As representatives of anchor institutions that contribute to their communities’ sustainability and advancement, university leaders must carefully consider how and to what extent myriad initiatives individually and collectively impact participants and communities. Through this rigorous investigation, universities may more holistically illustrate their significance and demonstrate appropriate stewardship of the vast public and private investment in higher education.

The authors provide substantial insight into the effects of community leadership development programs; however, even they acknowledge their contribution falls short in providing a general theory of community leadership. Pigg and his colleagues provide a model of the effects of community leadership development programs on individuals and communities that, although limited, presents a more comprehensive conceptual framework for leadership development education. Their model synthesizes multiple disciplinary literatures related to civic leadership capacity and demonstrates the interconnectivity and complexity of community leadership development. Community effects of leadership development education: Citizen empowerment for civic engagement is crucial addition to the literature on community leadership development, civic engagement, and community capacity building that will advance the influence of citizen leaders in their communities. The volume is also a valuable resource for university leaders who advocate for campus-community partnerships and those whose work directly impacts such collaborations that are designed to have positive impacts on communities.

AcknowledgmentThe editorial team of the Journal of

Community Engagement and Scholarship would like to acknowledge and thank the West Virginia University Press for providing copies of the book for this review.

About the ReviewerJennifer W. Purcell is an assistant professor

of Leadership Studies and faculty consultant for community engagement at Kennesaw State University.


INSTRUCTIONS TO AUTHORS

The JCES Editorial Board invites the submission of manuscripts that relate to its mission: to provide a mechanism through which faculty, staff, and students of academic institutions and their community partners may disseminate scholarly works from all academic disciplines, with the goal of integrating teaching, research, and com-munity engagement. All forms of writing and analysis will be acceptable with consideration being given to research and creative approaches that apply a variety of methodologies. Manuscripts that demonstrate cen-tral involvement of students and community partners and advance community-engaged scholarship will be given favorable consideration. Manuscripts should be free of all forms of bias, including racial, religious, gender, or ethnic.

Submission of a manuscript implies com-mitment to publish in this journal if accepted for publication. Manuscripts must have been submitted for exclusive publication in JCES and not simulta-neously submitted elsewhere. Manuscripts should not have been published elsewhere in substantially the same form. Authors in doubt about what con-stitutes prior publication should consult the editor. Paper submissions are not accepted. Inquiries and submissions should be emailed to [email protected].

Nick Sanyal, Ph.D.Professor of Natural Resources and Society

University of Idaho

Manuscripts should be submitted in Microsoft Word with a separate cover page. They should be double-spaced in Times New Roman 12-point type. Article length, except in rare instances, should not exceed 25 pages, including text, tables, and referenc-es. Tables and other graphics should be submitted as separate documents with their place in the man-uscript indicated. Do not include the abstract and cover pages in the page count.

Manuscripts should follow the Publication Manual of the American Psychological Association (sixth edition). See brief guidelines at http://www.apa.org/pubs/authors/instructions.aspx.

A separate cover sheet with the name(s), affiliation(s), and other identifying information and contact information (address, phone numbers, and email addresses) for each contributing author should be supplied. Additionally, authors should include four to six keywords at the bottom of the cover sheet.

All identifying information or references to the author(s) must be removed from the manuscript.

Manuscripts that include identifying informa-tion will not be reviewed prior to correction. Each

manuscript must also include an abstract of 150 words or less that summarizes the major themes of the manuscript.

Manuscripts not meeting these criteria will be returned to the author before being sent out for review.

Authors are required to submit written per-mission from the original publisher for any quoted material of 300 words or more from a single source; any quoted material from a newspaper, a poem, or a song (even a phrase); and any table, figure, or image reproduced from another work.

Images must also be submitted electronically, in JPEG format, with no less than 200 pixels per inch resolution for black and white images and 300 for color. Manuscripts will undergo masked peer review. It is the intention of this journal to assign manuscripts to reviewers within two to four weeks of compliant and correct submission. Authors will be notified of a decision in a timely manner consis-tent with thorough scholarly review.

If you wish the editor to consider using video related to your research, first upload it to YouTube, then send the relevant link or links to [email protected].

MANUSCRIPT PREPARATION


Additional Submission DetailsThe editorial board of the Journal of Commu-

nity Engagement and Scholarship (JCES) invites the submission of manuscripts that relate to its mission: To provide a mechanism through which faculty, staff, and students of academic institutions and their community partners may disseminate scholarly works from all academic disciplines. A goal of the publication is to integrate teaching, research, and community engagement. All forms of writing, analysis, creative approaches, and methodologies are acceptable for the journal.

Regular and Research From the Field man-uscripts that demonstrate central involvement of students and community partners and that advance community engagement scholarship will be given favorable consideration. Manuscripts should be free of racial, religious, gender, ethnic, or any other form of bias. Manuscripts submitted are for the exclusive publication in JCES and must not have been simultaneously submitted elsewhere. Sub-mission of a manuscript implies commitment to publish in this journal. Authors submitting man-uscripts to the journal should not simultaneously submit them to another journal. Nor should man-uscripts have been published elsewhere in substan-tially the same form. Authors in doubt about what constitutes prior publication should consult the editor.

All manuscripts (regular, Research From the Field, Book Reviews, Student Voices, and Com-munity Perspectives) are accepted on an ongoing basis. All submissions and inquiries should be sent to [email protected].

Regular ManuscriptsManuscripts should be typed double-spaced

in 12-point Times New Roman in Microsoft Word with a separate cover page. Article length, except in rare instances, should not exceed 25 pages including text, tables, and references. Tables should be on separate pages at the end of the manuscript with instructions to indicate their placement. Submit graphics and images as separate JPEG files, being sure to indicate, in the manuscript, where they should be placed. The abstract and cover pages are not included in the page count. The Publication Manual of the American Psychological Associa-tion (6th edition) is the official style and reference guide. For online help with APA style, see https://owl.english.purdue.edu/owl/resource/560/08/.

The separate cover sheet must contain the name(s), affiliation(s), and other identifying and contact information (address, phone numbers, and email addresses) for each contributing author. Additionally, authors should include four to six keywords at the bottom of the cover sheet. All iden-tifying information or references to the author(s) must be removed from the submitted manuscript. Manuscripts that include identifying information will not be reviewed prior to correction. Each manuscript must also include an abstract of no more than 200 words that summarizes the major themes of the research. Manuscripts not meeting these criteria will be returned to the author before being sent out for review.

Images should be JPEGs, 300 resolution, no larger than 6 by 9 inches. Most will be smaller. Please send your images as RGB color so that we will have the option of color. All photos must have captions as a separate document or imbedded in the Photoshop document under File Info.

Please include two versions of the manuscript, one masked and one with authors and places identified. The masked version should use brackets to indicate masking of particular identifying information. For example, do not refer to the University of Alabama as the location of your engagement project. Instead, refer to the [univer-sity]. This will help to protect your anonymity and preserve the integrity of the peer-review process. Our reviewers are aware that brackets indicate masked information.

Research From the FieldThis section of the journal is reserved for

reports that are likely to have a practice or case study orientation. Manuscripts appropriate for this section provide a venue for certain kinds of research projects involving partnerships between academic institutions, communities, and students. Manuscripts that share best practices, practice wisdom, and applied knowledge are especially appropriate for Research From the Field. Context is an essential part of all engaged scholarly activ-ities, and the reporting of them needs to be situ-ated philosophically, historically, and theoretically if they are to systematically extend our knowledge and understanding. All submissions need to be described and examined through these lenses or some combination of them. Unique partnerships have the potential to make highly interesting pieces


for Research From the Field. Lessons or practice principles learned should be a part of the manu-script. Research methodologies of all types are accepted, and research projects with strong ap-plication and practice implications will be given favorable consideration. Research From the Field manuscripts also must follow APA style.

Student VoicesStudents from all disciplines are invited to

submit original work to the Student Voices section. All forms of writing, analysis, creative approaches, and methodologies are acceptable for this section. Specific types of submissions appropriate for the Stu dent Voices section include commentaries, critical reflections, and opinion pieces related to community engagement and/or engaged scholar-ship. Given that engagement scholarship is such an inter disciplinary field in which there are many appropriate ways to best “tell the sto ry,” scholarly contributions of many kinds related to the field of engagement scholarship are welcome and will be con sidered for publication. Manuscripts should be between 750–2,000 words. You can also see ex-amples of published submissions on our website—jces.ua.edu.

Community PerspectivesCommunity members working with academic

partners from all disciplines are invited to sub-mit original work to the Communi ty Perspectives section. All forms of writing, analysis, creative ap proaches, and methodologies are acceptable for this section. Spe cific types of submissions appro-priate for Community Perspectives include com-mentaries, critical reflections, and opinion pieces re lated to community engagement and/or engaged scholarship. Giv en the interdisciplinary nature of the field, scholarly contributions of many kinds related to the field of engagement scholarship are encouraged and will be considered for publication in JCES. Community Perspectives submissions are accepted on an ongoing basis. Manu scripts should be between 750–2,000 words in length. Examples may be found on the JCES website — jces.ua.edu.

Book ReviewsBook reviews submitted to JCES should give the reader a well-developed sense/description of the

book, but should also go beyond description to discuss central issues raised by the text. Reviewers are encouraged to address how the reviewed book addresses theory, current scholarship, and/or current issues germane to the subject of the book and engagement scholarship. Reviewers may reference other material that has bearing on the book being reviewed, particularly when these sources have the ability to position the book within larger discourses regarding the topic. Like other submissions to JCES, book review manuscripts should be typed double-spaced in 12-point and follow APA reference style.

First Review• Editorial staff sends manuscript to reviewers, with review form, return due date• Editor reassigns manuscript if reviewer unable to complete review• Editorial staff sends reminder email one week after due date if review not yet received • Editor receives reviewers’ evaluation and rating forms

Editor Options• Editor accepts manuscript (proceed to Edit for Publication) • Editor sends corresponding author the recommended revisions

and requests resubmission • Editor rejects manuscript (end of process)

Resubmitted Manuscript and Second Review• Editor scans for compliance with reviewer evaluation; if necessary, author

is sent request for further revision

• Editorial staff sends to the original reviewers the revised manuscript, a copy of the original manuscript with editor’s and reviewers comments and tracked changes, the review form, along with a copy of the letter to the corresponding author

• Editorial staff requests return of second review within 2 weeks

Editor Options Following Second Review• Editor accepts manuscript (proceed to Edit for Publication)• Editor accepts manuscript with minor revisions

Publication*Authors’ names on all documents viewed by reviewers, including manuscripts, letters, and emails, and other identifying information, are masked throughout the process.

Manuscript Receipt*• Editor scans for style and documentation standards; requests revisions if necessary

• Editorial staff sends acknowledgment email to corresponding author (usually, but not always, the first author)

• Editor selects appropriate reviewers

• Editorial staff assigns manuscript number

JCESREVIEW PROCESS

Revise and Resubmit Instructions

• Editorial staff sends to corresponding author a letter regarding the decision, reviewer comments, manuscript with edits and tracked changes

• Editorial staff requests resubmission within four weeks

• Editorial staff notifies author of publication decision

Accept with Minor Revisions

• Editor ensures minor revisions have been made (proceed to Edit for Publication)

• Editor sends corresponding author notification of decision to accept with minor revisions and requests a final revision within two weeks

Edit for Publication• Final editing and proof reading by editor and editorial staff• Editorial staff sends proof to corresponding author• Editorial staff negotiates editorial changes with corresponding author• Editorial staff sends official copyright forms for corresponding author’s signature

JOURNAL of COMMUNITY ENGAGEMENT and SCHOLARSHIPwww.jces.ua.edu | www.uapress.ua.edu

Published by the Office of Community Affairs at The University of AlabamaDistributed by The University of Alabama Press

ISSN 1944-1207Published twice a year.

Also in e-edition. ISBN: 978-0-8173-8514-9

JCES is a peer-reviewed international journal through which faculty, staff, students, and community partners disseminate scholarly works. JCES integrates teaching, research, and community engagement

in all disciplines, addressing critical problems identified through a community-participatory service.

Subscription rate per issue:$50 institutional$30 individual$10 student

For subscription information, contact:

Allie L. HarperBox 870380

Tuscaloosa, AL 35487-0380Phone: 205.348.1564

[email protected]

For more information about licensing JCES: Claire Lewis Evans

Box 870380Tuscaloosa, AL 35487-0380

Phone: [email protected]

The University of AlabamaOffice of Community AffairsCenter for Community-Based PartnershipsBox 870372Tuscaloosa, AL 35487-0372

The Journal of Community Engagement and Scholarship is published at The University of Alabama by the Division of Community Affairs for the advancement of engaged scholarship worldwide.

PARTNERSHIP

RESEARCHTEACHING

FACULTY STUDENTS COMMUNITY

Nonprofit OrganizationUS Postage Paid

Tuscaloosa, ALPermit 16