alumni perceptions of institutional …€¦ · practical skills 3. personal and social...
TRANSCRIPT
A Clark University College-Wide
Assessment Report
Vol. 2, No. 2, Spring 2015
1
2
3
45
1. Knowledge of the natural world and human cultures and societies
2. Intellectual and practical skills
3. 3. Personal and socialPersonal and socialresponsibilityresponsibility
4. Ability to integrate4. Ability to integrateknowledge and skillsknowledge and skills
C L A R K ’ S D E F I N I N G C L A R K ’ S D E F I N I N G C O N T R I B U T I O NC O N T R I B U T I O N 5. Capacities of effective practice 5. Capacities of effective practiceincluding creativity and imagination, including creativity and imagination, self-directedness, resilience self-directedness, resilience and persistence, and the ability and persistence, and the ability to collaborate across di� erencesto collaborate across di� erencesand manage complexityand manage complexity
A L U M N I P E R C E P T I O N S O F I N S T I T U T I O N A L S U P P O R T F O R T H E L E E P L E A R N I N G O U T C O M E S :A Comparative Analysis Using the Higher Education Data Sharing (HEDS) 2011-2012 & 2013-2014 Survey Data
������
N�
S
WWW
Michelle BataLEEP Center
Katerine BielaczycHiatt Center
Nancy BudwigAcademic A� airs
Greg DoerschlerInstitutional Research
Li HanMath & Computer Science
Je� HimmelbergerInstitutional Research
Matt MalskyVisual & Performing Arts
Robert RiddlesCorporate & Foundation Relations
Marianne SarkisInternational Development, Community, and Environment (IDCE); Center for Excellence in Teaching and Learning (CETL)
Cheryl Turner ElwellITS/Academic Technology
The LEEP Compass Advisory Committee provides feedback to and assists the academic administration with developing Clark’s research and evaluation of the LEEP Learning Outcomes.
Published by:
Clark University 950 Main Street
Worcester, MA 01610
This report has been provided by Clark’s Compass Advisory Committee. For additional reports and information about LEEP assessment, please visit clarku.edu/leepcompass.
L E E P C O M PA S S A D V I S O R Y C O M M I T T E E
TABLE OF CONTENTS�
N
�
S
E
W
�
�
Clark LEEP Compass Report
VOL. 2 , NO. 2 | SPRING 2015
Nancy Budwig
Co-Editor
Marianne Sarkis
Analyst and Co-Editor
Je� Himmelberger
HEDS Data Management
Greg Doerschler
HEDS Data Management
Melissa Lynch
Copy Editor
SECTION�I: ABOUT THE REPORT
1 Liberal Education at Clark University
2 About the Data in this Report
2 The Organization of this Report
SECTION II: A SNAPSHOT
3 Clark Alumni Compared to HEDS Peers (5-year-out and 10-year-out Cohorts)
SECTION III: SUMMARY AND TRENDS
5 LEEP Outcome #2: Intellectual and Practical Skills
7 LEEP Outcome #3: Personal and Social Responsibility
9 LEEP Outcome #4: Integrative and Applied Learning
11 LEEP Outcome #5: Capacities of E� ective Practice
APPENDICES
13 Appendix A: 2013-2014 HEDS Alumni Survey Participating Institutions
14 Appendix B: 2013-2014 Clark and HEDS Respondent Characteristics)
15 Appendix C: 2013-2014 HEDS Alumni Survey
17 Appendix D: Specifi c HEDS Questions Used to Access the LEEP Learning Outcomes
18 Appendix E: Quantitative Analyses for Longitudinal Comparisons of Clark Alumni and HEDS Cohorts
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 1
1. Liberal Education and America’s Promise: see College Learning for the New Global Century (AAC&U, 2007). The LEAP Outcomes are summarized at http://www.aacu.org/leap/vision.cfm.
THE CLARK UNIVERSITY TASK FORCE ON UNDERGRADUATE EDUCATION’S FINAL REPORT, “Liberal Education and E� ective Practice: A Plan for the 21st Century,” identifi ed the use of survey data as an important piece of Clark’s assessment approach. This report focuses on a baseline analysis of what our alumni have shared about institutional support for what we now call the LEEP Learning Outcomes at a time before LEEP was implemented. The Association of American Colleges and Universities’ (AAC&U) LEAP Initiative developed a set of essential “Learning Outcomes” that provided a starting point for the outcomes used by Clark1, adopted by a vote of the Clark faculty in 2009.
LIBERAL EDUCATION AT CLARK UNIVERSITY
SECTION I : ABOUT THE REPORT
2
1
3
4
5
Knowledge of Human Cultures and the Natural World�—�including the ability to employ di� erent ways of knowing
Practiced extensively across the curriculum, in the context of progressively more challenging problems, projects, and standards for performance
LEARNING OUTCOME 1:
Intellectual and Practical Skills —�including inquiry and analysis, the generation and evaluation of evidence and argument, critical and creative thinking, written and oral communication, quantitative literacy, information literacy, teamwork and problem-solving
Focused by rigorous engagement with big questions, both contemporary and enduring
LEARNING OUTCOME 2:
Personal and Social Responsibility —�including ethical reasoning and action, the intercultural understanding and competence to participate in a global society, civic knowledge and engagement locally as well as globally, and the lifelong habits of critical self-refl ection and learning
Anchored through active involvement with diverse communities and real-world challenges, taking particular advantage of Clark’s urban location and global connections
LEARNING OUTCOME 3:
Ability to Integrate Knowledge and Skills —�including synthesis and advanced accomplishment across general and specialized studies, bridging disciplinary and interdisciplinary thinking, and connecting the classroom and the world
Experienced through progressively more advanced knowledge creation, contextual reasoning, and the construction of shared meaning and opportunities for refl ection
LEARNING OUTCOME 4:
Capacities of Effective Practice�—�including creativity and imagination, self-directedness, resilience and persistence, and the abilities to collaborate with others across di� erences and to manage complexity and uncertainty
Demonstrated by application of knowledge and skills to issues of consequence and by emerging membership in larger communities of scholarship or practice
LEARNING OUTCOME 5:
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 2
This Clark University college-wide assessment report uses Higher Education Data Sharing (HEDS) 2013-2014 alumni survey data to assess alumni’s perceptions of institutional support for the LEEP Learning Outcomes. This national survey has been administered every other year
since 20112.
This report includes a comparative analysis of Clark alumni and HEDS peer cohort at two points in time (5-year-out and 10-year-out), as well as a longitudinal analysis of Clark’s 5-year-out cohort at two points in time (2011-2012 and 2013-2014). Prior to 2013, HEDS provided data from four cohorts (less than 5-year-out, 5-year-out, 10-year-out, and all other alumni). With the 2013-2014 Survey, HEDS limited its sample to alumni who were 5-year-out and 10-year-out (only).
For this report, we had su� cient responses from the 10-year-out category to include them in the analysis, which was not the case in the previous report in this series (HEDS Alumni Survey, 2015 volume 2 issue 1).
The 2011-2012 and 2013-2014 alumni survey reports include data from cohorts of Clark alums who attended Clark before the adoption of the LEEP Learning Outcomes voted on by the faculty in 2009. Therefore, the primary purpose of these two reports is to provide a way for us to obtain a baseline assessment of alumni perceptions before and after LEEP has been implemented.
For the 2012-2013 alumni survey, Clark attempted to reach 1,128 alums who graduated from Clark between ’03 thru ’09 with known email addresses. Of the 1,128 alumni contacted, 220 responded (20% response rate) who were at 5-year-out (N=137) or 10-year-out (N=82). One survey response was discarded due to errors in the data.
For a detailed look at participating institutions, respondent characteristics and number of respondents, please refer to Appendix A and Appendix B.
SECTION 0NE provides a summary of the Clark LEEP Learning Outcomes, the data used, and some of the limitations of the analysis.
SECTION TWO provides a snapshot of a more detailed comparative analysis that examines Clark alumni responses from the
5-year-out cohort on 14 questions pertaining to institutional support for the LEEP Learning Outcomes as compared to responses from the 5-year-out cohort of the HEDS national sample.
SECTION THREE presents an in-depth analysis of the HEDS questions aligned with
LEEP Learning Outcomes 2-5. Three kinds of analyses are presented: Clark compared to HEDS at two time points (5-year-out and 10-year-out); Clark comparisons for fi ve year vs. 10-year-out; and Clark fi ve-year-out perceptions over longitudinal time.
THE ORGANIZATION OF THIS REPORT
ABOUT THE DATA IN THIS REPORT
SECTION I : ABOUT THE REPORT
H E D S Q U I C K FA C T S
Participating Institutions: In 2013-2014, 33 colleges and universities participated in the HEDS Alumni Survey.A full listing of those schools can be found in Appendix A.
Student Participation: In 2013-2014, 10,237 alumni completed a HEDS Alumni Survey from HEDS participating institutions.
220 alumni from Clark participated in the survey, including 137 alumni at 5-year-out and 82 at 10-year-out.
Additional alumni respondent characteristics for the 5-year and 10-year-cohorts can be found in Appendix B.
Sample Questions:A copy of the complete HEDS instrument can be found in Appendix C. The 14 HEDS questions used in this report can be found in Appendix D.
2. Since none of the HEDS Alumni Survey questions focused on Learning Outcome 1, this Outcome will not be discussed in this report.
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 3
CLARK ALUMNI COMPARED TO HEDS PEERS (5-YEAR-OUT AND 10-YEAR-OUT COHORTS)
SECTION I I : A SNAPSHOT
The snapshot in Table 1 compares frequencies of Clark alumni with their HEDS peers at 5-year-out and 10-year-out who answered “Quite a bit” or “Very much” (the top two highest response categories on the survey) regarding their perceptions of institutional support on 14 survey questions that link to LEEP Learning Outcomes 2-5. These questions in the HEDS alumni survey were included based on recommendations adopted in a prior LEEP Assessment Report (see NSSE issue 1, volume 1, 2013) and recommendations from the AAC&U Making Progress Report3.
Table 1 compares frequencies between Clark alumni against the national sample, and assesses whether and how Clark di� ers from alumni peers. Two questions in Table 1 showed that average scores of Clark alumni at fi ve years who responded “Quite a bit” or “Very much” were higher than HEDS peers. These di� erences both tied to Learning Outcome 3: Personal and Social Responsibility. No other di� erences between the Clark cohort and HEDS peers were noted.
It should be noted that this representation does not indicate whether Clark students give high marks for institutional support or rate Clark lower than the peers. That will be discussed in Section III. The snapshot reported in Section II is neutral with regard to quality of institutional support. For instance, alumni from Clark and the peer schools might all rate their institutions similarly high or low and thus are said to be similar. Section III describes in more detail what Clark alums thought of institutional support for skill sets and capacities related to the LEEP Learning Outcomes. For additional information about frequencies, average scores, and other statistical measures, refer to Appendix E.
Two further caveats should be noted. First, Clark alumni from this sample (5-year-out and 10-year-out) were at Clark before the LEEP Learning Outcomes were adopted and before the LEEP curricular framework was implemented. Second, there could be many reasons why populations di� er in assessment of institutional support and some may be unrelated to the quality of education students receive.
3. The HEDS Alumni Survey did not include any questions targeting Learning Outcome 1. Excerpts from the full survey can be found in Appendix C.
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 4
CLARK ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT COMPARED TO HEDS PEERS (5-YEAR-OUT AND 10-YEAR-OUT COHORTS)
SECTION I I : A SNAPSHOT
TA B L E 1 : CLARK ALUMNI (5-year-out AND 10-year-out COHORTS) PERCEPTIONS OF INSTITUTIONAL SUPPORT COMPARED TO HEDS PEERS
C L A R K
5 -Y E A R - O U T
C L A R K
1 0 -Y E A R - O U T
L E E P O U T C O M E 2 : Intellectual and Practical Skills
• Writing clearly and e� ectively
• Speaking clearly and e� ectively
• � inking critically and analytically
• Analyzing quantitative problems
• Using computing and information technology
L E E P O U T C O M E 3 : Personal and Social Responsibility• Voting in local, state, or national elections
• Understanding people of other racial and ethnic backgrounds
• Developing a personal code of values and ethics
L E E P O U T C O M E 4 : Integrative and Applied Learning
• Worked on a paper or project that required integrating ideas or information from various sources
• Put together ideas or concepts from di� erent courses when completing assignments or during class discussions
• Used course content to address a problem not presented in the course
L E E P O U T C O M E 5 : Capacities of Effective Practice
• Problem-solving
• Working e� ectively with others
• Creative thinking
K E Y:Frequencies of Clark alumni who answered “Very Much” or “Quite a bit” (the top two highest response categories) on their perceptions of institutional support compared to HEDS cohorts:
Clark scores were higher than HEDS scores by ≥ 10%
Clark scores were higher than HEDS scores by ≥ 5%
No di� erences in frequencies between Clark alumni compared to HEDS cohorts
Clark scores were lower than HEDS scores by ≤ 5%
Clark scores were lower than HEDS scores by ≤ 10%
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 5
1 2 3 4 5LEEP OUTCOME #2
“Intellectual and Practical Skills”
S U M M A R Y Five of the 14 HEDS survey questions that are the focus of this report aligned with Learning Outcome 2: Intellectual and Practical Skills. These included questions pertaining to the alums’ perceptions of the support that Clark provided regarding “e� ective writing,” “e� ective speaking,” “critical thinking,” “quantitative literacy,” and “information literacy.”
Figure 1 presents fi ndings on perceived institutional support by alumni who are 5-year-out and 10-year-out at Clark and HEDS peer institutions. For Learning Outcome 2, all of the responses showed the majority of alumni tending to report high levels of support. At 10-year-out the ratings of Clark alums and their HEDS peers was similar, with e� ective writing, critical thinking, and information literacy all getting especially high scores. At 5-year-out, the scores for institutional support were lower for Clark alums than HEDS peers by at least 5% for “critical thinking” and “information literacy,” and by at least 10% for “e� ective speaking” and “quantitative literacy.” Further information on these fi ndings is discussed in the next set of analyses.
“Intellectual and practical skills”
CLARK 5-year-out � HEDS 5-year-out
SECTION I I I : SUMMARY AND TRENDS
Figure 1: Percent of Clark alumni and HEDS cohorts who answered “Quite a bit” or “Very much” (the top two highest response categories) on their perceptions of institutional support on questions aligned with LEEP Outcome 2.
85%
60%
87%81%
63%
89%
72%
92%87%
75%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Effective writing Effective speaking Critical thinking Information literacy Quantitative literacy
CLARK 10-year-out � HEDS 10-year-out
88%
70%
93%88%
70%
88%
66%
90%84%
72%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Effective writing Effective speaking Critical thinking Information literacy Quantitative literacy
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 6
1 2 3 4 5LEEP OUTCOME #2
“Intellectual and Practical Skills”
SECTION I I I : SUMMARY AND TRENDS
Figure 2 compares Clark alums at two data points (5-year-out and 10-year-out) in 2013-2014 sample. Here we see more institutional support reported by the 10-year-out class.
85%
60%
87%
81%
63%
88%
70%
93%
88%
70%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%
Effective writing
Effective speaking
Critical thinking
Information literacy
Quantitative literacy
Clark 5-year-out
Clark 10-year-out
Figure 2: Percent of Clark alumni who answered “Quite a bit” or “Very much” on their perceptions of institutional support on questions aligned with LEEP Outcome 2.
Figure 3 helps tease apart time away from college and cohort, comparing 5-year-out alums from Clark and their HEDS peers at two points in time (2011-2102 and 2013-2014). Here we fi nd that Clark alums’ perceptions at 5-year-out tend to trail their HEDS peers and that the gap between the two increases by 2013-2014. Why this is the case is worthy of further examination.
CLARK 5-year-out � HEDS 5-year-out
60%
70%
80%
90%
100%
2011-2012 2013-2014
Effective writing
60%
70%
80%
90%
100%
2011-2012 2013-2014
Effective speaking
60%
70%
80%
90%
100%
2011-2012 2013-2014
Critical thinking
60%
70%
80%
90%
100%
2011-2012 2013-2014
Information literacy
60%
70%
80%
90%
100%
2011-2012 2013-2014
Quantitative literacy
Figure 3: Longitudinal comparison of Clark alumni and HEDS cohorts who answered “Quite a bit” or “Very much” on their perceptions of institutional support on questions aligned with LEEP Outcome 2.
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 7
1 2 3 4 5LEEP OUTCOME #3
“Personal and Social Responsibility”
S U M M A R Y Three questions on the HEDS survey tapped into LEEP Learning Outcome 3 including questions about “civic engagement,” “intercultural knowledge and competence,” and “ethical reasoning.” Clark alumni at 5-year-out again noted strong institutional support for these areas, with the majority of alums reporting that they believed the institution o� ered “quite a bit” or “very much” support. Both 5-year-out and 10-year-out Clark alumni distinguished themselves from their alumni peers in reporting that as a cohort they felt more institutional support for intercultural knowledge during their time at Clark. This echoes fi ndings from our analyses of seniors at Clark on similar questions found in the National Survey of Student Engagement (NSSE), in that Clark seniors also noted a perception of greater institutional support when compared with their national peers on these dimensions. Civic engagement and ethical reasoning were perceived to receive institutional support by both the Clark alums and HEDS peers and showed no di� erences.
“Intellectual and practical skills”
CLARK 5-year-out � HEDS 5-year-out
SECTION I I I : SUMMARY AND TRENDS
Figure 4: Percent of Clark alumni and HEDS cohorts who answered “Quite a bit” or “Very much” (the top two highest response categories) on their perceptions of institutional support on questions aligned with LEEP Outcome 3.
60%
77% 76%
55%
68%74%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Civic engagement Intercultural knowledge and competence Ethical reasoning
CLARK 5-year-out � HEDS 5-year-out
57%
77%73%
53%
62%71%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Civic engagement Intercultural knowledge and competence Ethical reasoning
CLARK 10-year-out � HEDS 10-year-out
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 8
1 4 5LEEP OUTCOME #3
“Personal and Social Responsibility”
SECTION I I I : SUMMARY AND TRENDS
2 3
60%
77%
76%
57%
77%
73%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
Civic engagement
Intercultural knowledge and competence
Ethical reasoning
Clark 5-year-out
Clark 10-year-out
Figure 5: Percent of Clark alumni who answered “Quite a bit” or “Very much” on their perceptions of institutional support on questions aligned with LEEP Outcome 3.
Figure 5 compares Clark alums at two time points: 5-year-out and 10-year-out. On questions pertaining to learning
outcome 3 there was no notable di� erence between these two groups.
50%
60%
70%
80%
90%
100%
2011-2012 2013-2014
Civic engagement
50%60%70%80%90%
100%
2011-2012 2013-2014
Intercultural knowledge and competence
50%
60%
70%
80%
90%
100%
2011-2012 2013-2014
Ethical reasoning
CLARK 5-year-out � HEDS 5-year-out
Figure 6: Longitudinal comparison of Clark alumni and HEDS cohorts who answered “Quite a bit” or “Very much” on their perceptions of institutional support on questions aligned with LEEP Outcome 3.
Figure 6 provides longitudinal perspective on Learning Outcome 3 showing that Clark alumni found more institutional support than their HEDS peers for this outcome, while both Clark alums and HEDS alums indicated less support for Learning Outcome 3 questions between 2011-2012 and 2013-2014 cohorts.
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 9
1 2 3 4 5LEEP OUTCOME #4
“Integrative and Applied Learning”
S U M M A R Y Questions pertaining to LEEP Learning Outcome 4 focus on integrative thinking, asking whether alums felt institutional support for connecting concepts across courses, and whether alums were encouraged to apply content to novel questions as students. We can see that the Clark alumni reported strong support for integrative thinking and connecting concepts across courses, but less support for applying content to novel problems. In addition, there were no conspicuous di� erences noted between Clark’s 5-year-out cohort of alums and their HEDS alumni peers about perceived institutional support for any of these three questions. Both Clark alums and HEDS alumni peers indicated that the institution supported integrative thinking and connecting concepts across courses. It should be noted that only about a third of alums 5-year-out (regardless of their institution) felt the institution supported their learning by having students apply concepts to a novel problem. Interestingly, while Clark’s 10-year-out alums indicated notably less support for applying novel concepts, more recent alums in the 5-year-out cohort felt institutional support at a level nearly identical to their national peers.
“Intellectual and practical skills”
CLARK 5-year-out � HEDS 5-year-out
SECTION I I I : SUMMARY AND TRENDS
Figure 7: Percent of Clark alumni and HEDS cohorts who answered “Quite a bit” or “Very much” (the top two highest response categories) on their perceptions of institutional support on questions aligned with LEEP Outcome 4.
78%71%
42%
83%
74%
43%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Integrative thinking Connected concepts across courses Applied content
CLARK 5-year-out � HEDS 5-year-out
75% 74%
35%
79%
69%
41%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Integrative thinking Connected concepts across courses Applied content
CLARK 10-year-out � HEDS 10-year-out
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 10
1 4 5LEEP OUTCOME #4
“Integrative and Applied Learning”
SECTION I I I : SUMMARY AND TRENDS
78%
71%
42%
75%
74%
35%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
Integrative thinking
Connected concepts across courses
Applied content
Clark 5-year-out
Clark 10-year-out
2 3
Figure 8: Percent of Clark alumni who answered “Quite a bit” or “Very much” on their perceptions of institutional support on questions aligned with LEEP Outcome 4.
Figure 8 provides a comparison of 5- and 10-year-out Clark cohorts, indicating almost no di� erences between the two samples. Clark 10-year-out alums were less likely to indicate strong support for applied content than their 5-year-out Clark alumni peers.
60%
70%
80%
90%
100%
2011-2012 2013-2014
Integrative thinking
60%
70%
80%
90%
100%
2011-2012 2013-2014
Connected concepts across courses
30%40%50%60%70%80%90%
100%
2011-2012 2013-2014
Applied content
CLARK 5-year-out � HEDS 5-year-out
Figure 9: Longitudinal comparison of Clark alumni and HEDS cohorts who answered “Quite a bit” or “Very much” on their perceptions of institutional support on questions aligned with LEEP Outcome 4.
Longitudinal comparisons of Clark and HEDS peer alums’ perceptions of institutional support fi ve years after graduation for Learning Outcome 4 at two time points can be found in Figure 9. For the most part no di� erences are found between the Clark and HEDS peer samples with everyone reporting more institutional support in 2013-2014. The one exception to this is with regard to integrative thinking, where Clark students looked similar to the HEDS peers in 2011-2012 but then dropped below them in 2013-2014.
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 11
1 2 3 4 5LEEP OUTCOME #5
“Capacities of Effective Practice”
S U M M A R Y Clark alumni at 5-year-out noted a strong sense of institutional support for their development of Capacities of E� ective Practice. Figure 10 shows that Clark alums (5- and 10-year-out) both reported less institutional support than that reported by their peers for problem-solving, while for the other two areas (creative thinking and teamwork), at both points in time Clark alums and HEDS peers look roughly similar.
“Intellectual and practical skills”
CLARK 5-year-out � HEDS 5-year-out
SECTION I I I : SUMMARY AND TRENDS
Figure 10: Percent of Clark alumni and HEDS cohorts who answered “Quite a bit” or “Very much” (the top two highest response categories) on their perceptions of institutional support on questions aligned with LEEP Outcome 5.
CLARK 5-year-out � HEDS 5-year-out
66%
82%
65%
76% 79%
67%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Problem-solving Creative thinking Teamwork
CLARK 10-year-out � HEDS 10-year-out
71% 74%69%
78% 80%
70%
0%
10%
20%
30%
40%
50%
60%
70%
80%
90%
100%
Problem-solving Creative thinking Teamwork
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 12
1 4 5LEEP OUTCOME #5
“Capacities of Effective Practice”
SECTION I I I : SUMMARY AND TRENDS
Figure 11 compares Clark alums at two data points: 5- and 10-year-out. Here we fi nd that for two items (problem-solving and teamwork), more recent alums give higher institutional ratings for support, while for creative thinking the reverse is true.
71%
74%
69%
66%
82%
65%
0% 10% 20% 30% 40% 50% 60% 70% 80% 90%
Problem-solving
Creative thinking
Teamwork
Clark 5-year-out
Clark 10-year-out
2 3
Figure 11: Percent of Clark alumni who answered “Quite a bit” or “Very much” on their perceptions of institutional support on questions aligned with LEEP Outcome 5.
Turning to Figure 12 we fi nd data on how two cohorts of 5-year-out alums compare with one another and with their HEDS peers. For the most part Clark 5-year-out alums trail their HEDS peers in reporting support for Learning Outcome 5 items, though Clark alums in 2013-2014 reported that teamwork received more support than those reporting in the 2011-2012 sample.
Keeping in mind these are baseline data (the alums had not been at Clark after LEEP was launched), it will be important to see how Clark compares to HEDS peers in the coming years.
60%
70%
80%
90%
100%
2011-2012 2013-2014
Problem-solving
60%
70%
80%
90%
100%
2011-2012 2013-2014
Creative thinking
60%
70%
80%
90%
100%
2011-2012 2013-2014
Teamwork
Figure 12: Longitudinal comparison of Clark alumni and HEDS cohorts who answered “Quite a bit” or “Very much” on their perceptions of institutional support on questions aligned with LEEP Outcome 5.
CLARK 5-year-out � HEDS 5-year-out
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 13
APPENDIX A
2013-2014 HEDS Alumni Survey Participating Institutions
# of Valid Respondents -
1-year-out Cohort
# of Valid Respondents -
5-year-out Cohort
# of Valid Respondents -
10-year-out Cohort
# of Valid Respondents -
More-than-10-year-out Cohort
Response Rate
Clark University 0 137 82 0 20.4%
DePauw University 143 177 59 2 18.2%
Earlham College 2 131 2 0 29.3%
Furman University 1 123 82 59 20.3%
Goshen College 40 79 50 136 36.9%
Haverford College 79 76 65 400 25.3%
Holy Names University 19 9 6 1 10.0%
Kenyon College 0 78 93 0 32.6%
Knox College 1 76 53 0 28.2%
Lee University 131 38 0 0 12.1%
Luther College 0 154 122 0 29.5%
McDaniel College 45 54 26 2 11.5%
Mills College 55 38 38 1 24.4%
Occidental College 228 208 168 96 *
Ouachita Bapti st University 105 67 71 836 26.5%
Pepperdine University 30 19 1 0 *
Sarah Lawrence College 97 121 82 871 19.9%
Scripps College 115 108 64 2 24.0%
Sewanee: The University of the South 1 127 127 257 21.7%
The College of Idaho 79 71 54 551 23.7%
Trinity University 97 100 71 1 19.4%
Union College 0 63 41 3 14.6%
University of Denver 174 89 45 5 10.1%
University of Indianapolis 47 16 25 4 10.3%
Washburn University 63 41 30 7 *
Whitti er College 85 30 28 463 16.6%
Willamett e University 59 183 114 50 20.2%
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 14
APPENDIX B
2013-2014 Clark and HEDS Respondent Characteristics
Clark University 5-year cohort
All other alumni in 5-year cohort
Clark University 10-year cohort
All other alumni in 10-year cohort
Age
25 or younger 2 1.5% 75 2.9% 0 0.0% 0 0.0%
26 to 29 129 97.7% 2454 93.3% 1 1.3% 16 0.9%
30 to 34 1 0.8% 29 1.1% 75 94.9% 1709 94.8%
35 to 39 0 0.0% 14 0.5% 2 2.5% 29 1.6%
40 to 44 0 0.0% 17 0.6% 0 0.0% 11 0.6%
45 to 49 0 0.0% 11 0.4% 0 0.0% 11 0.6%
50 to 54 0 0.0% 10 0.4% 0 0.0% 9 0.5%
55 to 59 0 0.0% 10 0.4% 1 1.3% 10 0.6%
60 or older 0 0.0% 9 0.3% 0 0.0% 7 0.4%
Total 132 100.0% 2629 100.0% 79 100.0% 1802 100.0%
Gender
Male 35 26.5% 883 33.0% 29 35.8% 607 33.1%
Female 97 73.5% 1779 66.6% 51 63.0% 1224 66.7%
Fill in: _________________________ 0 0.0% 10 0.4% 1 1.2% 3 0.2%
Total 132 100.0% 2672 100.0% 81 100.0% 1834 100.0%
Race/ethnicity
American Indian or Alaska Nati ve 0 0.0% 1 0.0% 1 1.2% 2 0.1%
Asian American/Asian 3 2.2% 78 2.8% 1 1.2% 51 2.7%
African American/Black 3 2.2% 55 2.0% 2 2.4% 42 2.2%
Nati ve Hawaiian/Pacifi c Islander 0 0.0% 4 0.1% 0 0.0% 4 0.2%
White 113 82.5% 2199 79.0% 62 75.6% 1511 80.0%
Hispanic/Lati no/a 3 2.2% 125 4.5% 4 4.9% 90 4.8%
Not a US citi zen or permanent resident 7 5.1% 48 1.7% 4 4.9% 33 1.7%
Two or more races 3 2.2% 97 3.5% 4 4.9% 58 3.1%
Unknown 5 3.6% 178 6.4% 4 4.9% 98 5.2%
Total 137 100.0% 2785 100.0% 82 100.0% 1889 100.0%
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 15
2013-2104 HEDS Alumni Survey
Please refer to the Clark Alumni report online for the full survey instrument: http://www.clarku.edu/leepcompass
APPENDIX C
Sample
© 2011 Higher Education Data Sharing (HEDS) Consortium Contact: [email protected]
Alumni Survey
Welcome to the HEDS Alumni Survey. We appreciate your willingness to participate.
This 15-minute survey is designed to assess the long-term impact of your undergraduate education. The
survey asks about your experiences both during your undergraduate years and during the years since
you have graduated.
Your participation is completely voluntary, and there are no penalties if you decide not to participate or if
you choose to skip any questions. All of your responses are strictly confidential. At no time will your
response to any question be publicly linked with your name or with any other identifying information. For
most of the questions below, you will be asked to select the response that best represents your
experiences or opinions. A few questions will ask you to "check all that apply.” Additional questions will
ask for a brief response in your own words.
1. In what year did you complete your undergraduate degree?
2. Please indicate whether each of the following describes your current primary activity:
3. Please indicate your plans for each of the following degrees:
Do not plan to pursue
Degree received
Currently enrolled or
working toward
Degree you hope to attain
Second bachelor’s degree □ □ □ □
Master’s degree (e.g., MA, MSW, MBA)
□ □ □ □
Doctoral degree □ □ □ □
Professional degree (e.g., JD, MD) □ □ □ □
Other degree □ □ □ □
Does not apply Applies
Employment full-time □ □
Employment part-time □ □
Employment multiple jobs □ □
Graduate/professional school full-time □ □
Graduate/professional school part-time □ □
Military service □ □
Not employed, but seeking employment, admission to graduate school, or other opportunity
□ □
Not employed by choice (homemaker, volunteer, traveling, etc.) □ □
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 16
Sample
© 2011 Higher Education Data Sharing (HEDS) Consortium Contact: [email protected]
4. If you answered "other degree" in the last item, please describe that degree here:
5. About how many hours each week do you engage in volunteer work or community service for
organizations?
□ None □ 1–5 hours □ 6–10 hours □ 11–15 hours □ 16 or more
6. What is your current personal status? (Choose one)
□ Married or living with partner
□ Widowed
□ Separated or divorced
□ Single
□ Other:
7. How many dependent children do you have?
□ None
□ 1 or 2
□ More than 2
The following questions are about your experiences at your undergraduate
institution.
8. Below are statements about your views of your undergraduate faculty’s interest in teaching
and students. Please indicate the extent to which you agree/disagree with each.
Most faculty with whom I had contact were . . .
Strongly disagree
Disagree Neutral Agree Strongly agree
Genuinely interested in students.
□ □ □ □ □
Interested in helping students grow in more than just academic areas.
□ □ □ □ □
Good at providing prompt and useful feedback.
□ □ □ □ □
Genuinely interested in teaching.
□ □ □ □ □
Willing to spend time outside of class to discuss issues of interest and importance to students.
□ □ □ □ □
2013-2104 HEDS Alumni Survey
Please refer to the Clark Alumni report online for the full survey instrument: http://www.clarku.edu/leepcompass
APPENDIX C
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 17
APPENDIX D
THE HEDS SURVEY CONTAINS 28 MAJOR QUESTIONS, a subset of which were selected as aligning with the AAC&U’s LEAP Essential Learning Outcomes. Because Clark’s fi rst four Learning Outcomes are drawn from these we have used the same 14 NSSE questions, selected by AAC&U and reported in Finley, 2012. The following is a list of the Essential Learning Outcomes along with the corresponding questions from the NSSE Survey.
Specifi c HEDS Questions Used to Access the LEEP Learning Outcomes
OUTCOME CATEGORY OUTCOME DETAIL Q U E S T I O N
O U TC O M E � 2 : Intellectual and Practical Skills
Including inquiry and analysis, the generation and evaluation of evidence and analysis, the generation and evaluation of evidence and argument, critical and creative thinking, written and oral communication, quantitative literacy, information literacy, teamwork and problem-solving
Q14. To what extent did your experience as an undergraduate contribute to your knowledge, skills, and personal development in the following areas?
Q14b: Critical thinking: Examination of ideas, evidence, and assumptions before accepting or formulating a conclusion.
1Q4d: Information literacy: Locating, evaluating, and using information e� ectively and responsibly for a particular purpose.
Q14e: Quantitative Literacy
Q14f: E� ective writing: Conveying accurate and compelling content in clear, expressive, and audience-appropriate prose.
Q14g: E� ective speaking: Conveying accurate and compelling content in clear, expressive, and audience-appropriate oral presentations.
O U TC O M E � 3 : Personal and Social Responsibility
Including ethical reasoning and action, the intercultural understanding and competence to participate in a global society, civic knowledge and engagement locally as well as globally, and the lifelong habits of critical self-refl ection and learning
Q14. To what extent did your experience as an undergraduate contribute to your knowledge, skills, and personal development in the following areas?
Q14j: Civic engagement: Promoting the quality of life in a community, through both political and non-political processes.
Q14k: Intercultural knowledge and competence: Information, skills, and commitments that support e� ective and appropriate interactions in a variety of cultural contexts.
Q14l: Ethical reasoning: Recognizing ethical issues, examining di� erent ethical perspectives, and considering the ramifi cations of alternative actions.
O U TC O M E � 4 : Ability to Integrate Knowledge and Skills
Including synthesis and advanced accomplishment across general and specialized studies, bridging disciplinary and interdisciplinary thinking, and connecting the classroom and the world
Q11. Below are descriptions of the types of exams or assignments you may have had as an undergraduate. About how often did you undertake each?
Q11d: Used course content to address a problem not presented in the course
Q11h: Connected what I learned in multiple courses
Q14m: Integrative thinking: The habit of connecting ideas and experiences, and the ability to transfer learning to novel situations.
O U TC O M E � 5 : Capacities of Effective Practice
Including creativity and imagination, self-directedness, resilience and persistence, and the abilities to collaborate with others across di� erences and to manage complexity and uncertainty
Q14. To what extent did your experience as an undergraduate contribute to your knowledge, skills, and personal development in the following areas?
Q14i: Problem-solving: Designing, evaluating and implementing a strategy to answer questions or achieve a goal.
Q14c: Creative thinking: Developing or combining ideas, images, or expertise in innovative ways.
Q14h: Teamwork: Contributing to a team, facilitating the work of team members, and fostering a constructive team climate.
A CLARK UNIVERSITY COLLEGE-WIDE ASSESSMENT REPORT OF ALUMNI PERCEPTIONS OF INSTITUTIONAL SUPPORT OF LEEP LEARNING OUTCOMES | 18
APPENDIX E
Quantitative Analysis for Longitudinal Comparisons of Clark Alumni and HEDS Cohorts
Frequencies Averages
5-year-out 10-year-out 5-year-out 10-year-out
Clark % HEDS % Di� Trend Clark
%HEDS
% Di� Trend Clark Avg.
HEDS Avg.
Clark Avg.
HEDS Avg.
OUTCOME 2: Intellectual and Practical Skills
E� ective writing 85% 89% -3% 88% 88% 0% 3.3 3.5 3.4 3.5
E� ective speaking 60% 72% -12% 70% 66% 5% 2.9 3.0 3.0 2.9
Critical thinking 87% 92% -6% 93% 90% 2% 3.5 3.6 3.6 3.5
Quantitative literacy 63% 75% -12% 70% 72% -2% 2.9 3.1 3.0 3.1
Information literacy 81% 87% -6% 88% 84% 4% 3.2 3.4 3.4 3.3
OUTCOME 3: Personal and Social Responsibility
Civic engagement 60% 55% 6% 57% 53% 4% 2.8 2.7 2.7 2.6
Intercultural knowledge and competence
77% 68% 10% 77% 62% 15% 3.1 3.0 3.1 2.8
Ethical reasoning 76% 74% 2% 73% 71% 2% 3.1 3.1 3.1 3.0
OUTCOME 4: Integrative and Applied Learning
Integrative thinking 78% 83% -5% 75% 79% -4% 3.2 3.3 3.1 3.2
Connected concepts across courses
71% 74% -3% 74% 69% 6% 3.0 3.1 3.0 2.3
Applied content 42% 43% -2% 35% 41% -5% 2.3 2.4 2.2 2.7
OUTCOME 5: Capacities of E� ective Practice
Problem-solving 71% 78% -7% 66% 76% -10% 3.0 3.2 3.0 3.1
Teamwork 69% 70% -1% 65% 67% -3% 3.0 3.0 2.9 2.9
Creative thinking74% 80% -6% 82% 79% 3% 3.1 3.2 3.3 3.2
K E Y:Frequencies of Clark alumni who answered “Very much” or “Quite a bit” (the top two highest response categories) on their perceptions of institutional support compared to HEDS cohorts:
Clark scores were higher than HEDS scores by ≥ 10%
Clark scores were higher than HEDS scores by ≥ 5%
No di� erences in frequencies between Clark alumni compared to HEDS cohorts
Clark scores were lower than HEDS scores by ≤ 5%
Clark scores were lower than HEDS scores by ≤ 10%