assessment in student affairs: some thoughts on a tricky business
DESCRIPTION
Assessment in Student Affairs: Some Thoughts on a Tricky Business. Peter T. Ewell National Center for Higher Education Management Systems (NCHEMS) Student Affairs Assessment Conference Emory University October 14, 2013. Logic of This Keynote. - PowerPoint PPT PresentationTRANSCRIPT
Assessment in Student Affairs:Some Thoughts on a Tricky Business
Peter T. Ewell
National Center for Higher Education Management Systems (NCHEMS)
Student Affairs Assessment Conference
Emory University
October 14, 2013
Logic of This Keynote
Challenges and Dilemmas of Assessment Practice in Student Affairs: What Makes This a Tricky Business
Growing Demand for the Kinds of Proficiencies at the Center of Student Affairs
Some (Occasionally Odd) Techniques for Getting at These Proficiencies
Using Results of Assessment to Improve Programming and Practice in Student Affairs
A Prefatory Comment
Two Worlds of Assessment:
Academic: Faculty Want to Know About that Small Piece of the Student that is Trying to Learn [Mathematics, History, etc.]
Student Affairs Professional: Want to Know About the Experience and Development of the Whole Student that is Experiencing an Emotional, Attitudinal, and Cognitive Process of Transformation
Some Particular Challenges of Assessment in Student Affairs
The Domains of Interest are Not Easy to “Measure”
Many of Them are Affected by Dispositions to Act in Particular Ways Even Though Students “Know” What they Should Do
Attributions of Cause are Hard to Relate to Individual or Isolable Programmatic Features or Elements
The Growing Demand for These Kinds of Proficiencies
Employer Complaints About Recent College Graduates [and What They Want Instead]
Demands for Integrated Abilities and Practical Competence
Reflected in Prominent Expectations Frameworks Like the Lumina Degree Qualifications Profile (DQP) and AAC&U’s LEAP Goals
Graduate Attributes: Top Choices for Employers from 2013 AAC&U Survey
Percent Reporting “Very” and “Somewhat” Important:
Ethical Judgment and Integrity – 96%
Comfortable with Diverse Backgrounds – 98%
Demonstrated Capacity for Professional Development – 94%
Interest in Giving Back to Communities Our Company Serves – 71%
Some Proposed Revisions to DQP 2.0
Confronts the ethical issues present in prominent problems in politics, economics, society, scholarship, or the arts and demonstrates how at least two different ethical perspectives, influence decision making with respect to such problems
Describes how knowledge from different cultural perspectives would affect inter pretations of prominent problems in politics, society, the arts and/or global relations
Some AAC&U LEAP Goals
Ethical Reasoning and Action
Inter-Cultural Knowledge and Competence
Teamwork and Problem Solving
Civic Knowledge and Engagement – Global and Local
Foundations and Skills for Lifelong Learning
Some Useful Approaches for Gathering Evidence About Values and Dispositions
Survey Items and Item Construction
Time Diaries and the “Beeper Study”
Formal Focus Groups and Interviews
Informal Observation and Imagery
Checklists
Rubrics Applied to Behavior and Work Products
Inventories of Good Practice
Survey Items and Item Construction
Scales with Even Numbers of Decision Points
“Semantic Differential” Items
Scenario Based Items (e.g. The Defining Issues Test)
Open-Ended Items (e.g. “Areas and Agents of Change”)
Semantic Differential Items Example
“Areas and Agents of Change” Example
Observation and Imagery
Develop and Pilot Clear Observational Protocols
Ensure What You are Looking at is Typical
For Ongoing Processes, Observe Regularly at Defined Points in Time
Consider Using Photographs or Videos to Capture Demographic Mixes or Ongoing Patterns of Behavior
Observation and Imagery
Develop and Pilot Clear Observational Protocols
Ensure What You are Looking at is Typical
For Ongoing Processes, Observe Regularly at Defined Points in Time
Consider Using Photographs or Videos to Capture Demographic Mixes or Ongoing Patterns of Behavior
Some Applications of Checklists
Academic Advising
Service Learning Courses
Internships, Field Placements
Student inquiries
Academic Skills Centers
Residence Hall and Student Group Advisors
AAC&U VALUE Rubric for Teamwork
TEAMWORK VALUE RUBRICfor more information, please contact [email protected]
Capstone4
Milestones3
2
Benchmark1
Contributes to Team Meetings Helps the team move forward by articulating the merits of alternative ideas or proposals.
Offers alternative solutions or courses of action that build on the ideas of others.
Offers new suggestions to advance the work of the group.
Shares ideas but does not advance the work of the group.
Facilitates the Contributions of Team Members Engages team members in ways that facilitate their contributions to meetings by both constructively building upon or synthesizing the contributions of others as well as noticing when someone is not participating and inviting them to engage.
Engages team members in ways that facilitate their contributions to meetings by constructively building upon or synthesizing the contributions of others.
Engages team members in ways that facilitate their contributions to meetings by restating the views of other team members and/or asking questions for clarification.
Engages team members by taking turns and listening to others without interrupting.
Individual Contributions Outside of Team Meetings Completes all assigned tasks by deadline;work accomplished is thorough, comprehensive, and advances the project.Proactively helps other team members complete their assigned tasks to a similar level of excellence.
Completes all assigned tasks by deadline;work accomplished is thorough, comprehensive, and advances the project.
Completes all assigned tasks by deadline;work accomplished advances the project.
Completes all assigned tasks by deadline.
Fosters Constructive Team Climate Supports a constructive team climate by doing all of the following:•Treats team members respectfully by being polite and constructive in communication.•Uses positive vocal or written tone, facial expressions, and/or body language to convey a positive attitude about the team and its work.•Motivates teammates by expressing confidence about the importance of the task and the team's ability to accomplish it.•Provides assistance and/or encouragement to team members.
Supports a constructive team climate by doing any three of the following:•Treats team members respectfully by being polite and constructive in communication.•Uses positive vocal or written tone, facial expressions, and/or body language to convey a positive attitude about the team and its work.•Motivates teammates by expressing confidence about the importance of the task and the team's ability to accomplish it.•Provides assistance and/or encouragement to team members.
Supports a constructive team climate by doing any two of the following:•Treats team members respectfully by being polite and constructive in communication.•Uses positive vocal or written tone, facial expressions, and/or body language to convey a positive attitude about the team and its work.•Motivates teammates by expressing confidence about the importance of the task and the team's ability to accomplish it. •Provides assistance and/or encouragement to team members.
Supports a constructive team climate by doing any one of the following:•Treats team members respectfully by being polite and constructive in communication.•Uses positive vocal or written tone, facial expressions, and/or body language to convey a positive attitude about the team and its work.•Motivates teammates by expressing confidence about the importance of the task and the team's ability to accomplish it. •Provides assistance and/or encouragement to team members.
Responds to Conflict Addresses destructive conflict directly and constructively, helping to manage/resolve it in a way that strengthens overall team cohesiveness and future effectiveness.
Identifies and acknowledges conflict and stays engaged with it.
Redirecting focus toward common ground, toward task at hand (away from conflict).
Passively accepts alternate viewpoints/ideas/opinions.
Evidence-Based Management: What Does a “Culture of Evidence” Require?
Respect for the Facts
Clear Goals and Objectives
Concrete Evidence About the Achievement of Goals and Objectives
An Attitude Toward Problem-Solving that Avoids “Finger-Pointing”
Clear Follow-Through On Decisions and Why They Were Made
Willingness to Stop Doing Things When They Don’t Work
Strategies for Addressing Key Difficulties in Building a Culture of Evidence
Keep the Outcomes Framework Simple
Look at Data Distributions, Not Just “Averages”
Adjust the Approach to Accommodate Important Differences in Context
Share Approaches Across Units
Don’t Wait for Perfection
Some “Points of Attack” in Building Cultures of Evidence
Expectations Exercises
Use Results to Inform and Initiate Discussions, Rather than “Give Answers”
Initiate Visible Opportunities to Consider Results and What They Mean
Present Results as Discrepancies
Disaggregate Results to Show Variations
Package Results Around Perceived Problems or Embed Them in Regular Decision Processes
Some Final Reminders
If It’s Important, Try to Assess It – No Matter How “Tricky” the Business Seems to Be
Dispositions Matter – So Experiment with Indirect and Unobtrusive Approaches
Use Multiple Forms of Evidence Wherever Possible
Be Creative and Have Fun