approaches to assessment of the sophomore-year experience: a panel discussion
DESCRIPTION
Approaches to Assessment of the Sophomore-Year Experience: A Panel Discussion. Institute on Sophomore Student Success Columbia, SC April 12-14, 2013 Jennifer R. Keup. Session Layout. Brief assessment primer: Jennifer Keup Panel of assessment colleagues to start discussion: - PowerPoint PPT PresentationTRANSCRIPT
Approaches to Assessment of the Sophomore-Year Experience:
A Panel Discussion
Institute on Sophomore Student SuccessColumbia, SC
April 12-14, 2013 Jennifer R. Keup
Session Layout
• Brief assessment primer: Jennifer Keup• Panel of assessment colleagues to start
discussion:– Facilitator: Jennifer Keup, National Resource
Center– Panelists:• Laurie Schreiner, Azusa Pacific University• Jimmie Gahagan, USC• David Sneed, Belmont University
ASSESSMENT AND THE SOPHOMORE-YEAR EXPERIENCE:
STARTING THOUGHTS AND
INITIAL IDEAS
What is Assessment?
• “The systematic collection, review & use of information about educational programs for the purposes of improving student learning and development.”
• “Any effort to gather, analyze, and interpret evidence, which describes institutional, divisional, or agency effectiveness.”
• “The process of gathering & discussing information from multiple & diverse sources in order to develop a deeper understanding of what students know, understand, & can do.”
CREATEACTION
GATHER LEARNFROM
USE
Is sophomore assessment different than other student assessment?
GENERALLY, NO. Good assessment
practices are
constant, but…
• Specific sample• Developmentally
appropriate outcomes• Can be one point,
starting point, or end point
• One point in process
Systematic Cycle of Assessment
• Identify learning outcomes• Gather evidence• Document what you already know• Identify appropriate methods of measurement• Acknowledge methodological limitations but focus
on results• Interpret evidence• Implement change
Identify Learning Outcomes• Characteristics:– Focuses on product rather than process– Must be measurable– Detailed and specific– Includes action verbs– Aspirational– Meaningful (i.e., must be useful not just interesting)
• Think-Pair-Share: What are some examples of sophomore learning outcomes?
Examples of Outcomes• Persistence• Analytical & critical thinking skills• Knowledge of university requirements• Interpersonal/Communication skills• Creation of a sense of community• Identity exploration and development• Develop educational career goals/Declare a major• Practical competence• Knowledge integration and application
How to involve faculty?
Examples of Outcomes (cont.)
• Multicultural competence• Civic engagement/Democratic citizenship• Ability to identify, seek, & utilize organizational
resources and student programs• Physical health & emotional wellness• Leadership skills• Moral and ethical development• Cognitive complexity• Academic achievement
Any others?
Gather Evidence• Needs assessment• Utilization data• Criterion-referenced– What are the criteria?
• Benchmarking– Who are your true or aspirational peers?– How to get their data?
• Trends• Value added
INPUTS
ENVIRONMENTS
OUTCOMES
• What data do you already have?– Who is collecting it?– Is it being used? If so, for what purposes?– Where is it being housed?– Are there plans for future data collection?
• How can you use data that has already been collected for your assessment needs?
• Challenge: List as many institutional sources of data on sophomores as you can.
Document What You Already Know
“Data lust turns into data dust”
• Admissions/Registrar’s data• Institutional participation in national surveys of first-
year students• Program/course evaluations• Utilization statistics• Program review reports• Satisfaction surveys• Campus/student profile• Focus group findings
Potential Sources of Data
Potential Data Sources (cont.)
• Home-grown IR surveys• SA departmental surveys, focus groups, or student
interviews• Self-study processes/CAS• Accreditation reports• Grants• Strategic planning• Student & faculty interview transcriptions• Other sources?
A series of important decisions
Identify Appropriate Methods of Measurement
• Formative assessment – Normative assessment• Quantitative methods – Qualitative methods• Direct measures – Indirect measures• National (“off-the-shelf”) instrument – Locally developed
instrument
Remain connected to learning outcomes
Acknowledge Methodological Limitations but Focus on Results
• Do what you can to minimize limitations but do not perseverate in the assessment planning stage
• There is always something to learn from assessment activities and findings
• Good assessment will typically raise more questions than it answers
• There is no “perfect” assessment plan; JUST GET STARTED & KEEP LEARNING!
Interpret Evidence
“You can’t fatten a pig by weighing it.”
• What conclusions can you draw from the data? How did you do?
• What other questions do the data raise? What other information might you need?
• What are the implications for practice or what policy decisions would you make?
Interpret Evidence
• Don’t interpret in isolation– Form an assessment committee– Ad hoc groups to process results– Benefits:
• More holistic thinking• Create buy in• Tool for dialogue
• Use multiple lenses (i.e. benchmarking, longitudinal, value added, quantitative, qualitative, etc.)
Implement Change• Purpose of assessment is to CREATE ACTION to:– Continue effective practice, or– Initiative change and improvement
• Effective dissemination of results– Who needs to know this information?– How do they need to know the information?
• Changes will be connected to new outcomes and assessment plans
Panelists
• Introduce themselves• Identify their role/experience with
sophomore-year assessment• Indicate the biggest challenge they see to
sophomore-year assessment• Best advice they know/have received with
respect to sophomore-year assessment• Open up for questions from audience
Discussion Questions: Challenges & Strategies
What are the biggest challenges to assessment of second-year students and
sophomore initiatives?
What are some strategies for overcoming these challenges?
Discussion Questions:Identifying Outcomes
How do I determine the best outcomes on which to focus for assessment of
sophomores?
Which outcomes would generate the greatest “buzz” on our campus?
Discussion Questions
Access to Population: How do I most effectively access sophomores for
assessment activities?
Instruments & Methods: What assessment tools already exist that are appropriate for
assessment of the second year?
Discussion Questions
Existing Data: What is our most important data point that we are likely not using?
Measurement: How do I create meaningful measures of complex outcomes (e.g.,
multicultural competence, leadership, civic engagement)?
Career Planning
Undergraduate Research
Learning Community
Class Events
Leadership Development
Discussion Questions:Indirect vs. Direct Measures
How do I best “sell” assessment outcomes generated from indirect measures?
How do I identify efficient direct measures for what is currently measured indirectly?
Discussion Questions:Creation of an Assessment Plan
Who are key campus partners with whom I should be communicating about
assessment?
How do I effectively integrate sophomore-year assessment efforts to a larger cycle
that includes the first-year and senior-year data collection efforts?
ReferencesAstin, A.W. (1991). What Matters in College. San Francisco:
Jossey-Bass.Gahagan, J., Dingenfelder, J., & Pei, K. (2010). A Faculty and
Staff Guide to Creating Learning Outcomes. Columbia, SC: University of South Carolina, National Resource Center for The First-Year Experience and Students in Transition.
Gansemer-Topf, A.M. & Pattengale, J.A. (2010). Evaluating Second-Year Programs. In M.S. Hunter, B.F. Tobolowsky, J.N. Gardner, S.E. Evenback, J.A. Pattengale, M.A. Schaller, L.A. Schreiner & Associates, Helping Sophomores Succeed (pp. 234-247).
Other Assessment ResourcesBresciani, M.J., Zelna, C.L., & Anderson, J.A. (2004). Assessing Student
Learning and Development. Washington, DC: National Association of Student Personnel Administrators.
Friedman, D.B. (2012). The First-Year Seminar: Designing, Implementing, and Assessing Courses to Support Student Learning and Success: Vol. V. Assessing the First-Year Seminar. Columbia, SC: University of South Carolina, National Resource Center for The First-Year Experience and Students in Transition.
Maki, P. (2004). Assessing for Learning: Building a Sustainable Commitment Across the Institution. Sterling, VA: Stylus.
Strayhorn, T.L., Creamer, D.G, Miller, T.K. (2006). Frameworks For Assessing Learning And Development Outcomes. Washington, DC: Council for the Advancement of Standards in Higher Education.
Upcraft, M.L. & Schuh, J.H. (1996). Assessment in Student Affairs: A Guide for Practitioners. San Francisco: Jossey-Bass.
Other questions for discussion?
Thanks!