data-driven student success programming in residence life - acuho-i 2013
TRANSCRIPT
Data-Driven Student Success Programming in Residence Life
Presented by: Ryan O’ConnellAssociate Director, Student Success Programs
June 2013
@CampusLabsCo #labgabwww.campuslabs.com/blog
WE LIVE AND BREATHE
dedicatedemployees
90+
CHANGED THE CAMPUS FOCUS FROM COLLECTING DATA TO CONNECTING DATA
700+Member campuses and
valued partnerships
Location: Buffalo, NY & Atlanta, GAm
an
y r
oo
ted
in
hig
her
ed
DATA
CAMPUS LABS SOLUTION
One system that collects and reports data, and brings data together for new insights.
Organize
planning,
program review
and
accreditation
Assess student
learning and
success
Elevate
teaching
and learning
Retain
students
through
early alert
Create
meaningful
involvement
experiences
This session will not be a product demo.
Rather, a
of residential life promising practices,
peppered with core competencies of
assessment.
CELEBRATION
Agenda
Intentional Program Design
Binghamton University Example
Learning Outcome Development
Assessing Outcomes
Student Completion Stats
NAU’s Non-Cognitive Approach
Continuous Assessment
NASPA/ACUHO-I Benchmark Data
Supported by data,
residential program design can impact your
students’ lives and your institution's retention
rate.
INTENTIONAL
Learning
Program Program Program Program Program Program
Learning
Student learning is not the
result of discrete experiences
but rather the product of
many different kinds of
experiences in and outside
the classroom over an
extended period of time.
Student Life
Classroom
Advising
Residential Life
College Experience
College Experience
Experience
College Experience
College Experience
College Experience
College Experience
College
ExperienceCollege
Experience
College ExperienceCollege Experience
College Experience
College
Experience
CAS & Learning Reconsidered 2
6 Domains:o Knowledge acquisition, construction, integration
and application
o Cognitive complexity
o Intrapersonal development
o Interpersonal competence
o Humanitarianism and civic engagement
o Practical competence
Dimensions within each domain provide further clarification and direction.
http://www.cas.edu/getpdf.cfm?PDF=D87A29DC-D1D6-D014-83AA8667902C480B
Other Sources
General education learning outcomes
Institutional mission statement
Accreditation standards
Other professional organizations
Strategies for Developing OutcomesStrategy 1: Mission Statements
Strategy 2: Professional Organizations
Strategy 3: Student Work
Strategy 4: An Epistemological and Ethnographic Process
Strategy 5: Deep and Surface Approach to Learning
Read Peggy Maki’s "Assessing for Learning: Building a
Sustainable Commitment across the Institution”
Chapter 3: Making Claims About Student Learning Within
Contexts for Learning
Process for Developing Own Framework
1. Research
2. Reflection
3. Collaboration
4. Consensus
Read Linda Suskie’s “Assessing Student Learning: A Common Sense Guide”
Chapter 8: Developing Learning Goals
Our Suggested Process
Gather documents
Assemble core group
to draft outcomes
Send draft to
leadership for
feedback and review
Use feedback to refine
and revise
Publicize and utilize outcome
framework
Continue to review
and revise
Jennifer Hapgood, Business
Manager and Assessment Specialist
Public Northeastern University
4-year Institution
Over 14,000 Students
10% Attrition after Freshman Year
Implemented Card Swiping System to Monitor Programming
Participation
Comprehensive Data Systems in Student Affairs to track Participation in Programs
Binghamton University
Pillars of Residential Life
Global
Assisting residents to
develop relationships
with others as well as
preparing them to
become informed
global citizens
Academic
Assisting residents to
successfully navigate
the collegiate
experience and
presenting them the
greatest chance of
being a successful
student
Pillars of Residential Life
Personal
Assisting individual
residents to further their
individual growth
Community
Development
EXERCISEWrite down no more than 7 learning domains that
you think are addressed through Residential Life
at your campus.
As a small group, share your domains and agree
on 4-6 domains along with a campus name.
Program Design
• Who does the outcome pertain to?Audience/Who
• What do you expect the audience to know/be able to do?Behavior/What
• Under what conditions or circumstances will the learning occur?Condition/How
• How much will be accomplished, how well will the behavior need to be performed, and to what level?
Degree/How much
ABCD Learning Outcomes
Learning outcome examples
1. After attending a financial aid session, students will be able to accurately fill out the FAFSA form.
2. As a result of three meetings with an academic coach, students will be able to compare study strategies that is appropriate for their learning style.
3. After participating in a leadership retreat, organization presidents will be able to differentiate between tasks to complete themselves versus those to delegate to their executive board before their first spring semester planning meeting.
Co-Curricular Mapping
Staff Training
• Module inserted into normal RA/RD training
processes
• Began emphasis on domains
• Review of each Hall’s programs and
demographic make-up prior the term
Event Promotion & Tracking
Scanfobs
Used at each Residential
Life Event
Very portable and easy
to use
Attendance added to B-
Engaged easily
EXERCISEList 5 programs that your RA staff could host that
map to the outcome domains that you resolved in
the previous exercise. As a group, discuss how
you might engage in this exercise on campus.
How do you train your RA’s to be good
programmers?
Assessing Learning Outcomes
Any process employed to gather data which
requires subjects to display their knowledge,
behavior, or thought processes.
Direct Methods Indirect Methods
Any process employed to gather data which asks subjects to reflect upon
their knowledge, behaviors, or thought
processes.
List three sections that should be listed on your
resume:
I can name all of the sections that should be
included when I create my resume.
Strongly agree
Moderately agree
Neither agree nor disagree
Moderately disagree
Strongly disagree
Direct or indirect?
As a result of three meetings with an academic
coach, students will be able to compare study
strategies that are appropriate for their learning
style.
Asking students to tell you how they are studying
and comparing that to the study strategies guide
Direct!
Direct or indirect?
As a result of three meetings with an academic
coach, students will be able to compare study
strategies that are appropriate for their learning
style.
At the conclusion of the meeting, asking students
to raise their hand if they know what study
strategy they will use (and tracking that)
Indirect!
Learning Surveys
• Examples from
Delaware
Ways to Gather Survey Self Report DataAs a result of ____, I can _____:Agreement: Strongly agree, Moderately agree, Neither agree nor disagree, Moderately disagree, Strongly disagree (another version removes the “moderately” qualifier and/or uses “neutral”)
To what extent did ____ help you_____:Extent (5 pt): A great deal, Considerably, Moderately, Slightly, Not at all
How would you rate your ability to _______:Proficiency: Beginner, Developing, Proficient, Advanced, Expert
What did you learn about _____?
Can you ____?Yes, No
Beginning
1
Developing
2
Accomplished
3
Exemplary
4
Score
Assessment of the
Situation
Staff did not identify the
key areas of concern in the
case study
Staff identified some of the
key areas of concern in the
case study
Staff identified the most
significant areas of
concern in the case study
Staff identified all areas of
concern in the case study
Approach with
Student(s)
Staff recommended an
approach with the
student(s) that is
inappropriate and/or
unprofessional
Staff recommended an
approach that was
acceptable, but could be
improved
Staff recommended an
approach that was
professional and suitable
for the situation
Staff recommended an
approach that was exactly
what is expected
Referrals to Campus
Resources
Staff did not make any
referrals to campus
resources
Staff acknowledged the
resident’s need for a
referral, but didn’t provide
any specific examples
Staff acknowledged
resident need for referral
and gave appropriate
examples
Staff acknowledged
resident need for referral,
provided appropriate ones,
and offered to personally
assist resident to obtain
services
Reporting
Staff did not recommend
reporting the situation to
the appropriate person and
did not write an incident
report
Staff only reported the
situation to the RD or
wrote an incident report –
not both.
Staff reported the situation
to the supervisor and
wrote an incident report
Staff informed the RD,
wrote an incident report,
and mentioned keeping
the RD continuously
informed of any follow up
Community
Education
Staff did not recommend
any community education
as a result of the scenario
Staff mentioned
community education but
did not provide examples.
Staff recommended
specific community
education components
Staff recommended a
variety of community
education components
Follow Up
Staff did not follow up with
the student(s) involved
after the initial contact
Follow up was done, but
not appropriately
Staff followed up with the
student(s) regarding the
initial concern.
Staff followed up with the
student regarding the initial
and also maintained a
positive relationship with
the student over time.
Comments: Overall
Average:
Break
Student Completion StatsNAU’s Non-Cognitive ApproachContinuous AssessmentACUHO-I Benchmark Data
First-year retention rate
http://www.higheredinfo.org/dbrowser
77%
Student Completion
6 year college graduation rate
55%
Student Completion
http://www.higheredinfo.org/dbrowser
Student CompletionPercentage of
students who
graduate from high
school in four years,
go directly to
college, are retained
to second year, and
graduate in six
years.
http://www.higheredinfo.org/dbrowser
Public Southwestern University
4-year Institution
Over 17,000 Students on the Flagstaff Campus and 1,100
Faculty
27% Attrition after Freshman Year
Goal to increase retention rate by 10%
Student academic services unit within Student Affairs
Northern Arizona University
A Non-Cognitive Approach
Non-cognitives
Non-cognitive attributes are those academically
and occupationally relevant skills and traits that
are not specifically intellectual or analytical in
nature. They include a range of personality and
motivational habits and attitudes that facilitate
functioning well in school. Non-cognitive traits,
skills, and characteristics include perseverance,
motivation, self-control, and other aspects of
conscientiousness (Borghans et al., 2008).
Literature
Student characteristics including psychosocial factors influence retention (Brown et al., 2008; Tinto 1975, 1993).
Influence of psychosocial factors on retention extend to two-year institutions (Napoli and Wortman, 1998)
Robbins, S., Lauver, K., Le, H., Davis, D., Langley, R., & Carlstrom, A. (2004) examined 109 studies and found that psychosocial and study skill factor (noncoginitives) were predictive beyond traditional factors.
The Student Strengths Inventory Framework
Risk Scale Definition Sample Item
Academic
Success
Academic Engagement The value an individual places on
academics and attentiveness to
school work.
I turn my homework in
on time.
Academic
Success
Academic Self-Efficacy An individual’s confidence in his or
her ability to achieve academically
and succeed in college.
I will excel in my
chosen major.
Retention Educational
Commitment
An individual’s dedication to college
and the value placed upon a college
degree.
I see value in
completing a college
education.
Retention/
Academic
Success
Resiliency An individual’s approach to
challenging situations and stressful
events.
I manage stress well.
Retention Social Comfort An individual’s comfort in social
situations and ability to communicate
with others.
I am comfortable in
groups.
Retention Campus Engagement Involvement in campus activities and
attachment to the college/university.
Being active in extra-
curricular activities in
college is important to
me.
Implement a readiness program to
increase
persistence and improve student
success.
NAU’s Goal
Institution Needs
Could identify some of their “at-risk” student groups
• First generation college students
• Ethnic minorities
• Low income students
• Lower SAT/ACT scoring students
BUT needed to know
• More about which students within those groups were
most in need.
• Which services might best help which students.
NAU needed to LEARN
MORE about
each INDIVIDUAL
STUDENT.
Strategic Intervention Model:Freshman Outreach Program
Implemented early in the first semester
Include groups known to be at risk
Collaborative and not duplicative
• academic advising and first year programs
• Summer bridge program and first year mentoring
• any key groups interfacing with new freshmen
Systematic
Whole student
Grounded in research
Program Model: Student Affairs
Prioritize Outreach
Connect Evaluate
Ethnic minority students
First generation college
students
Low income students
Low academicpreparedness
Predictive Model Scores
Meeting one-one
Review results
Refer resources
Track connections
Student perspectives
Track referrals and use
Primary Data Sort
Creating lists of students to be seen by each office
“Waterfall”– Summer Bridge Students (FG, LI)
– Native American Students
– Other Ethnic Minorities
– Other First Generation/Low Income
– Other Low Scoring (low ACT or HS Core GPA)
Secondary Data Sort
Non-cognitive Results– Low Index Scores
– Low Academic Scale Scores
In House Predictive Modeling Scores
– Likelihood of Leaving
Prioritizing Student Groups
One-on-One Meetings in Student Affairs
Systematic Outreach– Look up schedules and set times– Postcard home with appointment time– Postcard to campus with appointment time– Personal e-mail with appointment time– Appointment time on their web calendar– Reminder phone call 1-2 days before appointment– If miss scheduled meeting, protocol for rescheduling
at least twice more– Social Media: Facebook
The student’s peer group is the single most potent source of
influence on growth and development during the undergraduate years.
Astin, A. What Matters in College? Four Critical Years Revisited. (p. 398). San Francisco:
Jossey-Bass, 1993.
Purpose of Meeting
Above all else, connect with student
Acknowledge student’s strengths and identify campus activities to reinforce them
Match student’s needs to campus resources
Gather some data on SSI experience
– How did you hear about your appointment?
– How seriously did you take the SSI when you completed it at orientation?
Incoming Data Fall 2010 Outcomes
Total
Students
Fall 2010 % Ret'd F11
Ave HS Core
GPA Ave ACT Ave SAT
Census Cr
Hrs
Earned Cr
Hrs Term GPA
Good Acad.
Standing
Freshman
Outreach
SA Target Met 786 74% 3.32 23 1049 15.0 13.3 2.90 88%
SA Target Not
Met 1030 67% 3.22 23 1043 14.9 12.4 2.65 80%
GSSC Target
Met 195 77% 3.01 20 949 14.9 13.3 2.78 88%
GSSC Target
Not Met 117 69% 3.12 20 956 15.1 13.2 2.84 85%
Expected to Do
Well 644 78% 3.43 26 1169 15.2 13.9 3.10 90%
No Survey 460 64% 3.17 22 1047 14.6 11.7 2.65 80%
Assessment of Approach
Provide all students with program/service information, and
target needy students for higher-level contact
EARLYDirect outreach and referrals to needed services must happen
STRATEGICALLY
< Student Success
Advisor
< Student Success
Advisor
< RA
Erin Grisham
Executive Director, Educational Support Services
EXERCISEReview the non-cognitive report of your partner
and hold a mock intervention conversation.
Graphical Representation of Results
Percentile Ranks
Result Descriptions
Recommendation Statements
The Student Report
EXERCISEReview the non-cognitive report of your partner
and hold a mock intervention conversation.
How would you approach RA training?
How could you incorporate non-cognitives into
your programming efforts?
This information seems very personal, I’m not
trained in psychology/counseling.
Should I share the risk indices with students?
What if my conversation reveals an urgent
issue?
Common Concerns
Ongoing Data Collection
Integrate key questions into your ongoing surveys
to serve as triggers for success outreach
Option to target groups of students based on
known issues (e.g., social comfort, academic
engagement)
Micro Surveys
Fall
How does it feel to be away
from home?
How are things going with your
roommate?
Have you made friends?
Spring
How do you feel about your academic performance last fall?
How much do you like your major/studies?
Academic resources are available on campus to help you succeed in your classes. Which of the following reflects your use of these support services?
Creates a time-
appropriate data
point that informs
intervention
activity.
EXERCISEList assessment items that you could embed into
operational, satisfaction, and follow-up surveys
that would be good success indicators.
As a result of ____, I can _____:Agreement: Strongly agree, Moderately agree, Neither agree nor disagree, Moderately disagree, Strongly disagree (another version removes the “moderately” qualifier and/or uses “neutral”)
To what extent did ____ help you_____:Extent (5 pt): A great deal, Considerably, Moderately, Slightly, Not at all
How would you rate your ability to _______:Proficiency: Beginner, Developing, Proficient, Advanced, Expert
What did you learn about _____?
Can you ____?Yes, No
Recap
National Data
The most important housing considerations when deciding to attend this institution
www.naspaconsortium.org
What are the most common ways you learn about programs and events offered by residence life?
www.naspaconsortium.org
Has your involvement in residence life programming and events provided you with skills and abilities you will use after college?
www.naspaconsortium.org
Which issues have you learned more about/become more aware since living on campus/in the residence halls?
www.naspaconsortium.org
HAVE A GREAT CONFERENCE!
campuslabs.com