using action research to ensure relevance and excellence
DESCRIPTION
Using Action Research to Ensure Relevance and Excellence. Student Success Symposium. Dan Friedman, Ph.D. Director, University 101 Programs University of South Carolina. Does it work?. Academic achievement – GPA and hours earned Retention persistence to second year - PowerPoint PPT PresentationTRANSCRIPT
![Page 1: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/1.jpg)
Using Action Research to Ensure Relevance and Excellence
Dan Friedman, Ph.D.Director, University 101 Programs
University of South Carolina
Student Success Symposium
![Page 2: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/2.jpg)
![Page 3: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/3.jpg)
Does it work?
Academic achievement
– GPA and hours earned
Retention persistence to second year
Utilization of campus resourcesPaul Fidler
![Page 4: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/4.jpg)
![Page 5: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/5.jpg)
A movement created
87% of institutions with a FYS
Padgett & Keup (2011). 2009 National Survey of First-Year Seminars.
![Page 6: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/6.jpg)
High-Impact Educational Practices (AACU)
![Page 7: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/7.jpg)
My Focus for Today
Using assessment/action research to demonstrate the value of our programs, and to continually improve what we do by understanding why our programs work and for whom.
![Page 8: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/8.jpg)
FAITH-BASED?
“Estimates of college quality are essentially "faith-based," insofar as we have little direct evidence of how any given school contributes to students' learning.” RICHARD HERSCH (2005). What does college teach?
ATLANTIC MONTHLY.
![Page 9: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/9.jpg)
Assessment Cycle
3) Interpret Evidence
4) Implement Change
2) Gather Evidence
1) Identify Outcomes
Maki, P. (2004). Assessing for learning: Building a sustainable commitment across the institution. Sterling, VA: Stylus Publishing.
![Page 10: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/10.jpg)
Easy Stuff!
![Page 11: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/11.jpg)
Friedman, D. (2012). Assessing the first-year seminar.
![Page 12: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/12.jpg)
Two Types of Assessment
1) Summative – used to make a judgment about the efficacy of a program
2) Formative – used to provide feedback in order to foster improvement.
![Page 13: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/13.jpg)
The Prescription
Relevance (doing the right things)
Excellence (doing things right)
![Page 14: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/14.jpg)
EEnvironments
(College)
I O
Astin’s Value-Added
I – E – O Model
Astin, A. (1991)
“outputs must always be evaluated in terms of inputs”
Outcomes Inputs (Students)
![Page 15: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/15.jpg)
EEnvironments
I O Inputs Outcomes
Common Mistakes
Just looking at inputs
15
![Page 16: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/16.jpg)
EEnvironments
I O Inputs Outcomes
Common Mistakes
Just looking at environment
16
![Page 17: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/17.jpg)
EEnvironments
I O Inputs Outcomes
Common Mistakes
Just looking at outcomes
17
![Page 18: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/18.jpg)
EEnvironments
I O Inputs Outcomes
Common Mistakes
E-O Only (No Control for Inputs)
18
![Page 19: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/19.jpg)
EEnvironments
(College)
I O
Astin’s Value-Added
I – E – O Model
Astin, A. (1991)
“outputs must always be evaluated in terms of inputs”
Outcomes Inputs (Students)
![Page 20: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/20.jpg)
Disaggregating the Inputs
![Page 21: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/21.jpg)
What does this tell us?
2002 2003 2004 2005 2006 2007 2008 2009 2010 201180
81
82
83
84
85
86
87
88
89
90
84.585.2
83.8
86.2
87.488.1
87.6
86.3 86.3
88
83.5
84.3
81.3
84.6
85.4
84.384.6
85.1
86.8
84.4
1-Year Retention
U101Non-U101
Year
Perc
enta
ge
![Page 22: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/22.jpg)
What does this tell us?
PGPA Quintile UNIV 101 (n) Non-101 (n) P-Value
5 (High) 93.8% (n=497) 94.6% (n=297) NS
4 90.5% (n=652) 84.4% (n=154) .02
3 88.3% (n=656) 86.5 % (n=163) NS
2 87.4% (n=657) 79.1% (n=148) .008
1 (Low) 83.1% (n=628) 70.8% (n=182) .001
All 88.0% (n=3483) 84.4% (n=1086) .002
![Page 23: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/23.jpg)
Positive Impact on Graduation
U101 (all) Non-101 (all) U101 (lowest ability)
Non-101 (lowest ability)
25%30%35%40%45%50%55%60%65%70%75% 71.1%
64.9%
56.9%
36.9%
5-Year Graduation Rates(2007 Cohort)
![Page 24: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/24.jpg)
Need to Disaggregate
Disaggregate data by input variables Predicted GPA
SAT/ACT High School grades
Race/Ethnicity Family Income (Pell eligible) First Generation
![Page 25: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/25.jpg)
Outcome(O)
College(E)
Student
(I)
High ImpactPractic
e
How to do it
WELL
Disaggregating Environmental Variables
![Page 26: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/26.jpg)
![Page 27: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/27.jpg)
Which factors predict persistence?
Used FYI data set & included variables from student data file (persistence and GPA) 2,014 responses (72% response rate)
A series of logistic regressions were conducted Controlled for gender, race, and high school grades
A standard deviation increase in Sense of Belonging & Acceptance increased the odds of persisting into the second year by 38% (p < .001), holding all other variables constant.
![Page 28: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/28.jpg)
Assessing Educational Methods
Compare methods to determine if one approach is better than another
![Page 29: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/29.jpg)
Continual Improvement
Identifying and replicating best practices
![Page 30: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/30.jpg)
![Page 31: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/31.jpg)
Structural Variables
FACTCRA
P
![Page 32: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/32.jpg)
Fact or Crap?
Student Affairs professionals had higher ratings on overall course effectiveness than other instructors.
![Page 33: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/33.jpg)
No statistically significant differences were found on any of the fifteen FYI factors or course evaluation factors for Division of Student Affairs employees versus non-division employees.
CRAP
![Page 34: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/34.jpg)
Fact or Crap?
Sections that met 3 days a week (MWF) had significantly higher course effectiveness ratings than sections that met twice a week.
![Page 35: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/35.jpg)
CRAP
Mean (n)
p
2 Days Per Week
4.14 (2153)
.286
3 Days Per Week
4.19 (668)
Overall Course Satisfaction by Days Per WeekData from fall 2011 Course Evaluations
![Page 36: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/36.jpg)
Fact or Crap?
Sections with a Peer Leader had significantly higher course effectiveness ratings than sections without a Peer Leader.
![Page 37: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/37.jpg)
FACT
Overall Course Satisfaction by Peer Leader StatusData from fall 2011 Course Evaluations
Mean (n)
p
Peer Leader 4.22 (1585)
.004
No Peer Leader 4.05 (530)
![Page 38: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/38.jpg)
“Can’t fatten pig without weighing it”
Need to use assessment data to drive continual improvement
![Page 39: Using Action Research to Ensure Relevance and Excellence](https://reader036.vdocuments.mx/reader036/viewer/2022062315/56816005550346895dcf0701/html5/thumbnails/39.jpg)
Contact Information
University 101 Programs 1728 College Street
Columbia, South Carolina 29208(803) 777-6029www.sc.edu/univ101