empowering faculty to improve students’ learning through ... · empowering faculty to improve...
TRANSCRIPT
Empowering Faculty to Improve Students’ Learning through Collaborative Assessment:
An Implementation Workshop for Faculty and Program Administrators
Come with a challenge
Work together 2:00 – 2:30 Presenters
2:30 – 3:30 Tables Work
3:30 – 3:45 Break, Independence Foyer
3:50 – 4:30 Tables Work
4:30 – 4:45 Whole Group Convenes
4:45 – 4:50 Conclusion, Next Steps
Leave with a plan
Faculty/instructional development tool …
• By, of, for instructors: formative vs. summative
Appendix B
• Patterns of beneficial teaching/learning methods
– Discipline, level, size, demographics (Appdx A)
Example: transparency around assignment design
… broader uses?
Discuss assignments' learning goals and design rationale before students begin work
• Chart out the skills students will practice in each assignment example
• Begin each assignment by defining the learning benefits to students: skills practiced, content knowledge gained
example
• Provide criteria for success in advance example 1 Business U of Illinois;
example 2 research paper U of Baltimore;
example 3 lab report Iowa State U
• Offer examples of successful work, and annotate them to indicate how criteria apply
example, plant Bio, U of Illinois
TEMPLATE: SAMPLE BUSINESS COURSE ASSIGNMENT due November 6
Purpose: Skills: The purpose of this assignment is to help you practice the following skills that are essential for your success in this course and in any business context:
– using information technology – analysis of diverse, multicultural, global economic contexts – assessing professional, ethical and legal aspects – judging and selecting best solutions
Knowledge: This assignment will also help you to become familiar with a particular
multicultural business context.
Task: Use the material in the video and the website to illustrate what you learned about subsistence
contexts. In particular, discuss the types and nature of challenges that individuals have to face in different areas such as food, etc. Suggest some ideas that businesses could pursue to help people living in poverty while also making a profit.
Criteria for Success: A successful assignment will have the following characteristics: – 250 written words maximum with at least 3 video clips (under 4 minutes total) – Comparison of at least two contexts – Identification of ethical dilemma or ethical considerations – Evidence to support why your suggestion could work
Project Scope
Since 2010, this initiative has involved
31000+ students
200+ courses (varied disciplines, levels)
24 higher ed institutions
(cc, public/private, large research, small college)
6 countries US, Mexico, Scotland, Australia, Indonesia, India
Voluntary Response Rates
Response rates for US students only (2012)
Course Size
category
Total
Enrollmt
Total N Avg Rate
of Return
(1)= or < 30 629 320 50.9%
(2) 31-65 1223 448 36.6%
(3) 66-299 3416 787 23.0%
(4) = or > 300 9841 862 8.8%
Benchmarks:
Our Race/Ethnicity distribution ≅ representative
White 61%
Black 4%
Hispanic 5%
Asian 17%
Other 7%
Unknown 6%
US Institutions in Transparency Sample 2009-2012
White 62%
Black 15%
Hispanic 14%
Asina/Pac Isander
6%
Two or more 2%
Native Am 1%
US Institutions in 2010
Benchmarks:
Our UG control group doesn’t overestimate
its ability to learn
Question Total Women Men
PSRI
Strongly agree: I developed
increased ability to learn from
diverse perspectives during college
52.6% 56.6% 48.1%
Current (TILT) Study
More likely (Much more likely): to
consider different opinions or points
of view as result of taking this
course
50.7%
(11.4%)
48.5%
(9.1%)
52.2%
(13.1%)
Benchmarks: Our UG control group doesn’t over-estimate
its learning
Question Scales (NESSE 4-pt / TILT 5-pt)
NSSE During the current
year, how much has your
coursework emphasized:
Very
Little Some
Quite
a Bit
Very
Much
Analyzing 2% 16% 44% 38%
Application 4% 20% 39% 38%
TILT How much has this
course helped you in: Not at
All
A
Little
Moderate
Amt A Lot
Great
Deal
Analyzing 6% 18% 36% 28% 12%
Application 6% 18% 32% 30% 14%
Benefits for underrepresented students: Are you better or worse at recognizing when you need help
with your academic work?
Mean Responses for students in intro Physical Science courses
Enrollment Size 66-299
1
2
3
4
5
Caucasian Non-Caucasian
Me
an R
esp
on
se
Race/Ethnicity
Control "Understanding" "Assignments" + "How Learn"
Faculty development tool … broader uses?
Institutions:
First-year student success/retention
Gregory Brown, Vice Provost for Faculty, Policy & Research
Advising / Student Success
Administrative and faculty advisors
Gayle Juneau, Executive Director, Academic Advising
Programs:
General Education first-year seminars
Jennifer Keene, Associate Dean, College of Liberal Arts
Worksheet
• Challenge / goal
• Existing data / future measure of success
• Strategic planning
– Timeframe, plan what happens when
– Barriers
– Resources you have
– Resources you need
Gregory Brown
Vice Provost for Faculty, Policy and Research
Challenge / goal
Retain more first-year students
Existing data: over 1/5 of 1st year class stops (700+ students)
Assignment design rationale
benefits 1st-yr all disciplines
Student Progression
Nu
mb
er
of
Stu
de
nts
0
500
1000
1500
2000
2500
3000
3500
EnteringFreshmen
Progressionrates
GraduatingCohort
24Credits
78%
60Credits
74%
75Credits
64%
90Credits
91%
Goal: Retain More First-year Students
Data: future measures of success
• Numbers of 1st year students; rising sophomores
• Transparency survey questions re: 1st year UULOs
5-10, 16. communication, application, evaluation
19. recognizing when you need help
17. confidence to succeed in school
Strategic planning
– Timeframe: 2d sem large enrollment and FYS instructors modify assignment design in test group
– Barriers: survey via FYS course evaluations
– Resources: workshops, data gathering, analysis
Gayle Juneau,
Executive Director of Academic Advising
Challenge / goal
Administrative and academic advisors collaborate more
Connect more with students’ classroom experiences
Existing data
1st-year student retention rate
Did advisor direct you to helpful resources? (sooner?)
Assignment design rationale benefits 1st-yr all disciplines
Measures of Future success
Transparency survey ?s:
19, 21. are you better/worse recognizing when you need help
23. understand what constitutes successful work
17, 18. confidence to succeed in school/field
Goal: advisors collaborate, connect more
with students’ classroom experiences
Strategic planning
Timeframe: 2d semester, ongoing
College-based workshops for advisors: Coaching students: strategies to ID successful course work
Coaching students: recognize sooner when need help / how to get it
Barriers:
Lack precedent for prof’l & academic advisors studying classroom data
Advantage?: Equal footing (unfamiliar) for both groups
Resources: workshops, data gathering, analysis
Jennifer Keene,
Associate Dean, College of Liberal Arts
Challenge / goal In non-standardized, discipline-based First Year Seminars,
improve students’ writing
Existing data
1st-year retention rate in College of Liberal Arts
Grades, baseline assignments: varied
Measures of Future success
Assignment design rationale benefits 1st-yr all disciplines
Survey ?s: 4. How much has this course helped you in writing effectively
8. How much … improving your ability to separate and examine
the pieces of an idea, experience or theory (Bean, Colomb)
23. How well do you understand what constitutes successful work …
Goal: improve students’ writing
Strategic planning
Timeframe: 2d sem, ongoing
workshops for instructors to modify assignment design
gather, analyze data
Barriers:
Adjunct population includes eager and unwilling
Advantage?: test group and control group
Scheduling: blended online/onsite format for workshops
Resources: workshops, data gathering, analysis
Table Groups
10 minutes
• Introductions (1 minute per person)
Pairs discuss, record on worksheet:
GOALS, MEASURES
20 minutes
• Your challenge / goal (5 m)
• Existing data, future measures of success (5m)
– National Transparency data that could inform (5m)
– Specific survey questions that could inform (5m)
Pairs discuss, record on worksheet:
STRATEGIC PLANNING
15 minutes
• Timeframe, simple plan (what happens when) (5 m)
• Barriers to success (5m)
• Resources you have / need (5m)
Break: Independence Foyer
3:45 pm Table groups reconvene
Individual Feedback
35 – 40 minutes (5 min per table member)
Briefly identify, gather feedback on:
– Goal
– Measures/data
– Timeframe, plan (what happens when)
– Resources
– How Transparency project can adapt to help
General Advice
Common insights from groups:
– Goal
– Measures/data
– Timeframe, plan (what happens when)
– Resources
– How Transparency project can adapt to help