First-Year STEM Student Cohorts:Assessment and Best Practices

• Janet Callahan, Doug Bullock, Gary Hunt – Boise State University;

• Edmund Tsang– Western Michigan University

Session Structure

• Introduction – Callahan • Part I: Building Large Cohorts – Tsang • Part II: Assessing Cohorts – Bullock • Wrap-up – Hunt

Cohorts – Introduction

• Why do we create cohorts?– Improved retention• Tinto: social and academic engagement

– Social: organized social activities; informal– Academic: performance; interactions with faculty & staff– Campus: resource connections

– Enrollment logistics• Learning communities can help campuses with

managing course offerings

Cohorts, examples

• Living Learning Communities• Bridge programs• Learning communities• UROPs (undergraduate research

opportunities programs)

Social Learning Theory: COPs• Communities of practice are formed by people who engage in

a process of collective learning in a shared domain of human endeavor: a tribe learning to survive, a band of artists seeking new forms of expression, a group of engineers working on similar problems, a clique of pupils defining their identity in the school, a network of surgeons exploring novel techniques, a gathering of first-time managers helping each other cope. In a nutshell:– Communities of practice are groups of people who share a concern

or a passion for something they do and learn how to do it better as they interact regularly.

http://www.ewenger.com/theory/

Communities of Practice(as Tania Siemens expressed last night)

• COP are groups of people (community)• Who share a concern or a passion for

something they do (domain)• And learn how to do it better (practice).

It seems to me that we’re trying to create COPs with most freshmen cohorts

Engineering Residential College

Questions this session addresses

– Logistics associated with large university creation of cohorts;

– Assessment: how does an institution or program know if their cohort program is cost effective?

Building and Assessing Learning Community/Cohorts

Since evidence shows learning community/cohort is a powerful strategy to support student success and retention• Ideally should include as many students as possible

Logistics of building LC involving several hundred students (particularly if they have diverse academic preparation background)

WMU/CEAS Objectives – students in a cohort take the same 3-5 courses together in fall semester and 2-4 courses in spring semester of their first year

• Constrained to small number of students Selection criteria are important – how to get the biggest impact Avoid “false positive” results

Highly Motivated Moderately Motivated Not Motivated

(Well/Moderately/ (Well/Moderately/ (Well/Moderately/

Poorly Prepared) Poorly Prepared) Poorly Prepared)

Build Cohorts Involving ~400 Students at WMU

Constraints at WMU/CEAS• Entering 1st-year CEAS students place into Algebra II (~25%),

Precalculus (~30%), and Calculus I (~45%) • 13 engineering and applied sciences program organized into 6

departments• No common first-year curriculum in CEAS• CCE, ECE, CHEG, PAPR have their own 1st-semester course • Mathematics, chemistry, physics courses outside CEAS and

they do not want to “zero-out” any courses/labs

What are the constraints on your campus?

Build Cohorts Involving ~400 Students at WMUFactors to consider in building cohorts • Chemistry lab sections have capacities of 24 students/lab• Math sections have capacity of 40 students/section• Physics lab sections have capacities of 20 students/lab• Engineering Graphics lab sections have capacity of 24

students/lab• Technical Communication sections have capacity of 24

students/section

What are the factors on your campus to consider?

Build Cohorts Involving ~400 Students at WMUStrategies to build cohorts at WMU• # students/cohort = 24 based on section size of technical

communication/engineering graphics lab, chemistry lab (take 12 seats from 2 labs) and other labs, and mathematics

• Importance of an anchor class• Use admit and historic yield data, as well as MATH ACT sub-

score to project # seats required• Submit seat requests in January for summer orientation/fall

registration; early October for spring registration; complete iterations in February/late October

• Keep departments informed of seat usage; release unused seats immediately

What strategies would work best on your campus?

Build Cohorts Involving ~400 Students at WMUExamples of cohort course templates

Build Cohorts Involving ~400 Students at WMUExamples of cohort course templates

Build Cohorts Involving ~400 Students at WMUExamples of cohort course templates

Build Cohorts Involving ~400 Students at WMU

Lessons learned• Placing students in cohorts has become standard practice in

CEAS summer orientation/registration since 2005, shifting focus of summer orientation from course registration to learning more about their programs and how to be successful

• Captures 85-90% of students (not placed in a cohort are students with many AP credits; athletes; students who opt out)

• Math, chemistry, and physics are very supportive of the practice reduces uncertainty because they know which sections will make

Part II – Assessing CohortsDoug Bullock, Boise State University

read & brainstorm assessment strategies;

rotate papers; discuss; select one or two that your group would implement

report out

Wrap-upGary Hunt, Boise State University

A specific cohort;how it is being assessed.

What we did --Stem Summer Adventure

• August 2012• 5 day extended field program

Why?

• Goal: to increase first year retention, leading to increased STEM graduation rates

• The lever we are pulling: Build community, increase student engagement into campus

• Hypothesis: Incoming freshmen STEM students who participate in the summer adventure and the associated seminar class will be more integrated into the university than the average student.

Treatment:

• 5 days/4 nights of rafting, camping, social and curricular activities– Team building and social activities led by the

outdoor program– Science curricular activities -- faculty lead with

volunteers• Weekly seminar class in fall

How to measure success?

• NSSE-like question survey (national survey of student engagement)

• What is NSSE? – Nsse.lub.edu

How to measure success?

• Qualitative: student feedback in form of student videos and question responses

• Quantitative: NSSE question survey (national survey of student engagement) – BSU had administered NSSE– We had aggregate scores for the campus to

compare with– Students in the summer adventure program took

survey at the end of their first semester

Qualitative Feedback

• Typical student responses to:– What did you like the best about this trip?– What needs improvement?– Would you recommend this trip to a friend?

http://www.youtube.com/watch?v=PAq6Cpzxqag&feature=plcp

Quantitative results

• Selected NSSE questions were used to measure student engagement

• Statistical results showed that Cohort participants showed greater engagement. The result was significant.

• χ2 (3, N = 12) = 8.9, p <.01

Discussion:

• What would you do to assess this program?• What do you like about this program?• What are you concerned about?• What is your best take away from this session?