use of the canadian graduate & professional student satisfaction survey: a local approach

27
Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach Joan Norris, Keith Flysak & Michael Bittle Faculty of Graduate & Postdoctoral Studies

Upload: linnea

Post on 24-Feb-2016

74 views

Category:

Documents


0 download

DESCRIPTION

Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach Joan Norris, Keith Flysak & Michael Bittle Faculty of Graduate & Postdoctoral Studies. CGPSS Intent: to investigate sources and levels of satisfaction among enrolled graduate students - PowerPoint PPT Presentation

TRANSCRIPT

Page 1: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Joan Norris, Keith Flysak & Michael BittleFaculty of Graduate & Postdoctoral Studies

Page 2: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

CGPSS• Intent: to investigate sources and

levels of satisfaction among enrolled graduate students• In both research-intensive and

professional programs

Page 3: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach
Page 4: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Why measure satisfaction?HEQCO perspective (Spence, 2009; Zhao, 2012):• Better understanding of graduate level education

processes;• Comparative analyses;• Provincial & national portraits of graduate education

with insights into funding, completion, institutional infrastructure & other areas of improvement;

• Promoting relevant changes & appropriate adaptations to maintain a competitive international edge.

Page 5: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

But keep in mind the limitations of the CGPSS:• Survey development not systematic:

• Many sources cited (an informal group of grad deans from Rutgers, Duke, Stanford; adopted and revised by MIT, Western & G13).•Despite its origins, not often used in the U.S.•Decision rules re: category and question choice unclear (“anointed correct”)•Reliability and validity unknown (although factor analyses have been carried out).•Different versions of the survey administered, so a true-cross sectional analysis difficult.

Page 6: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach
Page 7: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

And be cautious:• Respondents’ answers to any measure of

“satisfaction” may be influenced by:• affective state; current context, future

expectations, past events and social comparisons

• Findings will be also affected by• Sample size restrictions, bias, missing data,

rewards and incentives to participants

Page 8: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Our goals at Laurier:• Examine stability of positive findings regarding faculty

mentoring and teaching strength;• Evidence for improvement in areas identified by first

two administrations;• Opportunities & challenges in individual programs;• Provide information for cyclical reviews, integrated

budgeting and planning exercise, strategic enrolment management;

• Benchmarking across similarly sized institutions.

Page 9: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach
Page 10: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Measures and Indices (HEQCO):General AssessmentGeneral SatisfactionBenchmarks of Satisfaction

Page 11: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

General Assessment:How would you rate the quality of--• your academic experience at this university?• your student life experience at this university?• your graduate/professional program at this

university?• your overall experience at this university?

Page 12: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

General Satisfaction:• If starting over, select same university?• If starting over, same field of study?• Would you recommend this university to

someone considering your program?• Would you recommend this university to

someone in another field?• If starting over, select same faculty supervisor?

Page 13: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Benchmarks of Satisfaction:(items selected from factor analyses by G13)• Quality of Teaching (3 items)• Opportunities to Present and Publish (5 items)• Research Training and Career Orientation

(9 items)• Supportive Dissertation Advisor (12 items)

Page 14: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach
Page 15: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Our analyses have included:• Frequencies (provided by Mosaic)• Snapshots of each administration• Development of unique indices• modelling• Cross-sectional analyses of indices• Program profiles and scorecards

Page 16: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Cross-sectional analyses:Variables:• Composite general assessment index• Composite satisfaction index• Four benchmark indices:

• Quality of Teaching• Opportunities to Present and Publish• Research Training and Career Orientation• Supportive Dissertation Advisor

Page 17: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Cross-sectional analyses (2007, 2010, 2013) with comparisons to mid-size and consortium groups (one-way ANOVAS with post-hoc comparisons)• Response rates: approx. 40% for research intensive

programs & 25% for professional programs at each administration

• Separately for master’s and doctoral students:• Could not separate master’s because of changes to the

survey•2007: with/without thesis•2010: regular/professional•2013: research & coursework streams/professional

Page 18: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Snapshot results:• Areas of strength include benefits of a small institution:

high quality faculty mentoring and teaching• Implications for expansion

• Areas of need included extracurricular training opportunities• Development of professionalization suite of workshops,

seminars, courses (ASPIRE) • Co-curricular record

Page 19: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Cross-Sectional Results:• Satisfaction ratings consistent with same-sized

universities in the consortium• Overall high quality maintained in the context

of rapid expansion (doubling of programs)

Page 20: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

• Differences in “general” measures often difficult to detect

• Persona and Program profiles and scorecards may be more useful :• Contribute to Strategic Enrolment Management

project•Developed persona groups: professional master’s, research intensive master’s, doctoral

• Individual program results provide insights into quality enhancement.

Page 21: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Supplemented satisfaction scores with:• Student demographics (e.g., age,

citizenship/visa status, gender, Canadian geographic area (KW, rest of ON, QC, East, West, North)

• Nonenrolment survey results • Admissions conversion scores: efficiency rates

Page 22: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Analyses of Variance in Persona GroupsUsing Satisfaction Indices

Page 23: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Research master’s persona group:Supervision satisfaction strengthening; publishing/presenting opportunities need more attention

Academic E

xperie

nce

Student L

ife

Program Exp

erience

Overal

l Exp

erience

General Sa

tisfacti

on

Teaching Q

uality

Present/Publish

Research Trai

ning

Supervi

sor

0

1

2

3

4

200720102013

Page 24: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Professional master’s persona group

Academic E

xperie

nce

Student L

ife

Program Exp

erience

Overal

l Exp

erience

General Sa

tisfacti

on

Teaching Q

uality

0

1

2

3

4

200720102013

Page 25: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Doctoral persona group:Significant improvement in Student Life & Quality;Supervision strong; presentation/publication opportunities need attention.

0

1

2

3

4

5

200720102013

Page 26: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Final thoughts:• Pressure to assess student views of their graduate

experience will remain;• Satisfaction surveys provide one useful, but limited,

means of assessment;• CGPSS will continue to develop as an assessment

method;• Benchmarking may be helpful, but within-institution

scorecards more likely to lead to quality improvement.

Page 27: Use of the Canadian Graduate & Professional Student Satisfaction Survey: A Local Approach

Some things do improve over time!They look pretty satisfied…