nsse.iub.eduNational Survey of Student Engagement
An Experience-Based View of University Quality in the USA:
The National Survey of Student Engagement
Alexander C. McCormick
Center for Postsecondary Research
Indiana University Bloomington, USA
HEPI-HEA ConferenceThe British Academy, London
22 May 2012
• Context: Conceptions of quality in U.S. tertiary education
• The National Survey of Student Engagement (NSSE)
• Low stakes, high yield assessment• Student engagement trends• Controversies and cautions
Overview
nsse.iub.eduNational Survey of Student Engagement
Context
Higher education in the USALimited understanding of quality
Student engagement as a new approach
nsse.iub.eduNational Survey of Student Engagement
Higher Education in the USA
No national system of higher education Autonomy of institutions Diversity of institutions 4,600 degree-granting tertiary institutions Public and private Universities (undergraduate and
postgraduate) Colleges (undergraduate only)
• 4-year bachelor’s degree• 2-year associate’s degree (community colleges)
nsse.iub.eduNational Survey of Student Engagement
Higher Education in the USA (2)
Diversity of students Full- and part-time Traditional college-age Older and returning students Widely varying levels of preparation
nsse.iub.eduNational Survey of Student Engagement
What Do We Really Know About University Quality?
nsse.iub.eduNational Survey of Student Engagement
Quality Assessment in US Higher Education
Accreditation Limited public understanding Limited disclosure Traditional emphasis on capacity and resources Limited attention to teaching & learning (until
recently)
Newsmagazine rankings Emphasize reputation, resources, other inputs
Research rankings Blind to undergraduate education
nsse.iub.eduNational Survey of Student Engagement
Origins of the “National” Survey of Student Engagement (NSSE)
A reaction to the dominant discourse about university quality in the USA
Shift the focus to teaching and learning
Promote evidence-based improvement
nsse.iub.eduNational Survey of Student Engagement
The Big Idea
Ask students about their experience
Focus on behavior: empirically confirmed effective practices
nsse.iub.eduNational Survey of Student Engagement
Why a Student Survey?
Direct assessment of learning is problematic Measurement challenges Inferential challenges Cost challenges Motivation challenges
Survey approach offers advantages Cost-effective and efficient Diagnostic utility of behavioral data Students are the experts on their experience
nsse.iub.eduNational Survey of Student Engagement
NSSE’s Key Aims
Enrich the impoverished national discourse about university quality Shift the focus to teaching & learning,
through the lens of effective practices
Provide diagnostic information that can be used to improve undergraduate education Actionable information based on
valid & reliable measures Meaningful comparisons to other institutions
nsse.iub.eduNational Survey of Student Engagement
What is Student Engagement?
The extent to which students engage in and are exposed to effective educational practices Activities and practices
known to be related to desired outcomes
nsse.iub.eduNational Survey of Student Engagement
OK, but what is Engagement?
Challenging academic work
Active learning Deep approaches to learning
High-impact practicesInvolvement with instructors
In a campus context that
promotes and supports success
nsse.iub.eduNational Survey of Student Engagement
Conceptual & Empirical Foundations
Time on task (Tyler, 1930s)
Quality of effort (Pace, 1960s & 70s)
Student involvement (Astin, 1984)
Academic & social integration (Tinto, 1975 & 1987)
Good practices in undergraduate education (Chickering & Gamson, 1987)
College impact (Pascarella, 1985)
Student engagement (Kuh, 1991, 2005)
nsse.iub.eduNational Survey of Student Engagement
Why Does Engagement Matter?
The impact of college is largely determined by individual effort and involvement in the academic, interpersonal, and extra-curricular offerings on a campus. It is important to focus on the ways an institution can shape its offerings to encourage student engagement.
Paraphrased from Pascarella & Terenzini (2005), p. 602
How Does it Work?
What Do We Ask?
nsse.iub.eduNational Survey of Student Engagement
How NSSE Works
• Paper or Web-based survey• First- & final-year
undergraduates• Uniform, centralized survey
administration• Institutions receive detailed
reports and an identified student data file
• Customizable comparison groups
• Results are confidential• Supported by institution fees
nsse.iub.eduNational Survey of Student Engagement
Participation
550 to 770 U.S. & Canadian institutions per year
About 1,500 since NSSE’s launch in 2000
Most repeat every 1-4 years
nsse.iub.eduNational Survey of Student Engagement
Survey Content
Academic activities & experiences• Academic
challenge• Reading and
writing• Active learning• Cognitive tasks in
courses• Enriching
educational experiences
Time use Study, work,
socializing…
Co-curricular activities
Institutional emphases
Quality of campus relationships
Cognitive and noncognitive gains
Satisfaction Demographic & enrollment characteristics
nsse.iub.eduNational Survey of Student Engagement
Sample Questions
Asked questions in class or contributed to class discussions?
Made a class presentation?Prepared two or more drafts of a paper or assignment before turning it in?
In your experience at your institution during the current school year, about how often have you…
nsse.iub.eduNational Survey of Student Engagement
Sample Questions
Put together ideas or concepts from different courses when completing assignments or during class discussions?
Discussed ideas from your readings or classes with faculty members outside of class?
In your experience at your institution during the current school year, about how often have you…
nsse.iub.eduNational Survey of Student Engagement
Sample Questions
Received prompt written or oral feedback from faculty on your academic performance?
Worked harder than you thought you could to meet an instructor’s standards or expectations?
In your experience at your institution during the current school year, about how often have you…
nsse.iub.eduNational Survey of Student Engagement
Sample Questions
Memorising facts, ideas, or methods…
Analysing the basic elements of an idea…
Synthesising and organizing ideas, information…
Making judgments about the value of information…
Applying theories or concepts…
During the current school year, how muchhas your coursework emphasized the following mental activities?
nsse.iub.eduNational Survey of Student Engagement
Sample Questions
Examined the strengths or weaknesses of your own views on a topic or issue?
Learned something that changed the way you understand an issue or concept?
During the current school year, about how often have you done each of the following?
nsse.iub.eduNational Survey of Student Engagement
Using NSSE
Results
How are we doing?
How can we improve?
nsse.iub.eduNational Survey of Student Engagement
Using NSSE Results
Diagnostic purpose: analysing the undergraduate experience
Identify strengths and weaknesses, areas where improvement is needed
Stimulates conversations about quality
Informs decision-making with “actionable” information
nsse.iub.eduNational Survey of Student Engagement
Many Ways to Use Results
Peer comparisons How do we measure
up to others?
Self comparison Are we improving?
Absolute standard How do we measure
up to our ideals?
Internal variation Who is least
engaged?
What to look at and how to look at it
NSSE results as a catalyst for campus discussions
about teaching & learning
nsse.iub.eduNational Survey of Student Engagement
Comparing Student andInstructor Perceptions
First-year students
Instructors Final-year students
Instructors0%
20%
40%
60%
80%
100%
69%
33%
62%
23%
Courses emphasised memorisation “Quite a bit” or “Very much”
Faculty Survey
nsse.iub.eduNational Survey of Student Engagement
Comparing Student andInstructor Perceptions
First-year students
Instructors Final-year students
Instructors0%
20%
40%
60%
80%
100%
69%
33%
62%
23%
Courses emphasised memorisation “Quite a bit” or “Very much”
Results at institutions with
several NSSE administrations show many with positive trends
nsse.iub.eduNational Survey of Student Engagement
Summary of Findings
• 41% of institutions demonstrated a pattern of improvement in at least one measure for first-year students
• 28% did so for final-year students
• Trends found across many institutional types Control (public & private), Size, Research emphasis
• Ratio of positive to negative trends: 7:1
nsse.iub.eduNational Survey of Student Engagement
Overview of Observed Changes: FY
Diversity Exp.
High-Impact
Integrative
Higher-Order
Supp. Campus
Stu-Faculty Int.
Active & Collab.
Academic Chall.
-10 0 10 20 30 40 50 60 70 80 90 100
Percentage with positive or negative trends
PositiveNegative
nsse.iub.eduNational Survey of Student Engagement
Overview of Observed Changes: Sr
Diversity Exp.
High-Impact
Integrative
Higher-Order
Supp. Campus
Stu-Faculty Int.
Active & Collab.
Academic Chall.
-10 0 10 20 30 40 50 60 70 80 90 100
Percentage with positive or negative trends
PositiveNegative
nsse.iub.eduNational Survey of Student Engagement
Cautions & Controversies
NSSE is no “magic bullet”The risks of high-stakes uses
“Participation” vs. “use”
nsse.iub.eduNational Survey of Student Engagement
Cautions
NSSE is no “magic bullet”: Best used in combination with other information
NSSE’s research foundation is based on studies of US university students
Most variation is between students, not institutions
nsse.iub.eduNational Survey of Student Engagement
Controversies
Confidentiality of results: Institutions control disclosure
Attaching high stakes to NSSE is risky Institutions will decline to participate if
public relations cost > institutional benefit Reliability of responses could be
compromisedMere participation is not sufficient Results must be analysed and used
nsse.iub.eduNational Survey of Student Engagement
Understanding Improvement
Organizations that improve… create and nurture agreement on what is worth achieving, and they set in motion the internal processes by which people progressively learn how to do what they need to do in order to achieve what is worthwhile.
(Elmore in Fullan, 2001)
Thank you.