Using National Using National Studies of Student Studies of Student

Engagement to Engagement to Support Institutional Support Institutional

ChangeChangeNathan Marti, CCSSE Nathan Marti, CCSSE

Todd Chamberlain, NSSETodd Chamberlain, NSSEFAIR ConferenceFAIR Conference

June 23, 2004June 23, 2004

What What ReallyReally Matters in Matters in

College:College: Student Student EngagementEngagement

““The research is unequivocal: The research is unequivocal: students who are actively students who are actively involved in both academic and involved in both academic and out-of-class activities gain more out-of-class activities gain more from the college experience than from the college experience than those who are not so involved.”those who are not so involved.”

Ernest T. Pascarella & Patrick T. Ernest T. Pascarella & Patrick T. Terenzini, Terenzini, How College Affects How College Affects StudentsStudents

Lessons from the Lessons from the ResearchResearch

What matters most to desired What matters most to desired outcomes is what students outcomes is what students dodo, , not who they arenot who they are

A key factor for student A key factor for student learning is the learning is the quality of effortquality of effort students devote to students devote to educationally purposeful educationally purposeful activitiesactivities

National Survey of National Survey of Student Student EngagementEngagement(pronounced “nessie”)(pronounced “nessie”)

Community College Community College Survey of Student Survey of Student EngagementEngagement(pronounced “sessie”)(pronounced “sessie”)

College student surveys that College student surveys that assess the extent to which assess the extent to which students engage in students engage in educational practices educational practices associated with high levels of associated with high levels of learning and developmentlearning and development

Penetrating NSSE/Penetrating NSSE/CCSSECCSSE FindingsFindings

Overview of ReportsOverview of Reports Start Broad, then Dig Start Broad, then Dig

DeeperDeeper Identify Significant ItemsIdentify Significant Items Focus on What is Important Focus on What is Important

to to YOURYOUR Institution’s Institution’s PrioritiesPriorities

Customized Institutional Customized Institutional ReportReport

Respondent Respondent CharacteristicsCharacteristics

National National benchmarksbenchmarks

Institutional dataInstitutional data Means and Means and

frequenciesfrequencies SubpopulationsSubpopulations Comparisons by Comparisons by

Carnegie/size, Carnegie/size, nationalnational

Respondent Respondent CharacteristiCharacteristi

cscs

Respondent Respondent Characteristics:Characteristics:

Does This Represent Your Does This Represent Your Campus?Campus? Response RateResponse Rate

Sampling ErrorSampling Error Student CharacteristicsStudent Characteristics

GenderGender Race/EthnicityRace/Ethnicity Class LevelClass Level Enrollment StatusEnrollment Status

Comparisons by Carnegie/size, Comparisons by Carnegie/size, national and consortium when national and consortium when relevantrelevant

Benchmarks: High Level Benchmarks: High Level Views of the DataViews of the Data

NSSE/NSSE/CCSSECCSSE developed five indicators, or benchmarks, to represent the multi-dimensional developed five indicators, or benchmarks, to represent the multi-dimensional nature of student engagement at the institutional, sector and national levelsnature of student engagement at the institutional, sector and national levels

Level of Academic Challenge

Active & Collaborative

Learning

EnrichingEducational Experiences/Student Effort

SupportiveCampus

Environment/Support for

Learners

Student-Faculty Interaction

Benchmark Benchmark ReportReport

Institution Consortium Carnegie Classification All NSSE Institutions

Nesseville State AAUDEDoc/Res-Extensive

National

First-Year 57.3 54.9 51.4 52.9Senior 57.3 56.2 54.6 56.6

Benchmark Scores

NSSE 2001 Institutional Benchmark Report

Nesseville State University

Level of Academic Challenge

20

30

40

50

60

70

80

First-Year Senior

Nesseville State

AAUDE

Doc/Res-Extensive

National

Level of Academic Challenge Items:

Preparing for class (studying, reading, writing, rehearsing, and other activities related to your academic program)

Number of assigned textbooks, books, or book-length packs of course readings

Number of written papers or reports of 20 pages or more

Number of written papers or reports of between 5 and 19 pages

Number of written papers or reports of fewer than 5 pages

Coursework emphasizes: Analyzing the basic elements of an idea, experience or theory

Coursework emphasizes: Synthesizing and organizing ideas, information, or experiences into new, more complex interpretations and relationships

Coursework emphasizes: Making judgments about the value of information, arguments, or methods

Coursework emphasizes: Applying theories or concepts to practical problems or in new situations

Worked harder than you thought you could to meet an instructor's standards or expectations

Campus environment emphasizes spending significant amounts of time studying and on academic work

The NSSE survey, The College Student Report, measures student engagement in many important activities that research studies show are positively related to learning and personal development. Forty-one questions from the survey are assigned to five clusters of similar activities and conditions to make up the national benchmarks of effective educational practice. The benchmarks are created on 100-point scales to make it easier to compare performance within and across sectors and institutional types.

These benchmarks are: (1) level of academic challenge, (2) active and collaborative learning, (3) student interactions with faculty members, (4) enriching educational experiences, and (5) supportive campus environment. The NSSE benchmark analysis is based on more than 105,000 randomly selected students at 467 four-year colleges and universities that participated in the spring of 2000 or 2001. The students represent a broad cross-section of first-year and senior students from every region of the country. The institutions are similar in most respects to the universe of four-year schools. More detailed information about the benchmarks can be found in the national report that was sent with this mailing and on the NSSE website at www.iub.edu/~nsse.

This report summarizes your institution’s performance in these five areas of effective educational practice. Your institution’s benchmark scores are presented and compared to schools in your consortium, your Carnegie Classification, and the NSSE national norms. Page 4 provides some additional information, including a standard score that represents the magnitude of the difference between your institution's score and the respective comparison group, and page 5 presents a table of National and Carnegie classification deciles against which you can gauge the relative performance of your institution on each of the benchmarks.

Mean Mean Summary Summary

ReportReport

FrequencyFrequencyDistributioDistributio

nnReportReport

3 1.8% 125 3.2% 1030 3.0% 3 1.8% 96 2.2% 629 1.7%

46 28.0% 1648 41.6% 13110 38.7% 41 25.0% 1364 31.2% 10244 27.3%

59 36.0% 1350 34.1% 11659 34.4% 53 32.3% 1439 32.9% 12245 32.6%

56 34.1% 839 21.2% 8062 23.8% 67 40.9% 1478 33.8% 14433 38.4%

164 100.0% 3962 100.0% 33861 100.0% 164 100.0% 4377 100.0% 37551 100.0%

14 8.6% 692 17.5% 5617 16.6% 4 2.4% 208 4.8% 1454 3.9%

106 65.0% 2223 56.2% 18751 55.4% 60 36.6% 1597 36.5% 13514 36.0%

34 20.9% 802 20.3% 7386 21.8% 69 42.1% 1619 37.0% 14027 37.4%

9 5.5% 240 6.1% 2072 6.1% 31 18.9% 947 21.7% 8532 22.7%

163 100.0% 3957 100.0% 33826 100.0% 164 100.0% 4371 100.0% 37527 100.0%

24 14.7% 587 14.8% 4444 13.1% 26 16.0% 730 16.7% 5930 15.8%

51 31.3% 1141 28.9% 10047 29.7% 69 42.3% 1658 37.9% 14151 37.7%

43 26.4% 1229 31.1% 10665 31.5% 38 23.3% 1172 26.8% 10021 26.7%

45 27.6% 996 25.2% 8674 25.6% 30 18.4% 811 18.6% 7422 19.8%

163 100.0% 3953 100.0% 33830 100.0% 163 100.0% 4371 100.0% 37524 100.0%

2 1.2% 149 3.8% 870 2.6% 2 1.2% 54 1.2% 365 1.0%

42 25.9% 944 23.9% 8002 23.7% 17 10.4% 641 14.7% 4820 12.8%

63 38.9% 1665 42.1% 14725 43.5% 66 40.2% 1708 39.1% 14703 39.2%

55 34.0% 1194 30.2% 10215 30.2% 79 48.2% 1968 45.0% 17629 47.0%

162 100.0% 3952 100.0% 33812 100.0% 164 100.0% 4371 100.0% 37517 100.0%

28 17.2% 647 16.4% 5805 17.2% 20 12.2% 640 14.6% 5554 14.8%

98 60.1% 2455 62.1% 20928 61.9% 105 64.0% 2690 61.6% 23113 61.6%

26 16.0% 634 16.0% 5114 15.1% 30 18.3% 738 16.9% 6244 16.7%

11 6.7% 219 5.5% 1952 5.8% 9 5.5% 302 6.9% 2589 6.9%

163 100.0% 3955 100.0% 33799 100.0% 164 100.0% 4370 100.0% 37500 100.0%

10 6.1% 446 11.3% 3502 10.4% 20 12.2% 433 9.9% 3467 9.2%

89 54.3% 1856 47.0% 15959 47.2% 70 42.7% 2043 46.8% 16938 45.2%

52 31.7% 1272 32.2% 10864 32.1% 55 33.5% 1257 28.8% 11671 31.1%

13 7.9% 376 9.5% 3474 10.3% 19 11.6% 632 14.5% 5406 14.4%

164 100.0% 3950 100.0% 33799 100.0% 164 100.0% 4365 100.0% 37482 100.0%

13 7.9% 588 14.9% 4831 14.3% 12 7.3% 304 7.0% 2468 6.6%

82 50.0% 1805 45.6% 15839 46.8% 58 35.4% 1460 33.4% 13553 36.1%

51 31.1% 1136 28.7% 9752 28.8% 64 39.0% 1445 33.1% 12963 34.6%

18 11.0% 429 10.8% 3403 10.1% 30 18.3% 1161 26.6% 8525 22.7%

164 100.0% 3958 100.0% 33825 100.0% 164 100.0% 4370 100.0% 37509 100.0%

83 50.6% 2113 53.4% 18347 54.3% 65 39.6% 2029 46.5% 17002 45.4%

59 36.0% 1278 32.3% 10963 32.4% 54 32.9% 1527 35.0% 13146 35.1%

14 8.5% 385 9.7% 3101 9.2% 24 14.6% 502 11.5% 4234 11.3%

8 4.9% 178 4.5% 1399 4.1% 21 12.8% 306 7.0% 3099 8.3%

164 100.0% 3954 100.0% 33810 100.0% 164 100.0% 4364 100.0% 37481 100.0%

110 67.1% 2973 75.3% 24690 73.1% 80 48.8% 2792 64.0% 22203 59.3%

37 22.6% 711 18.0% 6518 19.3% 53 32.3% 1117 25.6% 10676 28.5%

12 7.3% 195 4.9% 1815 5.4% 21 12.8% 298 6.8% 2983 8.0%

5 3.0% 69 1.7% 742 2.2% 10 6.1% 157 3.6% 1575 4.2%

164 100.0% 3948 100.0% 33765 100.0% 164 100.0% 4364 100.0% 37437 100.0%

33 20.1% 775 19.6% 6605 19.5% 18 11.0% 580 13.3% 5402 14.4%

44 26.8% 1131 28.6% 9676 28.6% 59 36.0% 1290 29.5% 10892 29.0%

43 26.2% 1043 26.4% 9009 26.6% 36 22.0% 1187 27.1% 10274 27.4%

44 26.8% 1004 25.4% 8526 25.2% 51 31.1% 1317 30.1% 10953 29.2%

164 100.0% 3953 100.0% 33816 100.0% 164 100.0% 4374 100.0% 37521 100.0%

7 4.3% 259 6.6% 2737 8.1% 10 6.1% 210 4.8% 2027 5.4%

54 32.9% 1265 32.0% 11025 32.6% 40 24.4% 1192 27.3% 10678 28.5%

59 36.0% 1357 34.4% 11215 33.2% 43 26.2% 1418 32.4% 12028 32.1%

44 26.8% 1068 27.0% 8830 26.1% 71 43.3% 1553 35.5% 12764 34.0%

164 100.0% 3949 100.0% 33807 100.0% 164 100.0% 4373 100.0% 37497 100.0%

9 5.5% 281 7.1% 2454 7.3% 9 5.5% 179 4.1% 1405 3.7%

66 40.2% 1785 45.1% 15113 44.7% 53 32.3% 1682 38.5% 13828 36.9%

61 37.2% 1321 33.4% 11198 33.1% 65 39.6% 1544 35.4% 13554 36.2%

28 17.1% 567 14.3% 5023 14.9% 37 22.6% 960 22.0% 8686 23.2%

164 100.0% 3954 100.0% 33788 100.0% 164 100.0% 4365 100.0% 37473 100.0%

32 19.6% 1059 26.8% 8215 24.3% 21 12.9% 801 18.3% 5902 15.8%

76 46.6% 1927 48.8% 16323 48.3% 59 36.2% 1932 44.2% 15773 42.1%

41 25.2% 718 18.2% 6685 19.8% 52 31.9% 1054 24.1% 9679 25.8%

14 8.6% 248 6.3% 2568 7.6% 31 19.0% 582 13.3% 6108 16.3%

163 100.0% 3952 100.0% 33791 100.0% 163 100.0% 4369 100.0% 37462 100.0%

55 33.5% 1826 46.2% 15045 44.5% 35 21.3% 1383 31.7% 10596 28.3%

75 45.7% 1563 39.6% 13590 40.2% 80 48.8% 2040 46.8% 17853 47.6%

25 15.2% 429 10.9% 3881 11.5% 31 18.9% 658 15.1% 6219 16.6%

9 5.5% 133 3.4% 1272 3.8% 18 11.0% 280 6.4% 2805 7.5%

164 100.0% 3951 100.0% 33788 100.0% 164 100.0% 4361 100.0% 37473 100.0%

16 9.8% 343 8.7% 2879 8.5% 7 4.3% 229 5.2% 1576 4.2%

48 29.3% 1503 38.1% 12621 37.4% 39 23.9% 1430 32.7% 11810 31.5%

67 40.9% 1519 38.5% 13125 38.9% 73 44.8% 1926 44.1% 16779 44.8%

33 20.1% 584 14.8% 5137 15.2% 44 27.0% 782 17.9% 7323 19.5%

164 100.0% 3949 100.0% 33762 100.0% 163 100.0% 4367 100.0% 37488 100.0%

13 8.0% 356 9.0% 2933 8.7% 6 3.7% 326 7.5% 2448 6.5%

49 30.2% 1570 39.7% 13406 39.7% 75 45.7% 1679 38.4% 14101 37.6%

75 46.3% 1463 37.0% 12456 36.9% 61 37.2% 1647 37.7% 14439 38.5%

25 15.4% 566 14.3% 4995 14.8% 22 13.4% 716 16.4% 6477 17.3%

162 100.0% 3955 100.0% 33790 100.0% 164 100.0% 4368 100.0% 37465 100.0%

89 54.6% 2608 66.0% 21408 63.4% 67 40.9% 2303 52.7% 17762 47.4%

45 27.6% 931 23.5% 8546 25.3% 55 33.5% 1324 30.3% 11878 31.7%

28 17.2% 312 7.9% 2713 8.0% 19 11.6% 458 10.5% 4927 13.2%

1 .6% 103 2.6% 1101 3.3% 23 14.0% 285 6.5% 2897 7.7%

163 100.0% 3954 100.0% 33768 100.0% 164 100.0% 4370 100.0% 37464 100.0%

8 4.9% 247 6.3% 1828 5.4% 6 3.7% 170 3.9% 1129 3.0%

46 28.0% 1499 37.9% 12480 36.9% 51 31.1% 1546 35.4% 12082 32.2%

69 42.1% 1409 35.7% 12317 36.4% 56 34.1% 1656 37.9% 14745 39.3%

41 25.0% 796 20.1% 7170 21.2% 51 31.1% 996 22.8% 9553 25.5%

164 100.0% 3951 100.0% 33795 100.0% 164 100.0% 4368 100.0% 37509 100.0%

Never

Sometimes

Often

Very often

Asked questions in class orcontributed to class discussions

Total

Never

Sometimes

Often

Very often

Made a class presentation

Total

Never

Sometimes

Often

Very often

Prepared two or more drafts of apaper or assignment before turning itin

Total

Never

Sometimes

Often

Very often

Worked on a paper or project thatrequired integrating ideas orinformation from various sources

Total

Never

Sometimes

Often

Very often

Came to class without completingreadings or assignments

Total

Never

Sometimes

Often

Very often

Worked with other students onprojects during class

Total

Never

Sometimes

Often

Very often

Worked with classmates outside ofclass to prepare class assignments

Total

Never

Sometimes

Often

Very often

Tutored or taught other students(paid or voluntary)

Total

Never

Sometimes

Often

Very often

Participated in a community-basedproject as part of a regular course

Total

Never

Sometimes

Often

Very often

Used an electronic medium (list-serv,chat group, Internet, etc.) to discussor complete an assignment

Total

Never

Sometimes

Often

Very often

Used e-mail to communicate with aninstructor

Total

Never

Sometimes

Often

Very often

Discussed grades or assignments withan instructor

Total

Never

Sometimes

Often

Very often

Talked about career plans with afaculty member or advisor

Total

Never

Sometimes

Often

Very often

Discussed ideas from your readingsor classes with faculty membersoutside of class

Total

Never

Sometimes

Often

Very often

Received prompt feedback fromfaculty on your academicperformance (written or oral)

Total

Never

Sometimes

Often

Very often

Worked harder than you thought youcould to meet an instructor'sstandards or expectations

Total

Never

Sometimes

Often

Very often

Worked with faculty members onactivities other than coursework(committees, orientation, student lifeactivities, etc.)

Total

Never

Sometimes

Often

Very often

Discussed ideas from your readingsor classes with others outside of class(students, family members,coworkers, etc.)

Total

Count Col %

Nesseville State Univ

Count Col%

Carnegie

Count Col%

National

First-year Students

Count Col %

Nesseville State Univ

Count Col%

Carnegie

Count Col%

National

Seniors

Means & Frequency Means & Frequency ReportsReports

Look at the Items that Make Look at the Items that Make Up Each BenchmarkUp Each Benchmark

Which Items Have Which Items Have Significantly Higher/Lower Significantly Higher/Lower Responses than Comparison Responses than Comparison Groups?Groups?

Practical Significance: Practical Significance: Identify Standardized Effect Identify Standardized Effect Sizes greater than .2Sizes greater than .2

Activity: Highlight Key Activity: Highlight Key FindingsFindings

Review Benchmark and Means Review Benchmark and Means Summary ReportSummary Report

Small groups by benchmarkSmall groups by benchmarkExamine a benchmark Examine a benchmark Which items differ?Which items differ?

Digging Deeper: Using the DataDigging Deeper: Using the Data

Identify important subpopulationsIdentify important subpopulations Determine the outcome that mattersDetermine the outcome that matters Determine the factors that influence the Determine the factors that influence the

outcomeoutcome

Principles for Data-Driven Principles for Data-Driven Learning-Centered Learning-Centered

ChangeChange

1. Get the ideas right1. Get the ideas right

Focus on a real problem Focus on a real problem (e.g., persistence, raising (e.g., persistence, raising expectations, success in expectations, success in major field courses)major field courses)

Concentrate on effective Concentrate on effective educational practiceseducational practices

Characteristics of Characteristics of Educationally Effective Educationally Effective

CollegesColleges

Organizational culture Organizational culture valuingvaluing

High expectations High expectations Respect for diverse talents Respect for diverse talents Emphasis on early years of Emphasis on early years of

studystudy

Characteristics of Characteristics of Educationally Effective Educationally Effective

CollegesCollegesCurriculumCurriculum Coherence in learningCoherence in learningSynthesizing experiencesSynthesizing experiencesIntegrating education Integrating education

and experienceand experienceOngoing practice of Ongoing practice of

learned skillslearned skills

Characteristics of Characteristics of Educationally Effective Educationally Effective

CollegesCollegesInstructionInstruction Active learning Active learning Assessment and feedbackAssessment and feedback CollaborationCollaboration Adequate time on task Adequate time on task Out-of-class contact with Out-of-class contact with

facultyfaculty

2. Get grass roots buy-in2. Get grass roots buy-in

Leaders endorse, but Leaders endorse, but don’t dictatedon’t dictate

Structures not (nearly) Structures not (nearly) as important as as important as relationshipsrelationships

Validate pockets of Validate pockets of qualityquality

The 10% ruleThe 10% rule

2. Get grass roots buy-in2. Get grass roots buy-in

ExamplesExamples Ask deans about their Ask deans about their

concernsconcerns Focus groupsFocus groups Get students “engaged” in Get students “engaged” in

the improvement effort the improvement effort (Illinois State, Oregon State)(Illinois State, Oregon State)

Faculty version of NSSE Faculty version of NSSE surveysurvey

3. Keep the stakes and 3. Keep the stakes and volume lowvolume low

Avoid “winners” & Avoid “winners” & “losers”“losers”

Suspend disbeliefSuspend disbelief Denial managementDenial management Go public later than Go public later than

soonersooner

4. It’s the culture 4. It’s the culture (stupid)(stupid)

Culture is (almost) always Culture is (almost) always (at least) part of (at least) part of the the problemproblem

Focus on “reculturing” Focus on “reculturing” and “revisioning” and “revisioning”

Use familiar (or at least Use familiar (or at least understandable) language understandable) language

5. Think and act 5. Think and act systemicallysystemically

Link innovations and Link innovations and change efforts from change efforts from different parts of the different parts of the campus (e.g., Greater campus (e.g., Greater Expectations, Gen Ed Expectations, Gen Ed reform, SOTL, NSSE, reform, SOTL, NSSE, service learning, diversity)service learning, diversity)

Presenting NSSE/CCSSE Presenting NSSE/CCSSE FindingsFindings

Potential Audiences? Potential Audiences? Consider Audiences Before Consider Audiences Before

Administration—It May Administration—It May Influence DecisionsInfluence Decisions

Internal Use & External Internal Use & External ReportingReporting

Be StrategicBe Strategic

Internal Sharing of NSSE 2003 Results

%

President 80

Administrative Staff 68

Department Chairs 59

Academic Advisors 51

Faculty 71

Governing Board 34

Students 32

Other (web site, fact book, etc.) 20

Internal Audiences

Source: NSSE 2003 Report Card

External AudiencesExternal AudiencesExternal Sharing of NSSE 2003 Data

%

Accreditation Agencies 31

State Agencies 11

Media 13

Web Site 25

Prospective Students 13

Parents 13

Alumni 13

Other 7

No External Disclosure 35

Source: NSSE 2003 Report Card

Ways to Communicate Ways to Communicate ResultsResults

Customize Report to Reach Campus Customize Report to Reach Campus AudiencesAudiences

Organize NSSE data by colleges, depts, Organize NSSE data by colleges, depts, programsprograms

Focus on items of interest to stakeholder and Focus on items of interest to stakeholder and communicate understandable percentagescommunicate understandable percentages

Use data to start discussionsUse data to start discussions Use data to correct misunderstandingsUse data to correct misunderstandings Assemble and distribute detailed information Assemble and distribute detailed information

on method and credibility of surveyon method and credibility of survey

Activity: Connecting Activity: Connecting Findings with Relevant Findings with Relevant

AudiencesAudiencesWhat was the most What was the most

interesting finding?interesting finding?Who needs to know about Who needs to know about

this?this?How can you let them know?How can you let them know?

National ContextNational Context

Putting Your Results in Context:Putting Your Results in Context: How good is good enough?How good is good enough? What would it mean to have 80% of What would it mean to have 80% of

your students indicate that they are your students indicate that they are satisfied with the college?satisfied with the college?

Sampling MethodsSampling Methods

Sampling Consistency Across Sampling Consistency Across Colleges:Colleges:

CCSSECCSSE takes a stratified random takes a stratified random sample of coursessample of courses

NSSE takes random samples of first- NSSE takes random samples of first- and fourth-year studentsand fourth-year students

Results are representative of the Results are representative of the populations from which they are drawnpopulations from which they are drawn

Over-sampling for targeted populationsOver-sampling for targeted populations

Comparing Your ResultsComparing Your Results

Sampling Consistency Across Sampling Consistency Across Colleges:Colleges:

Because results are comparable across Because results are comparable across institutions, institutions can situate their institutions, institutions can situate their resultsresults

Anything on the survey can potentially be Anything on the survey can potentially be benchmarkedbenchmarked

BenchmarkingBenchmarkingTwo Approaches:Two Approaches: Normative - compares your students’ Normative - compares your students’

responses to those of students at responses to those of students at other colleges and universities. other colleges and universities.

Criterion - compares your school’s Criterion - compares your school’s performance against a performance against a predetermined value or level predetermined value or level appropriate for your students, given appropriate for your students, given your institutional mission, size, your institutional mission, size, curricular offerings, funding, and so curricular offerings, funding, and so forth.forth.

Criterion Referenced Criterion Referenced ApproachApproach

Most valued activities Most valued activities Ask faculty & staff what is most Ask faculty & staff what is most valued in institution, department, valued in institution, department, then present relevant datathen present relevant data

Eliminate “Nevers”Eliminate “Nevers”Work on reducing or eliminating Work on reducing or eliminating reports by students of reports by students of nevernever doing specific engagement doing specific engagement activitiesactivities

Assessment PurposesAssessment Purposes

AccountabilityAccountability ImprovementImprovement

Activity: Identify Activity: Identify Appropriate BenchmarksAppropriate Benchmarks

Internal Differences: How do Internal Differences: How do part- and full-time students part- and full-time students differ?differ?

External Examples: How does External Examples: How does your institution differ from your institution differ from similar colleges? similar colleges?

Principles for Good Practice Principles for Good Practice in Undergraduate Educationin Undergraduate Education

(Chickering & Gamson, 1987)(Chickering & Gamson, 1987)

Student-faculty contactStudent-faculty contact Active learningActive learning Prompt feedbackPrompt feedback Time on taskTime on task High expectationsHigh expectations Respect for diverse learning Respect for diverse learning

stylesstyles Cooperation among studentsCooperation among students

Lessons from the Lessons from the ResearchResearch

What matters most is what What matters most is what students do, not who they arestudents do, not who they are

A key factor is the quality of A key factor is the quality of effort students expendeffort students expend

Educationally effective Educationally effective institutions channel student institutions channel student energy toward the right energy toward the right activitiesactivities

Principles For Promoting Principles For Promoting Student EngagementStudent Engagement

Intentionality - Not Intentionality - Not leaving serendipity to leaving serendipity to chancechance

Alignment (mission, Alignment (mission, curriculum, student curriculum, student experiences)experiences)

Collaboration (pull many Collaboration (pull many levers)levers)

Assessment and feedback Assessment and feedback to guide/document to guide/document improvementimprovement

How is Engagement How is Engagement Measured?Measured?

Switch benchmarksSwitch benchmarks How would you measure this How would you measure this

construct?construct?

Psychometric Findings: ReliabilityPsychometric Findings: Reliability

Both instruments have been shown to Both instruments have been shown to be reliable and validbe reliable and valid

Good construct reliability: benchmarks Good construct reliability: benchmarks items measure the same constructitems measure the same construct

Structural equation models have been Structural equation models have been used to demonstrate that there is used to demonstrate that there is strong consistency across strong consistency across subpopulations (gender & part- v. full-subpopulations (gender & part- v. full-time) and across yearstime) and across years

Psychometric Findings: ValidityPsychometric Findings: Validity

Psychographic DataPsychographic Data

NSSE data used to form clusters of NSSE data used to form clusters of studentsstudents

http://mypage.iu.edu/~chuzhao/air03dpi.pdfhttp://mypage.iu.edu/~chuzhao/air03dpi.pdf Psychographic data consistently was Psychographic data consistently was

more effective at predicting outcomes more effective at predicting outcomes than demographic datathan demographic data

The Florida OpportunityThe Florida Opportunity

How does engagement relate to How does engagement relate to outcome?outcome?

PersistencePersistence Goal AttainmentGoal Attainment

Activity: Expectations and Activity: Expectations and Desires in Student Desires in Student

EngagementEngagement How many students responded often or How many students responded often or

very often?very often? What would you like to see?What would you like to see? How did they respond?How did they respond? How can you increase engagement?How can you increase engagement?

Questions and SuggestionsQuestions and Suggestions

Questions for NSSE and Questions for NSSE and CCSSECCSSE Suggestions for NSSE and Suggestions for NSSE and CCSSECCSSE