1 mpi mission perception inventory ellen boylan, ph.d. and sister jane wakahiu, lsosf marywood...

36
1 MPI MPI Mission Perception Inventory Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the Mission Perception Inventory (MPI): Linking Mission Goals and Learning Environment North Carolina State University Undergraduate Assessment Symposium Aligning Pedagogy, Curriculum & Assessment April 24-26, 2009

Upload: aubrey-little

Post on 18-Dec-2015

218 views

Category:

Documents


1 download

TRANSCRIPT

Page 1: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

11

MPIMPIMission Perception InventoryMission Perception Inventory

Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF • Marywood University

Assessing Institutional Effectiveness with the Mission Perception Inventory (MPI): Linking Mission Goals and Learning Environment

North Carolina State University Undergraduate Assessment Symposium

Aligning Pedagogy, Curriculum & Assessment

April 24-26, 2009

Page 2: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

22

What’s in a mission?What’s in a mission?

Page 3: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

33

AcademicExcellence Leadership

Community/Service

Professional Development

Spirituality/Ethical

Develop. Other

Develop intellectually

Prepare for leadership

Sense of community

Develop professionally Develop ethically

Respect & understanding

Develop Acedemically Critical Thinking

Sense of Community

Professional Growth Personal Growth

Research & Expertise

ScholarshipIndependent

Learning Public ServiceSpecialized Expertise

Personal Advisement Diversity

Rigorous & Engaging

Compassionate Service

Truth & Social Justice

Care and Respect

Develop Intellectually

Prepare for Leadership Lives of Service

Prepare for Prof. Careers

Develop Spirit & Heart

Values-Based Education

ScholarshipAnalysis &

ComunicationSense of

CommunityDiciplines & Professions

Morally Responsible

Freedom & Diversity

Educational Excellence

Responsibility as Leaders

Care & Dignity for Community

Career Development Ethical Presence

Historical Remembrance

Comitment to Teach & Learn Service to Society

Professional Competence

Make Connactions

AcademicExcellence Leadership Skills

Involvment in Community

Develop Whole Person

Cultural & Social Growth

Develop Intellectually

Lifelong Learning CommitmentCompassion &

EmpathyDevelop

ProfessionallyPromote Ethics &

Spirituality Human Dignity

Sense of Community Truth & Justice

Diversity & Dignity

Intellectual InquiryResponsibility for

DecisionsCommitment to

ServiceCareer

Preparation Personal Growth Diversity

AcademicExcellence Leadership

Community/Service

Professional Development

Spirituality/Ethical

Develop. Other

Develop intellectually

Prepare for leadership

Sense of community

Develop professionally Develop ethically

Respect & understanding

Develop Acedemically Critical Thinking

Sense of Community

Professional Growth Personal Growth

Research & Expertise

ScholarshipIndependent

Learning Public ServiceSpecialized Expertise

Personal Advisement Diversity

Rigorous & Engaging

Compassionate Service

Truth & Social Justice

Care and Respect

Develop Intellectually

Prepare for Leadership Lives of Service

Prepare for Prof. Careers

Develop Spirit & Heart

Values-Based Education

ScholarshipAnalysis &

ComunicationSense of

CommunityDiciplines & Professions

Morally Responsible

Freedom & Diversity

Educational Excellence

Responsibility as Leaders

Care & Dignity for Community

Career Development Ethical Presence

Historical Remembrance

Comitment to Teach & Learn Service to Society

Professional Competence

Make Connactions

AcademicExcellence Leadership Skills

Involvment in Community

Develop Whole Person

Cultural & Social Growth

Develop Intellectually

Lifelong Learning CommitmentCompassion &

EmpathyDevelop

ProfessionallyPromote Ethics &

Spirituality Human Dignity

Sense of Community Truth & Justice

Diversity & Dignity

Intellectual InquiryResponsibility for

DecisionsCommitment to

ServiceCareer

Preparation Personal Growth Diversity

AcademicExcellence Leadership

Community/Service

Professional Development

Spirituality/Ethical

Develop. Other

Develop intellectually

Prepare for leadership

Sense of community

Develop professionally Develop ethically

Respect & understanding

Develop Acedemically Critical Thinking

Sense of Community

Professional Growth Personal Growth

Research & Expertise

ScholarshipIndependent

Learning Public ServiceSpecialized Expertise

Personal Advisement Diversity

Rigorous & Engaging

Compassionate Service

Truth & Social Justice

Care and Respect

Develop Intellectually

Prepare for Leadership Lives of Service

Prepare for Prof. Careers

Develop Spirit & Heart

Values-Based Education

ScholarshipAnalysis &

ComunicationSense of

CommunityDiciplines & Professions

Morally Responsible

Freedom & Diversity

Educational Excellence

Responsibility as Leaders

Care & Dignity for Community

Career Development Ethical Presence

Historical Remembrance

Comitment to Teach & Learn Service to Society

Professional Competence

Make Connactions

AcademicExcellence Leadership Skills

Involvment in Community

Develop Whole Person

Cultural & Social Growth

Develop Intellectually

Lifelong Learning CommitmentCompassion &

EmpathyDevelop

ProfessionallyPromote Ethics &

Spirituality Human Dignity

Sense of Community Truth & Justice

Diversity & Dignity

Intellectual InquiryResponsibility for

DecisionsCommitment to

ServiceCareer

Preparation Personal Growth Diversity

AcademicExcellence Leadership

Community/Service

Professional Development

Spirituality/Ethical

Develop. Other

Develop intellectually

Prepare for leadership

Sense of community

Develop professionally Develop ethically

Respect & understanding

Develop Acedemically Critical Thinking

Sense of Community

Professional Growth Personal Growth

Research & Expertise

ScholarshipIndependent

Learning Public ServiceSpecialized Expertise

Personal Advisement Diversity

Rigorous & Engaging

Compassionate Service

Truth & Social Justice

Care and Respect

Develop Intellectually

Prepare for Leadership Lives of Service

Prepare for Prof. Careers

Develop Spirit & Heart

Values-Based Education

ScholarshipAnalysis &

ComunicationSense of

CommunityDiciplines & Professions

Morally Responsible

Freedom & Diversity

Educational Excellence

Responsibility as Leaders

Care & Dignity for Community

Career Development Ethical Presence

Historical Remembrance

Comitment to Teach & Learn Service to Society

Professional Competence

Make Connactions

AcademicExcellence Leadership Skills

Involvment in Community

Develop Whole Person

Cultural & Social Growth

Develop Intellectually

Lifelong Learning CommitmentCompassion &

EmpathyDevelop

ProfessionallyPromote Ethics &

Spirituality Human Dignity

Sense of Community Truth & Justice

Diversity & Dignity

Intellectual InquiryResponsibility for

DecisionsCommitment to

ServiceCareer

Preparation Personal Growth Diversity

scope of researchscope of research

Purpose

Develop an instrument to measure student perception

of institutional mission.

Test instrument reliability.

Uncover constructs (factors).

Observe constructs longitudinally.

GRANTS IN HIGHER EDUCATION FOR IMPROVED ASSESSMENT METHODS

Page 4: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

44

scope of researchscope of research

Purpose

Develop an instrument

Test instrument reliability.

Uncover constructs.

Observe constructslongitudinally.

Research Questions

1. Is the Mission Perception Inventory (MPI) a valid and reliable reliable measure of student perception of institutional mission?

2. What are the factors in the MPI?

3. Do the factors recur in repeated administrations of the revised MPI?

4. Are the factors equally reliable over time?

Page 5: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

55

leaders of public and private institutions alike are

thinking about spirituality these days, as the

data suggest that's what their students are thinking about, too

(Inside Higher Ed, 2009).

There is strong connection between institutional

programs and student learning environment

(Pascarella, 2001). …institutions influence levels of engagement on

campus as a result of structural features, programs, policies,

and organizational culture (Kuh et al., 2005).

background

Page 6: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

66

Select the National Survey of

Student Engagement (NSSE) as

the vehicle for inserting research

questions.

Assemble a NSSE consortium to

jointly engage in research to explore

student perception of mission.

Develop question items;

administer them to the consortium

as a NSSE attachment.

Test the questions: reliability, factor

analysis, and correlation analysis. Repeat annually.

design

GRANTS IN HIGHER EDUCATION FOR IMPROVED ASSESSMENT METHODS

Page 7: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

77

developing the question items…developing the question items…

1. Conduct a qualitative analysis of mission statements 2. Distill common constructs3. Draft question items based on mission statement constructs4. Vet with participating institutions, experts, prospective subjects

ConceptsConcepts common to common to mission statement of mission statement of

consortium colleges, like this consortium colleges, like this one, were drawn out, one, were drawn out,

compared, and distilled into compared, and distilled into 20 20 questions.questions.

…The University roots

itself in the principle of

justice and a belief that

education empowers

people…offers students a

welcoming and

supportive community

that encourages men and

women of all backgrounds

to shape their lives as

leaders in serviceleaders in service to

others. Proud of its liberal

arts tradition and host of

professional disciplinesprofessional disciplines,

… challenging students to

broaden their

understanding of global

issues and to make

decisions based on

spiritual, ethical, and

religious values…

Page 8: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

88

AcademicExcellence Leadership

Community/Service

Professional Development

Spirituality/Ethical

Develop. Other

Develop intellectually

Prepare for leadership

Sense of community

Develop professionally Develop ethically

Respect & understanding

Develop Acedemically Critical Thinking

Sense of Community

Professional Growth Personal Growth

Research & Expertise

ScholarshipIndependent

Learning Public ServiceSpecialized Expertise

Personal Advisement Diversity

Rigorous & Engaging

Compassionate Service

Truth & Social Justice

Care and Respect

Develop Intellectually

Prepare for Leadership Lives of Service

Prepare for Prof. Careers

Develop Spirit & Heart

Values-Based Education

ScholarshipAnalysis &

ComunicationSense of

CommunityDiciplines & Professions

Morally Responsible

Freedom & Diversity

Educational Excellence

Responsibility as Leaders

Care & Dignity for Community

Career Development Ethical Presence

Historical Remembrance

Comitment to Teach & Learn Service to Society

Professional Competence

Make Connactions

AcademicExcellence Leadership Skills

Involvment in Community

Develop Whole Person

Cultural & Social Growth

Develop Intellectually

Lifelong Learning CommitmentCompassion &

EmpathyDevelop

ProfessionallyPromote Ethics &

Spirituality Human Dignity

Sense of Community Truth & Justice

Diversity & Dignity

Intellectual InquiryResponsibility for

DecisionsCommitment to

ServiceCareer

Preparation Personal Growth Diversity

Concepts become question items...

GRANTS IN HIGHER EDUCATION FOR IMPROVED ASSESSMENT METHODS

Page 9: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

99

A2. Every institution has a mission statement. We would like to know how familiar you are with your school's mission. Please indicate your agreement with each of the following statements:

Strongly agree Agree

Neither agree/

disagree Disagree Strongly disagree

a. The mission of this institution is widely understood

by students.

b. Ethical and spiritual development of students is an important part of the mission at this institution.

c. This institution offers opportunities for volunteering and community service.

d. Social and personal development of students is an important part of the mission at this institution.

e. This institution offers opportunities for developing leadership skills.

f. The heritage of the founding religious community of this institution is evident here.

g. The faculty, staff, and students here are respectful of people of different religions.

h. The faculty, staff, and students here are respectful of people of different races and cultures.

i. People of different sexual orientations are accepted socially here.

j. Students feel free to express their individual spirituality here.

k. The environment here encourages students to develop an appreciation of diversity.

l. At this institution, there are opportunities for students to strengthen their religious commitment.

m. The professors at this institution discuss the ethical implications of what is being studied.

A peek at the questionnaire

See consortia questions by year: http://nsse.iub.edu/html/consortia-list_2009.cfm

Page 10: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

1010

The NSSE survey with attached Mission Perception Inventory (MPI) questions has been administered 155 times to more than 44,198 first-year and senior students at 112 unique institutions across the United States

every year since 2004.

administration administration

GRANTS IN HIGHER EDUCATION FOR IMPROVED ASSESSMENT METHODS

Page 11: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

1111

• Reliability analysis of the mission questions (20 items) is performed. Low-loading items kicked out.• A Mission Perception Inventory (MPI) is produced (17 to 19 items).

Is the Mission Perception Inventory (MPI)

valid and reliable?

Page 12: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

1212

• Factor analysis of MPI is conducted each year.• By year, three or four subscales are produced:

•Sense of mission.•Respect for diversity.•Individual values.•Religious practice/Spirituality.

What are the factors (scales) in the Mission

Perception Inventory?

Page 13: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

1313

Mission

Perception

Inventory(MPI)

(~19 items)

SenseOf

Mission

Respect for

Diversity

Individual

Values

SpiritualPractice

Administer

Consortium

mission

questions

(20 items)

derivation of the MPI and subscalesderivation of the MPI and subscales

Page 14: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

1414

sense of mission sense of mission (10 items = .90) • The mission of this institution is widely

understood by students.• Social and personal development

is an important part of the mission.• Ethical and spiritual development

of students is important.• This institution offers opportunities for

volunteering and community service.• This institution offers opportunities for developing

leadership skills.• There are opportunities for students to

strengthen their religious commitment.

• This institution’s religious heritage is evident.

• Professors here discuss the ethical implications of what is being studied.

• As a result of my experience here, I am more

aware of my own personal values.• The mission of this institution is reflected in

course offerings.

Page 15: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

1515

respect for diversity (5 items = .878)

The faculty, staff, and students here…

• respect different religions

• respect different races and cultures ………………………………………………………………………………

• Students feel free to express individual spirituality.

• Different sexual orientations are accepted.

• The environment encourages appreciation of diversity.

Page 16: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

1616

Consortia institutions and Consortia institutions and respondents by yearrespondents by year

Respondents

Institutions First Year Senior

2004 15 2,000 1,827

2005 16 1,279 1,332

2006 24 2,684 2,854

2007 36 4,533 4,331

Catholic 2008 35 4520 5063

Indepen’t 2008 29 6805 6970

  155* 21,821 22,377

*(112 unique institutions)

Page 17: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

1717

Do the factors recur in repeated administrations of Do the factors recur in repeated administrations of

the revised Mission Perception Inventory (MPI) the revised Mission Perception Inventory (MPI)

2004 2005 2006 2007 2008

Sense of Mission 8 10 10 10 10

Respect for Diversity 5 5 5 5 5

Individual Actions 2 0 0 0 0

Religious Practice/Spirituality 2 3 3 2 2

Mission Perception Inventory 17 18 18 19 19

Number of items in factors

Page 18: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

1818

(MPI)(MPI) Mission Mission PerceptioPerceptionnInventory Inventory

ReportReport

2008 Mission Perception Inventory Report My University - Sample

Sense of Mission (Cronbach’s alpha=__)

First Year Mean 0.00 Median 0.00 25%ile 0.00 75%ile 0.00 Senior Year Mean 0.00 Median 0.00 25%ile 0.00 75%ile 0.00 Rank (out of x) First Year: Senior Year:

3.8

4.0

4.2

4.4

4.6

Sen

se o

f M

issi

on

Sco

re

My University 4.69 4.56

Consortium 4.02 3.97

First Year Senior

Social and personal development of students is an important part of the mission at this institution.

Ethical and spiritual development of students is an important part of the mission at this institution.

This institution offers opportunities for volunteering and community service. This institution offers opportunities for developing leadership skills. As a result of your experience here, you are more aware of your own personal values. The heritage of the founding religious community of this institution is evident here. The mission of this institution is widely understood by students. The mission of this institution is reflected in its course offerings At this institution, there are opportunities for students to strengthen their religious commitment. The faculty at this institution discusses the ethical implications of what is being studied. As a result of my experience here, I am more aware of my own personal values.

Respect for Diversity

(Cronbach’s alpha=__) First Year Mean 0.00 Median 0.00 25%ile 0.00 75%ile 0.00 Senior Year Mean 0.00 Median 0.00 25%ile 0.00 75%ile 0.00 Rank (out of x) First Year: Senior Year:

3.5

3.6

3.7

3.8

3.9

4.0

4.1

Res

pec

t fo

r D

iver

sity

Sco

re

My University 3.92 3.60

Consortium 3.98 3.87

First Year Senior

The faculty, staff, and students here are respectful of people of different religions. The faculty, staff, and students here are respectful of people of different races and cultures. Students feel free to express their individual spirituality here. People of different sexual orientations are accepted socially here. The environment here encourages students to develop an appreciation of diversity.

NSSE 2008 Mission Perception Inventory Report Mission Engagement Consortium for Independent Colleges My University- Sample

In spring 2008, My University participated in the National Survey of Student Engagement (NSSE) Mission Engagement Consortium for Independent Colleges, and administered additional questions on mission to the survey’s first year and senior student respondents. Of ___ total survey respondents at My University, there were ___ first year students and ___ seniors.

Reliability and factor analysis of the instrument based on results from administrations of consortium questions have consistently produced a highly reliable scale measure called the Mission Perception Inventory (MPI)*. Further analysis reveals three highly correlated and reliable subscales within the MPI:

Sense of Mission ( 11 items) Respect for Diversity (5 items) Spirituality (2 items)

Also presented are mean comparisons on another scale, NSSE Beliefs and Values (NSSE/BV). The

Consortium administrator created the NSSE/BV scale for this study by a process of selecting, a priori, the question items from the NSSE instrument that seemed most related to mission and then grouping them.

Scale mean comparisons between My University and Consortium schools are presented in the bar graphs. The graphs compare My University first year and senior means compared to Consortium group means on the study’s five scales or benchmarks. The mean scores are shown on the vertical or “y” axis according to the Likert scale range used for items in that scale. Sense of Mission and Respect for Diversity items were rated on a scale from 1 – 5 (1=Strongly disagree, 5=Strongly agree).

In the column to the left of each graph, the rank (high to low) and semi-interquartile range are given for both first year and senior scale means. Note carefully that the y-axis interval is 0.5 for some scales, and 0.1 for others. In the columns under and/or to the right of the graph, the items contained in that scale are listed.

When comparing first year and senior mean scores, recall that the results of the NSSE 2008 produce a snapshot in time of student responses. Results do not indicate growth or increase in scale means from first to senior years, but simply a comparison of those cohort classes at one moment.

Spirituality (Cronbach’s alpha=__)

First Year Mean 0.00 Median 0.00 25%ile 0.00 75%ile 0.00 Senior Year Mean 0.00 Median 0.00 25%ile 0.00 75%ile 0.00 Rank (out of “x” schools) First Year: ___ Senior Year:

0.0

0.5

1.0

1.5

2.0

Rel

igio

us

Pra

ctic

e S

core

My University 1.96 1.92

Consortium 2.07 2.08

First Year Senior

Within the past week, have you participated in a religious service? Within the past week, have you spent time in private prayer or meditation?

2008 Mission Perception Inventory Report

GRANTS IN HIGHER EDUCATION FOR IMPROVED ASSESSMENT METHODS

Page 19: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

1919

Are the factors equally reliable Are the factors equally reliable

over time?over time?

Subscales 2004 2005 2006 20072008

Catholic2008

Indept

cronbach cronbach cronbach cronbach cronbach cronbach

Sense of Mission .87 .88 .88 .90 .91 .88

Respect for Diversity .85 .84 .87 .86 .88 .87

Individual Actions .67 n/a n/a n/a n/a n/a

Religious Practice/Spirituality

.62 .55 .54 .62 .62 .64

MPI scale .88 .89 .90 .91 .91 .90

Page 20: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

2020

2009…

Teagle Foundation Research Teagle Foundation Research ContinuesContinues

• Develop the Mission Engagement Index.• Obtain NSSE 2008 consortia

data • Test using reliability analysis to produce factors• Distribute MPI reports • Compare consortia results• Conduct regression analysis to

create the new index

• Produce Mission Engagement Index (MEI) Reports for consortia institutions.

GRANTS IN HIGHER EDUCATION FOR IMPROVED ASSESSMENT METHODS

Page 21: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

2121

Does the Mission Engagement Does the Mission Engagement IndexIndex describe causal relationships among describe causal relationships among

variables that affect mission perception? variables that affect mission perception?

• Dependent variable (Institution score)• Mission Perception Inventory• Sense of Mission scale • Respect for Diversity scale

• Independent variables (need 15 cases per)*

• Selectivity 15 institutions• Enrollment 30 institutions• Urbanicity 45 institutions• Resident % 60

institutions• Female % 75 institutions• Another? 90 institutions

*…a recommended ratio of subjects to IVs of at least 15 to 1 will provide a reliable regression equation (Stevens, 1992).

Page 22: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

2222

Selecting IVs for regression Selecting IVs for regression analysisanalysis

Institution regionCarnegie

Class%

resident%

female

%ethnic/

cauUG

enroll

%Part-time Setting

%Accept

a North U Master's 39 61 95 2119 25 Suburb 74

b Midwest U Master's 33 67 70 928 40 Suburb 57

c South National U 32 68 39 4130 42 Suburb 54

d South Bacc.. Coll 43 57 76 1011 21 Rural 66

e Midwest U Master's 43 57 70 1692 23 Suburb 85

f Midwest Bacc. Coll 42 58 89 978 83 Urban 70

g Midwest U Master's 31 69 78 2985 37 Suburb 93

h West U Master's 32 68 23 1096 60 Urban 94

i North Bacc. Coll 92 58 62 8568 16 Suburb 42

j Midwest U Master's 28 72 91 1502 37 Urban 79

l North U Master's 24 76 79 1320 815 Suburb 60

Page 23: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

2323

participating institutions by region

Midwest31%

North43%

South17%

West9%

2008 = 2 consortia, 54 institutions

Is there Is there sufficient sufficient variabilityvariability

??

Page 24: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

2424

“urbanicity” of participating institutions 2008

Rural15%

Suburban54%

Urban31%

Is there Is there sufficient sufficient variabilitvariabilit

y?y?

Page 25: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

2525

considering elegance

Consistency of factors is affirmed. Institution sizes reasonably varied. Data sufficient to conduct analysis. Variables, initially selected on an a priori basis, tested. Variability investigated. Useful.

ββeta juiceeta juice

Page 26: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

2626

MPI Scales:MPI Scales:Comparison of means by institution Comparison of means by institution

typetype

Mean Score Scale Independent Catholic t(df) 2-tailed sig

Sense of Mission 3.77 4.04 -2.637(14.356) .019* Respect for Diversity 4.03 4.00 0.330(40.000) NS Spiritual Practice 1.37 1.46 -1.310(14.856) NS MPI 3.46 3.63 -2.296(14.756) .037* NSSE Beliefs & Values 2.57 2.62 -0.882(15.876) NS

*p≤.05

Page 27: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

2727

Correlation of regression Correlation of regression variables with MPI meanvariables with MPI mean

Correlations

usn_setting pct_residence mpi_mean

Pearson Correlation -.203 -.258 .429**

Sig. (2-tailed) NS NS .005

religious_affliation

N 42 42 42

Pearson Correlation .590** -.484**

Sig. (2-tailed) .000 .001

usn_setting

N 42 42

Pearson Correlation -.456**

Sig. (2-tailed) .002

pct_residence

N 42

**. Correlation is significant at the 0.01 level (2-tailed).

Page 28: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

2828

Deriving coefficientsDeriving coefficientsfor the Mission Engagement Index for the Mission Engagement Index

(MEI)(MEI)Model Summary

Model R R Square Adjusted R

Square Std. Error of the Estimate

1 .484a .234 .215 .163 2 .591b .349 .315 .153

a. Predictors: usn_setting b. Predictors: usn_setting, religious_affliation

Page 29: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

2929

Correlation of regression Correlation of regression variables variables withwith Mission scale mean Mission scale mean

Correlations

usn_setting pct_residence mission_mean

Pearson Correlation -.203 -.258 .486**

Sig. (2-tailed) NS NS .001

religious_affliation

N 42 42 42

Pearson Correlation .590** -.429**

Sig. (2-tailed) .000 .005

usn_setting

N 42 42

Pearson Correlation -.268

Sig. (2-tailed) .086

pct_residence

N 42

**. Correlation is significant at the 0.01 level (2-tailed).

Page 30: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

3030

Predictive Equation*Predictive Equation*

Institution Predicted MPI Score =

(B1)*(Value of USN_setting)+(-0.414)*(1, 2, or 3)

(B2)*(Value of Religious Affiliation)+(0.345)*(1 or 2)

Constant(3.687)

*Mortenson, T. (1997). Actual Vs Predicted institutional graduation rates for 1100 Colleges and universities. Opportunity, 58.

Page 31: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

3131

MockMock Mission Engagement Index Mission Engagement Index ReportReport

The MEI will show an institution’s actual versus predicted scores on mission constructs.

Progress on mission effectiveness can be assessed by comparing MEI outcomes to institutional goals.

Sense of Mission Respect for Diversity Values and Beliefs Mission Perception Inventory

Page 32: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

3232

ReferencesReferences

Inside Higher Education (2009). Spiritual accountability. Retrieved from http://www.insidehighered.com/layout/set/pri...assessment/01/02/2007/News

Kuh, D. G., Kinzie, J., Schuh, J. H., and Whitt, E. J. (2005). Never let it rest: lessons about student success from high-performing colleges and universities. Change, 37(4), 44-51.

Mortenson, T. (1997). Actual Vs predicted institutional graduation rates for 1100 colleges and universities. Opportunity, 58.

Pacarella, E. T. (2001). Identifying excellence in undergraduate education. Change, 33(3), 18-27.

Stevens, J. P. (1992). Applied multivariate statistics for the social sciences (2nd edition). Hillsdale, New Jersey: Erlbaum.

Page 33: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

3333

Carlow University Pittsburgh PA Carroll College Helena MT Catholic Theological Union Chicago IL Chaminade University of Honolulu Honolulu HI Chatfield College Saint Martin OH Chestnut Hill College Philadelphia PA Christ the King Seminary East Aurora NY Christendom College Front Royal VA Christian Brothers University Memphis TN Clarke College Dubuque IA College of Mount Saint Joseph Cincinnati OH College of Mount Saint Vincent Riverdale NY College of New Rochelle New Rochelle NY College of Notre Dame of Maryland Baltimore MD College of Saint Benedict St. Joseph MN College of Saint Catherine St. Paul MN College of Saint Elizabeth Morristown NJ College of Saint Joseph in Vermont Rutland VT College of Saint Mary Omaha NE College of Saint Rose Albany NY College of Saint Scholastica Duluth MN College of Saint Thomas More Fort Worth TX College of Santa Fe Santa Fe NM College of the Holy Cross Worcester MA Creighton University Omaha NE De Paul University Chicago IL DeSales University Center Valley PA Divine Word College Epworth IA Dominican College Orangeburg NY Dominican House of Studies Washington DC Dominican School of Philosophy and Theology Berkeley CA Dominican University River Forest IL Dominican University of California San Rafael CA

Donnelly College Kansas City KS Duquesne University of the Holy Spirit Pittsburgh PA D'Youville College Buffalo NY Edgewood College Madison WI Elms College Chicopee MA Emmanuel College Boston MA Fairfield University Fairfield CT Felician College Lodi NJ Fontbonne University St. Louis MO Fordham University Bronx NY Franciscan School of Theology Berkeley CA Franciscan University of Steubenville Steubenville OH Gannon University Erie PA Georgetown University Washington DC Georgian Court University Lakewood NJ Gonzaga University Spokane WA Gwynedd-Mercy College Gwynedd Valley PA Hilbert College Hamburg NY Holy Apostles College & Seminary Cromwell CT Holy Cross College Notre Dame IN Holy Family University Philadelphia PA Holy Names University Oakland CA Immaculata University Immaculata PA Iona College New Rochelle NY Jesuit School of Theology Berkeley CA John Carroll University Cleveland OH John Paul the Great Catholic University San Diego CA King's College Wilkes-Barre PA La Roche College Pittsburgh PA

The United College of San Antonio San Antonio TX Thomas Aquinas College Santa Paula CA Thomas More College Crestview Hills KY Thomas More College of Liberal Arts Merrimack NH Trinity University Washington DC Trocaire College Buffalo NY University of Dallas Irving TX University of Dayton Dayton OH University of Detroit Mercy Detroit MI University of Great Falls Great Falls MT University of Mary Bismarck ND University of Notre Dame du Lac Notre Dame IN University of Portland Portland OR University of Sacramento Sacramento CA University of Saint Francis Joliet IL University of Saint Francis Fort Wayne IN University of Saint Mary Leavenworth KS University of Saint Mary of the Lake/ Mundelein Seminary Mundelein IL University of Saint Thomas St. Paul MN University of Saint Thomas Houston TX University of San Diego San Diego CA University of San Francisco San Francisco CA University of Scranton Scranton PA University of the Incarnate Word San Antonio TX Ursuline College Cleveland OH Villa Maria College of Buffalo Buffalo NY Villanova University Villanova PA Viterbo University La Crosse WI Walsh University North Canton OH Washington Theological Union Washington DC Weston Jesuit School of Theology Cambridge MA Wheeling Jesuit University Wheeling WV Wyoming Catholic College Lander WY Xavier University Cincinnati OH Xavier University of Louisiana New Orleans LA

Our Lady of Corpus Christi Our Lady of Holy Cross College Our Lady of the Lake College Our Lady of the Lake University Presentation College Providence College Queen of the Holy Rosary College Quincy University Regis College Regis University Rivier College Rockhurst University Rosemont College of the Holy Child Jesus Sacred Heart School of Theology Sacred Heart University Saint Ambrose University Saint Anselm College Saint Anthony College of Nursing Saint Bernard's School of Theology and Ministry Saint Bonaventure University Saint Catharine College Saint Charles Borromeo Seminary Saint Edward's University Saint Elizabeth College of Nursing Saint Elizabeth School of Nursing Saint Francis College Saint Francis Medical Center College of Nursing Saint Francis University Saint Gregory's University

Our Lady of Corpus Christi Our Lady of Holy Cross College Our Lady of the Lake College Our Lady of the Lake University Presentation College Providence College Queen of the Holy Rosary College Quincy University Regis College Regis University Rivier College Rockhurst University Rosemont College of the Holy Child Jesus Sacred Heart School of Theology Sacred Heart University Saint Ambrose University Saint Anselm College Saint Anthony College of Nursing Saint Bernard's School of Theology and Ministry Saint Bonaventure University Saint Catharine College Saint Charles Borromeo Seminary Saint Edward's University Saint Elizabeth College of Nursing Saint Elizabeth School of Nursing Saint Francis College Saint Francis Medical Center College of Nursing Saint Francis University Saint Gregory's University

Our Lady of Corpus Christi Our Lady of Holy Cross College Our Lady of the Lake College Our Lady of the Lake University Presentation College Providence College Queen of the Holy Rosary College Quincy University Regis College Regis University Rivier College Rockhurst University Rosemont College of the Holy Child Jesus Sacred Heart School of Theology Sacred Heart University Saint Ambrose University Saint Anselm College Saint Anthony College of Nursing Saint Bernard's School of Theology and Ministry Saint Bonaventure University Saint Catharine College Saint Charles Borromeo Seminary Saint Edward's University Saint Elizabeth College of Nursing Saint Elizabeth School of Nursing Saint Francis College Saint Francis Medical Center College of Nursing Saint Francis University Saint Gregory's University

DiscussionDiscussion

Walsh University

Bennington CollegeBerry CollegeBirmingham-Southern CollegeClark UniversityClarkson UniversityDaemen CollegeFaulkner UniversityFlorida Hosp College of Health SciManchester CollegeManhattanville CollegeMarlboro CollegeMonmouth UniversityMountain State UniversityNaropa UniversityPaul Smiths CollegeSpringfield CollegeStetson UniversityWebster University WorldwideYeshiva University

Bennington College North

Berry College South

Birmingham-Southern College South

Clark University North

Clarkson University North

Daemen College North

Faulkner University South

Florida Hosp College of Health Sci South

Manchester College Midwest

Manhattanville College North

Marlboro College North

Monmouth University North

Mountain State University Midwest

Naropa University West

Paul Smiths College North

Springfield College North

Stetson University South

Webster University Worldwide South

Yeshiva University North

Page 34: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

3434

Ellen Boylan, Ph.D.Director of Institutional Research and Assessment

[email protected]

Sister Jane Wakahiu, LSOSF, MAGraduate Assistant

[email protected]

Office of Planning and Institutional Researchhttp://cwis.marywood.edu/instresearch/activity.stm

Assessing Institutional Effectiveness with the Mission Perception Inventory (MPI):

Linking Mission Goals and Learning Environment

Supported by a grant from

http://www.teaglefoundation.org/grantmaking/grantees/assessmentmethods.aspx

Marywood University

Page 35: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

3535

Page 36: 1 MPI Mission Perception Inventory Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF Marywood University Assessing Institutional Effectiveness with the

3636

MPIMPIMission Perception InventoryMission Perception Inventory

Ellen Boylan, Ph.D. and Sister Jane Wakahiu, LSOSF • Marywood University

Assessing Institutional Effectiveness with the Mission Perception Inventory (MPI): Linking Mission Goals and Learning Environment

North Carolina State University Undergraduate Assessment Symposium

Aligning Pedagogy, Curriculum & Assessment

April 24-26, 2009