marvin w. peterson & thomas e. perorazio the national center for postsecondary improvement,...
Post on 19-Dec-2015
220 Views
Preview:
TRANSCRIPT
Marvin W. Peterson & Thomas E. PerorazioMarvin W. Peterson & Thomas E. PerorazioThe National Center for Postsecondary Improvement, Project 5.2The National Center for Postsecondary Improvement, Project 5.2
The University of MichiganThe University of Michigan
Ross A. Griffith, Ross A. Griffith,
Wake Forest UniversityWake Forest University
American Association for Higher Education American Association for Higher Education Assessment ConferenceAssessment Conference
Denver, CO - June 25, 2001Denver, CO - June 25, 2001
Institutional Strategies for IntegratingInstitutional Strategies for IntegratingStudent Assessment with Student Assessment with
Academic Planning and ManagementAcademic Planning and Management
Session Purposes
Discuss Initial Findings from the Case Study Phase of NCPI Research Project on Institutional Support for Student Assessment
Provide Detailed Examples of Two Institutions That Have Integrated Assessment Into their Academic Management Processes
Outline the Lessons Learned Applicable to Assessment Practitioners
The National Center for Postsecondary Improvement
Main Project Web Site
http://www.stanford.edu/group/ncpi/
University of Michigan - Project Five
http://www.umich.edu/~ncpi/
NCPI Project Area 5:
Student Learning and Assessment
5.3 Academic Programs and Students:
Educational Reform and Innovation in Teaching, Learning, and Assessment Practices
Metadata Project: Review of National Student, Faculty, and Institutional Data Resources
5.1 State Policies and Regional Accreditation Practices of Assessment for Student Learning
5.2 Institutional Support for Enhancing Student Assessment and Performance
5.2 Institutional Support for Enhancing Student Assessment and Performance
Project 5.2’s Purpose
To Examine How
Postsecondary Institutions
Support and Promote the
Uses and Impacts of Student
Assessment Information for
Institutional Improvement
To Examine How
Postsecondary Institutions
Support and Promote the
Uses and Impacts of Student
Assessment Information for
Institutional Improvement
Overview of Project 5.2
Phase OnePhase One
•Review & Synthesis of the Literature •Developed a Conceptual Framework of Institutional Support
Phase TwoPhase Two
National Survey of Institutions•Comprehensive Inventory Based on Conceptual Framework•1394 Usable Responses -- 55.1%
Phase ThreePhase Three
Case Studies of Institutions with Comprehensive Student Assessment Efforts
Phase ThreePhase Three
Case Studies of Institutions with Comprehensive Student Assessment Efforts
Institutional Context
Assessment Culture and Climate
Institutional Approach to Student Assessment
Institution-wide Support for Student Assessment
Assessment Management Practices & Policies
External Influences on Student Assessment
Uses & Impacts of Student Assessment
Phase I: Conceptual Framework of Organizational and Administrative Support for
Student Assessment
Phase II: National InventoryResearch Questions
What Approaches to Student Assessment Have Institutions Adopted?
What Organizational and Administrative Support Patterns and Assessment Management Practices for Student Assessment Have Institutions Developed?
How Have Institutions Used Student Assessment Information and What Impacts Has It Had?
How Do Patterns of Student Assessment Approach, Organizational and Administrative Support Patterns and Practices, and Uses and Impacts of Student Assessment Vary by Institutional Characteristics?
How Do External Groups Influence Institutional Adoption of Various Measures of and Approaches to Student Assessment?
How Are External Influences on Institutional Approaches to and Organizational and Administrative Support Patterns and Practices for Student Assessment Related to the Use and Impact of Student Assessment Information?
Phase III: Case Study Research
Seven Institutions With Extensive Involvement in Student Assessment
•Review of Response Patterns to National Inventory of Institutional Support for Student Assessment (ISSA) •Campus Site Visits by Research Teams•Collection & Synthesis of Documents, & Records •Interviews & Focus Groups With Faculty, Administrators, & Students
Research Purposes
1) To Examine the Internal Dynamics of Institutional Support for Student Assessment
2) To Identify Institutional Strategies for Supporting, Promoting & Integrating Student Assessment
3) Assess the Culture & Climate for Using Student Assessment Data for Academic Improvement
Integrating Student Assessment with Academic Management for Institutional
Improvement
Institution’s Academic Management
Approach
Use in Academic Decision Making
AssessmentPurposeStructureApproach
AssessmentLeadershipSupportCoordination
Links to Management Process/
Improvement
Discussion of Institutions
Student Assessment Strategy Purposes of Assessment Assessment Structure Approach to Assessment Leadership, Support, & Coordination Links to Academic Management Processes Utilization in Academic Decision Making Summary - Culture for Student Assessment
Brief Institutional Storyline Institution’s Academic Management Approach
•Primary Mission & Governance Pattern•Key Academic Management Processes
The Story of
Institution Developed a Quality Initiative Based on a Baldridge
Model
Assessment Adopted as a Means of
Academic ImprovementAssessment Activity Gradually
Integrated into Culture of Quality and Improvement
for Many Years
Primary Mission & Governance Pattern
Vision
•Emphasis on Culture of Quality; Sets Goals
Statement of Mission•Focuses on Continuous Improvement•Committed to Using Assessment to Enhance
Performance
Cultural Core Values•Define the University; Drive the Mission & Vision
Decision Drivers•Priorities Regarding Essential Functions
Strategic Planning Council
Seven StepPlanning Process
Program Management
Student AffairsKQIs
Office of Assessment, Information, & Analysis
Approach to Academic Improvement
Key Academic Management Processes
Assessment Purpose -- NMSU
Initiating Conditions
Internal- President’s Focus on Culture of Quality (early
1980s)
External- State Mandated Assessment by Governor (late
1980s)
President’s Early Efforts ResistedCulture of Quality
Focused on Academic Improvement in Undergraduate Education•Adapted From Baldridge National Quality Award Criteria
Assessment Structure -- NMSU
Formal, University-wide Strategy
Uses Baldridge Quality Award Criteria As an Assessment Framework
Comprehensive Approach -- Cognitive and Affective Learning
College and Post-college Experiences
Collected by by Office of Assessment, Information, and Analysis (OAIA)
Seven Step Planning Process
Assessment Approach -- NMSU
Quantitative & Qualitative Data Gathered From Recruitment Until Post-graduation
Battery of Cognitive Measures During College Years
Regular Collection Schedule
Data on Needs or Expectations of Students Using Local and National Surveys & Listening Posts Assessments are required--Explained in UG Catalog
Student progress tracked and enforced through restrictions on registration and final transcripts
Leadership, Support, & Coordination
Leadership
Long-term, Continuous Presidential Promotion
Strong Support by Provost Through OAIA•Other Groups:
•President’s Cabinet & Deans, Baldridge Category
Council•Faculty Senate Assessment Committee•Student Success Task Force
•Faculty Provide Significant, but Hesitant, Support Coordination
•OAIA-- Compiles and Distributes Assessment Data
•Formal Grid Details Methods, Uses, & Frequencies for Assessment Activity
Links to Academic Management Process/
Improvement
Annual Departmental Reports & Course Reviews
Primary Link Between Action Planning & Assessment
Deans Uses Assessment Data to Identify Areas for
Improvement
Focus Is on Student Performance and Experience DataPresident's Cabinet Reviews Reports & Provides Feedback to Unit
Leaders
Strategic Planning Council
Representatives From Administrative, Academic, and Student Affairs
Assessment Info Used:
•To Evaluate Organizational Performance on Strategic Initiatives (SI) •The Development of New Key Quality Indicators (KQI) for Each SI
Links to Academic Management Process/ Improvement
OAIA Produces Three Types of Academic Information:
1) Outcomes and Performance;
2) Student and Stakeholder Satisfaction Ratings;
and
3) Operational Effectiveness
Data Is Distributed Through Dashboards and Profiles
Tracking Systems to Indicate Deviations From Intended Targets
Can Be Accessed Through Campus Network
Measurement of of Key Quality Indicators
Seven Step Planning Process
•To Meet Stakeholder Requirements
Utilization in Academic Decision Making-- NMSU
Planning
Phase I Review of Mission and Analysis of Progress on Current Goals
Phase II Identification of Strategic Initiatives
Phase III Action Planning and Development
Actions Plans Are Developed to Improve Performance on Key Quality Indicators
•Purpose Is to Increase Organizational Performance
•Assessment Information Part of KQIs in Dashboards & Profiles Critical to Developing Agenda for Planning Efforts
Utilization in Academic Decision Making
Program Management
•Annual Reports & Reviews•Benchmarking•Stretch Goals
Quality
Student Affairs-Academic Affairs Alliance
Development of Student Life & Experiences Surveys/Studies
Links to Educational uses of Assessment Information
NMSU Applies Annually for Baldridge Quality Award
Summary-- Culture for Assessment
Baldridge Quality Criteria Drive Assessment FrameworkIntegration of Assessment into Quality, Planning,
and Program Management Processes
Data-Driven System & Reports
Strong, Central Administrative Support for
Quality Approach & Assessment
Key Coordination Role Played by OAIA --
Commitment to Assessment•As a Means of Visibility for Quality Emphasis•Increasingly for Institutional Improvement
The Story of
Assessment Data Essential to Cycles of Institutional Planning,
Program Review, & Evaluation
Assessment Information Used by Decision Makers at All Levels to
Evaluate Progress Towards Goals
Administration Expects that Assessment Activity Will Occur for
the Purpose of Academic Improvement
Primary Mission & Governance Pattern
No Formal Statements Regarding Assessment
WFU Administration believes Assessment is a Necessary Management Tool to Maintain & Improve its Quality and Reputation
•Obtaining Data Crucial to Decision Making
Institutional Strategic Planning
Evaluation Committee
Institution-wideAssessment
Departmental Review
Student Life
Office of InstitutionalResearch
Approach to Academic Improvement
Key Academic Management Processes
Assessment Purpose -- WFU
Initiating Conditions•Internal- Foundations for Institutional Cycles of Planning Initiated by President & Vice-president
•External- Accreditation Focus on Institutional Effectiveness
Program Planning & Evaluation Committees Use Assessment Data
•To Address Major Campus Needs •Ongoing Strategic Planning
Program Planning & Evaluation Committees Use Assessment Data
•To Address Major Campus Needs •Ongoing Strategic Planning
Evaluate Strategic Plan The Plan for the Class of 2000
•Assessment Now a Management Tool in a Continual Process of Planning, Implementation, and Evaluation
Assessment Structure -- WFU
1) University-wide- OIR- Oversees Assessment of the Process
•Administers National & Local Surveys to Students & Faculty•Analyzes, Stores, & Disseminates Data•Works With Evaluation Committee & Other Depts. •Only Office to Have Dedicated Funds for Assessment
1) University-wide- OIR- Oversees Assessment of the Process
•Administers National & Local Surveys to Students & Faculty•Analyzes, Stores, & Disseminates Data•Works With Evaluation Committee & Other Depts. •Only Office to Have Dedicated Funds for Assessment
Three Level, Decentralized Process
3) Student Life- Focuses on its own programs
•Focus is on student experiences & satisfaction•Important for Intellectual Climate Planning
2) Departmentally Based- Program Review Process •Primary Planning Unit on Campus •13-month Process, 7 Year Cycle for All Programs •Review of Curriculum & Department/ Program Performance•$2,500 Budgeted for Each Review
Assessment Approach-- WFU
Departments Conduct Own Assessment of Programs & Satisfaction
Significant Autonomy & Discretion Granted to Evaluate Their Students
Primarily Quantitative Focus on Satisfaction & Experiences
Little Emphasis on Cognitive or Affective Measures Some Determined by Dept. In Departmental Review
Institution-wide Surveys of Students, Faculty, & Alumni
National Surveys Administered by Office of Institutional
ResearchAlso Some Locally Developed Instruments & Course
Evaluations
Leadership & Support
Institutional Support from Executive
•President’s and Provost’s Seminars, Conferences, & Retreats
•Executive Council & Reynolda Cabinet Receive Reports & Use the Information in Strategic Planning
Coordination
OIR Shares Its Information With Departments Upon Request
Student Life Provides Its Data to Departments
Information Systems Office Sends Course Evaluation Results to Department Heads Online
OIR Role in Accreditation Allows It to Maintain Focus on Institutional Effectiveness Criteria
OIR Has the Primary Role In:
•Administering Institution-wide Student Assessments•Coordinating Program Planning•Conducting Data Analysis
Information Flow/ Link to Improvement
Data Driven Approach to Management•Measuring Progress on Strategic Plan Goals Results in Steps Toward Improvement
OIR & Departments Report to Evaluation Committee (EC) for the Strategic Plan
Reviews Departmental Reports on How SA Info Was Used to Monitor Progress on Goals
EC Monitors 27 Areas of QualityEC Members Serve As Liaisons to Other Committees and Departments
Annual Reports by Departments to Deans•Must State Progress on Goals and Set New Ones
Utilization in Academic Decision Making
Institutional Strategic Planning•Prior Data Streams Produced the Development, Implementation
and Evaluation of Plan for the Class of 2000
•Recent Student Life Data Critical to Next Wave of Planning on Intellectual Climate
Program Review & Departmental Evaluations
•One Full Cycle Completed
•Departmental and Curricular Changes, including Program
Restructuring or Elimination
•Departments use OIR Data & their own to Improve Teaching & Learning
Utilization in Academic Decision Making
Evaluation Committee•Assesses Laptop Initiative and Effect of Technology on Academics
•Purpose Remains Measurement of Progress Towards Academic & Educational Goals
Student Life•Units Engage in Planning & Evaluation
•Many Changes to Services Made Based on Satisfaction Survey Results
Summary-- Culture for Assessment
Academic Management is Well-designed
Integrated Institutional Planning, Program Review, & Academic Evaluation Approach that is Improvement Driven
Student Assessment Integrated into All Academic Management Processes
Strong, Central Support for Data Driven Management
Intensive Involvement of Departments in Program Review
Key Support & Coordination Roles Played by OIR
Commitment to Assessment is Embedded in Culture, but Not as an End in ItselfWide Support for Student Assessment
Conclusions
Lessons & Insights
Emphasis is on Integrating Student Assessment into Key Elements & Academic Decision Making Arenas:
Assessment Used at All Levels of Institution
Tied to Academic Improvement
Embedded in Academic Management Policies & Practices
Assessment Data Key to Management & Evaluation Processes
Need a Clear Understanding of: The Institution’s Academic Management Approach
& its Key Elements
The Role of Assessment in Their Various Elements
Lessons & Insights An Effective Student Assessment Strategy Requires:
Clearly Understood Purpose for Assessment
Committed, Upper-Level Leadership
Coordination Across Support Offices &
Departments
Flow of Information to Decision Makers at All
Levels
Link of Data to Problem Identification and Goal
SettingBuilding a Culture that Values Assessment
Requires: Long-term Effort Linking Assessment to Academic Management Processes Concerned with Academic Improvement
Wake Forest Case Study Visit September 2000
Wake Forest Honored to Be Selected for the Case Study
Faculty, Students and Administrators Were Most Cooperative in Participation of Interviews and Focus Groups
Interviews Included President, Vice Presidents, Deans,
Associate Provost, Advising Director, Director of Teaching
& Learning Center, Director of Institutional Research,
Chief Information Officer, Faculty Senate President, Chair
of Curriculum Committee, Chairs of Academic
Departments and Members of Evaluation Committee Focus Groups Included Four Faculty Members and Four
Departmental Chairs Active in Assessment and 10 “Good” Students
Lessons Learned Considerable Assessment With Decided
Outcomes Took Place at Institutional Level, e.g. Program Review and Surveys of Students and Faculty
Assessment at Departmental Level Is Decentralized and Varies Considerably
Qualitative Assessment Could Be Emphasized More at the Institutional Level
Combination of SACS 1995-97 Self Study and 1992-95 Strategic Plan Strengthened Assessment at Wake Forest
top related