Transcript
Page 1: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

MeasuringInstitutional Success:

An Electronic Portfolio forAccreditation and Assessment

Outcomes

Vivienne CarrRosemary SkeeleDavid Middleton

Page 2: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Context - Seton Hall University

• Private, mid-sized, northeastern university• Technology accepted & promoted – mobile,

wireless computing in place• IT courses for University located in College of

Education

University Mission Statement

Seton Hall University is a major Catholic university. In a diverse and collaborative environment it focuses on academic and ethical development. Seton Hall students are prepared to be leaders in their

professional and community lives in a global society and are challenged by outstanding faculty, an evolving technologically advanced setting and values-centered curricula.

Page 3: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

What We Have Learned

• A portfolio is not a container – It is a process

• A portfolio is adaptable to any university program

• “Every act of creation is an act of destruction”

• The results of the process are revealing

Page 4: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Electronic Portfolio Institutional Initiative

History of Electronic Portfolios at Seton Hall

Curriculum Development Initiative (CDI):Electronic Portfolio

6 Branches of Project:

• College of Education and Human Services

• Psychology Department

• English Department

• Honors Program

• Freshmen Studies

• Outcomes Assessment Team

Page 5: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Electronic Portfolio in Higher Education - Considerations

• Academic Freedom

• New Role of the Teacher

• Technological Equity

• Resources

Page 6: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Faculty Evolution

• Personal development – skills, comfort, needs, exploration – CBAM Model

• Curriculum development – teaching, modeling, pedagogical reflection

• Team development – assisting, supporting,

collaborating

Page 7: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Outcomes Assessment Team (OAT)

• Established by the Office of the Provost in 1991

• Examining methods of integration across University Community

• Developing matrices of assessment in conjunction with SHU’s Core Curriculum Committee

Process:• Begin process at the department level• Department chairs to head the process• Devise a process

1. Construct syllabi with outcomes and related assessments2. Develop a program matrix3. Concurrently devise a university-wide content management system

Page 8: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Numeracy

Oral Comm.

Writing

Information fluency

Critical thinking

Core skills

•Specific metrics developed for integration into at least 50% of a student’s courses

•Conscious development of these habits of mind as part of coursework

Page 9: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Electronic Portfolios and the Core Curriculum

• Database of courses certified as fulfilling the Core requirements

• Ongoing checklist of student’s fulfilling Core requirements for advising

• Depository of objects attesting to student’s skill development

• Opportunity for student reflection on the learning process

Page 10: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Accountability of all parties

System of institutional assessment

Public accountability of effectiveness of

programs

Faculty accountability for quality of work products

Record for evaluation of success

of Core

Assessment of student learning

Assessmentvia

ElectronicPortfolios

Page 11: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

College of Education and Human Services

Educational Studies Department

• History of paper portfolios

• PT3 Grant in 1999 sparked commitment to technology- based student portfolios

• Evaluation by professional societies impacts delivery decisions

• Current leader in SHU: Electronic Portfolio Initiative

Page 12: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

College of Education and Human Services

Required Flexibility in Portfolio Development

• NCATE Influenced – Performance Evidence

• Demonstrate & Motivate Student Learning

• Reflection of the Student as a Teacher

• Development of Student Professional Portfolios

Page 13: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Portfolio Product Evolution

• Paper

• Mini disks (floppy)

• CD-ROM

• Web-Based (Front Page, On-line free & low cost services)

• Specialized product

• Out of the Box product customized by SHU

Page 14: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

CEHS Standards Electronic Portfolio

• Vehicle for assessment in demonstrating learning outcomes linked to accreditation standards

• Student products yield accomplished work in the field of education

• All work is coupled with student reflections stating how products meet accreditation standards

• This practice demonstrates the students’ knowledge and understanding of the standards

Page 15: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

How We Met Our Goals: A Process

Step 1: Determine what a portfolio means to your program. Define goal

Step 2: Identify what we want students to learn. Set objectives• Align with mission• Look at institutional, state, and national standards• Research other programs

Step 3: Describe content and tasks. Develop curriculum & activities• Map Curriculum 1 2• Learn and integrate new teaching practices• Revise syllabi

Step 4: Determine whether we met our objectives. Assess & revise learning

• Develop evaluation instruments; Create rubrics• Analyze results• Revise curriculum based on results

Page 16: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

• College of Education and Human Services is NCATE Accredited.

• Also meets INTASC (Interstate New Teacher Assessment and Support Consortium) and NJPS standards (New Jersey Professional Standards).

Standards Building Block

Student’s Dilemma

Page 17: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Why Blackboard Electronic Portfolios?• Comfort level – students, faculty, staff• Training and support issues• Multiple uses for files• Easy to share files across courses, Electronic Portfolio,

and people• Provides security not available on Web• Accepts variety of media• Let’s you build a homepage

Page 18: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Content ManagementSystem

Student/Faculty

Content Management

Peer Review

Collaboration

AssessmentResource Sharing

Page 19: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Blackboard Course

Blackboard Course

Blackboard Course

Faculty Content Management System

Page 20: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Content Management System

Course/Core Portfolio

Professional Portfolio

Faculty

AdvisorDepartment Alumni Employer

Career Center

Page 21: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Students can access the Electronic Portfolio from their Main Page

Page 22: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Linking Student Products to Pertinent Standards

Page 23: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Products are connected to the Blackboard Content and Learning Management Systems

Page 24: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Our application for participation in the Center for Academic Transformation's Roadmap to Redesign Programs is an example of our understanding of the importance of effective utilization of Blackboard as a Mission Critical Application.

Adding/Editing Products and Reflections

Products can be connected to multiple standards

Page 25: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Automatically Generated Electronic PortfolioCombines Products and Reflections

Page 26: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Standards Data Reporting

The system is built to allow for custom

reporting according to needs.

Page 27: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Standards Data Reporting

The data can be exported for

analysis in either Excel or SPSS.

Page 28: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Standards Electronic Portfolio Version 2.0

Page 29: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

1. Faculty Member creates “Standards Based Assignment”

2. Assignment Appears as Typical Blackboard Assignment

3. Students Complete the Assignment by attaching the artifact

4. Entry in the grade book is the same as others. Faculty Grade this entry as normal

5. Grading UI is the same as typical Assignment

6. The Achievement of a specific level grade would generate the ePortfolios

Standards Electronic Portfolio Version 2.0Workflow

Page 30: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

• Standards Based Assignment

• Rubric Maker & Grading Mechanism

• Assessment Tools:• Ability to assess how students are doing across multiple sections and on specific subject matter - example: lesson planning

• Filters – e.g. department/program; gender, race, etc; rubric portions; specific dates

• Demographics

• Track individual development

Standards Electronic Portfolio Version 2.0Future Features

Reporting will provide more granular data for Inter-rater

reliability.

Page 31: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Plans for Institutional Change

• Development and Implementation of the Standards Electronic Portfolio completed Summer 2005

• Used by over 50% of the faculty from College of Education and Human Services - Summer 2005

• Implementation and customization of the system across the Institution - Fall 2005

• Integration of the system in the assessment plans for the new University Core Curriculum

Page 32: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

• Demonstrates faculty excellence

• Shows support from administration• Initiates a culture change• Provides Accreditation Reporting • Demonstrates Institutional-wide • Contributes to Assessment and Program Review• Meets Standards & Core Curriculum Proficiency Measures• Provides a bridge to the Outcomes Assessment Team, Institutional Research & Budget Offices• Provides Personal and Professional growth for students

Outcomes

Page 33: Measuring Institutional Success: An Electronic Portfolio for Accreditation and Assessment Outcomes

Electronic PortfoliosMoving our Students into the Future


Top Related