MeasuringInstitutional Success:

An Electronic Portfolio forAccreditation and Assessment

Outcomes

Vivienne CarrRosemary SkeeleDavid Middleton

Context - Seton Hall University

• Private, mid-sized, northeastern university• Technology accepted & promoted – mobile,

wireless computing in place• IT courses for University located in College of

Education

University Mission Statement

Seton Hall University is a major Catholic university. In a diverse and collaborative environment it focuses on academic and ethical development. Seton Hall students are prepared to be leaders in their

professional and community lives in a global society and are challenged by outstanding faculty, an evolving technologically advanced setting and values-centered curricula.

What We Have Learned

• A portfolio is not a container – It is a process

• A portfolio is adaptable to any university program

• “Every act of creation is an act of destruction”

• The results of the process are revealing

Electronic Portfolio Institutional Initiative

History of Electronic Portfolios at Seton Hall

Curriculum Development Initiative (CDI):Electronic Portfolio

6 Branches of Project:

• College of Education and Human Services

• Psychology Department

• English Department

• Honors Program

• Freshmen Studies

• Outcomes Assessment Team

Electronic Portfolio in Higher Education - Considerations

• Academic Freedom

• New Role of the Teacher

• Technological Equity

• Resources

Faculty Evolution

• Personal development – skills, comfort, needs, exploration – CBAM Model

• Curriculum development – teaching, modeling, pedagogical reflection

• Team development – assisting, supporting,

collaborating

Outcomes Assessment Team (OAT)

• Established by the Office of the Provost in 1991

• Examining methods of integration across University Community

• Developing matrices of assessment in conjunction with SHU’s Core Curriculum Committee

Process:• Begin process at the department level• Department chairs to head the process• Devise a process

1. Construct syllabi with outcomes and related assessments2. Develop a program matrix3. Concurrently devise a university-wide content management system

Numeracy

Oral Comm.

Writing

Information fluency

Critical thinking

Core skills

•Specific metrics developed for integration into at least 50% of a student’s courses

•Conscious development of these habits of mind as part of coursework

Electronic Portfolios and the Core Curriculum

• Database of courses certified as fulfilling the Core requirements

• Ongoing checklist of student’s fulfilling Core requirements for advising

• Depository of objects attesting to student’s skill development

• Opportunity for student reflection on the learning process

Accountability of all parties

System of institutional assessment

Public accountability of effectiveness of

programs

Faculty accountability for quality of work products

Record for evaluation of success

of Core

Assessment of student learning

Assessmentvia

ElectronicPortfolios

College of Education and Human Services

Educational Studies Department

• History of paper portfolios

• PT3 Grant in 1999 sparked commitment to technology- based student portfolios

• Evaluation by professional societies impacts delivery decisions

• Current leader in SHU: Electronic Portfolio Initiative

College of Education and Human Services

Required Flexibility in Portfolio Development

• NCATE Influenced – Performance Evidence

• Demonstrate & Motivate Student Learning

• Reflection of the Student as a Teacher

• Development of Student Professional Portfolios

Portfolio Product Evolution

• Paper

• Mini disks (floppy)

• CD-ROM

• Web-Based (Front Page, On-line free & low cost services)

• Specialized product

• Out of the Box product customized by SHU

CEHS Standards Electronic Portfolio

• Vehicle for assessment in demonstrating learning outcomes linked to accreditation standards

• Student products yield accomplished work in the field of education

• All work is coupled with student reflections stating how products meet accreditation standards

• This practice demonstrates the students’ knowledge and understanding of the standards

How We Met Our Goals: A Process

Step 1: Determine what a portfolio means to your program. Define goal

Step 2: Identify what we want students to learn. Set objectives• Align with mission• Look at institutional, state, and national standards• Research other programs

Step 3: Describe content and tasks. Develop curriculum & activities• Map Curriculum 1 2• Learn and integrate new teaching practices• Revise syllabi

Step 4: Determine whether we met our objectives. Assess & revise learning

• Develop evaluation instruments; Create rubrics• Analyze results• Revise curriculum based on results

• College of Education and Human Services is NCATE Accredited.

• Also meets INTASC (Interstate New Teacher Assessment and Support Consortium) and NJPS standards (New Jersey Professional Standards).

Standards Building Block

Student’s Dilemma

Why Blackboard Electronic Portfolios?• Comfort level – students, faculty, staff• Training and support issues• Multiple uses for files• Easy to share files across courses, Electronic Portfolio,

and people• Provides security not available on Web• Accepts variety of media• Let’s you build a homepage

Content ManagementSystem

Student/Faculty

Content Management

Peer Review

Collaboration

AssessmentResource Sharing

Blackboard Course

Blackboard Course

Blackboard Course

Faculty Content Management System

Content Management System

Course/Core Portfolio

Professional Portfolio

Faculty

AdvisorDepartment Alumni Employer

Career Center

Students can access the Electronic Portfolio from their Main Page

Linking Student Products to Pertinent Standards

Products are connected to the Blackboard Content and Learning Management Systems

Our application for participation in the Center for Academic Transformation's Roadmap to Redesign Programs is an example of our understanding of the importance of effective utilization of Blackboard as a Mission Critical Application.

Adding/Editing Products and Reflections

Products can be connected to multiple standards

Automatically Generated Electronic PortfolioCombines Products and Reflections

Standards Data Reporting

The system is built to allow for custom

reporting according to needs.

Standards Data Reporting

The data can be exported for

analysis in either Excel or SPSS.

Standards Electronic Portfolio Version 2.0

1. Faculty Member creates “Standards Based Assignment”

2. Assignment Appears as Typical Blackboard Assignment

3. Students Complete the Assignment by attaching the artifact

4. Entry in the grade book is the same as others. Faculty Grade this entry as normal

5. Grading UI is the same as typical Assignment

6. The Achievement of a specific level grade would generate the ePortfolios

Standards Electronic Portfolio Version 2.0Workflow

• Standards Based Assignment

• Rubric Maker & Grading Mechanism

• Assessment Tools:• Ability to assess how students are doing across multiple sections and on specific subject matter - example: lesson planning

• Filters – e.g. department/program; gender, race, etc; rubric portions; specific dates

• Demographics

• Track individual development

Standards Electronic Portfolio Version 2.0Future Features

Reporting will provide more granular data for Inter-rater

reliability.

Plans for Institutional Change

• Development and Implementation of the Standards Electronic Portfolio completed Summer 2005

• Used by over 50% of the faculty from College of Education and Human Services - Summer 2005

• Implementation and customization of the system across the Institution - Fall 2005

• Integration of the system in the assessment plans for the new University Core Curriculum

• Demonstrates faculty excellence

• Shows support from administration• Initiates a culture change• Provides Accreditation Reporting • Demonstrates Institutional-wide • Contributes to Assessment and Program Review• Meets Standards & Core Curriculum Proficiency Measures• Provides a bridge to the Outcomes Assessment Team, Institutional Research & Budget Offices• Provides Personal and Professional growth for students

Outcomes

Electronic PortfoliosMoving our Students into the Future