judith herb college of education - university of toledo · judith herb college of education...

Judith Herb College of Education

Assessment Plan

Noela A. Haughton, Ph.D.

Spring 2010

I. Overview of the College of Education

Officially, the Judith Herb College of Education (JHCOE) is the professional education unit at

The University of Toledo, and the JHCOE Dean is responsible for all professional education

programs within this college. He also oversees 4 professional education programs in the College

of Health and Human Service (HSHS) and dual degree programs offered in collaboration with

the College of Arts and Sciences (A&S). During fall 2009 semester, the unit enrolled 1,492 full-

and part-time candidates in initial programs and 716 candidates in advance programs. An

additional 115 advanced candidates were also enrolled in professional pupil services programs at

the advanced HSHS. The college and its programs are accredited by the National Council for

Accreditation of Teacher Education (NCATE). In addition to meeting state program standards, the

professional education unit at The University of Toledo is committed to meeting the standards of the

Specialized Professional Associations (SPAs) and specialty area accreditation agencies. Currently, 31

programs in the unit are either recognized by their respective SPAS or nationally accredited by the

following organizations:

American Council on the Teaching of Foreign Languages

American Association for Health Education

Council for Exceptional Children

Educational Leadership Constituent Council

International Reading Association

International Society for Technology in Education

National Association for Gifted Children

National Association for Sport and Physical Education

National Association of School Psychologists

National Association for the Education of Young Children

National Council for the Social Studies

National Council of Teachers of English

National Council of Teachers of Mathematics

National Middle School Association

National Science Teachers Association

American Speech and Hearing Association

Council for the Accreditation of Counselling and Related Programs

National Association of Schools of Art and Design

National Association of Schools of Music

All programs are also approved by the Ohio Board of Regents (OBOR) / Ohio Department of

Education (ODE).

a. Vision

The intent of the College of Education is to provide learning opportunities that place students at

the center of their own learning. To achieve this, the College provides students with rich

intellectual learning experiences characterized by choice. The intent is to produce, place, and

sustain well-prepared educators capable of making decisions and crafting rich educational

experiences for students in their charge. These competent professional educators will have both

the knowledge base and experiences to participate fully in creating educational systems that

respond fully to the needs of citizens in diverse situations, including the metropolitan community

we serve.

b. Mission

The mission of the College of Education is to develop highly skilled, knowledgeable, and

creative teachers and other educational leaders through high quality undergraduate, graduate and

professional development programs. These programs are grounded in professional standards of

excellence and contemporary research reflecting experimentation, innovation, accountability and

informed by the wisdom of professional practice. The College of Education is committed to

continuous improvement through assessment, inquiry, reflection, and lifelong learning. The

College’s mission is consistent with the University’s student centered mission that promotes

integrated learning, discovery and engagement, celebration of human diversity, a disposition of

respect for individuals, freedom of expression, and community engagement.

c. Conceptual Framework

Four distinctive and interrelated elements:

Standards based curricula

Inquiry, reflection, assessment and accountability

Metropolitan focus

Engagement with professional practice.

Candidate Dispositions

Meeting professional and institutional standards

Continuous improvement & lifelong learning

Ongoing assessment, reflection & inquiry in professional practice

Responsiveness to individual & cultural differences

• Connecting with schools & community

• Accountability for student learning & professional development

II. Introduction to Assessment Plan

a. Defining the Unit of Assessment

The unit is defined as the administrative body at the university that has primary responsibility for

all degree programs (i.e., graduate and undergraduate programs which may or may not include

teacher licensure). Some programs (i.e., School Counseling, School Psychology, School

Speech/Language Pathology, School Nurse, Visual Art Education, Music Education, and health

education) may be administratively housed in a college other than education. Therefore, for the

purpose of this plan, the previously-named programs will be excluded because they will be

included in their respective administrative home colleges’ plans.

b. Defining the Level of Assessment

Assessment of unit operations is based on student outcomes (candidate performance) and, to a

lesser extent, faculty performance, and overall unit efficiency and accomplishments. Outcomes

are primarily tracked at the department level, within programs, with some at the unit level. Most

program completion outcomes lead to the traditional academic degree – bachelors, masters,

specialist, and doctorate. Other outcomes are influenced by professional and state licensure

requirements, and may not lead to degree completion.

Student outcomes, faculty performance, and unit effectiveness are so intertwined that acceptable

performance in one, and ultimately target performance, can only be achieved if the

accomplishments of the other two are of equivalent merit. The following represent the different

sources of evidence and types of assessments used currently used by the JHCOE to evaluate its

overall performance.

i. Individual Records

Individual records of student performance and academic outcomes are used to determine

admission to the College, admission to Professional Education, readiness to student teach, and

recommendation for initial licensure. Individual records for advanced program candidates

include admission to Graduate School and the College of Education, completion of academic

core requirements, and readiness for culminating experience.

ii. Surveys

Surveys provide systematic, comprehensive feedback to investigate program and unit

effectiveness. These surveys provide data from both internal and external constituents.

iii. Institutional Data

Institutional data such as courses grades and grade point averages provide information to

evaluate performance and effectiveness of the Education unit within the University and in

comparison with other colleges of education throughout Ohio and the nation.

III. The Assessment Plan (Licensure Programs)

Appendices A through F provides a synopsis of the assessment plans for each program area. the

following sections provides additional details regarding the nature of and uses of assessment

data.

a. Assessments Directed Toward External Agencies

The assessment systems for candidates in initial and advanced licensure programs include

components to monitor candidate performance that have been implemented over a period of

many years. Consequently, the external data collection is ongoing as required by external

accreditation agencies.

i. Timelines

Table 1 details the unit’s current timeline for developing, implementing, and analyzing the

individual components of its assessment plan and for outlining plans to use assessment data for

external reviews.

Table 1: External Assessment Timelines – Initial & Advanced Licensure

External Assessments From

Whom

Level Sub-

System

Technology Responsibility Collected Summarized

& Analyzed

Formats

ODE Report of PRAXIS

II Pass Rates: Title II

candidates Initial &

advance

Licensure electronic ETS /

licensure

officer

Each

semester

Annually

(Title II) and

As needed

Tables /

reports /

charts

ODE Report of PRAXIS

III Pass Rates

candidates Initial Licensure electronic ODE Annually Annually Tables /

data

Teacher Quality

Partnership (TQP) Pre-

service survey

candidates Initial External

Surveys

electronic TQP project /

Assessment

Coord

2003 - 2008 Each

semester

Tables /

reports

Teacher Quality

Partnership (TQP) In-

service survey


Surveys

electronic TQP project 2003 - 2008 Each

semester

Tables /

reports

Other National Surveys

(NSSE, CIRP, etc.)


Surveys

electronic University

institutional

research

Every two

years

Every two

years

Tables /

reports

ii. National Council for Accreditation of Teacher Education (NCATE)

1. Process of Assessment

This review includes all programs for the preparation of professionals, both teachers and other

school personnel, to work in P-12 school settings. The College had its onsite visit in January

2010. The preliminary results are that the College passed all six standards for initial

(undergraduate) and advanced (graduate) programs. The final decision will be rendered in

October 2010 when the NCATE Unit Accreditation Board (UAB) meets.

The review is based on the NCATE Unit Standards, a set of research-based national standards

developed by all sectors of the teaching profession. In NCATE’s performance-based system,

accreditation is based on evidence that demonstrates that teacher candidates know the subject

matter and can teach it effectively so that students learn. In the NCATE system, units must

prove that candidates can connect theory to practice and be effective in an actual P-12 classroom.

The education unit must be engaged in continuous assessment and development.

2. Modifications Made to Date

The Adolescent/Young Adult Integrated Language Arts program is currently under

review. A tighter alignment of courses with the state standards and Praxis II content

specialty test is required in the program.

All programs at the initial licensure level have continued the development and

assessment of the eight critical performances to determine their value and

appropriateness.

Departments are reviewing additional candidate dispositions that were not included in

the conceptual framework for possible inclusion as program requirements.

iii. Ohio Department of Education Review for Program Approval


The Ohio Department of Education regularly reviews all state licensure programs, both

undergraduate and graduate initial licensure, for alignment with state standards. Changes in

programs or in coursework are reported annually, and additional documents validating

compliance with new standards or regulations are submitted as directed by the state. UT’s

professional education unit has complied with all state requirements for program review.

2. Modification Made to Date

Note: The ODE Accreditation Review was conducted entirely in tandem with the

NCATE Review

The Adolescent/Young Adult Science programs are currently in revision. The

excessive number of hours required to achieve two bachelor’s degrees in the science

content areas was discussed during the ODE review. Concern was expressed due to the

high need for science teachers in the state of Ohio. Single degree programs in science

are currently moving through the college, university, and state approval processes.

Alignment of the content in Middle Childhood and AYA science and social studies

programs with K-12 content standards has begun as required by ODE. Matrices will be

submitted to ODE by June for all levels. Elective courses that do not meet the ODE or

SPA standards will be eliminated as course options for students to take in these revised

programs.

iv. Specialized Professional Associations Review of Programs


The professional education unit submitted a majority of its eligible programs for review and

approval, or for accreditation, by the specialized professional associations recognized by

NCATE. Programs that are “recognized” by the SPAs, are considered to be nationally

recognized and the program meets the current national standards for program quality in that

field. Many of the SPAs require candidate performance assessments in which students

demonstrate competency in the discipline. At present, 30 professional education programs at UT

are approved or accredited by 12 specialized professional associations (SPAs) in partnership with

NCATE.


Greater attention paid to alignment with standards in content courses outside of COE.

Revision of syllabi in courses outside of the COE to validate content coverage.

v. Ohio Department of Education Report of PRAXIS II


The Higher Education Act (Title II) requires that each state submit to the U.S. Department of

Education an annual report regarding the quality of its teacher education programs. Each year,

the Ohio Department of Education compiles pass rate data on its licensure test, the PRAXIS II

for The University of Toledo and the other 50 Ohio colleges and universities that prepare

individuals to teach pre-kindergarten through grade 12, both public and private. The PRAXIS II

measures a candidate’s knowledge of the subject matter he/she will be teaching as well as

general knowledge. The Report on the Quality of Teacher Education in Ohio represents the

performance of all program completers on the PRAXIS II from the state’s teacher education

programs. This report is designed to assist institutions of higher education and other education

stakeholders in identifying strengths and determining issues that need to be addressed if the state

is to succeed in its efforts to provide caring, competent and highly-qualified teachers to serve

Ohio students.

Candidates seeking Ohio licensure take the PRAXIS II examination, as required by ODE. At this

time, UT does not require successful completion of the PRAXIS II for admission to student

teaching or as a requirement for graduation. The pass rates for program completers seeking Ohio

licensure have ranged between 89% and 92% over the years since the testing requirement was

first implemented. Inasmuch as Educational Testing Service does not provide information that

will permit an item analysis of candidates’ scores, the measure does not readily lend itself to

formative assessment and/or overall program improvement. The state has been unsuccessful in

its attempts to leverage ETS for this information.


Faculty in the Career and Technical Education (CTE) program found that their students

were having a difficult time passing the PRAXIS II exam. As a result of examining their

test data, in the spring of 2003 the CTE faculty developed a series of review sessions that

were delivered via interactive video-conferencing. Using this delivery system, students

participated from remote sites across Northwest Ohio. All program areas are considering

this and other methods of PRAXIS II exam preparation.

The College is currently considering requiring students to submit a copy of their PRAXIS

II test scores when they apply for licensure. However, these data would not include the

students’ prior attempts to pass the PRAXIS II, therefore, would not be a clear indicator

of program weaknesses.

vi. Ohio Department of Education Report of PRAXIS III


The State of Ohio is currently transitioning away from the PRAXIS III performance-based

assessment. New metrics are currently being piloted. The remainder of this plan refers to the

PRAXIS III licensure framework. However, the College will begin its transition to the new

metrics once these are officially adopted by the State.

Forty-one graduates of The University of Toledo’s initial licensure programs were evaluated as a

part of the pilot and first year of implementation using the PRAXIS III. Ninety-five percent

scored at or above the cut-off score of 9.5 on Domain A (Organizing Content Knowledge for

Student Learning). All participants scored at or above the cut-off scores for Domain B (Creating

an Environment for Student Learning), Domain C (Teaching for Student Learning), and Domain

D (Teacher Professionalism), and all 41 exceeded the cut-off for the total score. Many UT

graduates were rated significantly higher than the minimum score for each of the domains.


UT had 27 beginning teachers complete the PRAXIS III assessment during its

implementation year. These candidates must successfully complete this assessment to

obtain their 5-year Professional License. Of that number 26 teachers passed which

resulted in a 96% pass rate. A subsequent PRAXIS III assessment of this UT graduate

resulted in a passing score in all areas. The pass rate for the state is 99.54%. All

programs are reviewing their course work to determine where the domains and criteria

for the PRAXIS III assessment process are taught and practiced by teacher candidates.

Course changes will be documented at the departmental level as they occur.

As more test results for PRAXIS III are available, the College will use its Electronic

Assessment System to correlate the students’ test scores on PRAXIS I, PRAXIS II, and

PRAXIS III to determine if success on PRAXIS I is a predictor of later success on

PRAXIS II or PRAXIS III.

b. Assessments Directed Toward Internal Review

In the COE the assessment systems for candidates in initial teacher preparation programs, for

candidates in advanced programs, and for faculty include internal components to monitor

candidate performance that have been implemented over a period of many years.

i. Timelines

Table 2 details the unit’s current timeline for developing, implementing, and analyzing the

individual components of its assessment efforts and for developing plans to use assessment data

for internal reviews.

ii. Assessment of Initial Licensure for Teacher Education Candidates

Assessment of candidates in initial programs begins with admission to the college and later into

professional education, and continues with assessment at appropriate transition points and at

program completion.

Table 2: Internal Assessment Timelines

Internal Assessments From

Whom

Level Sub-

System

Technolog

y

Responsibility Collected Summarized

&

Analyzed

Formats

Praxis I (current cut score

requirement)

candidates U/grad

initial

Licensure CDs /

candidate

records

ETS / Student

Services

Twice per

year /as

needed

Case by case Tables /

datasets

Advance Candidate Survey candidates grad Internal

Surveys

electronic Graduate dean

/ assessment

coord

annually annually Tables /

reports

UT Cooperating Teacher

Survey

Coop

teachers

Initial Internal

Surveys

electronic Assessment

coord

Each

semester

Each

semester

report

UT Supervisor Survey supervisor Initial Internal

Surveys


coord

Each

semester

Each

semester

report

UT Student Field

Experience Exit Survey

candidates Initial Internal

Surveys


coord

Each

semester

Each

semester

Tables /

reports

UT Graduate Program candidates Advance Internal

Surveys


coord /

graduate dean

Annually annually Tables /

reports

UT Early Career Teacher

Survey

candidates All levels Internal

Surveys


coord

Every 3-5

years

Every 3-5

years

Tables /

reports

Critical Performances candidates Initial EAS electronic Assessment

coord /

program

faculty

Each

semester

As needed Tables /

reports

Student Teaching Portfolio candidates Initial EAS electronic Assessment

coord /

Program

faculty

Each

semester

As needed Tables /

reports

Advance program

assessments

candidates Advance EAS /

Excel


coord /

Program

faculty

Each

semester

As needed Tables /

reports

Course grades candidates All levels Banner electronic University IT Ongoing As needed Tables /

reports

Record of Formal Student

Complaints

candidates All levels Student

Records

paper Undergraduate

dean / student

services

Ongoing Ongoing Reports


Undergraduate candidates must pass the PRAXIS I examination with a minimum score of 172 in

reading, writing, and mathematics for admission to professional education. Since the PRAXIS I

is a basic skills exam, students in initial licensure at the master’s level are not required to take it.

It is assumed that competency has occurred through the completion of a bachelor’s degree.

Following acceptance into professional education, the College of Education has developed a set

of Critical Performances that all candidates must successfully complete. These Critical

Performances were identified as the major tasks that are necessary for success in the classroom.

Proficiency is demonstrated by submission of an artifact that is scored on a three-point rubric

(i.e., Unsatisfactory, Proficient, and Distinguished). The Critical Performances are:

1. Community/School Context

2. Unit Plan

3. Videotaped Lesson with Reflection

4. Assessment of Student Learning

5. Analysis of Teaching and Professional Growth

Artifacts for each Critical Performance may vary by program area, but all artifacts provide clear

evidence of the knowledge and skills necessary to meet the standards.

Candidates in initial licensure programs are also required to complete a Student Teaching

Portfolio while they are student teaching. Based on the unit’s critical performances, portfolios

created by UT’s candidates have been adapted from the work of the Interstate New Teacher

Assessment and Support Consortium (INTASC). Each candidate portfolio includes multiple

samples of teaching that reflect actual tasks, knowledge, and skills deemed necessary for

effective practice. Candidates are directed to systematically document their work by including

goals for student learning, plans for achieving their goals, lesson plans, activities, materials,

assessment tools, samples of actual teaching, and reflection and analysis of their own teaching.

The protocol used for assessment of the field-based experiences and student teaching is based on

PRAXIS/Pathwise standards. Candidates are assessed at mid-term and at end-of-term by both

University Supervisors with PRAXIS/Pathwise training, and Cooperating Teachers using the

four domains and nineteen criteria of this system.

The candidate portfolio assessment system was piloted during the Spring Semester 2003, with

approximately 100 candidates in varying stages of program completion. Limited portfolios also

were piloted for candidate assessment in the Educational Administration and Supervision

program. Both paper and electronic portfolios were produced during this period. Data collected

from this pilot will analyzed to refine and revise the piloted system and then alter programs.

Lastly, the Ohio Department of Education initiated the PRAXIS III evaluation for all first year

teachers during the academic year 2002-2003. PRAXIS III evaluations are conducted by trained

observers and involve both observation and paper assessments of teaching competence. The

protocol used for this assessment is similar to the system used for all candidates during their

field-based experiences during their preservice training.


Undergraduate students must meet all program requirements before they are fully

admitted into or are permitted to advance through the professional education series of

classes.

Undergraduate students must demonstrate proficiency on critical performances

throughout their programs.

Additional training programs for university supervisors and cooperating teachers are

under development.

Advisers are assigned to specific program areas to handle advising issues in the College.

The unit has introduced PLATO, a computer based program designed to prepare

candidates for the PRAXIS I. Candidates can take PRAXIS I practice tests in reading,

writing, and mathematics and receive immediate results. This information should assist

students in determining if they are ready to take the PRAXIS I.

It was also noted during Spring Semester 2000 that applicant performance on the writing

section of the PRAXIS I was below preferred levels. Following up on this observation,

student programs of study were reviewed and it was determined that potential candidates

who successfully completed two semesters of composition earned significantly higher

scores on the writing subtest than those who undertook only one course in composition.

Consequently, students in Education were advised to complete two semesters of

composition prior to taking the PRAXIS I writing test. Since this change in advising

occurred, the percentage of candidates passing the writing test of the PRAXIS I on their

first attempt has improved.

iii. Assessment of Candidate Performance, Courses, and Programs

Students are assessed on knowledge of content, professional and pedagogical skills, and

dispositions outlined in the Ohio Performance Based Teacher Licensure Standards. The College

of Education has developed a comprehensive assessment plan utilizing multiple measures for the

evaluation of candidate performance for undergraduate and master’s initial licensure. Students

are assessed in a variety of ways, both in the classroom and in diverse clinical/field settings.

These candidate assessments provide evidence that the individual candidates have met the

expectations described in the licensure standards. These data on candidate performance also

provide information regarding course and program effectiveness. The College of Education has

developed additional assessments that provide data about Unit Quality and Operations.

iv. Assessment of Advanced Candidates and other School Personnel

The assessment of advanced candidates and other school personnel is at the initial stages of

development. A framework has been created that establishes benchmarks and outlines four

Advanced Critical Performances based on the graduate core courses and a fifth Advanced

Critical Performance that would measure the research and writing skills of advanced candidates.

Committees within the College have been formed to develop Advanced Critical Performances.

The Electronic Assessment System will be used to gather and analyze the data from these

Advanced Critical Performances. Department Chairpersons and the NCATE Coordinator will

coordinate efforts to assess all programs that are not licensure programs.

Assessment of advanced candidates and other school personnel begins with admission to

Graduate School and continues with assessment at appropriate transition points and at program

completion.

Assessment of candidates in non-licensure, teacher education programs at the advanced level has

been aligned with the National Board for Professional Teaching Standards. This assessment

focuses on the ability of the experienced teacher to manage and monitor student learning, think

systematically about their practice, participate as members of learning communities, and apply

research and research methods in their professional practice.

Candidates in UT’s advanced programs that prepare other professional school personnel are

assessed in accordance with standards of their respective specialized professional associations.

Assessment of advanced level teacher candidates in music education is conducted in accordance

with the standards of the National Association of Schools of Music (NASM). Candidates in

School Speech/Language Pathology and in School Counseling are enrolled in programs that are

accredited by the American Speech and Hearing Association (ASHA) and the Council for

Accreditation of Counseling Related Educational Programs (CACREP), respectively.

Candidates in School Psychology are assessed based on the standards of the National

Association of School Psychologists (NASP) and candidates in School Administration and

Supervision are assessed as delineated in the Standards for Educational Leadership of the

Educational Leadership Constituent Council (ELCC) and the Inter-State School Leaders

Licensure Competencies (ISSLC). School Nursing candidates are evaluated in accordance with

Ohio state standards.

IV. Data Collection Methods

a. Technology Used in Tracking Assessment

The Electronic Assessment system is a web-based system for submission, evaluation, and review

of student documents for individual, program, college, or University-wide assessment purposes.

i. Electronic Assessment System (EAS)

The College of Education has developed an electronic database that is being used to maintain our

assessment system. The Electronic Assessment System (EAS) enables the professional education

unit to manage data gathered from assessments from both internal and external sources (i.e.,

applicants, candidates, recent graduates, faculty and other members of the professional

community). The EAS stores the actual candidate work (i.e., undergraduate and graduate) along

with the reviewer’s scores and his or her comments. The system also provides regular and

comprehensive information on applicant qualifications, candidate proficiencies, competence of

graduates, unit operations, and program quality. For both undergraduate and advanced

candidates, Critical Performances will be uploaded into the EAS by students and then assessed

online by faculty and/or external reviewers.

Figure 1 is an EAS student screen that shows a Critical Performance assignment and document

upload requirement.

Figure 1: Student Screen for Critical Performance Assignment

The EAS is a self-paced system. All students enrolled in any professional education coursework

automatically have access to “upload” artifacts to their instructor for reviewing, scoring, and

reporting. Additionally, every instructor automatically has access to his/her own students’

records, but no access to records of students in other classes. Entry to the EAS system is

password protected for all users. Multiple evaluators (i.e., reviewers who are not instructors) can

be given access to selected student records when this is judged necessary for program review

purposes.

Figure 2 is an EAS faculty assignment evaluation screen that shows the scoring rubric used to

assess a student’s Critical Performance assignment. Faculty can easily access student

assignments, score the work, and provide detailed comments regarding its quality.

Figure 2

1. Individuals Responsible for Gathering the Data

Multiple individuals are responsible for gathering the individual record data. The Office of

Admissions collects preliminary data for admission to the University and College. This

information is stored in the Student Information System and can be accessed through Data

Warehouse. PRAXIS I and II test scores and other Professional Education admission

requirements are collected and documented by the College advisers. Personnel in the Office of

Student Field Experiences collect additional state compliance documents (i.e., BCI check and

Statement of Good Moral Character). Data on Critical Performance are inputted into EAS by the

faculty of each identified courses. The College hired a Systems Analyst to provide technical

support and create reports using the data from the electronic data collection system.

2. Use of EAS Data in the Cycle of Assessment

Candidate assessment data are used in two ways. First, candidates are assessed individually so

that the unit can arrive at decisions about admission and continuation in professional education,

graduation, and/or licensure. Disaggregated data are used to help make these decisions. Second,

aggregated data about overall candidate performance are used to evaluate unit performance and

to arrive at decisions that will improve professional education programs and/or unit function.

Where candidates appear to have inadequate understanding of specific concepts and/or practices,

as indicated by grades, test scores, and/or clinical performance, the unit attempts to determine the

source(s) of these problems, with the expectation of correcting any apparent program

weaknesses. Program/department faculty meet on a monthly basis to review this information to

determine if the program needs to be changed and where the changes should occur.

Use of the unit’s comprehensive Electronic Assessment System (see pages 11 & 12 for EAS

description and screen samples) for Professional Education Candidates began in August 2003. It

has the scope and flexibility to provide both disaggregated data for individual student assessment

and aggregated data for unit evaluation and program improvement. Data analysis on candidate

performance will occur during Spring Semester 2004. Results of this analysis will be forwarded

to department chairs and to program faculty for their review. The data will provide evidence to

drive program and course modifications.

3. Individuals Responsible for Analyzing Gathered Data

The Electronic Assessment System provides both individual and aggregated candidate data

ranging from demographic information to reports of candidate performance and records of

follow-up studies. Facts and statistical data derived from this system are available for state and

national reporting as well as for program improvement. The unit’s Electronic Assessment

System for Professional Education Candidates was piloted during the summer term and was

implemented Fall Semester 2003. Analyzing the unit’s assessment findings, which includes EAS

data and the surveys listed in Tables 2 is the responsibility of the College of Education’s NCATE

coordinator. A graduate student from the College is currently assigned to the NCATE

coordinator to assist him/her in data analysis. It is the intent of the College to continue to

support the NCATE coordinator (Assessment) position and a graduate assistant in this capacity.

4. Work Incorporating EAS into COE Assessment Efforts

The EAS is used to run end of semester reports about individual students. These reports,

forwarded to the appropriate program staff and/or faculty, are used to make decisions about

students meeting specific criteria at each checkpoint. These data are also aggregated at the end of

each semester at the course level to look for patterns in student strengths and weaknesses and are

used to modify courses content.

The EAS will also be used to compile, summarize, and analyze additional data on student

performance, program strengths and weaknesses, and unit operations. These reports are then fed

back to different levels of the professional education unit (i.e., faculty, department chair,

program coordinators, deans) to improve candidate performance, program quality, and unit

operations. Future use of EAS will include linking student entry data (i.e., incoming grade point

average, standardized test scores, earned grades in Composition I and II, etc.) Once EAS is fully

implemented, the COE will use the data to develop a long-term recruitment and retention plan.

ii. Follow-up Surveys

Follow up studies will be conducted at appropriate intervals to investigate program and unit

effectiveness in the preparation of professional educators. Some of these studies are mandated by

the SPAs and/or accrediting bodies. The unit’s most recent follow-up studies include the Survey

of Candidate Impact on Student Learning, Faculty Assessment Survey, UT Cooperating Teacher

Survey, UT Supervisor Survey, and focused studies conducted by accredited programs. The

follow up forms used in School Counseling are contained in the program’s CACREP report.


Results of these surveys have been forwarded to department chairs and to program

faculty for their review. The data will provide evidence to drive program and course

modifications.

The Office of Student Field Experiences is developing a handbook for all education

students, which will describe all program requirements and expectations.

The Director of Student Field Experiences and program faculty will develop an extended

training program for all UT field supervisors and update the Student Teacher Handbook

to include information on program expectations and requirements and the Electronic

Assessment System.

The College is working with Career Services to develop a more comprehensive system to

track graduates.

c. Use of Institutional Data

Current use of institutional data to drive program improvement and increase accountability for

education candidate learning is limited. At this time institutional data are used to evaluate unit

diversity, assess incoming candidate preparedness, and estimate program enrollment trends.

Future development of the Electronic Assessment System will include using institutional data to

create reports that provide profiles of both students and programs. Systematic assessments using

institutional data will be expanded to include program design, to increase faculty collaboration,

and to enhance greater alignment within and between colleges and programs. These data are

critical in determining future program development, potential program elimination, and resource

allocation.

iii. Recruitment and Retention Plan

The data collection and assessment system described in this document will be used to develop a

recruitment and retention plan for the COE. The intent is to determine the appropriate size of the

College, the graduate and undergraduate programs, and the academic programs within the

departments that can be sustained with the available resources.

Once this “right” size has been determined, the College will examine student data to determine

the characteristics of candidates who are academically successful. Recruitment activities will

then be focused upon students, both undergraduate and graduate, who are most likely to enter

and complete Education programs at the initial and advanced levels. These data will provide

evidence to improve first-year advising, design academic and emotional support structures for

students, focus resources in areas of need and growth, and improve overall unit quality.

The College’s Recruitment and Retention Plan will focus on opportunities where “Students are at

the center of their own learning within a rich, intellectual environment characterized by choice.”

This vision will drive the crafting of a set of rich educational experiences for students throughout

their programs and educational experience at The University of Toledo.

V. The Assessment Plan (Non-Licensure Graduate Programs)

1. Curriculum & Instruction, and Early Childhood, Physical and Special Education

(ECPSE)

The doctoral program in Curriculum and Instruction is shared between two departments,

Curriculum and Instruction and ECPSE. The current doctoral program was approved in the

spring semester, 2007, and was implemented that summer. The previous program had a

minimum requirement of 76 semester hours and the current program has a minimum requirement

of 60 semester hours. With the reduction of 16 semester hours, a number of foundations courses

were eliminated along with a doctoral minor. Instead, the program encouraged, but did not

require, students to take foundations courses as part of the 30 semester hour “Interdisciplinary

Studies” component of the program. The program is comprised of four major components (1)

Pro-Seminar I, II, and III (9 semester hours), (2) the research core (12 semester hours), (3)

Interdisciplinary Studies (30 semester hours), and (4) the dissertation (9 semester hours,

minimum). The program also requires a two semester (fall/spring, spring/summer, summer/fall)

residency of at least 9 semester hours each semester/summer. The program has both full-time

and part-time students enrolled currently and has a strong emphasis on diversity among the

student population including racial and ethnic minorities and international students.

Program Oversight and Evaluation: Feedback Loop

Aside from the College Doctoral Monitoring Committee, the C&I doctoral program has an

advisory committee that meets once or twice each semester to discuss issues related to policy,

governance, assessment of learning outcomes, and program structure. Membership on the

advisory committee is made up of the chairs of the two departments, faculty who have significant

interest in the doctoral program and the associate dean for graduate studies in the JHCOE who is

an ex-officio member of the committee.

2. Educational Foundations and Leadership

This department is comprised of five “wings”. These are: Theory and Social Foundations,

Research and Measurement, Educational Psychology, Higher Education, and Educational

Administration. Programs of study are summarized in Table 3. As previously reported, School

Administration and Supervision are assessed as delineated in the Standards for Educational

Leadership of the Educational Leadership Constituent Council (ELCC) and the Inter-State

School Leaders Licensure Competencies (ISSLC). This section will therefore address programs

in Foundations of Education and Higher Education.

Foundations of Education Context and Methodology

The program is authorized to offer both the Ph.D. and Ed.D. However, with the change in the

Ph.D. program in fall 2005 to be more focused, students have not been interested in the Ed.D.

Based on comments from students, the major reason for not applying to the Ed.D. has been the

difference in minimum semester hour requirements: 61 (Ph.D.) vs. 76 (Ed.D.).

All of the concentrations share the same general approach to assessing their students. That

approach involves the following assessment items, in the general sequence presented.

1. Individual course requirements

2. Major written examination (may occur after minor written examination)

3. Minor written examination (may occur before minor written examination)

4. Oral examination covering the major and minor areas

5. Oral defense of a dissertation proposal

6. Evaluation of the written dissertation

7. Oral defense of the written dissertation

Table 3

Doctoral Programs by Concentration, Major Code, and Degree

Program Concentration Major Code Ph.D. Ed.D.

Educational

Administration &

Supervision

EDAS X

Foundations of

Education

Educational

Psychology

FEEP X X

Educational

Research &

Measurement

FERM X X

Educational

Sociology

FEES X X

Foundations of

Education

FOED X X

History of

Education

FEHE X X

Philosophy of

Education

FEPE X X

Higher Education HED X

These assessments are intended to assess fulfillment of the following learning objectives:

Mastery of the disciplinary content of the major area of study

Mastery of the disciplinary content of the minor area of study

Understanding of the place of one’s major and minor fields of study in the overall

disciplines of Foundations of Education concentrations

Ability to articulate and defend various theoretical perspectives

Knowledge of and skill in the identification and implementation of research

methodologies

Ability to analyze and interpret research data, both quantitative and qualitative

Knowledge of and skill in theory development and theoretical analysis

Ability to articulate and defend conclusions drawn from empirical research and

theoretical analysis

Knowledge of and skill in the articulation and implementation of original educational

research (quantitative, qualitative, and/or theoretical)

Higher Education

The Higher Education Program has all master’s students write a master’s project or thesis,

and all doctoral students complete a major exam and a dissertation study. Doctoral level

assessments share the same general approach to Foundations of Education as outlined in the

previous section. Depending on program level, all of these academic projects are read every

year by at least two of the program faculty, if not all of the program faculty.

In addition to the review of these culminating documents from master’s and doctoral students

on a yearly basis, the Higher Education Program faculty developed a matrix that listed the

student goals for the Higher Education Program in the fall of 2004. These goals included

teamwork, technology, research, writing, finance/budgeting, strategic planning, organization

and administration, historical understanding, critical thinking, and presentation skills (see

Table 42). The courses in the master’s and doctoral programs were then analyzed to see

where these desired outcomes were covered in the curriculum of the two degree programs. It

was agreed that the program faculty would begin using the Electronic Assessment System to

review student documents written in classes to determine whether students had mastered

these desired student outcomes.

Table 4 – Matrix of Desired Higher Education Program Student Outcomes

Teamwork Technology Research Writing Finance Budget Planning

Organization Administration

Historical Understanding

Critical Thinking

Applied Experience

Present. Skills

Finance of HED (online) Paper & Presentation X X X X X State Porfolio Analysis X X X X X IPEDS Report X X X X X Student Activity Fees Project X X X X X X Virtual University Assignments X X X X X Group Virtual University Assignments X X X X X X Building Academic Culture Culture Project- Individual X X X X Culture Study Group Project X X X X X X Institutional Culture Presentation X X X X X Governance and Administration of HED ASHE Reader Chapter Presentation X X X Executive Governance Project- Group X X X X X Research Paper- Individual X X X X Faculty Senate Meeting Critique X X X Faculty Issues in HED (Online) Written Introduction X




Critical Thinking

Applied Experience

Present. Skills

Letters to the Editor Assignment X X X X Opinion Piece Assignment X X X X Beat Article (topical analysis) X X X X Faculty Senate Critique X X X X X Faculty Member Feature Story X X X X X Weekly Postings X X X X Leadership Theory in HED Seminar Paper and Presentation X X X X X Independent Virtual University X X X X X X Group Virtual University X X X X X X X Seminar Participation




Critical Thinking

Applied Experience

Present. Skills

Master's Capstone Written Bibliography Critique of Project/Thesis X X X X Progress Presentations X X Project Presentation X X Written Proposal X X X Institutional Advancement in HED

Course Content Presentations X X X X X X X

Final Examination X X X X

I.A. Research Team Project X X X X X

Issues in Access to HED

Group Presentation X X X X X

Research Paper X X X X X X

Final Exam X X X

The American College Student (On-line)

Participation X

Chronicle Report X X X X

Personal Reflection Paper X X

Topical Analysis X X X X X

Topical Analysis 2nd Assignment X X X X X

Student Profile Analysis X X X X

Campus Newspaper Analysis X X X

Book Report X X X

Case Study X X X X X X

Independent Project/Analysis X X X X X

Research in HED

Conference Proposal X

Seminar Presentation X X X X X X

Critique of Qualititative Article X X

Critique of Quantitative Article X X

History of HED

Class Presentations X X X X X X

Historical Briefing Paper X X X X




Critical Thinking

Applied Experience

Present. Skills

Significant Historical Figure Paper X X X X X X Research Paper-Contemporary Issues X X X X X

Book Report X X X

Final Examination X X X

Federal and State Policy

Group Written Action Plan X X X X X X X

Final Examination X X X X X

Managing College & University Personnel University Personnel Research Paper X X X X X

University Personnel Topic Present. X X X X X X


Student Development Theory

Journal Entries X X

Analysis of Student Dev. Theory Paper X X X X

Student Dev. Theory Presentation X X X X X Org. & Management of Student Affairs

Annotated Bibliography Assignment X X

Oral Group Presentation X X X X X X X

Statement of Personal Philosophy X X

Book Reviews X X X

Internet Presentation X X X X X X

Reading Quizzes X X X





Critical Thinking

Applied Experience

Present. Skills

College and University Curriculum

Chapter Presentations X X X X X Curriculum Group Project Presentation X X X X X X

Take-Home Final Examination X X X

Strategic Planning

Individual Chapter Presentation X X X X X

Group Project and Presentation X X X X X X X X

Take-Home Final Examination X X X X X

Practicum

Journal X X X

Reflective Paper X X X X

The Community College (On-line)

Campus Visit and Portfolio X X X X X X X Chronicle of Higher Education Report X X X X

College System Website Analysis X X X X X X

Final Examination X X X

Issue Paper X X X X

Critical Issues in Higher Education

Daily Question Emails X X

Group Overview of Current Issues X X X X X X X X X X

In-depth Analysis of Current Issues X X X X X X X

VI. Summary

All of the assessments described in this report are integral to the College of Education’s

Assessment Plan. Each assessment will be modified when necessary and continue to be used in

the required cycle as the College develops and refines its plan. A critical element of this

assessment plan is the creation of the Electronic Assessment System. The EAS was a result the

creativity of the COE faculty, the financial and moral support of the Provost’s Office, and the

technical expertise of the Division of Institutional Technology. The College will continue to

enhance and develop the system’s capabilities while expanding the uses of this electronic system.

The findings from the numerous assessments described in this document have been forwarded to

the college’s Strategic Planning Committee. It is anticipated that this committee will make

extensive use of the findings as it outlines a course for the unit to follow over the next several

years. Additionally, each department is reviewing the data to make informed decisions regarding

program modifications. By focusing upon internal and external assessments to evaluate both

candidate and unit performance and effectiveness, the College of Education is prepared to

engage in self-assessment and develop a culture of continuous improvement.

Appendix A

Judith Herb College of Education Assessment Plan Reporting Structure

The Judith Herb College of Education (JHCOE) is administratively organized in three

departments: Curriculum & Instruction; Early Childhood, Physical, & Special Education; and

Educational Foundations & Leadership. The reporting structure is outlined below.

Curriculum & Instruction (C&I)

Initial Licensure Programs

o Adolescent to Young Adult (Language Arts, Mathematics, Science, & Social

Studies)

Undergraduate Programs

Graduate Programs

o Middle Childhood


Graduate Programs

o Career and Technical – Route B only (Agrisciences, Business, Family &

Consumer Science, Marketing, and Trade & Industry)


Graduate Programs

o Foreign Language (French, German, & Spanish)


Graduate Programs

Advanced Licensure Programs

o Reading Endorsement

o Literacy Specialist Endorsement

o Adult Education Endorsement

o Career-based Intervention Endorsement

o Transition to Work Endorsement

o Worksite teacher/Coordinator Endorsement

o Early Childhood Generalist Endorsement

Non-Licensure Programs (degrees); with the Early Childhood, Physical & Special

Education Department

o Master's Degree Programs

o Education Specialist Degree Programs

o Doctoral Degree Programs in C & I

Early Childhood, Physical, & Special Education

Initial Licensure Programs

o Early Childhood Education

Undergraduate Program

Graduate Program

o Physical education

Undergraduate Program

o Special Education (Mild/Moderate, Moderate/Intensive, Visually Impaired)


Graduate Programs

Advanced Licensure Programs

o Early Childhood Intervention Specialist

o Gifted Education

o Adapted Physical Education Endorsement

o Prekindergarten Endorsement

o Undergraduate Programs

Early Childhood (Licensure)

Early Childhood (non-licensure)

Special Education

Physical Education

Educational Foundations & Leadership (all Graduate Programs)

Educational Administration & Supervision

o Advance Licensure Programs (Building & District Leadership)

Administrative Principal (Grades PreK-6)

Administrative Principal (Grades 4-9)

Administrative Principal (Grades 5-12)

Administrative Superintendent

o Non-Licensure Graduate Programs

Master's Degree Programs

Education Specialist Degree Programs

Doctoral Degree Programs in Educational Leadership

Higher Education



o Doctoral Degree Programs in Higher Education

Educational Psychology



o Doctoral Degree Programs in Educational Psychology

Research & Measurement



o Doctoral Degree Programs in Research & Measurement

Licensure Programs outside of the Judith Herb College of Education

The following programs are a part of the JHCOE “accreditation unit” for licensure and

accreditation purposes, but will be reported in their respective colleges for institutional reporting

purposes.

Health Sciences & Human Services

Initial Licensure

o Health Education undergraduate program

Advanced Licensure

o School Psychology

o School Counseling

o School Nurse

o Speech Language Pathology

Arts & Sciences

Initial Licensure

o Music Education

Undergraduate program

Graduate program

o Visual Art Education

Undergraduate program

Graduate program

Appendix B

Initial Licensure (Undergraduate) Key Assessments and Transition Points

Entry to UT Entry to Professional

Ed. Student Teaching Eligibility Graduation

Recommendation for Two-year Licensure

Professional Licensure

high school diploma / general education development (GED) diploma

Core Courses

Completion of 90% credits or minimum 100

Completion of all course work Pass Student Teaching Entry year program

(OH)

ACT test scores >= 19 or SAT test scores (Math >= 900 and Reading >= 900)

Comp II (& Comp I if required)

TSOC requirement

3. Student Teaching Portfolio Tab1: School/Community Context

Tab 2: Unit Planning Tab 3 Assessment Planning Tab 4: Video with Reflection

Tab 5: Analysis of Teaching and Professional Growth

NCATE #6

1. PRAXIS II PLT (& other SPA requirements)

NCATE #1

3. PRAXIS III

NCATE #7

Math Requirement

EDP Requirement 1. PRAXIS II Content

NCATE #1

DHS students High school grade point average (HSGPA) >= 2.0

5. Technology Requirement

GPA >= 2.7 GPA >= 2.7

2. GPA >= 2.7 NCATE #2

School Context (CP1)

Student Teaching Portfolio Tab 6: Value-Added & Ohio Operating Standards (Ohio

Requirement)

Reading Content (Ohio Requirement)

Transfer students GPA requirements < 30 credits, >=2.2 30–59 credits, >=2.3 > 59 credits, >= 2.7

4. Pass PRAXIS I 1. Unit Planning (CP2)

NCATE #3 Posted Baccalaureate

Voluntary Hours 2. Assessment Planning

(CP3)

4. Video & Reflection (CP4) 2. Final Supervisor Student Teaching Observation

NCATE #4 Methods Observations

Pass Methods Field 4. Cooperating Teacher

Evaluation

Surveys

Cultural Sensitivity I Retention I Course Evaluations Course Evaluations

Early Career Course Evaluations Course Evaluations

Retention II

Cultural Sensitivity II 4. TQP In-Service

1. TQP Pre-Service

NCATE #5 Graduate

3. Student Exit Employer

3. Cooperating Teacher

Appendix C

Initial Licensure (Graduate) Key Assessments and Transition Points Entry to

Professional Ed. Student Teaching

Eligibility Licensure

Recommendation for Two-year Licensure

Degree Requirements

Professional Licensure

Core Courses as determined by

Transcript Audit

Completion of 90% of licensure coursework

Completion of all course work Pass Student Teaching Research &

Measurement Requirement

Entry year program (OH)

Completion of Licensure Core Requirements

TSOC requirement

3. Student Teaching Portfolio

Tab1: School/Community Context

Tab 2: Unit Planning Tab 3 Assessment Planning Tab 4: Video with Reflection Tab 5: Analysis of Teaching

and Professional Growth NCATE #6

1. PRAXIS II PLT

(& other SPA requirements) NCATE #1

Capstone Experience

3. PRAXIS III

NCATE #7

CUM GPA >= 2.7 in Undergrad Content

EDP Requirement 1. PRAXIS II Content

NCATE #1

Technology Self-Assessment

GPA >= 2.7 (Undergraduate)

GPA >= 3.0 (Graduate)

GPA >= 2.7 (Undergraduate) GPA >= 3.0 (Graduate)

2. CUM Grad GPA >=

3.0 NCATE #2 School Context (CP1)

Student Teaching Portfolio Tab 6: Value-Added & Ohio Operating Standards (Ohio

Requirement)

Reading Content (Ohio Requirement)

1. Unit Planning (CP2)

NCATE #3 Completion of licensure requirements

2. Assessment Planning

(CP3)

4. Video & Refection (CP4) 2. Final Supervisor Student

Teaching Observation NCATE #4 Methods Observations

Pass Methods Field 4. Cooperating Teacher

Evaluation

Surveys

Cultural Sensitivity I Retention I Course Evaluations Course Evaluations

Early Career Course Evaluations Course Evaluations

Retention II

Cultural Sensitivity II 4. TQP In-Service

1. TQP Pre-Service

NCATE #5 Graduate

3. Student Exit Employer

3. Cooperating Teacher

Appendix D

Master of Education Key Assessments and Transition Points

Entry Requirements Completion of Masters Coursework Degree Requirements

Entry to UT Core (9 credits)

(TSOC, RESM, EDP)

Curriculum with embedded field

(3 credits)

Specialization (15 credits)

Culminating Experience (3 credits)

Graduate School Requirements GPA > = 3.0

Complete Curriculum Project

GPA > 3.0

Capstone Experience (applied content &

professional knowledge)

CUM Undergraduate GPA >= 2.7

GRE scores if GPA < 2.7

Course-embedded field experience

Graduate Survey

TOEFL Requirement (International Students)

Teaching Experience

Requirement

Appendix E

Education Administration Building Leadership Key Assessments and Transition Points

Entry to UT Completion of Masters

Coursework Completion of Licensure

Coursework Clinical

Experience Degree Requirements Licensure

Requirements

Graduate School Requirements

GPA > = 3.0 2. Community and School

Action Plan 4. Internship

Portfolio

Research & Measurement Requirement

1. PRAXIS II

CUM Undergraduate GPA >= 2.7

6. School Culture Improvement Plan

Capstone Experience


GPA > = 3.0

Teaching Experience Requirement

3. Supervisory Plan for Professional Growth

7. Building Climate and Culture Evaluation Plan

Surveys

Course Evaluations Course Evaluations Course Evaluations Course Evaluations Course Evaluations

Retention 5. Post-Field

Candidate Survey

8. Post-Field

Supervisor Survey

Appendix F

Education Administration Building Leadership Key Assessments and Transition Points

Entry to UT Completion of Licensure

Coursework Clinical Experience

Degree Requirements

Licensure Requirements

Graduate School Requirements

2. District Plan to Improve Student Learning

6. Internship Portfolio Research &

Measurement Requirement

1. PRAXIS II (if not taken at Building Level)

CUM Graduate GPA >= 3.0

Course work GPA > = 3.0


GPA > = 3.0

Capstone Experience

Principal License 3. District-wide Title 1 Plan

7. District Strategic Plan

Surveys

Course Evaluations Course Evaluations Course Evaluations Course Evaluations

5. Post-Field

Candidate Survey

4. Post-Field

Supervisor Survey

judith herb college of education - university of toledo · judith herb college of education...

Documents