edtpa at unc charlotte: a report of edtpa data from fall ... report of edtpa...a report of edtpa...

edTPA at UNC Charlotte:A Report of edTPA data from Fall 2014 through Spring 2016

Prepared by: Office of Assessment and Accreditation

College of Education, UNC Charlotte Laura C. Hart, Ed.D., Director

John Williams, Graduate Assistant, Ph.D. candidate Ashley Flatley, University Program Associate

UPDATED: September 15, 2016

P a g e | 1

Contents

Table of Contents Introduction ………………………………………………………………………………………………………………………………………………………………………………………………2

UNC Charlotte Candidate Demographics …………………………………………………………………………………………………………………………………………….. 5 Number of Candidates, by handbook, ethnicity (White vs. non‐White), gender, age group

Institutional edTPA Comparisons ………………………………………………………………………………………………………………………………………………………………8

UNC Charlotte, North Carolina Universities, and National Task Means (Fall 2014 – Spring 2015: Year 1) ……………………………………………..8 UNC Charlotte, North Carolina Universities, and National Task Means (Fall 2014 – Spring 2015: Year 2) ……………………………………………11 UNC Charlotte edTPA Task Means for the AY 2014‐2015 (Year 1) and AY 2015‐2016 (Year 2) 1………………………………………………………….14

UNCC edTPA Passing Percentage by demographic group ……………………………………………………………………………………………………………………17

Passing Percentage by Gender Passing Percentage by Ethnicity (White vs. non‐White) Passing Percentage by Age

UNCC Program Handbook Scores, by Individual Rubric (by academic year) …………………………………………………………………………………………25 Condition Codes/Incomplete Scores …………………………………………………………………………………………………………………………………………………..43 Discussion ………………………………………………………………………………………………………………………………………………………………………………………….47 Moving Forward with edTPA at UNC Charlotte.…………………………………………………………………………………………………………………………………..49

Any questions about this report should be directed to the Office of Assessment and Accreditation, COED 131, 704‐687‐8163.

P a g e | 2

Introduction As a measurement tool developed by educators for educators, edTPA is a performance‐based portfolio that assesses subject‐

specific material. edTPA is designed to answer the question, “Is the candidate ready to teach?” Successful completion of edTPA serves as one indicator that the candidate has the necessary skills and knowledge that a beginning teacher needs to enter the classroom. Candidates are required to demonstrate skills and knowledge on subject‐area assessments. These assessments are comprised of three Tasks: Task 1 is planning for instruction, Task 2 is implementation of instruction, and Task 3 is assessment of instruction. Each task is assessed on an average of five rubrics, and each rubric measures a different aspect of teaching, such as assessing student learning, understanding language acquisition of content material, utilizing student feedback, and creating a supportive learning environment. While the individual rubric indicators are not the same for all content areas, all candidates are assessed on common overarching constructs. In addition, by utilizing a portfolio approach similar to what is used in the National Boards process, edTPA provides multiple artifacts to assess candidate readiness, including lesson plans, video clips of candidates as they teach lessons, reflections, and data analysis. By utilizing a multifaceted approach to gathering and analyzing data on candidates, instructors have the ability to strengthen candidates’ capability to work with differentiated learning styles of students, refine teachers’ pedagogical practices in the classroom, and improve the entire teacher education program. This technical report on the UNCC’s edTPA standing should serve as a tool in identifying possible areas of improvement, answering the following questions:

What are the demographics of candidates who are participating and passing (as defined at UNC Charlotte)? How does UNC Charlotte compare by Task and by Handbook to all other North Carolina Teacher Education Programs and

Educator Preparation Programs (EPPs) across the nation? Within the College of Education, what are the individual programs’ strengths and areas of growth with specific rubrics? For candidates who received incomplete scores, is the data a reflection of instructional misalignment or candidate

misunderstanding? The range of the data begins with the fall 2014 semester and ends with spring 2016. It is important to note that the data contained

in this report are not psychometrically evaluated; still, there are important trends and information for each program. The initial offerings of this report begin with an overview of number on candidates who completed the edTPA each semester, by their respective content area. Next is a visual representation of institutional comparisons between UNC Charlotte, other North Carolina Educator Preparation Programs (EPPs) (aggregated), and National Teacher Education Programs (aggregated). This is followed by UNC Charlotte’s pass percentages by

P a g e | 3

demographic, and UNC Charlotte’s rubric scores by handbook. Although data collection of incomplete scores only began in fall 2015, a brief synopsis of condition codes and an analysis of trends within incomplete scores by candidates is provided. Finally, this report provides recommendations based on the trends within the data to allow instructors, individual programs and the College of Education at UNC Charlotte to strengthen the curriculum, refine pedagogical content within the coursework, and merge evidence‐based interdisciplinary methods and techniques. Ultimately, it is our hope that this information serves to assist faculty with improving candidates’ skills and knowledge prior to entering the teaching profession.

Interpretive Notes 1. This report includes no statistical analysis. Faculty who wish to undergo psychometric analysis as part of their research agenda are

encouraged to work with the Office of Assessment and Accreditation to accomplish this. 2. 2014‐15 was Year 1 (Y1) of the initiative; 2015‐16 was Year 2 (Y2). 3. In fall 2014, Elementary Education (ELED) candidates completed the Elementary Literacy only handbook (15 rubrics). Beginning in

spring 2015, ELED candidates completed the “combination” handbook, which includes both literacy and math tasks (18 rubrics), while the SPEL Dual majors (Special Education + Elementary Majors) complete the Elementary Literacy only handbook. It is important to note this in reviewing the data.

4. Not all institutions who use edTPA utilize official scoring. Comparisons at the state and national level are only for institutions with official scores. No local evaluation comparisons are included.

5. As of July 2016, there is no official requirement by the state of North Carolina to meet a minimal required cut score for “passing” the edTPA. In order to provide a common understanding and goals for program improvement, UNC Charlotte opted to determine our own criteria for passing. In doing so, we followed SCALE recommendations for standard setting. As NC policy changes, these criteria may also change, so that we are in alignment with NC policy.

P a g e | 4

As of July 2016, “passing” edTPA at UNC Charlotte is defined as the following:

Content/Handbook Area Number of rubrics in a scored edTPA product

Required edTPA Passing score

Rubric Average

Elementary Education (combo) 18 44 2.44 World Language 13 32 2.46 All other content areas, including Middle Grades, Secondary, Special Education, SPEL (duals), B‐K, TESL, and all Arts areas

15 37 2.46

6. edTPA scores are converted into a course assignment grade by the university supervisor. Scores are incorporated into part of the

student teaching grade. edTPA, therefore, is a part of the student teaching grade, one “piece of the puzzle” in determining candidate competency. Candidates who do not pass edTPA may still pass student teaching and receive a licensure recommendation under the current policy. The NC State Board of Education will determine official edTPA policy in fall 2016. Any decision by the state board to require edTPA for licensure will necessitate a change to UNC Charlotte procedures.

P a g e | 5

UNC Charlotte edTPA Candidate Demographics Table 1 Number of UNC Charlotte candidates, by program, number attempted/passed edTPA, white/non‐white, Y1‐Y2

Program

2014‐15 (Y1) 2015‐16 (Y2) TOTAL

ATTEMPT edTPA TO DATE White Non‐White

TOTAL TO

ATTEMPT 14‐15

White Non‐White

TOTAL TO

ATTEMPT 15‐16

Attempt Pass Attempt Pass Attempt Pass Attempt Pass Early Childhood 18 18 8 7 26 10 10 15 14 25 51 Elementary Education 155 133 52 43 207 136 118 34 28 170 377 SPEL (Dual) 11 11 1 1 12 15 15 1 1 16 28 Eng. As Additional Lang 3 3 12 10 15 7 7 3 3 10 25 K‐12 Performing Arts 8 7 2 2 10 10 K‐12 Visual Arts 11 10 3 3 14 14 Middle Child ELA 12 10 5 3 17 10 10 4 4 14 31 Middle Child Math 13 9 12 7 25 20 11 3 3 23 48 Middle Child Science 9 6 4 3 13 9 7 4 3 13 26 Middle Child Soc Studies 5 4 3 2 8 11 9 5 4 16 24 Secondary English 20 16 10 6 30 15 14 5 3 20 50 Secondary Math 10 6 3 2 13 10 7 6 5 16 29 Secondary Science 4 2 2 1 6 6 5 1 1 7 13 Secondary Social Studies 25 22 6 4 31 22 20 2 1 24 55 Special Education 50 21 11 5 61 44 26 10 5 54 115 World Language 8 8 7 4 15 2 2 3 3 5 20 TOTALS 343 269 136 98 479 337 278 101 83 438 917

P a g e | 6

Table 2 Number of UNC Charlotte candidates to attempt edTPA, by program, by gender, Y1‐Y2.

Program

2014‐15 (Y1) 2015‐16 (Y2) TOTAL TO DATEMale Female TOTAL Male Female TOTAL

Early Childhood 0 26 26 0 25 25 51 Elementary Education 11 196 207 11 159 170 377 SPEL 0 12 12 1 15 16 28 Eng. As Add. Language 1 14 15 1 9 10 25 K‐12 Performing Arts 5 5 10 10 K‐12 Visual Arts 2 12 14 14 Middle Child ELA 6 11 17 2 12 14 31 Middle Child Math 6 19 25 5 18 23 48 Middle Child Science 3 10 13 5 8 13 26 Middle Child Social Studies 0 8 8 8 8 16 24 Secondary English 5 25 30 6 14 20 50 Secondary Math 8 5 13 7 9 16 29 Secondary Science 2 4 6 4 3 7 13 Secondary Social Studies 19 12 31 15 9 24 55 Special Education 6 55 61 5 49 54 115 World Language 4 11 15 3 2 5 20 TOTALS 71 408 479 80 358 438 917

P a g e | 7

Table 3 UNC Charlotte edTPA Candidates by program, by age group, Y1‐Y2.

Program Early Childhood Elementary Ed SPEL (Duals) SPED

Age Range 17‐22

23‐28

29‐40

40 + TOT 17‐

22 23‐28

29‐40

40 + TOT 17‐

22 23‐28

29‐40

40 + TOT 17‐

22 23‐28

29‐40

40 + TOT

2015‐16 (Y2) 2 13 7 3 25 79 56 24 11 170 14 2 0 0 16 23 17 10 4 54

2014‐15 (Y1) 7 12 5 2 26 78 90 30 9 207 8 4 0 0 12 17 18 14 12 61

TOTAL TO DATE 9 25 12 5 51 157 146 54 20 377 22 6 0 0 28 40 35 24 16 115

Program K‐12 Performing Arts K‐12 Visual Arts Eng. As Addit Language World Language

Age Range 17‐22

23‐28

29‐40

40 + TOT 17‐

22 23‐28

29‐40

40 + TOT 17‐

22 23‐28

29‐40

40 + TOT 17‐

22 23‐28

29‐40

40 + TOT

2015‐16 (Y2) 4 5 1 0 10 1 7 3 3 14 0 5 5 0 10 1 1 1 2 5

2014‐15 (Y1) 0 0 0 0 0 0 0 0 0 0 0 7 5 3 15 3 4 3 5 15

TOTAL TO DATE 4 5 1 0 10 1 7 3 3 14 0 12 10 3 25 4 5 4 7 20

Program Middle Child ELA Middle Child Math Middle Child Science Middle Child Soc Studies

Age Range 17‐22

23‐28

29‐40

40 + TOT 17‐

22 23‐28

29‐40

40 + TOT 17‐

22 23‐28

29‐40

40 + TOT 17‐

22 23‐28

29‐40

40 + TOT

2015‐16 (Y2) 7 3 3 1 14 7 6 6 4 23 1 5 4 3 13 4 5 3 4 16

2014‐15 (Y1) 4 5 5 3 17 1 14 6 4 25 4 6 2 1 13 2 4 2 0 8

TOTAL TO DATE 11 8 8 4 31 8 20 12 8 48 5 11 6 4 26 6 9 5 4 24

Program Secondary English Secondary Math Secondary Science Secondary Social Studies

Age Range 17‐22

23‐28

29‐40

40 + TOT 17‐

22 23‐28

29‐40

40 + TOT 17‐

22 23‐28

29‐40

40 + TOT 17‐

22 23‐28

29‐40

40 + TOT

2015‐16 (Y2) 9 7 3 1 20 6 4 3 3 16 1 3 3 0 7 7 13 1 3 24

2014‐15 (Y1) 8 13 6 3 30 3 5 5 0 13 1 3 1 1 6 8 17 3 3 31

TOTAL TO DATE 17 20 9 4 50 9 9 8 3 29 2 6 4 1 13 15 30 4 6 55

P a g e | 8

Institutional edTPA Comparisons UNC Charlotte, North Carolina Universities, and National Task Means for 2014‐15 Academic Year (Year 1) AY 2014‐15 was the first year of data collection of edTPA at UNC Charlotte. Elementary, Middle Grades, and Secondary programs began in fall 2014, while Early Childhood and Special Education were added in spring 2015. Handbook data are reported at each level, with the exception of K‐12 Performing Arts and Visual Arts (these programs were added in fall 2015). Task 1 – Y1

Figure 1. Task 1 Mean Scores for UNC Charlotte, North Carolina Universities, and National Task Means, 2014‐15 (Year 1).

11

12

13

14

15

16

17

18

19

20

UNCC State National

P a g e | 9

Task 2 – Y1

Figure 2. Task 2 Mean Scores for UNC Charlotte, North Carolina Universities, and National Task Means, 2014‐15 (Year 1).

11

12

13

14

15

16

17

UNCC State National

P a g e | 10

Task 3 – Y1

Figure 3. Task 3 Mean Scores for UNC Charlotte, North Carolina Universities, and National Task Means, 2014‐15 (Year 1). Summary of Tasks 1‐3 in Year 1 (2014‐15) In Year 1 of the edTPA initiative, UNC Charlotte scores were similar to other North Carolina institutions and National institutions on average on Task 1. UNC Charlotte exceeded their peers in Middle Grade Childhood Science, World Language, and Elementary Education on Task 1. The UNC Charlotte highest mean for Task 1 is English as an Additional Language. UNC Charlotte Task 2 mean scores remained within one point of North Carolina and National institutions for each handbook, except for Secondary Science. While Task 3 means overall

10

11

12

13

14

15

16

UNCC State National

P a g e | 11

are the lowest of the three tasks, UNC Charlotte Task 3 mean scores exceeded the National EPP mean scores in Elementary Literacy, Middle Childhood Science, World Language, and Elementary Education handbooks. In comparison to the state means, UNC Charlotte scored higher in Middle Childhood Science, Secondary Mathematics, Secondary Science, World Language, and Elementary Education. UNC Charlotte, North Carolina Universities, and National Task Means for the 2015‐16 Academic Year (Year 2) The following three (3) charts represent task mean scores for the UNC Charlotte, North Carolina, and National Education Preparation Programs (EPPs) that utilize edTPA to gauge teacher competency in 2015‐16 (Y2). Task 1 – Y2

Figure 4. Task 1 Mean Scores for UNC Charlotte, North Carolina Universities, and National Task Means, 2015‐16 Academic Year (Y2).

11.0

12.0

13.0

14.0

15.0

16.0

17.0

18.0

19.0

UNCC State National

P a g e | 12

Task 2 – Y2

Figure 5. Task 2 Mean Scores for UNC Charlotte, North Carolina Universities, and National Task Means, 2015‐16 Academic Year (Y2).

11.0

12.0

13.0

14.0

15.0

16.0

17.0

18.0

UNCC State National

P a g e | 13

Task 3 – Y2

Figure 6. Task 3 Mean Scores for UNC Charlotte, North Carolina Universities, and National Task Means, 2015‐16 Academic Year (Y2). Summary of Tasks 1‐3 – Year 2 (2015‐16) When compared to North Carolina EPPs on Task 1 in 2015‐16 (Y2), UNC Charlotte exceeded the state means in English as an Additional Language, Middle Childhood English Language Arts, Secondary Mathematics and Visual Arts. UNC Charlotte exceeded national means for Task 1 in Early Childhood, Elementary Literacy, English as an Additional Language, Visual Arts, and World Language. For UNC Charlotte, English as an Additional Language contained the highest mean for Task 1. On Task 2, UNC Charlotte candidates’ means exceeded North Carolina and National EPPs in English as an Additional Language, K‐12 Performing Arts, Middle Childhood English‐Language Arts and World Language. As in Task 1, UNCC’s highest mean scores were in English as an Additional Language. For Task 3, UNC Charlotte outperformed North Carolina and National EPPs in English as an Additional Language. As with the previous two tasks, UNC Charlotte’s highest performing program was English as an Additional Language. For most all other programs, UNC Charlotte scores were below the state and national means. Overall Task 3 means were the lowest of the tasks assessed at UNC Charlotte (and at the state and national level as well).

10.011.012.013.014.015.016.017.018.0

UNCC State National

P a g e | 14

UNC Charlotte edTPA Task Means for the AY 2014‐2015 (Year 1) and AY 2015‐2016 (Year 2) Task 1 – Difference between Y1‐Y2

Figure 7. Task 1 Mean Scores of UNC Charlotte for the 2014‐2015 AY (Year 1) and 2015‐2016 AY (Year 2). UNC Charlotte Year 1 mean exceeded the Year 2 mean in the handbooks of Middle Childhood Math, Middle Childhood Social Studies, Middle Childhood Science, Secondary English Language Arts, Secondary History/Social Studies, Secondary Science and Elementary Education for Task 1.

11

12

13

14

15

16

17

18

19

UNCC Year 1, Task1 UNCC Year 2, Task 1

P a g e | 15

Task 2 ‐ Difference between Y1‐Y2

Figure 8. Task 2 Mean Scores of UNC Charlotte for the 2014‐2015 AY (Year 1) and 2015‐2016 AY (Year 2). UNC Charlotte Year 2 means in this task exceeded Year 1 means in the handbooks of Early Childhood, English as an Additional Language, Middle Childhood English, Middle Childhood Social Studies, Middle Childhood Science, Middle Childhood Mathematics, Secondary Science, Special Education and World Language.

11

12

13

14

15

16

17

18

UNCC Year 1, Task 2 UNCC Year 2, Task 2

P a g e | 16

Task 3 ‐ Difference between Y1‐Y2

Figure 9. Task 3 Mean Scores of UNC Charlotte for the 2014‐2015 AY (Year 1) and 2015‐2016 AY (Year 2). Year 2 candidates scored lower than Year 1 in the handbooks of Middle Childhood Science, Secondary Mathematics and Secondary Science in Task 3.

9

10

11

12

13

14

15

16

17

18

UNCC Year 1, Task 3 UNCC Year 2, Task 3

P a g e | 17

UNC Charlotte edTPA Passing Percentages – Demographic breakdown Gender

Figure 10. UNCC edTPA passing percentages, by gender, all handbooks combined. Summary of pass percentage gender comparison Male candidates passing scores rose from 55.56 percent in fall 2014 to 89.36 percent in spring 2016. Similar to all demographic indicators, male candidates obtained their highest passing percentage in spring 2016. Female candidates passing percentage decreased from 79.17 percent in fall 2014 to 75.89 percent in fall 2015, however, they rebounded in spring 2016 to reach an 89.36 passing percentage. The overall female passing percentage exceeds the male passing percentage by 5.97 percent.

40.00%

50.00%

60.00%

70.00%

80.00%

90.00%

100.00%

Cumulative %Spring 2016Fall 2015Spring2015

Fall2014

% Passing Female % Passing Male

P a g e | 18

Ethnicity (White vs. non‐White), pass percentages overall

Figure 11. UNCC edTPA passing percentages, white vs. non‐white, all handbooks combined.

50.00%

55.00%

60.00%

65.00%

70.00%

75.00%

80.00%

85.00%

90.00%

95.00%

Cumulative %Spring 2016Fall 2015Spring2015

Fall2014

% Passing White % Passing Non‐white

P a g e | 19

Ethnicity comparison, by handbook – Y1

Figure 12. UNCC edTPA passing percentages, by handbook, white vs. non‐white, AY 2014‐15 (Year 1).

0

10

20

30

40

50

60

70

80

90

100

2014‐2015 White Pass Percentage 2014‐2015 Non White Pass Percentage

P a g e | 20

Ethnicity comparison, by handbook – Y2

Figure 13. UNCC edTPA passing percentages, by handbook, white vs. non‐white, AY 2015‐16 (Year 2). Summary of pass percentage White vs. non‐White comparison Overall, non‐White candidates passed at rate of 71.33 percent and Whites had a passing rate of 78.93 percent. White candidates’ overall pass percentage (78.93) is less than their fall 2014 percentage (80.20), whereas non‐White candidates’ overall percentage (71.33) is higher

0

10

20

30

40

50

60

70

80

90

100

2015‐2016 White Pass Percentage 2015‐2016 Non White Pass Percentage

P a g e | 21

than their initial passing percentage (63.04) in fall 2014. Each ethnic group obtained the highest passing percentage in spring of 2016, with non‐White candidates (92.42) exceeding their White peers (90.31) by 2.11 percent. Special Education/Elementary Dual had the highest passing rates for all ethnicities across both academic years at 100 percent. Non‐White candidates from Year 1 to Year 2 increased their passing percentage in Early Childhood, Middle Childhood English Language Arts, Middle Childhood Math, Secondary Math, Special Education and World Language programs. These increases culminated in the pass percentages for non‐White candidates exceeding that of their White peers in spring 2016. For non‐White candidates, their mean totals for both academic years were the highest in the following programs: SPEL, Early Childhood, and Elementary Education. In seven programs (English as an Additional Language, K‐12 Visual Arts, Middle Childhood Science, Middle Childhood Social Studies, Secondary English and Secondary Social Studies) there was a 27 percent decrease in average for non‐White candidates from Year 1 to Year 2. Elementary Education was the only program where White candidates pass percentages decreased from the previous 2014‐2015 academic year (1 percent). Calculating the mean for both academic years, White candidates performed the highest in the following programs: Early Childhood, Special Education/Elementary Dual, and English as an Additional Language and World Language.

P a g e | 22

Age Group – GRAPHIC UPDATED SEPTEMBER 15, 2016

Figure 14. UNC Charlotte edTPA passing percentages, by age group, all handbooks combined.

50.00%

55.00%

60.00%

65.00%

70.00%

75.00%

80.00%

85.00%

90.00%

95.00%

Cumulative %Spring 2016Fall 2015Spring 2015Fall 2014

% Passing 17‐22 % Passing 23‐28 % Passing 29‐40 % Passing 41 +

P a g e | 23

Table 4 UNC Charlotte 40+ “not passing,” by handbook, by track (YLI, GTA, GST, GLE) and distance education (DE) status*. *YLI = undergraduate, GTA = graduate teacher assistant, GST = graduate student teacher, and GLE = graduate lateral entry student. Note that some students in the graduate tracks may also be distance education students.

Program 40+ # Did Not Pass Y1 YLI GTA GST GLE DE

40 + # Did Not Pass Y2 YLI GTA GST GLE DE

Early Childhood 0 0 0 0 0 0 0 0 0 0 0 0 Elementary Ed 1 1 0 0 0 0 2 1 1 0 0 0 SPEL 0 0 0 0 0 0 0 0 0 0 0 0 Eng. As Additional Language 1 0 0 1 0 0 0 0 0 0 0 0 K‐12 Performing Arts 0 0 0 0 0 0 0 0 0 0 0 0 K‐12 Visual Arts 0 0 0 0 0 0 1 0 0 1 0 0 Middle Child ELA 0 0 0 0 0 0 0 0 0 0 0 0 Middle Child Math 2 1 0 1 0 0 2 0 0 1 1 1 Middle Child Sci 1 1 0 0 0 0 1 0 0 1 0 1 Middle Child SS 0 0 0 0 0 0 2 0 0 2 0 1 Secondary English 1 0 0 1 0 0 0 0 0 0 0 0 Secondary Math 2 0 0 2 0 0 1 0 0 1 0 0 Secondary Science 1 0 0 0 1 0 0 0 0 0 0 0 Secondary SS 0 0 0 0 0 0 2 0 0 1 1 1 SPED 5 0 0 1 4 5 2 0 0 1 1 0 World Language 2 0 0 1 1 0 0 0 0 0 0 0

TOTALS 16 3 0 7 6 5 13 1 1 8 3 4 Summary of pass percentage age group comparison As of spring 2016, candidates in the 17‐22 age cohort have the highest passing percentage (91.79), and the 40+ age cohort has the lowest (82.35). Each cohort saw an increase in passing percentages in spring 2016, which is consistent with the college’s overall upswing in passing percentages. Of all the cohorts, the 29‐40 group had the largest increase in passing percentage (16.69) over the data collection

P a g e | 24

period. As the 40+ group performed lower over the two‐year period than the other age groups, disaggregated data for this group are provided in Table 4. Traditional graduate certificate candidates overall struggled the most in this group.

P a g e | 25

UNC Charlotte Program Handbook Scores, by Individual Rubric All programs except for World Language and Elementary Education are scored based on 15 rubrics. Elementary Education is scored based on 18 rubrics, and World Language is scored based on 13 rubrics. In the following table, an explanation of each task, and corresponding rubric is provided. This is followed by each program’s overall candidate performance in the individual rubrics, separated by academic years. Table 5 Explanation of edTPA rubrics.

Task 1 Planning Rubric 1 The candidate’s ability to plan for a safe environment, and build on each other and include tasks that develop

students’ competencies in the psychomotor, cognitive, and affective domains. Rubric 2 The candidate’s ability to use knowledge of his/her students to target support for students to develop

competencies and knowledge in the psychomotor, cognitive, and affective domains. Rubric 3 Candidates use of knowledge of his/her students to justify instructional plans. Rubric 4 (omitted in World Lang)

Candidate’s ability to identify and support language demands associated with a key physical learning task.

Rubric 5 How the planned assessments are selected or designed to provide evidence of student progress toward the standards/objectives.

Task 2 Instruction Rubric 6 The candidate’s ability to promote a supportive and respectful learning environment that supports students. Rubric 7 The candidate’s ability to actively engage students in developing specific competencies in the psychomotor,

cognitive and/or affective domains. Rubric 8 The candidate’s ability to actively monitor students’ actions to further develop their competencies in the

psychomotor, cognitive, and/or affective domains. Rubric 9 Candidate’s use of pedagogical content knowledge to develop students’ competencies in psychomotor,

cognitive, and/or affective domains Rubric 10 Evidence of the candidate’s use of student learning, to evaluate and change teaching practice to better meet

students’ varied learning needs. Task 3 Assessing Student Learning

P a g e | 26

Rubric 11 How does the candidate analyze evidence of student learning of specific competencies in the psychomotor, cognitive, and/or affective domains?

Rubric 12 What type of feedback does the candidate provide to students? Rubric 13 How does the candidate provide opportunities for students to use the feedback to guide their further

learning? Rubric 14 (omitted in World Lang)

How does the candidate analyze students’ use of language to develop content understanding?

Rubric 15 How does the candidate use the analysis of what student do to plan next steps instruction? Task 4 Mathematics Assessment (Elementary Education only)

Rubric 16 Candidates’ ability to analyze whole class evidence to identify patterns of student learning. Rubric 17 The candidates’ ability to use student work to analyze mathematical errors, confusions, and partial

understanding. Rubric 18 The candidates’ ability to examine the re‐engagement lesson to further student learning.

P a g e | 27

Early Childhood

Figure 15. Early Childhood Handbook Rubric Average, by academic year. Candidates performed higher on Rubrics 1, 5, 6, 8, 9, 11, 12, and 13 in the 2015‐2016 (Y2) academic term than in the previous 2014‐2015 (Y1) academic term. Rubric 9 and Rubric 13 accounted for the lowest mean score, and Rubric 1 and Rubric 5 accounted for the highest mean score. The largest change within a rubric between the 2014‐2015 and 2015‐2016 academic terms occurred in Rubric 9, which had a .71 increase in mean score.

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

2014‐2015 Academic Year 2015‐2016 Academic Year

P a g e | 28

Elementary Literacy * *note: in fall 2014, all ELED candidates completed the Elementary Literacy handbook. Beginning in spring 2015, only SPEL Duals completed the Elementary Literacy handbook.

Figure 16. Elementary Literacy Handbook Rubric Average, by academic year. Elementary Literature candidates registered their highest totals in Rubric 12 and Rubric 15 respectively. The two lowest scores were in Rubric 14 and Rubric 10. Between both academic years, the candidates who participated in the edTPA examination in 2014 – 2015, performed better in each rubric, than their 2015 – 2016 counterparts; however, it is important to note that the population for this handbook changed in 2014‐15 (see note above). Rubric 9 had the most significant change, dropping more than .50 between both academic years.

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

Rubric 1 Rubric 2 Rubric 3 Rubric 4 Rubric 5 Rubric 6 Rubric 7 Rubric 8 Rubric 9 Rubric 10 Rubric 11 Rubric 12 Rubric 13 Rubric 14 Rubric 15


P a g e | 29

Elementary Education (Combination)

Figure 17. Elementary Education (Combination) Handbook Rubric Average, by academic year. Elementary Education candidates performed higher on 13 rubrics in 2014 – 2015, than in the 2015 – 2016 academic term. However, it is important to note that because of the population change described earlier in the document, scores were only recorded in spring 2015 on this handbook for the 2014‐15 academic year. The two rubrics with the lowest scores are 13 and 14, and the two rubrics that scored the highest are 6 and 9. The most noteworthy change occurred in Rubric 11, with a .19 increase between 2014 – 2015 and 2015 – 2016 academic terms.

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

Rubric 1 Rubric 2 Rubric 3 Rubric 4 Rubric 5 Rubric 6 Rubric 7 Rubric 8 Rubric 9 Rubric 10 Rubric 11 Rubric 12 Rubric 13 Rubric 14 Rubric 15 Rubric16

Rubric 17 Rubric 18


P a g e | 30

English as an Additional Language

Figure 18. English as an Additional Language Handbook Rubric Average, by academic year. English as an Additional Language candidates performed higher on all rubrics in the 2015 – 2016 academic term, with the exception of Rubric 9. Rubric 9 and Rubric 10 produced two lowest scores, and Rubric 1 and 3 had the highest scores amongst the 15 rubrics. The most notable change occurred in Rubric 11, where there is a .98 increase in mean score between the 2014 – 2015 and 2015 – 2016 academic terms.

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

4.50



P a g e | 31

K‐12 Performing Arts

Figure 19. K‐12 Performing Arts Handbook Rubric Average, by academic year. K‐12 Performing Arts produced its first edTPA scores during the 2015 – 2016 academic year. The lowest two rubrics are 13 and 14. Candidates performed the highest in Rubric 1, 5, 6, and 9 (each rubric registered a score of 3.29).

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00


2015‐2016 Academic Year

P a g e | 32

Middle Childhood English Language Arts

Figure 20. Middle Childhood English Language Arts Handbook Rubric Average, by academic year. Middle Childhood English Language Arts candidates in the 2015 – 2016 academic term, scored higher in all rubrics with the exception of Rubric 4 and Rubric 12. The largest improvement between the academic terms occurs in Rubric 11, with a .63 difference. The two highest rubrics are 1 and 7. The lowest scoring rubrics are Rubric 10, 14, and Rubric 13.

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00



P a g e | 33

Middle Childhood Math

Figure 21. Middle Childhood Math Handbook Rubric Average, by academic year. Middle Childhood Math candidates registered higher scores in all but one rubric (Rubric 1) in the 2015 – 2016 academic year, than in the previous academic year. The two lowest averages among all rubrics were in 10 and 14. The two highest rubrics were 6 and 12. Candidates’ scores in Rubric 15 registered the largest improvement of all rubrics, with a .41 difference between academic terms.

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50



P a g e | 34

Middle Childhood Science

Figure 22. Middle Childhood Science Handbook Rubric Average, by academic year Middle Childhood Science candidates scored the lowest in Rubric 9 and Rubric 11. The two highest scoring rubrics overall are Rubrics 2 and 4. The largest difference in scores between academic years occurred in Rubric 12, with a .86 score decrease between the 2014 – 2015 and 2015 – 2016 academic terms. Of the 15 rubrics measured, candidates scored higher in the 2014 – 2015 academic term in ten rubrics (2, 3, 4, 5, 8, 10, 11, 12, 14, and 15).

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00



P a g e | 35

Middle Childhood Social Studies

Figure 23. Middle Childhood Social Studies Handbook Rubric Average, by academic year. Middle Childhood Social Studies candidates scored higher in the 2014 – 2015 academic term in 11 rubrics, with candidates in the 2015 – 2016 only scoring higher in four rubrics (1, 8, 9, and 12). The highest difference between scores occurs in Rubric 11, with a decrease of .63 between academic terms. The two highest scoring rubrics are 5 and 11, and the lowest scoring rubrics are 13 and 14.

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00



P a g e | 36

Secondary English

Figure 24. Secondary English Handbook Rubric Average, by academic year. Secondary English candidates scored the highest in Rubric 1 and Rubric 12. The lowest scores are located in Rubric 13 and Rubric 14. Out of the fifteen rubrics, candidates in the 2015 – 2016 academic year outperformed their counterparts on twelve rubrics. Rubric 14 had the highest change in score, with candidates’ scores increasing .60 between academic years.

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00



P a g e | 37

Secondary Math

Figure 25. Secondary Math Handbook Rubric Average, by academic year. Secondary Math candidates’ highest score averages were in Rubric 6 and Rubric 12, and the lowest scores were in Rubric 13 and Rubric 14 respectively. Between the two academic years, among all rubrics, candidates in the 2015 – 2016 cohort scored higher on eight rubrics. Candidate scores in Rubric 3 produced largest average change by increasing by .36 between the two academic terms.

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50



P a g e | 38

Secondary Science

Figure 26. Secondary Science Handbook Rubric Average, by academic year. Candidates in Secondary Science registered their highest averages in Rubric 12 and Rubric 5. The lowest score averages were in Rubric 9 and Rubric 15. Out of the fifteen rubrics, candidates in the 2014 – 2015 academic term outperformed candidates in the 2015 – 2016 academic term in nine rubrics. Rubric 3 contains the largest change with a .85 decrease in score averages between the academic terms.

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00



P a g e | 39

Secondary Social Studies

Figure 27. Secondary Social Studies Handbook Rubric Average, by academic year. Secondary Childhood Social Studies candidates received their lowest scores in Rubric 13 and Rubric 14. Candidates performed the highest on Rubric 5 and Rubric 6. Overall, candidates’ scores for the 2015 – 2016 academic year were lower than the previous academic year. The most significant change occurred in Rubric 3, where there was a .33 decrease in candidates’ score average.

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50



P a g e | 40

Special Education

Figure 28. Special Education Handbook Rubric Average, by academic year. Special Education candidates produced the two lowest averages in Rubric 11 and Rubric 15, and the highest two averages in Rubric 6 and Rubric 7. Rubric 14 had the highest change, with the mean score improving .40 between the two academic years. All mean scores increased from the 2014 – 2015 to the 2015 – 2016 academic year.

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50



P a g e | 41

Visual Arts

Figure 29. Visual Arts Handbook Rubric Average, by academic year. Visual Arts Program completed its first complete academic year in 2015 – 2016. The lowest scoring rubrics were 13 and 15. The two highest scoring rubrics were 3 and 5.

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00


2015‐2016 Academic Year

P a g e | 42

World Language *World Language only has 13 rubrics so the mean scores overall are lower than other handbooks.

Figure 30. World Language Handbook Rubric Average, by academic year. World Language candidates performed better in the 2015 – 2016 academic term in seven of the thirteen rubrics (2, 6, 7, 8, 9, 10, and 13). The two strongest rubrics are 1 and 2, and the lowest scoring rubrics are 9 and 13. Rubric 8 highlights the largest change in scores, with a .68 improvement between the 2014 – 2015 and 2015 – 2016 academic term.

0.00

0.50

1.00

1.50

2.00

2.50

3.00

3.50

4.00

Rubric 1 Rubric 2 Rubric 3 Rubric 5 Rubric 6 Rubric 7 Rubric 8 Rubric 9 Rubric 10 Rubric 11 Rubric 12 Rubric 13 Rubric 15


P a g e | 43

Condition Codes / Incomplete Scores Beginning in fall 2015, SCALE changed the policy for scoring candidate work products that are submitted with errors that render them impossible to score. For example, work products with an edited video or with fewer than the requisite number of lesson plans do not meet the criteria established in the project directions, and therefore may not be scored accurately. Instead of rubric ratings, these products receive a condition code, a letter rating in lieu of a rubric score. The letters correspond to a defined set of explanations as to why the candidate’s work was judged unscorable. Work products that earned more than two condition codes in any given task are reported as Incompletes on the score reports. For the past two semesters (fall 2015 and spring 2016), incomplete scores were reviewed by Dr. Tom Fisher in the Office of Field Experiences (OFE) for any commonalities. Dr. Fisher worked with student teachers with who received Incompletes to develop an individual remediation plan. Each student teacher was provided a week to complete the plan, resolve the errors that led to the Incomplete, and then resubmit the edTPA product for local evaluation. Evaluators were faculty in OFE and in department programs with edTPA experience. In fall 2015, 18 student work products (one in Birth‐Kindergarten, eight in Elementary Education, and nine in the Middle‐Secondary Program) received Incompletes. In spring 2016, 17 student work products were rated as Incomplete because of indicated condition codes. There were seven candidates in Elementary Education, one in Special Education, three in Child and Family Development, five students in Middle‐Secondary Education and one student in Foreign Language Education. In order to better understand the nature of the condition code errors, the following table is provided, disaggregated by department. Arts numbers are reported with MDSK. Please note that it is typical for students to receive multiple condition codes for a single project. Table 6 Number and description of edTPA condition codes, by department, reported for all semesters, fall 2014‐spring 2016 (Y1‐2).

Total MDSK ELED SPCD Code Name Descriptor

Count of A's 40 10 20 10 Rubrics for

Planning Task are Unscorable

Your edTPA portfolio documents your planning, instruction and assessment within a learning segment designed to meet the learning needs of your students. Condition Code A may be assigned to Rubrics 1–5 for the following reasons: ‐ Fewer than three lesson plans are submitted, or ‐ for candidates teaching within a large block of 3–5 hours, the lesson plans cover less than 3 hours of instruction.

P a g e | 44

Total MDSK ELED SPCD Code Name Descriptor

Count of B's 4 4 0 0 Video technical issues

Any required video components of your submission must conform to technical specifications (e.g., format, file type), and be viewable by scorers. After uploading or transferring your video, you must review it in the Pearson ePortfolio system prior to submitting your assessment, even if the video file was playable prior to upload or transfer. You are responsible for ensuring that your submitted video meets the technical specifications and can be viewed by scorers by confirming successful playback prior to submission. Condition Code B will be assigned if your video cannot be played at all or if it is of insufficient technical quality to view.

Count of C's 21 12 0 9 Audio technical issues

The audio for required video components must be intelligible to scorers. Review your video(s) prior to submission for potential background noise or poor audio quality that would interfere with a scorer’s ability to hear you and your students. Please note that some environments (e.g., auditorium, gymnasium, pool) are less conducive to audio recording, and make any appropriate adjustments when recording your video to ensure adequate audio quality. Condition Code C will be assigned if your video contains no audio or if the audio associated with your video is of insufficient technical quality to review.

Count of D's 42 8 26 8

Insufficient or excessive

information to score

Your materials must provide sufficient evidence so that scorers can evaluate your performance according to the edTPA rubrics. Condition Code D may be assigned for the following reasons: • No response to the commentary prompt(s) provided or response is unrelated to edTPA tasks • The total time of your video clip(s) submitted for Task 2 is less than three minutes • The response to the commentary prompt was not within the allowable page limits (i.e., substantive response to the commentary prompt appeared after the maximum page limit for the document)

Count of E's 87 52 28 7 Incorrect or

missing, blank, or otherwise

Each file must be uploaded or transferred to the appropriate designated location in the Pearson ePortfolio system. Blank files cannot be scored. All required documents must be legible, including

P a g e | 45

Total MDSK ELED SPCD Code Name Descriptor inaccessible file handwriting in student work samples as well as candidate‐typed

responses. You must submit all materials required for your content area. You must review your uploaded or transferred materials prior to submission to ensure that you are submitting the correct, unique file(s) for each Part. Condition Code E may be assigned for the following reasons: • The incorrect file was uploaded to the designated location. • The file was blank, containing no text or graphics, or the video clip or audio file contained no content. • The file could not be opened for technical reasons (e.g., file extension does not match file type). • Documents are illegible (e.g., handwriting not legible, images are blurry, reduced in size, or otherwise cannot be read). • No referenced artifact submitted (e.g., Evidence of Feedback is referenced or described in the Commentary, but not submitted). • Hyperlinks to external documents are provided in lieu of the actual documents.

Count of F's 49 8 36 5 Video is edited

Submitted video clips must be continuous and unedited, with no interruption in events (e.g., you may not edit a video clip to remove “quiet time” when students are working independently, edit out physical transitions, or edit out disruptive behavior). Condition Code F may also be assigned if your face cannot be seen or your face has been blurred. You must review your uploaded or transferred video clip(s) prior to submission to ensure that they meet the video requirements described in the handbook.

Count of G's 104 41 19 44

Materials unrelated to

Handbook or fail to

conform to Handbook instructions

The submitted materials must correspond to the edTPA content area for which you have registered. Materials related to a content area for which you have not registered cannot be scored. Materials completely unrelated to edTPA cannot be scored. Condition Code G may be assigned for the following reasons: • The learning objectives are not in the registered content area. • The submission corresponds to a different content area than the area for which you registered (e.g., you registered for the Physical

P a g e | 46

Total MDSK ELED SPCD Code Name Descriptor Education edTPA assessment, but all submitted materials are related to a Health Education content area). • Artifacts are unrelated to the learning segment focus. • The number of students does not meet the minimum group size as specified in the Handbook. • The learning segment assessment is not completed by the whole class (or group within the class) as directed in the Handbook. • The student work samples are different for one or more focus students than described for the whole class. Exception: Differences in assessment are permitted if the assessments address the same central focus but are adapted to be more appropriate for the focus student(s) needs. (not applicable to Special Education and only applicable to the common assessment for Early Childhood)

Count of H's 0 0 0 0 English translation requirement not

met

If a portion of your submission (written or verbal) is not in English, the required translation/transcript must be included. Translations are not required for the following content areas: • a "World Language" or “Classical Language” field/specialty area • Elementary Education–Spanish • Elementary Literacy–Spanish • Elementary Mathematics–Spanish • Special Education using American Sign Language (ASL) in a Deaf & Hearing Impaired setting. • English as an Additional Language (EAL) permits some use of languages other than English by candidates. See the handbook for the extent to which translations are required.

P a g e | 47

Discussion edTPA has been in use as a key assessment at UNC Charlotte since fall 2014. With four semesters of data collection to date, there exists a rich source of information to use for program improvement purposes. Overall, it is helpful to consider specific areas for growth that could influence coursework instruction and/or clinical experiences as part of candidate training. Rubric Analysis Of the rubrics assessed in edTPA, the highest scoring rubrics overall at UNC Charlotte are found in Task 1 (Planning): Rubric 1 (Planning for Understandings: How do the candidate’s plans build on each other and include tasks that develop students’ competencies in the content area?), and Rubric 5 (Planning Assessments to Monitor and Support Student Learning: How are formal and informal assessments selected or designed to monitor student understanding?). Six of the 16 programs using edTPA saw their highest rubric scores on Rubric 1 or Rubric 5 or both. While Task 3 overall was the lowest scoring task, Rubric 12 (Providing Feedback to Guide Learning ) was also a high‐scoring rubric for five of the 16 programs. Of the rubrics assessed in edTPA, the two lowest scoring rubrics overall at UNC Charlotte are found in Task 3 (Assessment): Rubric 13 (Student Use of Feedback: How does the candidate provide opportunities for students to use the feedback to guide their future learning?), and Rubric 14 (Student Language Use: How does the candidate analyze students’ use of language to develop content understanding?) For Rubric 13, the guiding question addresses how candidates explain how they will help focus students to understand and use the feedback provided in order to improve their own learning. For a level 3, candidates describe how focus students will use the feedback related to the learning objectives. The feedback might include corrections of misunderstandings or extensions of learning related to the learning objectives. “Related to the learning objectives” is key. The feedback must be focused and aligned directly back to the objectives of the lesson, otherwise it may be considered too vague. For Rubric 14, the guiding question addresses how the candidate analyzes the use of the academic language demands (planned for in Task 1) to develop content understanding. This rubric is closely aligned with Rubric 4, where the academic language demands are planned. In Rubric 14, candidates explain the opportunities students had to use the academic language demands associated with the identified language function from Task 1, and how these opportunities support understanding of the central focus. It is imperative that candidates have a clear understanding of the concepts of “academic language demands” planned for in Task 1. Candidates must make explicit connections between the lesson activities and identified language demands. Candidates who lack conceptual understanding of the academic language demands will certainly struggle on this rubric.

P a g e | 48

Given that overall candidates score well on Rubric 12 (Providing Feedback to Guide Learning) but score poorly on Rubrics 13 and 14 in the same Task, faculty who are teaching assessment to candidates may wish to consider making deeper and deliberate connections to these concepts. The Understanding Rubric Progression documents available on our edTPA Moodle Resources site are excellent for exploring and understanding these concepts in more detail. Passing Rate Analysis In considering overall passing rates for specific demographic indicators, UNC Charlotte has worked to strengthen the passing rates for a majority of gender, age and ethnic groups. The data show that overall, non‐Whites, males, and older candidates have improved their scores from Year 1 to Year 2. As familiarity with the requirements of edTPA by faculty and candidates has increased, the pass rates have improved for all groups overall, with the highest scores occurring in the most recent semester (spring 2016). White candidates perform at higher rates than non‐White candidates do, which is on par with the state and national trends in edTPA. Still, the pass percentage of non‐White students who obtained a passing score within the college exceeded the number of White students who obtained a passing score in spring 2016. This helped shrink the overall gap in passing scores between White and non‐White candidates to just under 10 percent. In Year 1, 479 candidates attempted edTPA; 367 passed, giving a passing rate of 76.62 percent. In Year 2, 438 candidates attempted edTPA; 361 passed, giving a passing rate of 82.42 percent. Overall, 917 candidates have attempted edTPA at UNC Charlotte, 728 of those have earned a passing score through external official scoring, giving an overall pass rate to date of 79.39 percent. UNC Charlotte pass rates rose 5.8 percent from Year 1 to Year 2, a positive sign for both students and faculty who have been working extremely hard to implement edTPA in the College of Education.

P a g e | 49

Moving Forward with edTPA at UNC Charlotte The results from the edTPA data provide critical feedback to the College of Education from an internal and external standpoint. For the College of Education, the results represent an opportunity to shape the learning of candidates entering teacher education programs. While resistance and objections to edTPA still exist, it is important to note that faculty at UNC Charlotte generally (though not completely) agree that the edTPA Tasks embody the fundamental core of effective teaching. This conclusion is drawn through documented conversations held at TPALs meetings from the last two years, where any faculty are encouraged to attend and participate in discussions on edTPA and our implementation at UNC Charlotte. While it is too soon to tell if there is a connection between edTPA and value‐added measures from P12 classrooms, the addition of a nationally normed and validated assessment like edTPA provides a high degree of external credibility for our programs. In addition, edTPA tasks closely mirror those of National Board certification available to practicing teachers. By exposing our candidates to the tenets of effective teaching via the edTPA framework, it seems logical that many of our candidates may feel more ready to tackle National Board certification in the future. It is our hope that faculty will use these data to continue the meaningful and ongoing dialogue on program improvement already occurring in our college. Thank you to all faculty who work so diligently and tirelessly for the benefit of our candidates.