a curriculum audit™ · phoenix elementary school district # 1 audit report page iii a curriculum...

A Curriculum Audit™of the

Phoenix Elementary School District #1

Phoenix, Arizona

Curriculum Management Solutions, Inc.5619 NW 86th Street, Suite 500

Johnston, IA 50131

August 2017

Phoenix Elementary School District # 1 Audit Report Page iii

A Curriculum Audit™

of the

Phoenix elementary School DiStrict #1

Phoenix, Arizona

Curriculum Management Solutions, Inc.5619 NW 86th Street, Suite 500

Johnston, IA 50131

Date Audit Presented: August 2017

Members of the Phoenix Elementary School District #1 Audit Team:

Lead AuditorRoger C. Antón, Jr., M.A.

AuditorsPenny Gray, Ph.D.

Holly J. Kaptain, Ph.D.Ronnie Thompson, M.Ed.Jeffrey Tuneberg, Ph.D.

Olivia Elizondo Zepeda, M.Ed.

Phoenix Elementary School District # 1 Audit Report Page iv

Phoenix Elementary School District # 1 Audit Report Page v

Table of Contents

I. INTRODUCTION ...........................................................................................................................................1

Background ...................................................................................................................................................1

Audit Background and Scope of Work ..........................................................................................................4

II. METHODOLOGY ..........................................................................................................................................7

The Model for the Curriculum Audit™.........................................................................................................7

A Schematic View of Curricular Quality Control .........................................................................................7

Standards for the Auditors .............................................................................................................................8

Technical Expertise .................................................................................................................................8

The Principle of Objectivity ...................................................................................................................8

The Principle of Consistency ..................................................................................................................8

The Principle of Materiality ....................................................................................................................9

The Principle of Full Disclosure .............................................................................................................9

Data Sources of the Curriculum Audit™ ....................................................................................................10

Standards for the Curriculum Audit™ ........................................................................................................10

III. EXECUTIVE SUMMARY .........................................................................................................................13

IV. FINDINGS ...................................................................................................................................................19

STANDARD 1: The School District Demonstrates Its Control of Resources, Programs, and Personnel. ........19

What the Auditors Expected to Find in the Phoenix Elementary School District #1: .................................19

Overview of What the Auditors Found in the Phoenix Elementary School District #1: .............................19

Finding 1.1: The organizational chart does not meet audit criteria for sound organizational design. Most job descriptions do not meet audit criteria for current, accurate, and clear specifications of authority, responsibilities, curriculum linkage, and role relationships in the district. ..........................20

Finding 1.2: Board policies are inadequate to provide for a sound, effective system of curriculum management and quality control to attain the mission and goals established for student achievement in terms of curriculum design and delivery, consistency in organizational operations, program structures, and accountability for roles and responsibilities. ..................................................................................29

Finding 1.3: Evidence of planning was found in the district, but system planning did not direct key district functions to promote the change needed to close existing gaps in student achievement. Planning is inadequate to direct curriculum management, assessment, professional development, technology, and budgeting to ensure system-wide congruence and drive improvement in student achievement. .........................................................................................................................................54

STANDARD 2: The School District Has Established Clear and Valid Objectives for Students. ......................67

What the Auditors Expected to Find in the Phoenix Elementary School District #1: .................................67

Overview of What the Auditors Found in the Phoenix Elementary School District #1: .............................67

Finding 2.1: Phoenix Elementary School District #1 lacks a comprehensive curriculum management plan and a documented process to direct and implement the design, delivery, revision, or evaluation of curriculum consistently across the school district. ...............................................................................68

Phoenix Elementary School District # 1 Audit Report Page vi

Finding 2.2: The scope of the PreKindergarten through grade 8 written curriculum is inadequate for the core and non-core content areas to direct teaching and learning. ...................................................82

Finding 2.3: The quality of the written curriculum (PreK-8) is inadequate to provide clear guidance for effective teaching and learning. The design of assessment to support effective delivery of the district curriculum is inadequate to support student learning and success on state and local tests. .....87

STANDARD 3: The School District Demonstrates Internal Consistency and Rational Equity in Its Program Development and Implementation. ..................................................................................................................123

What the Auditors Expected to Find in the Phoenix Elementary School District #1: ...............................123

Overview of What the Auditors Found in the Phoenix Elementary School District #1: ...........................123

Finding 3.1: All students in the district do not have equal access to all programs. Certain subgroups are disproportionately represented in the accelerated learning and special education programs, and other subgroups are more likely to be retained or suspended. Although there is direction for ELL programming and MTSS, these programs are not consistently implemented in schools. Support services for students who are behind are not based on aligned assessment data and are not designed in response to specific student need. .......................................................................................................123

Finding 3.2: The district has no clearly defined instructional model or expectations for delivery. Curriculum delivery is typified by whole-group approaches or seatwork, basic types of cognition, limited use of differentiation and technology, and minimal use of powerful instructional strategies. Monitoring the delivery of curriculum is inadequately defined and inconsistently implemented to ensure the improvement of instructional practices leads to increased student achievement. .............144

Finding 3.3: Professional development is mostly site-based, and is neither coordinated nor planned at the district level in response to curriculum delivery needs. Teachers report different levels of support from leaders depending on the campus at which they work and are dissatisfied with the quality of the training provided at the district-level. .................................................................................................171

STANDARD 4: The School District Uses the Results from System-Designed and/or -Adopted Assessments to Adjust, Improve, or Terminate Ineffective Practices or Programs. ..................................................................179



Finding 4.1: District-wide student assessment data are below state averages in English language arts and mathematics and show significant gaps in various subgroups. ....................................................180

Finding 4.2: The scope of the district’s student assessment program is inadequate to provide sufficient data for curriculum and instructional changes to improve student achievement. ...............193

Finding 4.3: The district lacks a comprehensive student assessment and program evaluation plan to guide instructional decision making for the improvement of student achievement. ..........................199

Finding 4.4: The district lacks a systemic process for selecting formative assessment instruments and using the data from those instruments for improving instruction and continuing or terminating programs. ............................................................................................................................................206

STANDARD 5: The School District Has Improved Productivity. ..................................................................211



Finding 5.1: The budget development and decision making processes are not tightly aligned to the district’s curricular goals and priorities, nor are there adequate cost-benefit analyses within the financial network to assure maximum productivity. ...........................................................................212

Phoenix Elementary School District # 1 Audit Report Page vii

Finding 5.2: Academic intervention programs are offered on school campuses to improve student achievement; however, programs lack district wide coordination and are not systemically selected, monitored, or evaluated for effectiveness. ..........................................................................................224

Finding 5.3: A comprehensive facility plan does not exist to guide major renovation, remodeling, and new construction to address facility needs. Some features of aging facilities diminish the quality of the learning environment for students. ...............................................................................................232

Finding 5.4: The district’s instructional technology efforts are not guided by a district technology plan. Efforts to increase technology usage among teaching staff are unfocused and lack direction and oversight. Recent efforts are underway to increase instructional technology use in classrooms. .....242

V. RECOMMENDATIONS OF THE CMSI CURRICULUM AUDIT™ TEAM FOR THE IMPROVEMENT OF THE PHOENIX ELEMENTARY SCHOOL DISTRICT #1 .....................................................................253

Recommendation 1: Establish system-wide expectations, direction, and system guidance through updated and new board policies and administrative regulations to support systemic quality control of the educational program and system operations. ................................................................................253

Recommendation 2: Develop a comprehensive system to support the effective delivery of curriculum in Phoenix Elementary School District #1. Clearly define expectations for instructional planning, classroom-based approaches, monitoring, and professional development to provide a clear picture of engaging, challenging, and effective instruction. ...............................................................................255

Recommendation 3: Develop and implement a comprehensive curriculum management system that coordinates and focuses all curriculum management functions and tasks across and within departments and schools. Deeply align current district curriculum to the new Arizona Standards; redesign and direct curriculum revision to ensure curriculum documents are of the highest quality and deeply aligned in content, context, and cognitive rigor with district and high stakes assessments. ...264

Recommendation 4: Develop and implement a comprehensive written formal student assessment plan to encompass proper scope and quality alignment of the written, taught, and tested curriculum to guide instructional planning across the district. ..................................................................................274

Recommendation 5: Establish a system for monitoring and supporting equity across the district. Ensure that areas of greatest need receive commensurate support with additional financial and human resources. Establish clear direction for ELL and Accelerated Learning Procedures and the MTSS that outlines program expectations and guidelines to ensure equity of access and effectiveness of services. Commit to effectively communicate with parents and proactively build relationships in the community. .........................................................................................................................................277

Recommendation 6: Develop and implement an expectation among all district staff that data are to be used to improve the district’s curriculum design, curriculum delivery measured by improvements in student achievement results, and to evaluate based on a quality designed formal assessment system. ..............................................................................................................................285

Recommendation 7: Design and implement a comprehensive, curriculum-driven budget development process that emphasizes cost-benefit analyses that links resources to instructional priorities. Update long-range facility planning, providing evidence of the tangible connections between the district’s educational goals for students and the facility needs within the plan. ................................................286

Recommendation 8: Develop, adopt, and implement policy to support the use of technology, both instructional and management, across the district that enhances student learning and staff productivity. Clearly identify expectations for technology usage in classrooms and provide adequate professional development and oversight to ensure its implementation and links to student learning. ...................290

Phoenix Elementary School District # 1 Audit Report Page viii

Recommendation 9: Revise, adopt, and implement job descriptions and the organizational chart to meet audit criteria, establish clear lines of authority, and improve the clarity of functional relationships. .......................................................................................................................................292

Recommendation 10: Develop and implement a policy and procedure that standardizes program and intervention selection based on diagnosed needs and design and implement evaluation of program objectives with feedback linked to student achievement. Develop and implement program decision-making in the initiation, modification, continuation, or termination of programs and interventions that are assessed on valid and impartial knowledge of potential value and measured results. ..................299

Recommendation 11: Develop comprehensive assessment procedures to provide reliable data for decision making at all levels of the system. Establish a formalized process for selection, implementation, and evaluation of programs and interventions utilizing data that will be used as feedback for initiation, modification, continuation, or termination of programs and practices. ........301

Recommendation 12: Develop and implement coordinated planning processes that produce congruent district and campus plans and link the planning for curriculum management, professional development, student assessment, program evaluation, and budget development with those plans. .303

VI. APPENDICES ...........................................................................................................................................307

Appendix A: Auditors’ Biographical Data .........................................................................................309

Appendix B: List of Documents Reviewed .......................................................................................311

Appendix C: Blooms vs. DOK Matrix ..............................................................................................313

Appendix D: Characteristics of Culturally Responsive Teaching .....................................................315

Appendix E: Characteristics of Cognitively Engaging Instruction ...................................................317

Phoenix Elementary School District # 1 Audit Report Page ix

Table of Exhibits

Exhibit 0.1 District Schools ..........................................................................................................................3

Exhibit 0.2 Student Enrollment Data ............................................................................................................3

Exhibit 1.1.1 Curriculum Audit™ Principles of Sound Organizational Management ..................................21

Exhibit 1.1.2 2016 Phoenix Elementary School District #1 Organizational Chart .......................................22

Exhibit 1.1.3 Comparison of Administrative Positions on District Organizational Charts and Personnel Roster Without Job Descriptions .............................................................................................26

Exhibit 1.1.4 Curriculum Audit™ Rating Indicators for Job Descriptions ...................................................26

Exhibit 1.1.5 Rating of the Quality of Job Descriptions ...............................................................................27

Exhibit 1.2.1 Board Policies Reviewed by Auditors .....................................................................................31

Exhibit 1.2.2 Auditors’ Analysis of Board Policy and Administrative Regulations on Audit Standard One to Determine Quality and Degree of Adequacy .......................................................................35

Exhibit 1.2.3 Auditors’ Analysis of Board Policy and Administrative Regulations on Audit Standard Two to Determine Quality and Degree of Adequacy .......................................................................39

Exhibit 1.2.4 Auditors’ Analysis of Board Policy and Administrative Regulations on Audit Standard Three to Determine Quality and Degree of Adequacy .......................................................................42

Exhibit 1.2.5 Auditors’ Analysis of Board Policy and Administrative Regulations on Audit Standard Four to Determine Quality and Degree of Adequacy .......................................................................45

Exhibit 1.2.6 Auditors’ Analysis of Board Policy and Administrative Regulations on Audit Standard Five to Determine Quality and Degree of Adequacy .......................................................................46

Exhibit 1.2.7 Summary Ratings of the Auditors’ Analysis of Board Policy to Determine Quality and Degree of Adequacy ................................................................................................................51

Exhibit 1.3.1 Key District Planning Documents Reviewed by Auditors ......................................................54

Exhibit 1.3.2 Level I: Rating Characteristics of Quality Planning Design, Deployment, and Delivery ......56

Exhibit 1.3.3 Level II: Rating Characteristics of District-wide Plan Quality for Design, Deployment, and Delivery ...................................................................................................................................59

Exhibit 1.3.4 Level III: Rating Characteristics of Department and School Improvement Plan Quality for Design, Deployment, and Delivery .........................................................................................62

Exhibit 2.1.1 Tightly Held vs. Loosely Held Curriculum Management Functions and Components ...........68

Exhibit 2.1.2 Key Curriculum Planning Documents Reviewed by Auditors ................................................69

Exhibit 2.1.3 Curriculum Management Plan Characteristics and Auditors’ Assessment of District Approach ..................................................................................................................................70

Exhibit 2.1.4 Job Descriptions with Responsibilities for the Design and Delivery of Curriculum ..............73

Exhibit 2.2.1 Scope of the Written Curriculum by Subject Area and Grade Level PreKindergarten through Grade 8 ............................................................................................83

Exhibit 2.3.1 District Curriculum Documents Reviewed PreKindergarten Through Grade 8 ......................89

Exhibit 2.3.2 Curriculum Management Improvement Model Frame One Analysis: Minimal Basic Components for Curriculum Document Quality and Specificity ............................................93

Phoenix Elementary School District # 1 Audit Report Page x

Exhibit 2.3.3 Auditors’ Ratings of English Language Arts Curriculum Documents PreKindergarten Through Grade 8...........................................................................................94

Exhibit 2.3.4 Auditors’ Ratings of Mathematics Curriculum Documents PreKindergarten Through Grade 8...........................................................................................96

Exhibit 2.3.5 Auditors’ Ratings of Science Curriculum Documents PreKindergarten Through Grade 8...........................................................................................98

Exhibit 2.3.6 Auditors’ Ratings of Social Studies Curriculum Documents PreKindergarten, Montessori, Grades 6 Through 8 ...............................................................100

Exhibit 2.3.7 Auditors’ Ratings of Non-Core Subject Curriculum Documents PreKindergarten Through Grade 8.........................................................................................102

Exhibit 2.3.8 Summary of Auditors’ Mean Ratings of District Curriculum Documents by Content Area .103

Exhibit 2.3.9 All Teacher Responses: Quality and User-Friendliness of District Curriculum ....................104

Exhibit 2.3.10 Description of Cognitive Types in the Revised Bloom’s Taxonomy .....................................107

Exhibit 2.3.11 Internal Consistency of District Galileo® Pre-Assessment and Benchmark Assessment Items to Arizona College and Career Readiness Standards in English Language Arts Grades 3, 5, and 8 ..................................................................................................................108

Exhibit 2.3.12 Internal Consistency of District® Pre-Assessment and Benchmark Assessment Items to Arizona College and Career Readiness Standards in Mathematics Grades 3, 5, and 8 ..................................................................................................................115

Exhibit 2.3.13 Summary of the Analyses of Topological Alignment and Cognitive Type of Galileo®

Assessments with the Arizona College and Career Readiness Standards English Language Arts and Mathematics ..............................................................................121

Exhibit 3.1.1 English Language Learner and Free/Reduced lunch Enrollments by School .......................125

Exhibit 3.1.2 Accelerated Learning Procedures Enrollment by Gender .....................................................126

Exhibit 3.1.3 Accelerated Learning Procedures Enrollment by Ethnicity ..................................................127

Exhibit 3.1.4 Special Education Enrollment by Gender ..............................................................................128

Exhibit 3.1.5 Special Education Enrollment by Ethnicity ...........................................................................128

Exhibit 3.1.6 Disciplinary Actions by Gender .............................................................................................129

Exhibit 3.1.7 Disciplinary Actions by Ethnicity ..........................................................................................130

Exhibit 3.1.8 Disciplinary Actions by Program English Language Learner Subgroup ...............................131

Exhibit 3.1.9 Disciplinary Actions by Special Education Program Subgroup ............................................131

Exhibit 3.1.10 Disciplinary Actions by Free/Reduced Lunch Program Subgroup ........................................132

Exhibit 3.1.11 Retention Data by Program Subgroup ...................................................................................132

Exhibit 3.1.12 Teachers’ Ratings of Quality of District Programs on Online Survey (N=93) ......................133

Exhibit 3.1.13 Principals’ Perceptions Regarding Special Programs on Online Survey (N=9) ....................134

Exhibit 3.1.14 Teachers’ Perceptions Regarding ELL Program on Online Survey (N=91) ..........................135

Exhibit 3.1.15 Principals’ Perceptions Regarding Teachers’ Preparedness for Diverse Students on Online Survey (N=11) .......................................................................................................................136

Exhibit 3.1.16 Teachers’ Perceptions Regarding Special Education Services on Online Survey (N=93) ....138

Phoenix Elementary School District # 1 Audit Report Page xi

Exhibit 3.2.1 Principals’ Perceptions Regarding District Direction for Instructional Delivery on Online Survey (N=9) .........................................................................................................................146

Exhibit 3.2.2 Teachers’ Perceptions Regarding District Direction for Instructional Delivery on Online Survey (N=95) .......................................................................................................................147

Exhibit 3.2.3 Teachers’ Perceptions Regarding District Communication of Expectations for Instructional Delivery (N=87) .....................................................................................................................148

Exhibit 3.2.4 Classroom Observation Characteristics and Definitions .......................................................149

Exhibit 3.2.5 Dominant Teacher Activity Observed (N=131) .....................................................................151

Exhibit 3.2.6 Dominant Student Activity Learning Arrangements (N=137) ...............................................152

Exhibit 3.2.7 Dominant Student Activity Observed (N=136) .....................................................................153

Exhibit 3.2.8 Type of Knowledge Observed (N=127).................................................................................154

Exhibit 3.2.9 Cognitive Process Dimension Observed on Online Survey (N=128) ...................................155

Exhibit 3.2.10 Teachers’ Responses Regarding Differentiation on Online Survey (N=96) ..........................156

Exhibit 3.2.11 Frequency of Differentiation on Online Survey (N=10) .......................................................157

Exhibit 3.2.12 Principals’ Perceptions of Instructional Effectiveness on Online Survey (N=9) ...................158

Exhibit 3.2.13 Principals’ Perceptions of Instructional Practices on Online Survey (N=9) ..........................159

Exhibit 3.2.14 Evidence of Differentiation During Classroom Visits on Online Survey (N=*) ...................160

Exhibit 3.2.15 Effective Strategies Observed on Online Survey (N=81 classrooms out of 138; 256 strategies noted overall) ..................................................................................................162

Exhibit 3.2.16 Frequency of Classroom Visits by Principals on Online Survey (N=10) ..............................164

Exhibit 3.2.17 Teacher Responses to the Frequency of Classroom Visits by Principals and Other School Administrators (N=89) ...........................................................................................................165

Exhibit 3.2.18 Principal Walk-Through Protocols (N=10) ............................................................................167

Exhibit 3.2.19 Usefulness and Frequency of Feedback to Teachers on Online Survey (N=88) ...................168

Exhibit 3.2.20 Teacher Perception of Campus Instructional Leadership on Online Survey (N=87) ............169

Exhibit 3.2.21 Effectiveness of Instructional Leadership (N=87) .................................................................170

Exhibit 3.3.1 Curriculum Management Improvement Model Staff Development Criteria Auditors’ Assessment of Staff Development Program ..........................................................172

Exhibit 3.3.2 Teachers’ Perceptions Concerning Adequacy of Professional Development on Online Survey (N=88) .......................................................................................................................173

Exhibit 3.3.3 Teachers’ Perceptions Concerning Quality and Relevance of Professional Development Offerings (N=89) ...................................................................................................................174

Exhibit 3.3.4 Principals’ Perceptions Concerning Professional Development (N=9) .................................175

Exhibit 3.3.5 Principals’ Perceptions Concerning Quality and Relevance of Professional Development Offerings (N=8) .....................................................................................................................176

Exhibit 4.1.1 Comparison of State and District English Language Arts Performance by Percentage of Students Scoring Proficient and Above in Grades 3-8 on the AzMERIT ..............................180

Exhibit 4.1.2 Comparison of State and District Mathematics Performance by Percentage of Students Scoring Proficient and Above in Grades 3-8 on the AzMERIT .............................................181

Phoenix Elementary School District # 1 Audit Report Page xii

Exhibit 4.1.3 Comparison of White and Hispanic Students Performance by Percentage of Students Scoring Proficient and Above in Grades 3-8 on the AzMERIT in English Language Arts ...182

Exhibit 4.1.4 Comparison of White and Hispanic Students Performance by Percentage of Students Scoring Proficient and Above in Grades 3-8 on the AzMERIT in Mathematics ...................183

Exhibit 4.1.5 Comparison of White and African American Students Performance by Percentage of Students Scoring Proficient and Above in Grades 3-8 on the AzMERIT in English Language Arts ...184

Exhibit 4.1.6 Comparison of White and African American Students Performance by Percentage of Students Scoring Proficient and Above in Grades 3-8 on the AzMERIT in Mathematics ...................185

Exhibit 4.1.7 Comparison of Hispanic and African American Students Performance by Percentage of Students Scoring Proficient and Above in Grades 3-8 on theAzMERIT in English Language Arts ........................................................................................................................................186

Exhibit 4.1.8 Comparison of Hispanic and African American Students Performance by Percentage of Students Scoring Proficient and Above in Grades 3-8 on the AzMERIT in Mathematics ....187

Exhibit 4.1.9 Comparison of District White Students and State White Students Performance by Percentage of Students Scoring Proficient and Above in Grades 3-8 on the AzMERIT in English Language Arts ........................................................................................................................188

Exhibit 4.1.10 Comparison of District White Students and State White Students Performance by Percentage of Students Scoring Proficient and Above in Grades 3-8 on the AzMERIT in Mathematics 189

Exhibit 4.1.11 Performance by Percentage of Low SES Students Scoring Proficient and Above in Grades 3-8 on the AzMERIT in English Language Arts .......................................................190

Exhibit 4.1.12 Performance by Percentage of Low SES Students Scoring Proficient and Above in Grades 3-8 on the AzMERIT in Mathematics .......................................................................191

Exhibit 4.1.13 “Years to Parity” Calculation Template .................................................................................192

Exhibit 4.2.1 Formal Assessments of Student Performance ........................................................................195

Exhibit 4.2.2 Formal Assessments Administered in PreK through Grade 8 by Content Area ....................196

Exhibit 4.2.3 Scope of Formal Assessment for Grades PreK-8 ..................................................................197

Exhibit 4.3.1 Characteristics of a Comprehensive Student Assessment and Program Evaluation Plan .....202

Exhibit 4.4.1 Teacher Response to the Statement: “How frequently do you use the results of assessments to plan instruction?” ...................................................................................................................208

Exhibit 4.4.2 Teacher Response to the Statement: “I use student assessment data for the following…” ...209

Exhibit 4.4.3 Characteristics of a Quality Program Evaluation Plan or Process and Auditors’ Assessment of the District’s Approach ......................................................................................................210

Exhibit 5.1.1 Fiscal Planning Documents Reviewed by Auditors ...............................................................213

Exhibit 5.1.2 Statement of Revenues and Expenditures Maintenance and Operations Fund .....................215

Exhibit 5.1.3 Budget Spending Percentages by Operational Category Percentage of Maintenance and Operations ..............................................................................................................................216

Exhibit 5.1.4 Components of a Performance-based Budget and Adequacy of Use in the Budget Development Process .............................................................................................................217

Exhibit 5.1.5 Results of Campus Leader Survey Question on District Budget Development ....................218

Exhibit 5.1.6 Results of Campus Leader Survey Question on Building Budget Development ..................219

Exhibit 5.1.7 Enrollment Trends: Resident vs. Non-Resident Students .....................................................220

Phoenix Elementary School District # 1 Audit Report Page xiii

Exhibit 5.1.8 Federal Title Grant Funds ......................................................................................................221

Exhibit 5.1.9 Arizona Tax Credit Revenue by School and Per Pupil ..........................................................222

Exhibit 5.1.10 Arizona Tax Credit Expenditures by Program and Building .................................................223

Exhibit 5.2.1 Intervention Programs: District and Campus Level Offerings ..............................................226

Exhibit 5.2.2 Comparison of Imagine Learning to Audit Intervention Design Criteria ..............................228

Exhibit 5.2.3 Comparison of the Imagine Learning to Intervention Implementation Criteria ....................230

Exhibit 5.3.1 Facility Documents Reviewed by Auditors ...........................................................................234

Exhibit 5.3.2 Comparison of Facilities Planning Efforts to Audit Components of a Comprehensive Long-range Facilities Plan .....................................................................................................235

Exhibit 5.3.3 Principals’ Responses Regarding Response Time to Maintenance Issues (N=10) ................237

Exhibit 5.3.4 Facility Construction Dates ...................................................................................................238

Exhibit 5.3.5 Recent and On-going Facility Projects ..................................................................................239

Exhibit 5.3.6 Student to Teacher Ratio by School ......................................................................................240

Exhibit 5.3.7 District Facility Efficiency Measures Compared to State and Peer Districts ........................241

Exhibit 5.4.1 Auditors’ Assessment of Quality Criteria for Instructional Technology Related Programs ..245

Exhibit 5.4.2 Computer Inventory by School and Computer/Student Ratio ...............................................248

Exhibit 5.4.3 Observed Teacher Use of Instructional Technology ..............................................................249

Exhibit 5.4.4 Teacher Use of Technology Based on SAMR Model ............................................................250

Exhibit R.2.1 Mastery Learning Model .......................................................................................................258

Exhibit R.9.1a Recommended District Organizational Chart ........................................................................295

Exhibit R.9.1b Curriculum and Instruction Organizational Chart .................................................................296

Exhibit R.9.2 Job Description for Chief Academic Officer .........................................................................297

Exhibit R.9.3 Recommended Modifications to the Organizational Chart ....................................................299

Phoenix Elementary School District # 1 Audit Report Page xiv

Phoenix Elementary School District # 1 Audit Report Page xv

Table of Photographs

Phoenix #1 Superintendent insists that all decisions enhance student learning. ................................................53

Teacher posts standards and objectives for daily lessons. .................................................................................85

Paired students work together on reading assignment. One students points to the words as the other reads aloud. ..................................................................................................................................................................96

Pairs of students work on math problem at the whiteboard. ..............................................................................98

Students use manipulatives in their mathematics lesson to calculate the perimeter of shapes. .........................98

Microscopes are available for students to use in science class. .......................................................................100

Students enjoy music education class on a regular basis. ................................................................................101

Students are regularly assessed with district and state test instruments. .........................................................198

Schools in the district take advantage of outdoor learning environment for student activities. ......................239

Imaginative furnishing in the classroom offers students a special reading corner...........................................239

Kenilworth Elementary School portrays the district’s pride in its historical facilities ....................................241

Recent district technology initiative has provided computer labs with headsets for all schools. ....................247

Teacher utilizes her SmartBoard to facilitate teaching her lesson. All district classrooms have been furnished with SmartBoards. ............................................................................................................................................247

Phoenix Elementary School District # 1 Audit Report Page xvi

Phoenix Elementary School District # 1 Audit Report Page 1

A Curriculum Audit™

of the

Phoenix Elementary School District #1

Phoenix, Arizona

I. INTRODUCTIONThis document constitutes the final report of a Curriculum Audit™ of the Phoenix Elementary School District #1. The audit was commissioned by the Phoenix Elementary School District #1 Board of Education/Governing Authority within the scope of its policy-making authority. It was conducted during the time period of May 1-4, 2017. Document analysis was performed off-site, as was the detailed analysis of findings and site visit data.

A Curriculum Audit™ is designed to reveal the extent to which officials and professional staff of a school district have developed and implemented a sound, valid, and operational system of curriculum management. Such a system, set within the framework of adopted board policies, enables the school district to make maximum use of its human and financial resources in the education of its students. When such a system is fully operational, it assures the district taxpayers that their fiscal support is optimized under the conditions in which the school district functions.

Background

Phoenix Elementary School District #1 is an urban community in Maricopa County, Arizona, in the center of the city of Phoenix. The district was the first school district created in the state of Arizona in 1871. It has celebrated its place of prominence in the state during its 146 years history.

While property values continue to rise, the student enrollment in Phoenix #1 has declined from a peak enrollment of 35,000 students in 1965 to slightly over 7,000 students in 2017, as surrounding cities have created their own school districts and over 50 charter schools have opened since 1994.

Phoenix Elementary School District #1 serves the area with 13 elementary schools and one early childhood center. Overall, enrollment in the district has declined over the last eight years. Its enrollment has decreased by 422 students since 2009-10, a decrease of 5.5%.

The board’s adopted mission statement, “Phoenix Elementary School District #1 provides each child we serve with educational experiences that ensure academic and social success.”

Our mission is aligned with state academic standards that include:

• Every child performing at grade level or above in reading, writing, mathematics, science and social studies.

• Every child prepared for high school.

The district’s goals, presented in the 2012 document Our Blueprint for Student Success (Strategic Plan), are:

• Student Academic Success

○ All Schools Exceeding State Standards

○ Students Performing at/or Above Grade Level

○ Students Successfully Promoted at Every Grade Level

○ Students are Prepared for High School, College and Career


• Stakeholder Satisfaction

○ Annual Parent Satisfaction Survey to Measure Quality of Instruction and Service

○ Effective Processional Development & Opportunities for Staff

• Enrollment

○ Increase Student Enrollment to Full Capacity by 2018

• Partnerships

○ Engage Parents as Partners in their Children’s Learning

○ Engage Local Business Leaders & Community Members to Gain Support

○ Develop Extended Learning Opportunities for Students through Partnerships

• Culture

○ Provide a Welcoming Environment for all Students, Staff, Parents, Families and Partners

○ Practice and Model Expert Communication and Customer Service Skills

School District Organization and Governance

Per Board Policy AA: School District Legal Status, “The legally designated name of the District is School District No. 1 of Maricopa County. The official name shall be Phoenix Elementary School District No. 1.”

Under the provisions of Arizona Revised Statutes (ARS) 15-341: General powers and duties; immunity; delegation, “A. The governing board shall: 1. Prescribe and enforce policies and procedures for the governance of schools…3. Manage and control the school property within its district…5. Prescribe the curricula and criteria for the promotion and graduation of pupils…25. Establish an assessment, data gathering and reporting system…26. Provide special education programs and related services…and, F. A governing board may delegate in writing to a superintendent, principal or head teacher the authority to prescribe procedures that are consistent with the governing board’s policies.”

Current Board of School Directors

The Governing Board of the Phoenix Elementary School District #1, consists of five members elected at large in even numbered years to staggered four-year terms. The District Governing Board routinely meets at least twice monthly.

The Governing Board members at the time of the audit were:

• Ms. Susan Benjamin, President – (2013 – Present)

• Ms. Lynné Almy, Board Member – (2017 – Present)

• Mr. Daniil Gunitskiy, Board Member – (2015 – Present)

• Dr. Ruth Ann Marston, Board Member – (2003 – Present)

• Dr. Louisa Stark, Board Member – (2005 – Present)

Enrollment and Trends

The Phoenix Elementary School District #1 is located in Maricopa County, Arizona and currently serves 7,208 students from Phoenix and surrounding areas in Head Start and grades PreK-8. The district’s enrollment includes students who reside outside of the Phoenix #1 boundary. Since the 2010-11 school year, an average of more than 34% of the district’s enrollment is nonresident students from surrounding districts.

Simultaneously, Phoenix ESD #1 students leave the district through open enrollment. In 2015, 1,270 district students exited to attend schools in other school districts or charter schools. Overall, district enrollment has benefited from the influx of nonresident student attendance.


The district has 14 campuses, as noted in Exhibit 0.1.

Exhibit 0.1

District SchoolsPhoenix Elementary School District #1

May 2017

Bethune School (PreK-8) Capitol School (PreK-8)Dunbar School (PreK-8) Edison School (PreK-8)Emerson School (PreK-8) Garfield School (PreK-8)Heard School (PreK-8) Herrera Arts & Dual Language School (K-8)Kenilworth School (PreK-8) Lowell School (PreK-8)Magnet Traditional School (K-8) Shaw Montessori School (PreK-8)Whittier School (PreK-8) Faith North Early Childhood Learning Center (PreK)

Exhibit 0.1 shows the following:

• Eleven schools are PreK-8.

• Two schools contain grades K-8.

• The early learning center enrolls only PreKindergarten.

Enrollment in the district has decreased 5.5% since the 2009-10 school year. Exhibit 0.2 summarizes district enrollment trends by grade level for eight years from 2009-10 to 2016-17.

Exhibit 0.2

Student Enrollment DataPhoenix Elementary School District #1

2009-10 through 2016-17

Grade Level 2009-10 2010-11 2011-12 2012-13 2013-14 2014-15 2015-16 2016-17 8 Years +/-

PercentageHead Start 225 237 235 278 173 161 143 159 <29.3%>Preschool 395 408 396 371 384 392 410 375 <5.1>

K 916 932 944 870 933 803 679 690 <24.7>1 914 878 915 900 834 875 759 707 <22.6>2 895 869 853 866 897 838 839 765 <14.5>3 856 828 831 803 861 842 762 832 <2.8>4 819 836 861 791 798 807 820 761 <7.1>5 756 810 808 775 746 780 779 783 +3.66 715 712 767 743 756 713 732 745 +4.27 583 662 685 711 708 709 666 710 +21.88 556 603 675 646 660 686 708 681 +22.5

Total Enr. 7,630 7,775 7,970 7,754 7,750 7,606 7,297 7,208 <5.5%>

Based on data provided by the district in Exhibit 0.2, student enrollments indicate the following:

• Total enrollment in the district decreased 422 students (5.5%) from 2009-10 to 2016-17, with overall decreases in Head Start through grade 4 (except Kindergarten) totaling 731 students offset by increases in grades 5-8 totaling 309 students.

• The enrollment data summary shows a significant decrease in student enrollment of 762 students (9.6%) since the 2011-12 school year’s peak enrollment.


• Head Start, Kindergarten, and first grades reflect declines in enrollment in excess of 20% (average 24.3%).

• Grades 5-8 reflect an increase in enrollment of 11.8%.

• District enrollment increased annually from 2009-10 through 2011-12 but has declined yearly since the 2012-13 school year.

• Cohort survival enrollments from grade to grade have all declined with the exception of second grade enrollments in the 2014-15 and 2016-17 school years that were four and six more students, respectively, than in the previous year.

• The 2009-10 Kindergarten class (916 students) has decreased its enrollment in the district by 206 students in the seventh grade in 2016-17.

Enrollment Data by Ethnic and Socioeconomic Subgroups

The student population of Phoenix Elementary School District #1 has remained relatively static in its diversity. Based on the data obtained from district personnel, the 2016-17 student population is represented as follows by ethnicity:

• Hispanic – 79.8%

• African American – 8.4%

• White – 5.8%

• American Indian – 2.9%

• Asian, Pacific Islander, other – 3.0%

Students designated as Low Income (Economically Disadvantaged) status revealed 5,955 students (82.6%) in the district for the 2016-17 school year.

Audit Background and Scope of Work

The Curriculum Audit™ is a process that was developed by Dr. Fenwick W. English and first implemented in 1979 in the Columbus Public Schools, Ohio. The audit is based upon generally-accepted concepts pertaining to effective instruction and curricular design and delivery, some of which have been popularly referred to as the “effective schools research.”

A Curriculum Audit™ is an independent examination of four data sources: documents, interviews, site visits, and online surveys. These are gathered and triangulated, or corroborated, to reveal the extent to which a school district is meeting its goals and objectives, whether they are internally or externally developed or imposed. A public report is issued as the final phase of the auditing process.

The audit’s scope is centered on curriculum and instruction, and any aspect of operations of a school system that enhances or hinders its design and/or delivery. The audit is an intensive, focused, “postholed” look at how well a school system such as Phoenix Elementary School District #1 has been able to set valid directions for pupil accomplishment and well-being, concentrate its resources to accomplish those directions, and improve its performance, however contextually defined or measured, over time.

The Curriculum Audit™ does not examine any aspect of school system operations unless it pertains to the design and delivery of curriculum. For example, auditors would not examine the cafeteria function, unless students were going hungry and, therefore, were not learning. It would not examine vehicle maintenance charts, unless buses continually broke down and children could not get to school to engage in the learning process. It would not be concerned with custodial matters, unless schools were observed to be unclean and unsafe for children to be taught.


The Curriculum Audit™ centers its focus on the main business of schools: teaching, curriculum, and learning. Its contingency focus is based upon data gathered during the audit that impinges negatively or positively on its primary focus. These data are reported along with the main findings of the audit.

In some cases, ancillary findings in a Curriculum Audit™ are so interconnected with the capability of a school system to attain its central objectives, that they become major, interactive forces, which, if not addressed, will severely compromise the ability of the school system to be successful with its students.

Curriculum Audits™ have been performed in over 500 school systems in more than 41 states, the District of Columbia, and several other countries, including Canada, Saudi Arabia, New Zealand, Bangladesh, Malaysia, and Bermuda.

The methodology and assumptions of the Curriculum Audit™ have been reported in the national professional literature for more than two decades, and at a broad spectrum of national education association conventions and seminars, including the American Association of School Administrators (AASA); Association of Supervision and Curriculum Development (ASCD); National Association of Secondary School Principals (NASSP); Association for the Advancement of International Education (AAIE); American Educational Research Association (AERA); National School Boards Association (NSBA); and the National Governors Association (NGA).

This audit was conducted in accordance with a contract between Phoenix Elementary School District #1 and Curriculum Management Solutions, Inc. All members of the team were certified by Curriculum Management Solutions, Inc.

The curriculum auditors in this audit were:

• Roger C. Antón, Jr., M.A.

• Penny Gray, Ph.D.

• Holly J. Kaptain, Ph.D.

• Ronnie Thompson, M.Ed.

• Jeffrey Tuneberg, Ph.D.

• Olivia Elizondo Zepeda, M.Ed.

Biographical information about the auditors is found in Appendix A.

System Purpose for Conducting the Audit

In July 2016, Assistant Superintendent of Business Services Larry Weeks was appointed as the Chief Executive Officer (CEO) of the Phoenix Elementary School District #1. In April 2017, the Governing Board appointed Mr. Weeks as the superintendent of the district. The new superintendent “saw the value in reviewing the district’s Strategic Plan as a means to set direction for the short-term and long-term, with a focus on 2025.”

During strategic planning committee meetings, the idea of a curriculum audit was supported by the Governing Board, CEO, and administrators to complement individual program and department audits that had been conducted by the Arizona Department of Education. In an effort to respond to rising external factors challenging education in the state and nation, the Governing Board and Superintendent have focused on the use of “instructional time to engaging students in meaningful learning experiences.”

Areas identified for inspection and investigation by the district include:

• The structure of the organization for success;

• If the district’s budget reflects its goals;

• The efficient use of time;

• The use of data to guide decisions;

• Equitable service to students for academic and social/emotional needs;


• Effective delivery systems for curriculum, instruction, and assessment;

• Sufficient professional development; and

• A communication system that provides information to families.

The superintendent intends for the curriculum audit to serve as a “guide for stakeholders to identify immediate action steps, as well as being long-term goals.” The Strategic Planning Steering committee will be tasked with consolidating the audit, the strategic plan, and the state’s newly required Integrated Action Plans. The superintendent stated, “Our goal will be to blend the results of our work from these three tools to develop one manageable plan to guide teaching for learning in our school system.”

Approach of the Audit

The Curriculum Audit™ has established itself as a process of integrity and candor in assessing public school districts. It has been presented as evidence in state and federal litigation concerning matters of school finance, general resource managerial effectiveness, and school desegregation efforts in Kansas, Kentucky, New Jersey, and South Carolina. The audit served as an important data source in state-directed takeovers of school systems in New Jersey, California, and Kentucky. The Curriculum Audit™ has become recognized internationally as an important, viable, and valid tool for the improvement of educational institutions and for the improvement of curriculum design and delivery.

The Curriculum Audit™ represents a “systems” approach to educational improvement; that is, it considers the system as a whole rather than a collection of separate, discrete parts. The interrelationships of system components and their impact on overall quality of the organization in accomplishing its purposes are examined in order to “close the loop” in curriculum and instructional improvement.


II. METHODOLOGY

The Model for the Curriculum Audit™

The model for the Curriculum Audit™ is shown in the schematic below. The model has been published widely in the national professional literature, including the best-selling book, The Curriculum Management Audit: Improving School Quality (1995, Frase, English, Poston).

A Schematic View of Curricular Quality Control

General quality control assumes that at least three elements must be present in any organizational and work-related situation for it to be functional and capable of being improved over time. These are: (1) a work standard, goal/objective, or operational mission; (2) work directed toward attaining the mission, standard, goal/objective; and (3) feedback (work measurement), which is related to or aligned with the standard, goal/objective, or mission.

When activities are repeated, there is a “learning curve,” i.e., more of the work objectives are achieved within the existing cost parameters. As a result, the organization, or a subunit of an organization, becomes more “productive” at its essential short- or long-range work tasks.

Within the context of an educational system and its governance and operational structure, curricular quality control requires: (1) a written curriculum in some clear and translatable form for application by teachers in classroom or related instructional settings; (2) a taught curriculum, which is shaped by and interactive with the written one; and (3) a tested curriculum, which includes the tasks, concepts, and skills of pupil learning and which is linked to both the taught and written curricula. This model is applicable in any kind of educational work structure typically found in mass public educational systems, and is suitable for any kind of assessment strategy, from norm-referenced standardized tests to more authentic approaches.

The Curriculum Audit™ assumes that an educational system, as one kind of human work organization, must be responsive to the context in which it functions and in which it receives support for its continuing existence. In the case of public educational systems, the support comes in the form of tax monies from three levels: local, state, and federal.

In return for such support, mass public educational systems are supposed to exhibit characteristics of rationality, i.e., being responsive to the public will as it is expressed in legally constituted bodies such as Congress, state legislatures, and locally elected/appointed boards of education.


In the case of emerging national public school reforms, more and more this responsiveness is assuming a distinctive school-based management focus, which includes parents, teachers, and, in some cases, students. The ability of schools to be responsive to public expectations, as legally expressed in law and policy, is crucial to their future survival as publicly-supported educational organizations. The Curriculum Audit™ is one method for ascertaining the extent to which a school system, or subunit thereof, has been responsive to expressed expectations and requirements in this context.

Standards for the Auditors

While a Curriculum Audit™ is not a financial audit, it is governed by some of the same principles. These are:

Technical Expertise

CMSi-certified auditors must have actual experience in conducting the affairs of a school system at all levels audited. They must understand the tacit and contextual clues of sound curriculum management.

The Phoenix Elementary School District #1 Curriculum Audit™ Team included auditors who have been school superintendents, assistant superintendents, directors, coordinators, principals and assistant principals, as well as elementary and secondary classroom teachers in public educational systems with combined experience for almost 200 years in California, Texas, Iowa, Ohio, Arizona, Guam, Tennessee, Oklahoma, and Indiana.

The Principle of Independence

None of the Curriculum Audit™ Team members had any vested interest in the findings or recommendations of the Phoenix Elementary School District #1 Curriculum Audit™. None of the auditors has or had any working relationship with the individuals who occupied top or middle management positions in the Phoenix Elementary School District #1, nor with any of the past or current members of the Phoenix Elementary School District #1 Board of Education.

The Principle of Objectivity

Events and situations that comprise the database for the Curriculum Audit™ are derived from documents, interviews, and site visits. Findings must be verifiable and grounded in the database, though confidential interview data may not indicate the identity of such sources. Findings must be factually triangulated with two or more sources of data, except when a document is unusually authoritative such as a court judgment, a labor contract signed and approved by all parties to the agreement, approved meeting minutes, which connote the accuracy of the content, or any other document whose verification is self-evident.

Triangulation of documents takes place when the document is requested by the auditor and is subsequently furnished. Confirmation by a system representative that the document is, in fact, what was requested is a form of triangulation. A final form of triangulation occurs when the audit is sent to the superintendent in draft form. If the superintendent or his/her designee(s) does not provide evidence that the audit text is inaccurate, or documentation that indicates there are omissions or otherwise factual or content errors, the audit is assumed to be triangulated. The superintendent’s review is not only a second source of triangulation, but is considered summative triangulation of the entirety of the audit.

The Principle of Consistency

All CMSi-certified curriculum auditors have used the same standards and basic methods since the initial audit conducted by Dr. Fenwick English in 1979. Audits are not normative in the sense that one school system is compared to another. School systems, as the units of analysis, are compared to a set of standards and positive/negative discrepancies cited.


The Principle of Materiality

CMSi-certified auditors have broad implied and discretionary power to focus on and select those findings that they consider most important to describing how the curriculum management system is functioning in a school district, and how that system must improve, expand, delete, or reconfigure various functions to attain an optimum level of performance.

The Principle of Full Disclosure

Auditors must reveal all relevant information to the users of the audit, except in cases where such disclosure would compromise the identity of employees or patrons of the system. Confidentiality is respected in audit interviews.

In reporting data derived from site interviews, auditors may use some descriptive terms that lack a precise quantifiable definition. For example:

“Some school principals said that…”

“Many teachers expressed concern that…”

“There was widespread comment about…”

The basis for these terms is the number of persons in a group or class of persons who were interviewed, as opposed to the total potential number of persons in a category. This is a particularly salient point when not all persons within a category are interviewed. “Many teachers said that…,” represents only those interviewed by the auditors, or who may have responded to a survey, and not “many” of the total group whose views were not sampled, and, therefore, could not be disclosed during an audit.

In general these quantifications may be applied to the principle of full disclosure:

Descriptive Term General Quantification RangeSome…or a few… Less than a majority of the group interviewed and less than 30%Many… Less than a majority, more than 30% of a group or class of people interviewedA majority… More than 50%, less than 75%Most…or widespread 75-89% of a group or class of persons interviewedNearly all… 90-99% of those interviewed in a specific class or group of personsAll or everyone… 100% of all persons interviewed within a similar group, job, or class

It should be noted for purposes of full disclosure that some groups within a school district are almost always interviewed in toto. The reason is that the audit is focused on management and those people who have policy and managerial responsibilities for the overall performance of the system as a system. In all audits an attempt is made to interview every member of the board of education and all top administrative officers, all principals, and the executive board of the teachers’ association or union. While teachers and parents are interviewed, they are considered in a status different from those who have system-wide responsibilities for a district’s operations. Students are rarely interviewed unless the system has made a specific request in this regard.

Interviewed Representatives of the Phoenix Elementary School District #1

All Governing Board Members SuperintendentDistrict Administrators and Staff Campus Principals and Co-administrators Teachers, Counselors, and Instructional Support Staff Classified District EmployeesParents Students

Approximately 117 individuals were interviewed during the site visit phase of the audit. An online survey regarding the district was completed by 181 teachers, 11 school leaders, and 111 parents.


Data Sources of the Curriculum Audit™

A Curriculum Audit™ uses a variety of data sources to determine if each of the three elements of curricular quality control is in place and connected one to the other. The audit process also inquires as to whether pupil learning has improved as the result of effective application of curricular quality control.

The major sources of data for the Phoenix Elementary School District #1 Curriculum Audit™ were:

Documents

Documents included written board policies, administrative regulations, curriculum guides, memoranda, budgets, state reports, accreditation documents, and any other source of information that would reveal elements of the written, taught, and tested curricula and linkages among these elements.

Interviews

Interviews were conducted by auditors to explain contextual variables that were operating in the school system at the time of the audit. Such contextual variables may shed light on the actions of various persons or parties, reveal interrelationships, and explain existing progress, tension, harmony/disharmony within the school system. Quotations cited in the audit from interviews are used as a source of triangulation and not as summative averages or means. Some persons, because of their position, knowledge, or credibility, may be quoted more than once in the audit, but they are not counted more than once because their inclusion is not part of a quantitative/mathematical expression of interview data.

Site Visits

All building sites were toured by the CMSi audit team. Approximately 135 classrooms were observed by audit team members. Site visits reveal the actual context in which curriculum is designed and delivered in a school system. Contextual references are important as they indicate discrepancies in documents or unusual working conditions. Auditors attempted to observe briefly all classrooms, gymnasiums, labs, playgrounds, hallways, restrooms, offices, and maintenance areas to properly grasp accurate perceptions of conditions, activities, safety, instructional practices, and operational contexts.

Online Surveys

Online surveys were administered to stakeholder groups, such as principals, teachers, parents, and sometimes students. The surveys allow stakeholders to provide auditors with valuable feedback regarding strengths and weaknesses in the system.

Standards for the Curriculum Audit™

The CMSi Curriculum Audit™ used five standards against which to compare, verify, and comment upon the Phoenix Elementary School District #1’s existing curricular management practices. These standards have been extrapolated from an extensive review of management principles and practices and have been applied in all previous Curriculum Audits™.

As a result, the standards reflect an ideal management system, but not an unattainable one. They describe working characteristics that any complex work organization should possess in being responsive and responsible to its clients.

A school system that is using its financial and human resources for the greatest benefit of its students is one that is able to establish clear objectives, examine alternatives, select and implement alternatives, measure results as they are applied against established objectives, and adjust its efforts so that it achieves a greater share of the objectives over time.


The five standards employed in the CMSi Curriculum Audit™ in Phoenix Elementary School District #1 were:

1. The school district demonstrates its control of resources, programs, and personnel.

2. The school district has established clear and valid objectives for students.

3. The school district demonstrates internal consistency and rational equity in its program development and implementation.

4. The school district uses the results from district-designed or -adopted assessments to adjust, improve, or terminate ineffective practices or programs.

5. The school district has improved productivity.

A finding within a Curriculum Audit™ is simply a description of the existing state, negative or positive, between an observed and triangulated condition or situation at the time of the CMSi audit and its comparison with one or more of the five audit standards.

Findings in the negative represent discrepancies below the standard. Findings in the positive reflect meeting or exceeding the standard. As such, audit findings are recorded on nominal and ordinal indices and not ratio or interval scales. As a general rule, audits do not issue commendations, because it is expected that a school district should be meeting every standard as a way of normally doing its business. Commendations are not given for good practice. On occasion, exemplary practices may be cited.

Unlike accreditation methodologies, audits do not have to reach a forced, summative judgment regarding the status of a school district or subunit being analyzed. Audits simply report the discrepancies and formulate recommendations to ameliorate them.


III. EXECUTIVE SUMMARYA Curriculum Audit™ is basically an “exception” report. That is, it does not give a summative, overall view of the suitability of a system. Rather, it holds the system up to scrutiny against the predetermined standards of quality, notes relevant findings about the system, and cites discrepancies from audit standards. Recommendations are then provided accordingly to help the district improve its quality in the areas of noted deficiency.

With the direction outlined in the district mission and belief statements, the school district is at a natural point for reviewing, evaluating, and affirming its practices and plans. This juncture is an ideal time to request an external review such as provided through the Curriculum Audit™ and its analysis protocol. The Phoenix Elementary School District #1 Governing Board and Superintendent took a bold step to have a curriculum audit as an objective external assessment of the system’s current status. This action expressed a strong commitment to moving the system forward with a clear vision of excellence.

As stated in the district’s statement of the purpose of the audit, “On July 1, 2016, the new Chief Executive Officer stepped into the district’s top leadership role following the retirement of our former superintendent….Our new CEO maintained continuity of district leadership, but saw the value in reviewing the district’s Strategic Plan as a means to set direction for the short-term and long-term, with a focus on 2025….With the support of the Governing Board, CEO, and administrators, the district decided to engage in a curriculum audit.”

The district cited “…A list of real factors that have significant impact on the work school districts are charged to accomplish for our students. We encourage staff to value every minute of instructional time to engage students in meaningful learning experiences. We expect the curriculum audit to provide the school system with information that will guide us to 2025….We also expect the curriculum audit to provide findings that we can’t anticipate. However, through the lens of external eyes, we believe the final audit report will be a useful guide for stakeholders to identify immediate action steps, as well as identify long-term goals….Our goal will be to blend the results of our work from these three tools [Strategic Plan, Comprehensive Needs Assessment, curriculum audit] to develop one manageable plan to guide teaching for learning in our school system.”

Over the past year, Phoenix Elementary School District #1 has undergone critical decisions regarding its personnel, the expansion of technology, a focus on closing the achievement gap, and is now developing goals that would enhance the status of the district in the Phoenix community.

The intent of this audit report is to identify areas in which the educational program and supporting district operations do not meet audit criteria for effectiveness and to offer reasonable recommendations related to the identified areas of need. The report contains 14 findings for which auditors have provided 12 comprehensive recommendations with action steps to help bring the specified conditions to a status of congruence with audit standards.

When reviewing the current status of the Phoenix Elementary School District #1 in relationship to the principles of Standard One, which addresses system control and oversight, the auditors found that the governing board does not have a comprehensive framework of governing policies that support effective planning and decision making to provide the leadership needed to direct the evolving expectations for student success. Without policies that provide clarity regarding roles of governance and management and the organizational structures needed to focus on the mission of teaching and learning, personnel run the risk of not meeting the needs of students.

Efforts in the district presently focus on identifying the status of student performance. Current board direction shows a preference for providing intervention strategies in an effort to remedy low student performance, rather than first examining the causes of that low performance. The use of data is needed to analyze and identify the causes for particular student performance and to support decisions to be made in the district. If the causes of student performance were identified, the district would be able to direct its improvement efforts to focus on the level of initial instruction rather than rely on interventions to “correct” student performance.

Board policies are inadequate to provide guidance for curriculum management and to establish quality control of the educational program. Board policies lack sufficient clarity and thoroughness of content to provide clear


expectations in such areas of educational program management as the design and delivery of the curriculum, professional development, monitoring of curriculum implementation, student assessment, program evaluation, instructional technology, and other related aspects of policy guidance for systemic direction.

Board policy direction for school improvement planning is very limited. What exists leans primarily on the completion of tasks rather than focusing on student learning results. The district has undertaken initial long-range planning by initiating a process for the development of a new strategic plan. The district’s 2012 Strategic Plan—Our Blueprint for Student Success continues to guide the district while a steering committee embarks on its planning efforts.

Board policies to accomplish present and future strategic goals are vague, do not provide direction on the roles and responsibilities of district personnel, and do not provide direction on improving student achievement for its majority minority and low socioeconomic population. Policy does not provide a philosophical direction for student learning, nor does it require the processes and systems to manage and support student growth. Current processes are focused on evaluating existing performance rather than changing a course for future improvement. Policies do not outline the board’s expectations for decision making, nor do they empower the superintendent to lead the development of processes, procedures, and programs to attain those expectations.

Auditors found that the organizational chart does not meet audit criteria for sound organizational management. Scalar relationships are not accurately represented, with positions at different levels of responsibility and compensation depicted on the same horizontal plane. Incongruities in the chain of command are a result of the discrepancies in the organizational chart. The district lacks a strong, “tightly held” control of its curriculum, professional development activities, and learning expectations.

The job descriptions presented to auditors are generic in the description of the tasks to be accomplished by a particular position and lack clarity and specificity related to the responsibilities and relationships as they pertain to achieving the district’s curricular focus for rigor and excellence. Descriptions lack a clear, causal connection between the efforts of personnel and the academic performance of students; rather, they focus on the completion of specific generic operational functions and tasks.

Under the parameters of Standard Two, the auditors examined the district’s direction for teaching and learning. Specifically, they looked for systematic curriculum management planning, high quality written curriculum guides, and a focus on aligning the written, taught, and tested curriculum. The auditors found that the Phoenix Elementary School District #1 does not have a comprehensive curriculum management plan to provide direction and expected processes for the design and delivery of the curriculum. A single plan that provides a comprehensive and cohesive approach to curriculum management was not provided to the auditors.

A strong curriculum management plan directs the design of comprehensive, district-wide curriculum guides to clearly define learning expectations and establish methods of measuring student achievement and the effectiveness of the written and taught curriculum. Without this clarity and the directive that teachers teach the centrally defined curriculum, student achievement becomes more a factor of their background and economic status than the result of a well-designed and delivered set of learning expectations. While some elements of curriculum design are in place, the scope of the written curriculum does not guide teaching across all content areas, quality is inadequate in most curriculum documents, and the rigor of new state standards and assessments has not been incorporated in the written curriculum.

Alignment of the written, taught, and tested curriculum is crucial to ensure student learning of the essential curriculum aligned to state high-stakes testing. Some general district documents are available to teachers online, but they lack the critical components of a quality written curriculum.

During brief visits to all schools and many classrooms, the auditors observed that the predominant instructional practices in the classroom are inconsistent with district expectations for rigor. The auditors observed that the predominant instructional practice was direct, whole-group instruction. Auditors observed limited examples of: differentiated instruction; a use of best practices and effective instructional strategies; and the effective use of instructional technology. Instruction reflected less demanding types of cognition.


Phoenix #1 has developed a culture that relies on providing “support services” and “interventions.” When students are struggling to learn, additional services are sought rather than performing a detailed analysis of the causes of the inadequate student performance or a reliance on improving initial instruction in the classroom. The district has committed extensive financial and personnel resources to intervention and support efforts that could be capitalized in order to reduce class sizes in all grade levels, from preschool through primary grades. School and district efforts are directed to finding “THE program” that will “correct” student learning, rather than developing a tightly held, district-wide, mandatory curriculum for all subjects and all grades across the district that provides teachers with the support and resources they need to more effectively meet student needs in the classroom.

In Standard Three analysis, the auditors studied equity, curriculum delivery and monitoring, and connectivity within the system. The auditors’ review of comparability of student access to programs and services surfaced several concerns about equity and equal opportunities. Auditors noted equity issues related to student enrollment in special programs, certain schools, and discipline. An identified imbalance was placement/enrollment patterns in special programs. The disproportional distribution of students by race, gender, and SES status is evident in special education and the Accelerated Learning Procedures (ALPS). Commensurate allocation of additional resources to ensure equity for students was not observed.

Auditors noted that enrollment by gender and ethnicity in the ALPS program dramatically differed from proportions of overall enrollment. Male students constituted 57.5% of the ALPS program enrollment, compared to only 42.5% of female students. African American students were proportionately lower enrolled (6% compared to 8.2% of enrollment) than White students. White students were over-enrolled in the ALPS program (8%) compared to their percentage of the district enrollment (5%).

Approximately two thirds of special education enrollment was comprised of male students, and Hispanic students in special education exceeded their district enrollment figure by more than 4%. Likewise, male students represented more than 75% of disciplinary actions, while African American students were suspended almost three times their enrollment percentage.

Connectivity addresses the consistency of curriculum being delivered across the district and within each school. A student’s education should not be dependent upon his/her school or teacher, but upon common district curricula in all subjects taught. The issue of connectivity focuses on professional development to assure the effective delivery of the curriculum. There is no district-level planning or coordinating framework that directs the content and methods for professional development, aligning them with district goals.

Auditors rated the delivery of curriculum mostly at the lowest types of cognition (remembering and understanding); less than 10% of classrooms were observed utilizing more challenging types of cognition (analyzing, evaluating, creating).

The quality of the delivery of the district curriculum should benefit from sustained professional development that equips teachers with the skills necessary to design lessons suited to the diverse needs of the students they teach. When training focuses on effective instructional approaches and a deepening of content knowledge, teachers are better prepared to move their students to deeper understanding of content. The design of professional development in Phoenix Elementary School District #1 was fragmented across the district and lacked connectivity to the realities of student achievement, characteristics of the district’s students (poverty, race), and the experience level of teachers. Professional development is not coordinated at the district level but is largely left to school sites to deliver. Needs assessments remain at a superficial level, and results were not aligned to projected professional development agendas. There was no evidence that existing practice focuses professional development on a district-wide set of priorities derived from student learning results that are sustained over time with coaching and a deepening of teachers’ understanding.

In Standard Four, the focus is on feedback of various types and how the system used those data. Typically, the information might come from student assessments and program evaluations. Auditors found that Phoenix #1 lacked a comprehensive student assessment and program evaluation system to guide decision making in the district.


A formal assessment system aligned to state standards and high-stake testing is essential in designing appropriate curriculum guides. Formal assessments constitute those assessments collected centrally by the district. However, the auditors did not find standard, locally-developed assessments for all core subjects. Such assessments provide district and campus administrators, as well as teachers, with immediate data needed to make critical instructional decisions. As a result, the district does not generate the information needed to adequately plan for the design and delivery of the curriculum to improve student learning and achieve the district’s goals. In Phoenix #1, the assessment program is limited to specific core courses. Board policies do not define an expectation for assessment to provide ongoing diagnostic data to be used in evaluating student learning, the effectiveness of the curriculum in use, instructional delivery mechanisms, interventions, the integration of technology, and the allocation of resources.

Many of the district’s assessment efforts are driven by state requirements. Formative assessment efforts occur randomly in the district but are inconsistently utilized across various schools; they do not reflect a district-wide requirement or effort. Course offerings in non-core subject areas do not include aligned assessments. The district collected assessment data as a status report but did not use data to affect change in instructional practices that were not effective. Without changes in instructional practice, the district cannot expect improvements in student performance.

By analyzing the implications of data trends reviewed, the auditors determined that educational changes are needed in order for the Phoenix Elementary School District #1 to reach desired levels of excellence. How a school district uses the data in making important decisions can significantly affect the quality and outcomes of those decisions. The district must use data in establishing goals and have targeted assessment information to help improve instruction.

In analyzing Standard Five, the auditors found that Phoenix Elementary School District #1 has not characteristically employed processes for budget development that meet audit criteria for a curriculum-driven, program-focused budget. Evidence did not indicate that student assessment results had influenced allocation decisions, and no systematic process was in place for the evaluation of discrete programs and the use of those results in budget planning. No cost-benefit analysis was systematically incorporated into the budget planning process to force consideration of various levels of funding for existing programs and services in the budget year being planned. The absence of these systematic steps may impede results over time by not maximizing use of data for results-based decisions.

Auditors found that Phoenix Elementary School District #1 has not employed an established process for program interventions. Auditors studied the district’s Imagine Learning program and it revealed the lack of a formal policy-described process for the identification, selection, implementation, and evaluation of intervention programs. A plethora of programs and interventions exist across various campuses; thus can impede the alignment of instruction with district goals, expectations, and priorities. The lack of an evaluation process for intervention programs leaves a void for campuses and the district as to whether or not those programs are effective in increasing student achievement. Auditors found that the selection and implementation of the Imagine Learning intervention was inadequate. Auditors concluded that the district should focus on the quality of district-wide instructional practices and effective differentiated Tier I classroom instruction. Such a focus would preclude the current perceived need for extensive interventions.

The state of facilities in the district received careful review by auditors. The district received a large influx of capital funds in 2015, and the overall condition of facilities in Phoenix #1 support educational programs. Auditors received some individual comments regarding potential overcrowding in some classrooms at particular schools. Overall, facilities in Phoenix #1 were functional and up-to-date. Due to declining enrollment numbers, the district may consider consolidating neighborhood schools in the future.

The Governing Board of Phoenix Elementary School District #1 had undertaken a considerable infusion of instructional technology within the district. District leadership aspires to integrate technology into all aspects of classroom instruction—teachers using instructional technology to enhance and improve student learning and students using technology for higher cognitive activities and rigor. Many aspects of district operations have identified professional development needs to promote and support the use of technology in the classroom.


The district has begun to utilize the SAMR designations of instructional technology usage and created a new Director of Educational Technology position to lead the district’s efforts.

Past practice clearly shows that these circumstances are simply barriers to be overcome, and that they can be overcome with a high quality, aligned curriculum; effective teaching practices; and system- and community-wide support of improved student achievement. With the professional backgrounds, experience, and commitment of so many district and campus leaders, staff members, and the community, the auditors firmly believe that the Phoenix Elementary School District #1 will move to higher levels of excellence.

The CMSi audit team has suggested numerous steps for improving all areas in which the current status precluded meeting audit criteria. While additional actions might be developed by the district administration and staff to implement these recommended changes, most of the recommendations that have been offered have a history of success in similar school systems. The first step is for the superintendent to develop a work plan responding to the findings and recommendations in concert with the efforts currently underway with the district’s strategic plan. With eventual approval by the governing board and active implementation by the administration over the next three to five years, this blueprint can bring organizational effectiveness and student achievement to new heights.


IV. FINDINGS

STANDARD 1: The School District Demonstrates Its Control of Resources, Programs, and Personnel.Quality control is the fundamental element of a well-managed educational program. It is one of the major premises of local educational control within any state’s educational system.

The critical premise involved is that, via the will of the electorate, a local board of education establishes local priorities within state laws and regulations. A school district’s accountability rests with the school board and the public.

Through the development of an effective policy framework, a local school board provides the focus for management and accountability to be established for administrative and instructional staffs, as well as for its own responsibility. It also enables the district to make meaningful assessments and use student learning data as a critical factor in determining its success.

Although educational program control and accountability are often shared among different components of a school district, ultimately fundamental control of and responsibility for a district and its operations rests with the school board and top-level administrative staff.

What the Auditors Expected to Find in the Phoenix Elementary School District #1:

A school system meeting CMSi Curriculum Audit™ Standard One is able to demonstrate its control of resources, programs, and personnel. Common indicators are:

• A curriculum that is centrally defined and adopted by the board of education;

• A clear set of policies that establish an operational framework for management that permits accountability;

• A clear set of policies that reflect state requirements and local program goals and the necessity to use achievement data to improve school system operations;

• A functional administrative structure that facilitates the design and delivery of the district’s curriculum;

• A direct, uninterrupted line of authority from school board/superintendent and other central office officials to principals and classroom teachers;

• Organizational development efforts that are focused to improve system effectiveness;

• Documentation of school board and central office planning for the attainment of goals, objectives, and mission over time; and

• A clear mechanism to define and direct change and innovation within the school system to permit maximization of its resources on priority goals, objectives, and mission.

Overview of What the Auditors Found in the Phoenix Elementary School District #1:

This section is an overview of the findings that follow in the area of Standard One. Details follow within separate findings.

When reviewing the current status of the Phoenix Elementary School District #1 in relationship to the principles of Standard One, system control and oversight, the auditors found the current organizational chart does not meet all audit criteria for sound organizational management. Functions are generally grouped logically, though problems were identified in the logical clustering of functions. Scalar relationships are not accurately represented on the organizational chart, with positions at different levels of responsibility and compensation depicted on the same horizontal plane. Some positions (teachers, some central office administrators, and non-professional staff) are not depicted on the organizational chart. The organizational chart does not clearly delineate the chain of command, especially with regard to school principals.


The district’s job descriptions are board adopted. Curriculum Audit™ criteria expect that the duties and responsibilities of the superintendent and most other administrative positions have current job descriptions outlining required qualifications, responsibilities, and linkage to the district’s curriculum. Most of the district’s job descriptions are more than 10 years old.

Auditors found that board policies are inadequate to provide overall control and guidance for curriculum management and to establish quality control. Auditors reviewed existing board policies and found that current policies, including policies reviewed and updated recently, were not sufficient to meet audit criteria. District policies fell short of audit expectations in all standard areas.

Policies lack sufficient clarity and content to adequately address expectations in such areas of educational program management as design and delivery of the curriculum, assessment, professional development, program evaluation, equitable student access to offerings, technology, program-focused budgeting, and other related aspects of policy guidance for systemic direction. The governing board has established the district’s mission and belief statements but its policies lack the clarity to direct written curriculum that is more rigorous than state and national standards. Such policies direct the decision-making responsibilities to ensure consistency, non-duplication of tasks, and product requirements.

The district planning process does not meet the audit characteristics for design, deployment, and delivery. There is no district plan for curriculum management, professional development, assessment, budget, or technology. The district strategic plan, Our Blueprint for Student Success (2012), identifies five major goals: Student Academic Success, Stakeholder Satisfaction, Enrollment, Partnerships, and Culture. However, beyond the strategic plan, the policies do not align with the characteristics of district-wide plan quality and auditors rated the policies as inadequate to direct planning efforts.

Auditors found little direction for planning in the areas of curriculum management, professional development, student assessment, program evaluation, instructional technology, and budget development. Current plans and related planning documents do not meet audit criteria to provide sufficient direction for ongoing improvement efforts and to cohesively link those efforts across units and functions within the school district.

Finding 1.1: The organizational chart does not meet audit criteria for sound organizational design. Most job descriptions do not meet audit criteria for current, accurate, and clear specifications of authority, responsibilities, curriculum linkage, and role relationships in the district.

Clarity of administrative role relationships is important to an organization in the productive grouping and management of its tasks and functions. A functional and accurate delineation of administrative relationships is generally depicted in graphic form and is called an “Organizational Chart” or “Table of Organization.” An organizational chart graphically depicts the line of authority and responsibilities from the board of education and superintendent to campus principals and classroom teachers for producing student learning.

In order to analyze the adequacy of the Phoenix Elementary School District #1 organizational chart, auditors reviewed appropriate board policies, the Phoenix Elementary School District #1 Organizational Chart, district-provided job descriptions, and other documents communicating information about roles and areas of responsibility. Auditors interviewed board members, district and campus administrators, and other staff regarding the organizational chart and job descriptions.

Clear delineation of lines of responsibility and authority is critical in guiding the design and delivery of a standard curriculum in the district. To serve as an effective guide in curriculum development, a school district’s policy framework must be specific so decisions can be made by referencing specific relevant policies. Auditors reviewed board policies for clear direction in this area but found no board policy that specifically requires an organizational chart. Board Policy CCB: Line and Staff Relations directs that “Lines of authority shall be clearly outlined by the Superintendent. This may be accomplished through the use of organization charts, job descriptions, and administrative regulations and directives.”

No board policies specifically address the design and implementation of an organizational chart. Additionally, the superintendent job description omits any reference to a responsibility to create and maintain an organizational chart.


Auditors were presented with a copy of the 2014-2015 district Organizational Chart. The organizational chart does not provide singular clarity in the design and delivery of the district’s curriculum. As a result, departments and individuals in the system operate in isolation from others resulting in inconsistent and disparate implementation of instruction and learning in the district.

The auditors found that the organizational chart does not meet the audit criteria for sound organizational management. A number of positions listed on the organization chart lacked job descriptions. Auditors identified a lack of clarity on the chart for key administrative areas of leadership such as curriculum, instruction, and assessment. There are a number of positions listed on the chart for which job titles are inconsistent with those listed in job descriptions presented to auditors.

Auditors found inconsistency in the format of job descriptions, and they were missing critical elements to ensure a standard format. Terminology for positions that may have supervisory responsibilities was unclear. Auditors were provided with job descriptions that utilized such titles as: director, coordinator, supervisor, administrator, specialist, interventionist, and coach without corresponding clarity of levels in a chain of command.

The district utilized the term “supervision” to indicate evaluative responsibilities as well as monitoring employee work performance. Similarly, the district’s usage of the term “support staff” caused confusion between professional instructional services provided by certificated personnel and classified, non-professional staff responsibilities for auxiliary services (e.g., clerical, food service, technical support, etc.). A position’s title does not necessarily indicate the position’s level of authority. Job descriptions most notably do not include clear linkage between job responsibilities and the district’s curriculum.

Organizational Chart

Auditors were presented one main organizational chart for Phoenix Elementary School District #1, plus an additional 11 charts for various divisions and departments. The overall district organizational chart was dated 2014-15. Only the organizational chart for Management Information Systems (MIS) was dated for the 2016-17 school year. Seven of the remaining division/department organizational charts were from 2013-14 and three from 2015-16. The auditors reviewed the district’s organization charts and other documents and used the Curriculum Audit™ design principles to examine the organizational structure. These Principles of Sound Organizational Management are presented in Exhibit 1.1.1. The audit expectation is that all design principles listed will be met.

Exhibit 1.1.1

Curriculum Audit™ Principles of Sound Organizational Management

1. Span of Control The span of control for effective day-to-day supervision requires direct responsibility for no more than 12 employees.

2. Chain of Command No employee should have more than one supervisor to avoid being placed in a compromised decision-making situation.

3. Logical Grouping of Functions

Tasks of similar nature should be grouped together. This keeps supervisory needs to a minimum (ensuring economy of scale).

4. Separation of Line and Staff

Line (principals and teachers) and staff positions (central office administrators, support staff, etc.) should be separate from curriculum design and program assessment functions. The administrators carrying out the primary mission of the district are not confused with those supporting it. Line administrators only report to line administrators.

5. Scalar Relationships All positions shown at the same level should have similar responsibilities, authority, and compensation.

6. Full InclusionAll central functions that facilitate quality control are included in the organizational structure. All persons working within the district carrying out its essential line and staff functions should be depicted on the organizational chart.


The district’s graphical depiction of its organizational structure, as presented to auditors, is shown in Exhibit 1.1.2.

Exhibit 1.1.2

2016 Phoenix Elementary School District #1 Organizational ChartMay 2017

Community

Governing Board

Superintendent

Assistant SuperintendentBusiness Services

Principals

Assistant SuperintendentCurriculum & Instruction

Director – Budget & Finance

PurchasingAccounts Payable

Payroll

Director – Child Nutrition

Director – Management Information Systems

DirectorHuman Resources

Director – Plant Services Transportation / Warehouse

Director of Assessment and Accountability

Director of Federal Programs

Director of Curriculum

Director of Principal Effectiveness

Director of Teacher Effectiveness

Director of Coaching Effectiveness

Director of Student Services

Instructional Enrichment Intervention Specialists

PHOENIX ELEMENTARY SCHOOL DISTRICT NO. 1ORGANIZATIONAL CHART

2014 - 2015

The following is the auditors’ assessment of the district’s organizational charts presented by district personnel based on the criteria presented in Exhibit 1.1.1.

1. Span of Control

Except for the Superintendent, district office positions on the chart appear to meet this criterion with no other position supervising more than 12 employees. On the district organizational chart, the Superintendent supervises 16 administrators and coordinates with five governing board members, making the position responsible for mission direction with a group of 21 individuals. The Assistant Superintendents of Curriculum and Instruction and Business Services supervises only seven and six, respectively. However, in the outdated Superintendent’s job description, only five supervision assignments are cited. Principals are not referenced although their job description does indicate that in their “supervision received” they are directly responsible to the Superintendent.

Therefore, the organizational chart and its corresponding job descriptions do not clearly delineate the supervisory responsibilities of district administrators.


2. Chain of Command

This principle is only partially applied in the current district-wide organizational chart. The chart extends from the Superintendent, Assistant Superintendents, and Principals to Directors, but not further. The chain of command does not extend beyond Directors and school Principals. A number of positions for which auditors received job descriptions do not appear on the organizational chart (K-8 Assistant Principal, Directors of Educational Technology, General Studies, Special Education, and Maintenance & Operations). The current district organizational chart omits Co-administrators, Classroom Teachers, Mentors, Coaches, and most Instructional Specialists. Organizational charts for specific divisions and departments are similarly incomplete.

While the district organizational chart shows that the Superintendent supervises Principals, such supervisory responsibilities are not described in the Superintendent’s job description. Rather, the job description cites supervision for only the Assistant Superintendents, Director of Human Resources, Administrator for Special Projects, and Public Relations Coordinator along with Classified Employees. The job description for the Director of Management Information Systems indicates that the position reports directly to the Superintendent but, on the organizational chart is represented to report to both the Assistant Superintendent for Curriculum and the Assistant Superintendent for Business Services.

On the district organizational chart, the Instructional Enrichment Intervention Specialist position (non-administrative) reports to the site Principal. However, the position’s job description also states that the position “reports to the Principal, with additional support provided from Central Office Administrators.” On the Student Services Organizational Chart, the Coordinator of School Social Work and Student Support Systems reports to the Director of Student Services. The job description states that the position will “supervise and evaluate Social Workers and Behavior Intervention Specialists in collaboration with building Principals and Director of Student Services.” The Curriculum Audit™ Principles of Sound Organizational Management expect all employees to report to no more than one supervisor, to avoid compromising or conflicting decision-making situations.

3. Logical Grouping of Function

Functions are basically grouped together logically on the organizational chart. The campus and curricular leadership and business functions are clustered together consistent with Curriculum Audit™ expectations, although campus principals should be centered on the organizational chart to easily depict the direct line of authority from the board to the superintendent to principals to teachers.

4. Separation of Line and Staff

On the district organizational chart, clear separation of line and staff is clearly depicted; therefore, the primary mission of the district is separate from those carrying out the mission to those supporting it.

5. Scalar Relationships

This principle is partially applied in the current district organizational chart but lacking on division/department charts. The district chart contains only positions of Assistant Superintendents, Principals, and Directors but does not include Coordinator positions. The chart lists “Instructional Enrichment Intervention Specialists” on the same parallel plane as Directors although the positions have varied degrees of responsibility, compensation, and authority.

On division/department organizational charts, supervisory positions (Directors and Coordinators) are located together on the same parallel plane occasionally including Supervisors, Specialists, and Classified Employee positions at different levels of responsibility, authority, and compensation.


6. Full Inclusion

A number of positions included on the organizational charts do not have corresponding job descriptions. Auditors were presented with administrative job descriptions for positions that do not appear on the organizational charts. A number of positions on the organization charts appear with different titles than those found on parallel job descriptions.

The district organizational chart is incomplete as it does not depict full inclusion of all positions responsible for the implementation and delivery of the curriculum to students, as noted under the Chain of Command.

Auditors interviewed board members, central office administrators, campus principals, support staff, and teachers concerning the delineation of responsibilities within the district. Comments about the lack of system-wide coordination shared with auditors included:

• “There is a disconnect between schools and the district.” (Central Office Administrator)

• “Everyone is doing their own thing (site-based). It’s hard to support for that reason.” (Support Staff)

• “Collaborative leadership team extends down to principals and department chairs.” (Central Office Administrator)

• “There doesn’t seem to be anybody in charge in Phoenix Elementary holding anyone accountable.” (Central Office Administrator)

• “No one is holding people accountable for doing the same thing.” (Board Member)

• “There’s not much conversation about district expectations. It’s done at the schools.” (Central Office Administrator)

• “These are smart people, but until recently they were all siloed. Principals went to whoever was most comfortable.” (Central Office Administrator)

• “It’s more go off and do my own thing.” (Teacher)

• “We need different leadership and culture to make it more about teaching and learning.” (Board Member)

Comments from stakeholders at all levels of the system indicated the fragmentation of decision making.

Job Descriptions

Job descriptions are clearly written summaries of duties and qualifications of persons employed by the school district. They provide information regarding the necessary background to qualify for specific jobs and how those positions function within the organization. The descriptions should include assignment of supervisory relationships and the critical components of the job duties. A clear set of job descriptions supports the district’s internal and external communication by explaining who performs what duties within the organization. Adequately designed job descriptions also support the implementation of the administrative relationships graphically depicted on the organizational chart.

To analyze job descriptions, auditors reviewed policies directing their role, existing job descriptions, and other related documents. Auditors also interviewed several staff regarding job descriptions, their respective job responsibilities, and the supervision/reporting structure. The following board policies/regulations provide general direction regarding the development, use, and approval of job descriptions.

• Board Policy CBA: Qualifications and Duties of the Superintendent states that the superintendent “Determines assignments, defines the duties, and coordinates and directs the work of all employees of the District.”

• Board Policy CCB: Line and Staff Relations indicates that “Lines of authority shall be clearly outlined by the Superintendent. This may be accomplished through the use of organization charts, job descriptions, and administrative regulations and directives.”


• Board Regulation GBA-RB: Equal Employment Opportunity provides administrative guidelines in ensuring equal employment opportunity including job analysis. “A job analysis of all employment positions will be completed and updated as necessary. Each position will be reviewed in terms of job descriptions, job structure, job requirements, job functions, selection procedures, and salary ranges.” The regulation implies that job descriptions will be created, maintained, reviewed, and updated.

• Board Policy GCA: Professional Staff Positions directs that “Professional staff positions are created only with the approval of the Board. Before recommending the establishment of any new position, the Superintendent will present a job description for the position that specifies the qualifications, the performance responsibilities, and the method by which the performance of such responsibilities will be evaluated.” The policy further states that “The establishment of any new position will require Governing Board approval. The Superintendent will maintain a comprehensive and up-to-date set of job descriptions of all positions in the school system.”

• Board Policy GDA: Support Staff Positions similarly mirrors Board Policy GCA for support staff positions.

Auditors were provided 36 job descriptions to analyze. Each has a connection to the design and delivery of the curriculum or is prominent on the organizational chart of the district. The job descriptions are dated with the year ranging from 1996 to 2016. Thirteen of the job descriptions were approved since 2012. The remaining 23 (64%) ranged from six to twenty years old and had not been updated. All of the job descriptions indicated the date they were approved by the governing board.

District personnel provided auditors with a roster of district level personnel separated by departments. Auditors found inconsistencies between the personnel roster, job descriptions, and the organizational charts provided.

When the selected job descriptions were compared to the district’s organizational chart (see Exhibit 1.1.2), no job descriptions were found for 11 (23%) of the positions analyzed from the organizational charts (see Exhibit 1.1.3). Auditors found that the titles of several administrative positions varied when comparing the job descriptions and the organizational chart. The use of different titles can result in confusion for employees as to the supervisory positions to whom they may report and by whom they are to be evaluated.

The job descriptions reviewed by the audit team represented 77% of the positions listed on the Phoenix Elementary School District #1 Organizational Chart. The titles of several positions on the organizational chart did not match the titles found on corresponding job descriptions. Auditors reviewed existing administrative positions to determine if there were written job descriptions, and looked to see if the positions appeared on an organizational chart. Exhibit 1.1.3 lists positions that were located on the district or division organizational charts.

Auditors did not receive job descriptions for a considerable number of positions listed on the district and/or division and department charts. Exhibit 1.1.3 lists position titles for which auditors did not receive a corresponding job descriptions and on which organizational chart(s) the positions were listed. Without job descriptions, auditors were unable to verify the accuracy of the chain of command for the organizational chart(s).


Exhibit 1.1.3

Comparison of Administrative Positions on District Organizational Charts And Personnel Roster Without Job Descriptions

Phoenix Elementary School District #1May 2017

Position Title District Org. Chart

Division/Department Org. Charts

Personnel Roster

Director of Curriculum X XDirector of Human Resources X XDirector of Student Services X XAdministrator Preschool & Autism X XBefore and After School Program Coordinator XEnergy Education Coordinator X XCoordinator IEP Compliance X XCoordinator of Psychological Services X XCoordinator of Gifted Programs X XMIS Manager X X XTechnology Coordinator X XSoftware Manager X X

The auditors rated each of the 36 job descriptions on four critical elements listed below:

• Qualifications;

• Links to Chain of Command;

• Functions, Duties, and Responsibilities; and

• Relationship to the Curriculum (where relevant).

There are five possible ratings on the four critical elements. The possible ratings are shown in Exhibit 1.1.4. For a job description to be considered strong, it must be rated as adequate or higher on all four of the critical elements.

Exhibit 1.1.4

Curriculum Audit™ Rating Indicators for Job Descriptions

Rating ExplanationMissing (M) No statement made.

Inadequate (I) A statement is made but is incomplete and missing sufficient details.

Adequate (A) The statement is more or less complete, usually missing curricular linkages or sufficient detail regarding curricular linkage/alignment.

Strong (S) A clear and complete statement is present, including linkage to curriculum where appropriate or, if not appropriate, otherwise quite complete.

Exemplary (E) A clear, complete statement is present with inclusive linkages to curriculum indicated in excellent scope and depth.


Exhibit 1.1.5 presents the job descriptions reviewed and the auditors’ assessment of their adequacy.

Exhibit 1.1.5

Rating of the Quality of Job DescriptionsPhoenix Elementary School District #1

May 2017

Job Description Date Qual. Chain of Command

Duties & Resp.

Curr. Linkage

Superintendent 1/96 A A A IK-8 Assistant Principal 4/06 A A A IAssistant Superintendent of Business Services 4/16 A A A N/AAssistant Superintendent of Curriculum & Instruction 3/11/03 I A A ABehavior Interventionist 5/26/11 I A A N/AClassroom Teacher 10/25/01 A A A ICoordinator of School Social Work & Student Support Services 4/25/13 I I A I

Director of Assessment and Accountability 6/11/09 I A A IDirector of Budget and Finance 4/21/05 I A A N/ADirector of Child Nutrition Services 9/03 A A A N/ADirector of Coaching Effectiveness 4/24/14 A A A IDirector of Educational Technology 8/11/16 A A A IDirector of Educator Effectiveness 4/12/12 I A A IDirector of Federal Programs 4/14/16 I A I IDirector of General Studies 5/02/03 I A I IDirector of Maintenance & Operations 4/04 A A A N/ADirector of Management Information Systems 1/96 A A I N/ADirector of Plant Services & Transportation 8/12 I A I N/ADirector of Principal Effectiveness 4/25/13 I A A IDirector of Special Education 12/10/09 I A A AElementary School Principal 11/03 I A I IEnglish Language Arts Implementation Specialist 4/15/13 I A A IEnglish Language Development (ELD) Interventionist 5/28/09 I A I IGrants Management Budget Specialist 4/25/13 A A A N/AHealth and Wellness Coordinator 4/12/12 A A A N/AHigh School Implementation Specialist 5/26/16 I A A IHuman Resources Specialist 3/12 A A A N/AInstructional Coach 5/10/12 I A A IInstructional Enrichment Intervention Specialist 4/25/13 A I A ILiteracy Coach 6-07-04 A A A IMaintenance Supervisor 10/25/12 I A A N/AMathematics Intervention Specialist 7/01/09 I A I INCLB Data Trainer/Auditor 5/10/12 A A A N/ANew Teacher Mentor 6/11/09 I A A AParent Engagement & Education Coordinator 7/16/14 I A A N/ASpecial Populations Coordinator 4/12/12 I A A IN/A = Not applicable


Auditors rated 62% of the critical elements as adequate. Of the 36 job descriptions, none were rated strong. Seven non-academic positions did received three “adequate” category ratings but a “N/A” for curriculum linkage. All other job descriptions received a rating of inadequate on at least one element. No job descriptions were provided for critical positions including the Director of Curriculum, Director of Student Services, and the Director of Human Resources. As noted in Exhibit 1.1.2, numerous other positions did not have written job descriptions. Further comments regarding the ratings follow:

1. Qualifications

Auditors found 21 (58%) of the descriptions of qualifications to be inadequate. Most of the job descriptions contain generic qualifications that do not address the specific expertise required for the position. Qualifications are generally limited to certification requirements. Few job descriptions required knowledge or experience related to the design and delivery of a district’s curriculum, an understanding of the role of assessment to measure student learning, or responsibility for professional development.

2. Chain of Command

Two job descriptions were rated as inadequate regarding the chain of command. These two have employees reporting to more than one supervisor, which may lead to uncertainty in decision-making situations. The Directors of Coaching Effectiveness, Educator Effectiveness, and General Studies are assigned supervisory responsibilities for instructional support staff (e.g., mentor teachers, resource teachers, implementation specialist, etc.) that are also shared by site principals.

Two job descriptions cited that the position reported to a supervisory position that was not on the district organizational chart and one job description (the Director of Management Information Systems) indicated the position reported to the Superintendent, which is different than reflected on the organizational chart.

Thirteen of the 36 job descriptions provided to auditors reflect no supervisory responsibilities including two director and one coordinator positions. The remaining 10 positions are titled such non-supervisory positions as specialist, coach, interventionist, etc.

3. Functions, Duties, and Responsibilities

This critical element had seven (19%) of the positions rated as inadequate. These descriptions of responsibilities were found to be missing sufficient details related to the design and delivery of the district’s educational program. Many job descriptions repeated a generic list of duties and responsibilities, thereby, restricting a clear understanding of the district’s expectations for performance.

Duties of personnel directly responsible for the delivery of the district’s curriculum do not include responsibilities for the design, organization, or delivery of aligned professional development or assessment. Such responsibilities are critical to coordinate the delivery of curriculum across the district. No position is designated fully responsible for professional development.

4. Curricular Linkage

Thirteen of the job descriptions reviewed were marked as N/A (not applicable), with no direct relationship to curricular matters. Of the remaining applicable job descriptions related to the district’s curriculum, only three were judged to be adequate. Other job descriptions contain generic references to administrative responsibilities with no linkage to curriculum and are absent a specific assignment for the design and/or delivery of the district’s curriculum. Auditors received a comment from a central office administrator regarding district job descriptions that shared, “We have job description but they’re more about other duties as assigned.”


Summary

Board policy does not include any requirement that the district produce an organizational chart. The organizational charts presented to auditors do not meet audit criteria for sound organizational management. Several positions have a vague designation of their place on the organizational chart. Some positions with job descriptions do not appear on the organizational chart. Scalar relationships are not accurately represented on the organizational charts with positions at different levels of responsibility and compensation depicted on the same plane on the chart.

The ambiguities of the organizational charts result in uncertainty and confusion regarding the authority of central office positions related to campus administrators. The scope of responsibilities and supervision depicts the superintendent’s span of control exceeding the audit criteria for sound organizational management. The organizational chart does not provide the clarity needed to not only develop a cohesive district curriculum, but also the clear authority of the central office to direct the implementation of a board adopted curriculum. The major factor impacting the effectiveness of the organizational charts results from all charts, except for the MIS department, being out of date and not current or consistent with the job descriptions or the administrative personnel roster provided.

With most of the job descriptions rated inadequate for curricular linkage and almost 20% rated inadequate for function, duties, and responsibilities, district job descriptions are insufficient to direct the design and delivery of the district’s curriculum. The job descriptions lack clear direction and do not outline responsibilities for the district’s professional development, assessment, instructional strategies, and instructional technology.

The district’s job descriptions do not meet audit criteria for accurate and clear specifications of responsibilities and relationships in the district. Not all positions on the organizational chart had job descriptions. Not all job descriptions presented to auditors were included on the organizational chart as would be expected.

Although job descriptions reflect the date of adoption, 64% have not been reviewed or revised in more than five years resulting in the district’s inability to maintain accurate and current expectations for its professional staff. Without current job descriptions, the district cannot ensure that the staff is cognizant of its professional responsibilities related to student learning, new initiatives, and state and local mandates. Auditors judged all job descriptions inadequate in at least one of the audit elements.

The organizational structure of the district is inconsistent among the documents reviewed by auditors. Job titles have been altered on some documents but are not consistent. New positions have been added to the district structure but are not necessarily reflected in job descriptions and organizational charts. Positions that appear on previous organizational charts have been eliminated although job descriptions remain on file in the district. Such a lack of clarity infuses confusion regarding the positions responsible for designated tasks.

Finding 1.2: Board policies are inadequate to provide for a sound, effective system of curriculum management and quality control to attain the mission and goals established for student achievement in terms of curriculum design and delivery, consistency in organizational operations, program structures, and accountability for roles and responsibilities.

Educational policy development is the essential responsibility of the governing board. Through its policies, the board establishes its responsibility for system control, direction, and ensures efficacy in determining curricular matters. Well written policies regarding curriculum management and direction for a school system are critical in providing control over a system’s functions and for communicating expectations to the administration and instructional staff. Such policies establish an operational framework for management of the curriculum and related functions by providing a structure for the development, delivery, and assessment of the district curriculum, ensuring the consistent connectivity of the curriculum across the district, and a means for managing innovation and change.

Comprehensive board policies provide foundational direction for the day-to-day operations of the school district. They must be specific, easily referenced for ongoing use by the system, and serve as first source documents to provide individual and system guidance. It is through policies that commonly understood standards are established and maintained over time for the design and delivery of all written, taught, and tested curriculum.


For policies to be effective decision-making guides at all levels of the organization, they must be broad enough to avoid micromanagement, but specific enough that managers can refer to them for direction in making decisions about teaching, learning, and related district operations. If policy direction is absent, outdated or unclear, or if it is not specific enough, educational decisions are left to the discretion of individuals. Results of these decisions may or may not be congruent with the intent of the district’s board.

Effective policies establish parameters for a system of curriculum management that provides clarity of learning objectives to be mastered, valid and reliable measures of monitoring student attainment of these objectives, program structures that allow equal access to learning for all types of learners, and the hiring and cultivation of high quality staff who are provided adequate flexibility to differentiate instruction to meet individual student needs to achieve established learning objectives. Responsibility for student results includes providing guidance within the written curriculum to support mastery by various types of learners, matching program design with student need (e.g., English language learners), and allowing for wide instructional differentiation that promotes teacher creativity within a delivery system that keeps the focus on student attainment of targeted district outcomes.

Board policies must establish clear expectations for each of the following:

• A centrally defined curriculum aligned to state standards and assessments, structured with clearly defined learner outcomes that do not permit exceptions that may jeopardize student achievement. Such a curriculum provides teacher guidance regarding differentiating the design of lessons to address varying needs of learners (second language learners, special education students, students of poverty).

• A wide umbrella of instructional differentiation that encourages creative teacher response to student learning needs—the “how” of teaching. Board policy needs to specify that textbooks and worksheets do not dominate instruction; teachers are expected to show great variation in the delivery of the adopted curriculum in the interest of student learning.

• The delineation that adoption of a program, such as a reading series, is not the adoption of a curriculum. Policy directs that the development of curriculum occur first and that the adoption of textbooks and other resources are to support delivery of the established district curriculum.

• The expectation that curriculum spirals from grade to grade and course to course to provide vertical articulation across grade levels and horizontal coordination among schools.

• An expectation that student performance results will be used to address student success and refinement of curriculum design and delivery. Achievement data form a feedback loop used to examine the appropriateness of curriculum design for all populations and to promote improvement delivery for all students.

• The expectation that professional development be connected to the effective delivery of the adopted curriculum for the district’s population. The “what” of professional development is responsive to district curriculum deployment, student performance results, and teacher needs.

• The expectation that programs adopted to address specific needs (e.g., reading deficits, English language development) will be evaluated in accordance with expectations of a board-adopted curriculum.

• The expectation that all of the above occur in the context of a planned approach to curriculum management that is anchored by system-wide priorities that are continuously evaluated and results used to target ongoing improvements.

The auditors examined Phoenix Elementary School District #1 policies and administrative regulations provided by school district personnel. They selected those policies most directly related to curriculum management and organizational support for analysis and assessed them by comparing their content to 26 policy criteria used in the Curriculum Audit™. Interviews were conducted with board members, administrators, and staff to identify the extent to which board policies are used in the district to guide decisions about educational programs and the curriculum.


The auditors found that current policies are inadequate to direct a sound curriculum management system and to establish quality control of the educational program and organizational operations. Existing board policies met legal requirements but lacked key elements necessary for sound management of the curriculum. This management of curriculum design and delivery must be a key focus to meet the needs of a student population characterized by increasing levels of poverty, academic challenges, and students with special needs. Additionally, auditors found that the use of policies to guide decision-making has been minimal, partly due to a lack of specificity regarding the lines of authority and clarity to ensure consistent implementation of the curriculum across the district.

Most of the Phoenix Elementary School District #1 policies are based on similar guidelines from Arizona Revised Statutes. However, the district has not issued its own policies to include and/or expand on its local priorities and goals. Access to state statutes is not convenient for local administrators to utilize on a day-to-day basis.

Auditors were presented with a number of administrative regulations, which are the directives approved by the superintendent to carry out the board policies. Regulations provide the actions, procedures, and processes in a precise manner to bring the policy to attainment and create an educational environment that is focused on student achievement.

Auditors found direction regarding the formulation, communication, and implementation of board policies and administrative regulations in the following current policies. Exhibit 1.2.1 lists 100 policies, 26 regulations, and five exhibits selected by the auditors for analysis because they are considered the ones most related to a curriculum management system and a fundamental support framework for educational program delivery.

Exhibit 1.2.1

Board Policies Reviewed by AuditorsPhoenix Elementary School District #1

May 2017

Policy/Adm Reg Number Policy Name Policy

Adopted AA School District Legal Status 2/24/2011

ABA Community Involvement in Education 2/24/2011ABAA Parent Involvement 2/24/2011

AC Nondiscrimination/Equal Opportunity 2/24/2011ADA School District Goals and Objectives 3/3/2011BA School Board Operational Goals 2/24/2011

BBAA Board Member Authority and Responsibilities 2/24/2011BDD Board - Superintendent Relationship 2/24/2011BDE Board Committees 2/24/2011BDF Advisory Committees 2/24/2011BE School Board Meetings 9/8/2011

BEDC Quorum 2/24/2011BEDB Agenda 2/24/2011

BG School Board Policy Process 2/24/2011BGB Policy Adoption 2/24/2011BGC Policy Revision and Review 2/24/2011BGD Board Review of Regulations 2/24/2011BGE Policy Communication/Feedback 2/24/2011

BGE-R Policy Communication/Feedback naBGF Suspension/Repeal of Policy 2/24/2011BHC Board Communication With Staff Members 2/24/2011CA Administrative Goals/Priority Objectives 2/24/2011


Exhibit 1.2.1 (continued)Board Policies Reviewed by Auditors



Adopted CB Superintendent 2/24/2011

CBA Qualifications and Duties of the Superintendent 9/8/2011CBI Evaluation of Superintendent 10/27/2011CCB Line and Staff Relations 10/27/2011

CCB-RA Line and Staff Relations (School Administration) naCCB-RB Line and Staff Relations (Shared Decision Making na

CD Management Team/Senior Staff 10/27/2011CE Administrative Councils, Cabinets and Committees 10/17/2011

CFD School-Based Management (School Councils) 10/27/2011CFD-R School-Based Management (School Councils) naCFD-E School Based Management (School Councils) na

CH Policy Implementation 10/27/2011CH-R Policy Implementation naCHCA Handbooks and Directives 10/27/2011

CM School District Annual Report 10/27/2011DA Fiscal Management Goals/Priority Objectives 7/16/2001DB Annual Budget 7/16/2001

DBC-R Budget Planning, Preparation, and Schedules naDBF Budget Hearings and Reviews/Adoption Process 7/16/2001DBI Budget Implementation 7/16/2001DD Funding Proposals, Grants, and Special Projects 7/16/2001

DD-E Funding Proposals, Grants, and Special Projects naDDA Funding Sources Outside the School System 7/16/2001DIC Financial Reports and Statements 1/8/2009DIE Audits/Financial Monitoring 7/16/2001EA Support Services Goals/Priority Objectives 7/16/2001

ECB Building and Grounds Maintenance 1/8/2009EDBA Maintenance and Control of Instructional Materials 7/16/2001

EF Food Service 7/16/2001FA Facilities Development Goals/Priority Objectives 7/16/2001FE Facilities Construction 7/16/2001

FEA Educational Specifications for Construction 7/16/2001GA Personnel Goals/Priority Objectives 4/7/2011GB General Personnel Policies 1/24/2013

GBA Equal Employment Opportunities 4/7/2011GBA-RB Equal Employment Opportunities na

GBEA Staff Ethics 4/7/2011GBEBB Staff Conduct with Students 4/7/2011

GCA Professional Staff Positions 4/7/2011GCBA Professional Staff Salary Schedule 4/7/2011GCH Professional/Support Staff Orientation and Training 4/7/2011GCI Professional Staff Development 4/7/2011

GCMF Professional Staff Duties and Responsibilities 4/7/2011GCO Evaluation of Professional Staff Members 8/22/2013GDA Support Staff Positions 4/7/2011





Adopted GDB Support Staff Contracts and Compensation 4/7/2011

GDBA Support Staff Salary 4/7/2011GDM Support Staff Career Development 4/7/2011HA Meet and Confer Goals (Administrative) 4/7/2011IA Instructional Goals and Objectives 12/2/2003

IGA Curriculum Development 12/2/2003IGA-RB Curriculum Development (Guides and Course Outlines) na

IGD Curriculum Adoption 12/2/2003IGE Curriculum Guides and Course Outlines 12/2/2003

IGE-R Curriculum Guides and Course Outlines naIHA Basic Instructional Program 2/6/2007

IHA-E Basic Instructional Program naIHA English Instruction 7/16/2001IHB Special Instructional Programs 6/12/2008

IHB-R Special Instructional Programs (Identification and Placement of Exceptional Students) na

IHBA Special Instructional Programs and Accommodations for Disabled Students (Section 504 of the Rehabilitation Act of 1973) 1/6/2004

IHBA-R Special Instructional Programs and Accommodations for Disabled Students (Section 504 of the Rehabilitation Act of 1973) na

IHBA-RB Special Instructional Programs and Accommodations for Disabled Students (Section 504 of the Rehabilitation Act of 1973) na

IHBA-E Special Instructional Programs and Accommodations for Disabled Students (Section 504 of the Rehabilitation Act of 1973) (Policy Memorandum) na

IHBB Gifted and Talented Education 12/2/2003IHBD Compensatory Education (Title 1) 7/16/2001

IHBD-R Compensatory Education (Title 1) naIHBE Bilingual Instruction/Native Language Instruction 7/16/2001IIB Class Size 7/16/2001

IIB-R Class Size (Special Education) naIJ Instructional Resources and Materials 12/2/2003

IJ-R Instructional Resources and Materials naIJJ Textbooks/Supplemental Materials Selection and Adoption 12/2/2003

IJJ-R Textbooks/Supplemental Materials Selection and Adoption naIJL Library Materials Selection and Adoption 2/6/2007

IJL-R Library Materials Selection and Adoption naIJM Special Interest Materials Selection and Adoption 7/16/2001IJNC Resource Centers/Media Centers/School Libraries 12/2/2003IJND Technology Resources (Movies/Videos) 7/16/2001

IJND-R Technology Resources (Movies/Videos) naIJNDB Use of Technology Resources in Instruction 7/12/2012

IJNDB-R Use of Technology Resources in Instruction naIJOA Field Trips 12/2/2003

IJOA-R Field Trips naIKA Grading/Assessment Systems 12/2/2003

IKAB Report Cards/Progress Reports 12/2/2003IKB Homework 7/16/2001





Adopted IKE Promotion and Retention of Students 9/25/2008

IKE-R Promotion and Retention of Students naIKE-E Promotion and Retention of Students naIKEB Acceleration 12/2/2003

IL Evaluation of Instructional Programs 2/6/2007ILB Test/Assessment Administration 2/6/2007IMA Teaching Methods 2/6/2007

IMA-R Teaching Methods (Lesson Plans) naJB Equal Educational Opportunities 7/16/2001

JB-R Equal Educational Opportunities naJC School Attendance Areas 5/15/2007JE Student Attendance 7/16/2001

JFAA Admission of Resident Students 1/8/2009JFAB Tuition/Admission of Nonresident Students 7/16/2001

JFAB-R Tuition/Admission of Nonresident Students naJFABB-R Admission of Homeless Students 1/8/2009JFABB-R Admission of Homeless Students na

JFB Open Enrollment 7/16/2001JIC Student Conduct 7/16/2001KA School, Community, Home Relations Goals/Priority Objectives 7/16/2001KB Parent Involvement in Education 1/8/2009

KB-R Parent Involvement in Education naKCD Public Gifts/Donations to Schools 7/16/2001

na = No date included

Auditors noted that the dates of the 100 policies selected for review ranged from July 16, 2001, to August 22, 2013, with most policies bearing dates from 2007 through 2011 or earlier. Twenty-seven of the policies indicated an adoption date as “date of manual adoption,” which was July 16, 2001. Auditors were told that the board does attempt to conduct regular reviews to revise, update, and add policies. Interviewees indicated that some policies are still in need of updating. Auditors found that three of the 100 policies reviewed are less than five years old. Ninety-seven percent of the 100 policies reviewed are more than five years old, 40% are more than 10 years old, and 27% are more than 15 years old.

Policy Design

Auditors analyzed the policies listed in Exhibit 1.2.1 for congruence with the CMSi Curriculum Audit™ standards using the Curriculum Management Improvement Model (CMIM) 26 criteria for adequate policies, each with three levels of analysis. The auditors assessed the quality of board policies by comparing the content to audit criteria for good curriculum management. The 26 criteria are organized into five categories: control, direction, connectivity and equity, feedback, and productivity, which mirror the five standards of the audit and reflect 78 policy characteristics.

The auditors examined each relevant policy to determine if the CMIM criteria were met. For each criterion, a point is given based on the characteristics of the policy. If a policy (or several together) met the descriptor, a point was given the corresponding descriptor. If a policy were considered too weak to meet the descriptors or if there were no policy regarding the criterion, no point was given. To be considered adequate, 70% of the total possible points for a standard (set of criteria) had to be awarded. The criteria and results of this analysis are presented in Exhibit 1.2.2 through Exhibit 1.2.7.


Exhibit 1.2.2 provides the auditors’ analysis of district board policies and the level of control provided by the policies related to the design and delivery of the written curriculum, long-range planning, and functional decision-making structures.

Exhibit 1.2.2

Auditors’ Analysis of Board Policy and Administrative Regulations On Audit Standard One to Determine Quality and Degree of Adequacy


Standard One—Provides for Control: Directs the superintendent or designee to oversee the development of board policy to ensure:

Audit Criteria and CharacteristicsRelevant

Policies & Regulations

Auditors’ Rating

1.1 A taught and assessed curriculum that is aligned to the district written curriculum• Requires the taught and assessed curriculum to be aligned to the district’s written

curriculumCFD, IA, IGA, IGA-RB, IGE, IGE-R, IHA, IHA-E, IHB, IL, IMA

0

• Addresses the alignment of the district’s written curriculum with state and national standards for all subject areas and grades (includes electives)

0

• Directs the district’s written curriculum documents to be more rigorous than state and national standards to facilitate deep alignment in all three dimensions with current and future high-stakes tests

0

1.2 Philosophical statements of the district instructional approach• Has a general philosophical statement of curriculum approach, such as standards-based,

competency-based, outcome-based, etc.IA, IGA-RB, IGE, IHA, IHA-E, IHBB, IJ, IJ-R, IJJ, IL, IMA, IMA-R

0

• Directs adherence to mastery learning practices for all content areas and grades involved in local, state, and national accountability

0

• Directs adherence to mastery learning practices for all grade levels and content areas, including electives

0

1.3 Board adoption of the written curriculum• Requires the annual review of new or revised written curriculum prior to its adoption IA, IGA,

IGA-RB, IGD, IGE, IGE-R, IL

0• Directs the annual adoption of new or revised written curriculum for all grade levels and

content areas1

• Directs the periodic review of all curriculum on a planned cycle over several years 11.4 Accountability for the design and delivery of the district curriculum through roles and responsibilities• Directs job descriptions to include accountability for the design and delivery of the

aligned curriculumCA, CBA, CCB-RA, CCB-RB, GCA, GCO, GCO-EA

0

• Links professional appraisal processes with specific accountability functions in the job descriptions of central office administrators, building administrators, and regular classroom teachers

0

• Directs professional appraisal processes to evaluate all staff in terms of gains in student achievement

1

1.5 Long-range, system-wide planning • As part of the district planning process, policy requires that the superintendent and staff

think collectively about the future and that the discussion take some tangible form (This allows for flexibility without prescribing a particular template)

ADA, CA, CFD, IHBB, IHBD, IHBD-R

0

• Requires the development of a system-wide, long-range plan that is updated annually; incorporates system-wide student achievement targets; and is evaluated using both formative and summative measures

0

• Expects school improvement plans to be congruent with the district long-range plan, to incorporate system-wide student achievement targets, and to be evaluated using both formative and summative measures

0


Exhibit 1.2.2 (continued)Auditors’ Analysis of Board Policy and Administrative Regulations

On Audit Standard One to Determine Quality and Degree of AdequacyPhoenix Elementary School District #1

May 2017Standard One—Provides for Control:

Directs the superintendent or designee to oversee the development of board policy to ensure:



Auditors’ Rating

1.6 Functional decision-making structure• Expects an organizational chart that is annually reviewed, presented to the board, and

approved by the superintendent ABA, BDF, BG, CA, CBA, CCB, CCB-RB, CE, CFD, CGD-R, CFO-E, GBA, GBA-RB, GDA

0

• Requires that job descriptions for each person listed on the organizational chart be present and updated regularly to ensure that all audit criteria, such as span of control, logical grouping of functions, etc., are met

0

• Directs and specifies the processes for the formation of decision-making bodies (e.g., cabinet, task forces, committees) in terms of their composition and decision-making responsibilities, to ensure consistency, non-duplication of tasks, and product requirements

0

Standard One Rating (number of points for the six criteria with a possibility of 18) 3Percentage of Adequacy (points divided by the number of possible points—18) 17%Note: One point was awarded for every characteristic met under each criterion for a maximum of three points. No points are awarded when policies do not meet any characteristics.©2013 CMSi

Audit standards require that 70% of the audit criteria be met for board policies to be considered adequate. Three of the eighteen characteristics in Exhibit 1.2.2 (17%) were rated as meeting the criteria. It should be noted that auditors rated a characteristic as adequate only if it were met completely. Below is the auditors’ analysis of each criterion for Exhibit 1.2.2:

Criterion 1.1: A taught and assessed curriculum that is aligned to the district’s written curriculum

Expectations in board policy do not direct the alignment of the district’s written, taught, and assessed curriculum. Policy did not address a planned design for curriculum, instructional delivery, or assessment. Board Policy IGE: Curriculum Guides and Course Outlines directs that “the district will develop its own curriculum guides to reflect alignment to the state standards and local needs.” However, there is no requirement for alignment of the written, taught, and tested curriculum for all subjects and grade levels.

While policies allude to state standards, no direction requires the district curriculum to be aligned to or more rigorous than the state and national standards, or that curriculum be developed for all subjects and grades, including electives. Board Policy IGA: Curriculum Development states “The need for and value of a systematic, ongoing program of curriculum development and evaluation.” Further, “It is essential that the school system continually develop and modify its curriculum to meet changing needs.” Policy, though, does not specifically require the district’s curriculum to be more rigorous that state and national standards, to ensure the alignment of the written, taught, and tested curriculum, or to facilitate deep alignment of the content, contexts, and cognitive types. This criterion was not met; no points were awarded by auditors.

Criterion 1.2: Philosophical statements of the district instructional approach

Board policies do not articulate a philosophy for an instructional approach. Board Policy IGE: Curriculum Guides and Course Outlines directs that “Curriculum guides shall be developed for the various subject areas. These guides shall present at least a minimal outline for instruction and a basis for further development of the particular courses. In accordance with requirements of the state of Arizona, the District will develop its own guides….The guides shall be designed to assist users in implementing the District philosophy regarding the teaching of a subject and will, when possible, suggest a variety of possibilities for instruction, patterns of


individualization, variations of approaches, and materials.” However, the auditors did not find a statement clarifying the district’s philosophy.

Auditors found no policies requiring mastery learning practices to be employed at all grade levels and for all content areas, including electives. No points were awarded for this criterion.

Criterion 1.3: Board adoption of the written curriculum

This criterion requires annual adoption of all curricula by the board. Board Policy IGD: Curriculum Adoption states that “All new programs and courses of study will be subject to Board approval, as will elimination of programs and courses and extensive alteration in their content.” Board Policy IGA: Curriculum Development directs “All curriculum changes shall be approved by the Governing Board.” However, no policy sets an expectation that the board annually review and adopt new and revised curriculum according to a planned cycle over several years. Two of the three characteristics of this criterion were met.

Criterion 1.4: Accountability for the design and delivery of the district curriculum through roles and responsibilities

Board Policy GCA: Professional Staff Positions does establish general expectations for “a job description to include qualifications, performance responsibilities and the method by which the performance of such responsibilities will be evaluated.” But no policy requires accountability for the design and delivery of aligned curriculum through roles and responsibilities outlined in district job descriptions.

Auditors examined board policies to determine whether policy linked the evaluation of staff to gains in student achievement. Board Policy GCO: Evaluation of Professional Staff Members directs that the district’s professional appraisal processes will “Include quantitative data on student academic progress that accounts for between thirty-three percent (33%) and fifty percent (50%) of the evaluation outcomes.” Expectations of principals, however, in district regulation CCB-RA: Line and Staff Relations (School Administration) simply cites principal duties to include, “A principal is responsible for the supervision and evaluation of the building staff members” without reference to student achievement and progress.

In Board Policy GCO, the appraisal of principals outlines options for evaluation of a principal’s performance that may include student progress. Focus and emphasis on student achievement is lacking in policy. Overall, PESD #1 board policies did not focus appraisal processes on student progress but, rather, on the improvement of professional staff’s instructional skills and operational duties and responsibilities. Generally, policies reviewed by auditors were mute regarding student achievement results. One point was awarded for this criterion.

Criterion 1.5: Long-range, system-wide planning

Phoenix Elementary School District #1 recently undertook a process to develop a new strategic plan. Our Blueprint for Student Success (2012) is the current plan in place. Board Policy ADA: School, District Goals and Objectives requires that “The District shall create a strategic plan which outlines the long term goals and objectives for the District. This plan shall be reviewed and updated at least every four years.” Board policies, though, do not establish an expectation for long-range, system-wide planning for the future, link system-wide planning to agreed targets, create coherence between district and campus priorities, or specify how planning would address student achievement.

Board Policy ADA establishes an expectation that, “The District’s strategic plan shall address key areas which will include, but not be limited to a focus on:

• Student academic success.

• Creating and maintaining a culture of success.

• Incorporating partners for success into our operations”

Board policy does not direct administrators to maintain a long-range, system-wide planning process on an ongoing basis at both the district and/or school levels. An expectation of long-range planning that would include system-wide student achievement targets to be evaluated annually both formatively and summatively


is included in the superintendent’s job description but not in board policy. No board policy exists that directs short- and long-range planning at the district and school site levels to be updated annually and incorporates system-wide achievement targets.

Because regular long-range planning and aligned campus improvement plans are not required by board policy, auditors awarded no points for this criterion.

Criterion 1.6: Functional decision-making structure

Policy does not establish an expectation that the organizational chart depicts lines of authority, be consistently used, and be reviewed annually (see Finding 1.1). Except as noted in Criterion 1.4 above, board policy does not specifically require a job description for each position listed on the organizational chart that aligns to all audit criteria for quality job descriptions.

The creation and formation of decision-making bodies was cited in Board Policy CFD: Administrative Councils, Cabinets, and Committees directing that “The Board authorizes the Superintendent to establish advisory councils, cabinets, and committees when deemed necessary…for the improvement of the total educational program. The number, composition, and work to be done by such cabinets, councils, and committees shall be defined by the Superintendent.”

Auditors found no policies defining a district planning process involving community members, although Board Policy CFD: School-Based Management does outline process, procedures, and membership for school-based management school councils.

In interviews, auditors learned that the lack of alignment of the written, taught, and assessed curriculum, tightly-held curriculum, consistent planning, and a functional decision-making structure hampered productivity of various staff:

• “There is not a lot of uniformity in this district. People work very hard, but in isolation.” (Campus Administrator)

• “The site-based issue has moved to the extreme. Need to decide what is tightly held and what is loosely held.” (Central Office Administrator)

• “I think our district needs a curriculum and to find ways to be unified. I almost feel like our school is more of an island than a part of the district.” (Teacher Survey Response)

• “We really don’t have anything that is tightly held.” (Central Office Administrator)

• “New teachers are confused because the district adopts a certain program and the site they are assigned to uses something else.” (Support Staff)

• “We have gotten better at planning. In the past, it’s ‘We need to do this so let’s get on it and do it.’ Now we have been doing focus groups.” (Central Office Administrator)

• “In previous years, they [principals] did not have to follow district regulations and they were not used to, ‘No, you can’t’.” (Central Office Administrator)

Without clarity for the development of an organizational chart, detailed job descriptions, or direction for creating district decision-making bodies, auditors awarded no points for this criterion.

After an analysis of the level of control provided by board policies, auditors analyzed board policies related to providing direction of the written curriculum, aligned assessment, instructional resources, and program interventions. Standard Two delineates the expectations that a school system has established clear direction through a valid and measurable set of learning objectives for students. This analysis is provided in Exhibit 1.2.3.


Exhibit 1.2.3

Auditors’ Analysis of Board Policy and Administrative Regulations On Audit Standard Two to Determine Quality and Degree of Adequacy


Standard Two—Provides for Direction: Directs the superintendent or designee to oversee the development of board policy to ensure:


Policies and Regulations

Auditors’ Rating

2.1 Written curriculum with aligned, criterion-referenced formative assessments for all subject areas at all grade levels• Requires enough specificity so that all teachers can consistently describe how students

will demonstrate mastery of the intended objectiveCBA, IA, IGA, IGD, IGE, IHA

0

• Requires formative assessment instruments that align to specific curriculum objectives 0• Directs that suggestions be provided to teachers for differentiating curriculum to meet

students’ needs as diagnosed by formative assessments0

2.2 Periodic review/update of the curriculum and aligned resources and assessments• Requires the development of procedures to both formatively and summatively review

the written curriculum for all grade levels and content areasIGA, IGE, IJ, IJ-R, IJJ, IKA, IL, JB

0

• Requires the annual review of test banks, benchmark assessments, and other assessment instruments for alignment with the district or state accountability system

0

• Evaluates assessment instruments for alignment to the district curriculum in all three dimensions: content, context, and cognitive type

0

2.3 Textbook/resource alignment to curriculum and assessment• Requires textbooks/resources to be regularly reviewed and the resource revision/

adoption cycle to align with the curriculum revision cycleIJ, IJ-R, IJJ, IJJ-R, IJL, IJNDB, IL, JB

0

• Directs review of all new instructional resource materials for content, context, and cognitive type alignment to the district curriculum and assessment

0

• Directs district staff to identify discrete areas where alignment is missing and provide teachers with supplementary materials to address gaps in alignment (missing content, inadequate contexts, etc.)

0

2.4 Content area emphasis• Directs the yearly identification of subject areas that require additional emphasis based

on a review of assessment resultsDA, DB, DEBC-R, IA, IHA, GA, GCI, IA, IKA, IL, ILB

0

• Within subject areas, requires identification by administration of specific objectives, contexts, cognitive types, and instructional practices to receive budgetary support

0

• Requires focused professional development and coaching to support the instructional delivery of the identified priorities within the content areas

0

2.5 Program integration and alignment to the district’s written curriculum• Directs that all subject-related (e.g., reading, Title I) and school-wide (e.g., tutoring,

DARE, AVID) programs be reviewed for alignment to the written and assessed curriculum

DD, IA, IGA, IHA, IHA-E, IGD, IHB, IHBD, IHBE, IJM, IL, JB, JB-R

1

• Requires written procedures for both formative and summative evaluation of all new subject-related and school-wide programs before submission to the board for approval

0

• Directs administrative staff to prepare annual recommendations for subject-related and school-wide program revision, expansion, or termination based on student achievement

0

Standard Two Rating (number of points for the five criteria with a possibility of 15) 1Percentage of Adequacy (points divided by the number of possible points—15) 7%Note: One point was awarded for every characteristic met under each criterion for a maximum of three points. No points are awarded when policies do not meet any characteristics.©2013 CMSi


Exhibit 1.2.3 indicates that the board policies that provide for direction received a rating of 1 out of 15 possible points and are considered inadequate to provide clear curriculum guidance.

The following presents information about the auditors’ analysis of policies as related to Standard Two.

Criterion 2.1: Written curriculum with aligned, criterion-referenced formative assessment for all subject areas at all grade levels

Auditors were presented with two policies, Board Policy IGA: Curriculum Development and Board Policy IGE: Curriculum Guides and Course Outlines, related to written curriculum. These policies do not require formative assessment aligned to specific curriculum objectives. Policies requiring differentiation that is in response to formative assessment techniques were not found. No points were awarded for this criterion.

Criterion 2.2: Periodic review/update of the curriculum and aligned resources and assessments

The two district policies described in Criterion 2.1 relate to review of the district curriculum, resources, and assessments. Board Policy IGA authorizes the superintendent to “develop the curriculum for the school system and to organize committees to review curriculum.” Board Policy IGE requires the curriculum guides be developed for “the various subject areas.” Also, these guides shall “suggest a variety of possibilities for instruction, patterns of individualization, variations of approaches, and materials.” Board Policy IL: Evaluation of Instructional Programs calls for a testing program that will be subject to regular review and evaluation, and evaluation of all proposed testing instruments and periodic evaluation. No policies were presented to auditors that expect the testing program to be periodically reviewed for alignment with the district and state accountability system in the three dimensions of content, context, and cognitive type. No points were awarded for this criterion.

Criterion 2.3: Textbooks/resource alignment to curriculum and assessment

Auditors found three policies that address textbooks and resources. Board Policy IJ: Instructional Resources and Materials requires that the district shall furnish all textbooks and supplies for students in grades K-8. Board Policy IJJ: Textbook/Supplementary Materials Selection and Adoption requires that the board will have final approval and adopt all new textbooks and supplementary course books. The superintendent “shall establish textbook selection procedures that shall provide for the appropriate involvement of staff members, students, and community members.” Resources recommended to the board for approval “will support and enrich the curriculum.” Board Policy IJNDB: Use of Technology Resources in Instruction describes the use of technology resources in instruction, but is basically an acceptable use policy covering staff and student use of district technology equipment, software, and networks. No policy requires the alignment of textbooks or resources to the curriculum or to assessment. No points were awarded for this criterion.

Criterion 2.4: Content area emphasis

Board Policy IA: Instructional Goals and Objectives states generally, “The district instructional program will be designed and implemented to provide for developing:

• Skills in communication—to include reading, writing, speaking, listening, and composition.

• Skills in computation.

• Appreciation of the world of work.

• Pride of workmanship and skills for economic survival.

• Appreciation of the importance of physical fitness.

• Research and problem-solving skills.

• Ability to think analytically, critically, and independently.

• Skills in foreign or Native American language.

• Ability leading to citizen responsibility.


• Understanding and respect for our cultural heritage.

• Appreciation for the intrinsic value of education.

• Appreciation of the fine arts.

• Skills in technology.”

Board Policy IHA: Basic Instructional Program states that The instructional program will include planned sequences in:

• Language arts—reading, spelling, handwriting, English grammar, composition, literature, and study skills.

• Mathematics experiences.

• Social studies—history, geography, civics, economics, world cultures, political science, and other social science disciplines.

• Science experiences.

• Fine and practical arts experience—art education, vocal and instrumental music, and vocational/business education.

• Technology skills.

• Health and safety education (to include skin cancer prevention as required by Senate Bill 1297).

• Physical education.

• Foreign or Native American language.”

Neither Board Policy IA nor Board Policy IHA requires the identification of subject areas that need additional emphasis and budgetary support.

Board Policy GCI: Professional Staff Development recognizes the board’s responsibility to provide opportunity for the continual professional growth of staff, but does not include policy statements requiring professional development in support of curriculum delivery. No points were awarded for this criterion.

Criterion 2.5: Program integration and alignment to the district’s written curriculum

No policy specifically directs that all subject-related programs be reviewed for alignment. Board Policy IHA-E does require that for students in Kindergarten through grade 3, the district will “review its reading program and take corrective action as specified by the State Board of Education whenever more than 20% of the third grade students do not meet the AIMS standards.” However, this requirement is limited to those students in Kindergarten through grade 3 for reading only. One point was awarded for this criterion.

Exhibit 1.2.4 presents auditors’ analysis of the adequacy of board policies in requiring connectivity and equity in the district curriculum, articulation, coordination, and professional development to deliver the adopted district curriculum, monitoring the delivery of the curriculum, and equitable student access to the curriculum, instructional resources, and learning environment.


Exhibit 1.2.4

Auditors’ Analysis of Board Policy and Administrative Regulations On Audit Standard Three to Determine Quality and Degree of Adequacy


Standard Three—Provides for Connectivity and Equity: Directs the superintendent or designee to oversee the development of board policy to ensure:



Auditors’ Rating

3.1 Predictability of written curriculum from one grade and/or instructional level to another• Requires the vertical articulation and horizontal coordination of the curriculum within

schoolsADA, CBA, IA, IGA, IGD, IGE, IHA

0

• Requires vertical articulation across grade levels and horizontal coordination among schools at a given level for all content areas 0

• Directs the identification of prerequisite skills and their placement in the written curriculum at the appropriate grade/instructional level 0

3.2 Training for staff in the delivery of the curriculum • Directs the development and implementation of a district professional development

plan, focused on effective curriculum delivery, that is congruent with the district long-range plan and annual goal priorities

ADA, CBA, GCI, GCI-R, GCO, IA, IGA, IGA-RB, IGD, IGE, IGE-R, IHA, IHB, IL

0

• Requires a process whereby staff are coached over time in the implementation of professional development initiatives 0

• Directs the regular evaluation of the impact of professional development on student achievement, using both formative and summative measures 0

3.3 Delivery of the adopted district curriculum• Requires all staff to deliver the curriculum as approved by the board ADA, CBA,

IA, IGA, IGA-RB, IGD, IGE, IGE-R, IL, ILB, IMA, IMA-R, JB

0• Requires building principals and all central office staff with curriculum responsibilities

to review disaggregated assessment results and identify areas where curriculum delivery may be ineffective

Partial*

• Requires an annual report for the board regarding the status of curriculum delivery0

3.4 Monitoring the delivery of the district curriculum• Directs building principals to develop and implement a plan to monitor the delivery of

the district curriculum on a weekly basisADA, CA, CBA, CBI, CCB, CCB-RA, GCI, IA, IL, ILB, IMA, IMA-R, JB

0

• Directs central office curricular staff to assist the principal in monitoring the delivery of the district curriculum 0

• Requires periodic school and classroom data-gathering reports from administrators detailing the status of the delivery of the curriculum across the district, with recommendations for the creation of professional development activities or curricular revisions

0



On Audit Standard Three to Determine Quality and Degree of AdequacyPhoenix Elementary School District #1

May 2017Standard Three—Provides for Connectivity and Equity:

Directs the superintendent or designee to oversee the development of board policy to ensure:



Auditors’ Rating

3.5 Equitable student access to the curriculum, instructional resources, and learning environment• Requires equal student access to the curriculum, appropriate instructional materials for

a variety of learning levels and modes, and appropriate facilities to support the learning environment necessary to deliver the district curriculum

ADA, CA, CBA, IHB, IHBB, IHBD, IHBE, IJM, IJNDB, JB, JI, JI-R

1

• Directs the development of procedures for fast-tracking students who lack sufficient prerequisite skills for courses such as AP, honors, etc., but need more challenging content

0

• Requires an annual review of equity data (such as access, racial isolation, rigor), the subsequent reporting to the board of those data, and the development of a plan for correcting equity issues

0

Standard Three Rating (number of points for the five criteria with a possibility of 15) 1Percentage of Adequacy (points divided by the number of possible points—15) 7%

Note: One point was awarded for every characteristic met under each criterion for a maximum of three points. No points are awarded when policies do not meet any characteristics.*Partial ratings are tallied as inadequate.©2013 CMSi

Auditors rated 1 of the 15 characteristics (7%) in Standard Three—Connectivity and Equity as adequate and one as partially adequate. Points are awarded only if the entire characteristic is met, and partially rated characteristics receive no points. Following is the auditors’ analysis of the criteria under Standard Three:

Criterion 3.1: Predictability of written curriculum from one grade and/or instructional level to another

While board policies exist that discuss curriculum, no policies addressed the specifics required in this criterion such as a curriculum management plan or the articulation and delineation of prerequisite skills for core content areas. No policies were found that required vertical articulation across grade levels or horizontal articulation among schools. Board policy does not direct vertical articulation between various grade levels in the district or horizontal coordination to ensure consistency at a particular grade level across all schools in the district. This criterion was not met.

Criterion 3.2: Training for staff in the delivery of the curriculum

Board Policy GCI: Professional Staff Development endorses the need for professional development and cites, “The Board recognizes its particular responsibility to provide opportunity for the continual professional growth of its certificated staff. Such opportunities include, within budgetary limitations, special in-service training courses, workshops, school or District visitations, conferences, professional library, and assistance from supervisors and consultants.” However, auditors were not presented with a professional development plan or any document coordinating efforts to meet their requirement. The policy does not align professional development with effective curriculum delivery or the district long-range plan and annual goals. It does not establish expectations and standards regarding the impact of professional development on student achievement using both formative and summative assessment measures and walk-through data. Ongoing coaching over time in the implementation of professional development initiatives is not required by the policy. This criterion received no points.


Criterion 3.3: Delivery of the adopted district curriculum

Board Policy IGD: Curriculum Adoption clearly requires that, “All new programs and courses of study will be subject to Board approval, as will elimination of programs and courses and extensive alteration of their content.” However, auditors found that the requirement that teachers teach the adopted curriculum is located in Administrative Regulation and the teacher job description, not in board policy. Auditors awarded partial credit for Board Policy CBA: Qualifications and Duties of the Superintendent that indicated that the superintendent “administers the development, coordination, maintenance, and evaluation of the educational program, including the special education program.” However, the policy lacks the specificity of requiring analysis of disaggregated assessment results to determine where curriculum delivery may be ineffective. The superintendent’s job description generally states that the superintendent “shall be responsible for the operation of the school system in order to establish optimum educational opportunities for every student.” The job description lacks the specificity to meet the characteristics of Criterion 3.3. Auditors awarded one point and one partial rating for this criterion.

Criterion 3.4: Monitoring the delivery of the district curriculum

No policy requires building principals to be responsible for curriculum delivery. However, Administrative Regulation CCB-RA: Line and Staff Relations does indicate that “The primary duty of a principal is to administer and supervise the instructional program.” The regulation cites duties that include, “A principal is responsible for the supervision and evaluation of building staff members.” In both instances, no clear direction to monitor teachers in a designed manner on a regular basis is present, only implied.

While campus administrators reported to auditors that they conducted classroom walk-throughs on a regular basis, no policy was found that directed administrators to monitor the delivery of the instruction with established regularity. This monitoring may be occurring at the various sites, but it is not indicated in policy. Auditors found no policy that directed central office staff to aid principals in monitoring the delivery of the curriculum, nor gathering periodic data as the basis for the creation of professional development activities. On the teacher survey, 38% of teachers indicated that their principal was in their classroom at least weekly while over 40% indicated being visited monthly. This criterion was not met.

Criterion 3.5: Equitable student access to the curriculum, instructional resources, and learning environment

Board Policy JB: Equal Educational Opportunities states, “The right of a student to participate fully in classroom instruction shall not be abridged or impaired because of race, color, religion, sex, age, national origin, and disability, or any other reason not related to the student’s individual capabilities.” Board Policy IHA: Basic Instructional Program directs that, “The various instructional programs will be developed to maintain a balanced, integrated, and sequential curriculum that will serve the educational needs of all school-aged children in the District.” Board Policy IHB: Special Instructional Programs indicates that “A long-range plan will be the basis for providing special education services for students with exceptional needs and education requirements. These services may include specialized programs, personnel, facilities, materials, and equipment needed to promote the individual physical, social, intellectual, and emotional growth of exceptional students.”

Likewise, Board Policy IHBB: Gifted and Talented Education states, “The Board requires that appropriate instructional programs be conducted to meet the needs of exceptionally gifted students of school age, in keeping with the District’s goal of developing the special abilities of each student.” Board Policy FA: Facilities Development Goals/Objectives requires that “Priority in the development of facilities shall be based on identified educational needs and on programs developed to meet those needs…including educational specifications for school buildings on identifiable learner needs.”

Auditors found no board policies requiring an annual review of equity data, a report to the board on such data, nor the requirement to develop a plan to correct equity issues. This criterion received one point.

Feedback regarding student progress is a critical tool for the district. Auditors analyzed the adequacy of board policies to direct the student assessment process, program assessment, and the use of data to determine program/


curriculum effectiveness and efficiency. A minimum of nine points are needed to be considered adequate. Exhibit 1.2.5 presents the auditors’ analysis of the level of feedback directed by board policies.

Exhibit 1.2.5

Auditors’ Analysis of Board Policy and Administrative Regulations On Audit Standard Four to Determine Quality and Degree of Adequacy


Standard Four—Provides for Feedback: Directs the superintendent or designee to oversee the development of board policy to ensure:


Policies and Regulation

Auditors’ Rating

4.1 A student assessment process• Requires the development and implementation of a district student assessment process

that goes beyond the state accountability assessment system and includes both formative and summative measures

IA, IKA, IKE, IKE-E, IL, ILB

0

• Requires the development and implementation of a district student assessment process that is differentiated to address variations in student achievement (both above and below grade level) and includes both formative and summative assessment measures

0

• Requires assessment instruments to be more rigorous in content, context, and cognitive type than external, high stakes assessments

0

4.2 A program assessment process• Directs the development and implementation of a district program evaluation process IA, IHA,

IHB, IHB-R, IJ, IL, ILB, JB, JB-R

0• Requires each proposed program to have an evaluation process (The process includes

both formative and summative evaluations) before that program is adopted and implemented

0

• Directs the program assessment process to link with district planning initiatives, including site improvement plans and the strategic/long-range plan

0

4.3 Use of data from assessments to determine program and curriculum effectiveness and efficiency• Requires the disaggregation of assessment data at the school, classroom, student

subgroup, and student level to determine program and curriculum effectiveness and efficiency

IA, IKA, IKAB, IKE, IKE-R, IKE-E, IKEB, IL, ILB

0

• Requires classroom teachers to track and document individual student mastery in core content areas

0

• Requires the development of modifications to the curriculum and/or programs as needed in response to disaggregated assessment data to bring about effectiveness and efficiency

0

4.4 Reports to the board about program effectiveness• Requires yearly reports to the board regarding program effectiveness for all new

programs for the first three years of operationADA, BA, IA, IGD, IHA, IHB, IL, ILB, JB

0

• Requires reports to the board every three years for long-term programs 0• Requires summative reports to the board every five years for all content areas before any

curriculum revisions or major materials acquisition, with the reports delivered prior to the curricular adoption cycle

0

Standard Four Rating (number of points for the four criteria with a possibility of 12) 0Percentage of Adequacy (points divided by the number of possible points—12) 0%Note: One point was awarded for every characteristic met under each criterion for a maximum of three points. No points are awarded when policies do not meet any characteristics.©2013 CMSi

Exhibit 1.2.5 shows the auditors’ ratings of district policies and procedures related to Standard Four. Auditors found a lack of clear direction and control to provide assessment feedback to determine program and curriculum


effectiveness and efficiency. At least 70%, or nine, of the characteristics must be met for policy to be considered adequate. Auditors found that none of the 12 characteristics were met.

The following presents information about the auditors’ ratings on policy for Standard Four:

Criterion 4.1: A student assessment process

No policies were found requiring district assessments to go beyond the required state accountability, nor to establish a system that is differentiated based on formative measures, or more rigorous than high stakes external assessments. No points were awarded in this criterion.

Criterion 4.2: A program assessment process

Auditors found no policies requiring the implementation of a program assessment process. Furthermore, no policies were found showing linkages to district planning that utilize formative and summative assessments to direct planning initiatives and long-range planning. No points were awarded for this criterion.

Criterion 4.3: Use of data from assessments to determine program and curriculum effectiveness and efficiency

No policies were found requiring the use of disaggregated assessment data to determine program and curriculum effectiveness and efficiency. Furthermore, no policies provide direction or control to document student mastery in core content areas nor do they require the modification of programs or curriculum based upon disaggregated assessment data. No points were awarded for this criterion.

Criterion 4.4: Reports to the board about program effectiveness

No policies were found requiring annual reports to the board regarding the effectiveness for new programs to the district. In addition, no policies require any ongoing reports for long-term effectiveness, curriculum revisions, or material acquisitions related to program effectiveness. No points were awarded for this criterion.

Exhibit 1.2.6 presents the auditor’s analysis and ratings of the district policies, rules and regulations related to Standard Five, which provides for productivity.

Exhibit 1.2.6

Auditors’ Analysis of Board Policy and Administrative Regulations On Audit Standard Five to Determine Quality and Degree of Adequacy


Standard Five—Provides for Productivity: Directs the superintendent or designee to oversee the development of board policy to ensure:



Auditors’ Rating

5.1 Program-centered budgeting• Directs development of a budget process that requires program evaluation,

identification of specific measurable program goals before the budget process begins, and documented costs to ensure that expenditures are aligned within revenues and cost-benefit analysis is facilitated

DA, DB, DBC-R, DBI, DIC, IL

0

• Requires adherence to a program-centered budgeting process that includes incremental budgeting based on different program types, delivery, and quality for all curriculum areas (The process provides evidence of tangible connections between allocations and anticipated program outcomes or accomplishments.)

0

• Directs full implementation of a program-centered budgeting process that includes incremental funding possibilities, a process for evaluating options, and the use of program evaluation data linked to budget allocations (This process enables program budget decisions to be based upon documented results and performance.)

0



On Audit Standard Five to Determine Quality and Degree of AdequacyPhoenix Elementary School District #1

May 2017Standard Five—Provides for Productivity:

Directs the superintendent or designee to oversee the development of board policy to ensure:Audit Criteria and Characteristics Relevant


Auditors’ Rating

5.2 Resource allocation tied to curriculum priorities• Requires a budget that allocates resources according to documented needs, assessment

data, and established district curriculum and program goals and prioritiesCA, CBA, CCB, IA, IGD, IHA, IL

0

• Requires a budget that may be multi-year in nature, provides ongoing support for curriculum and program priorities, and connects costs with program expectations and data-based needs

0

• Directs a budget that provides resources needed to achieve system priorities over time and demonstrates the need for resources based on measurable results and/or performance of programs and activities

0

5.3 Environment to support curriculum delivery• Directs facilities that enable teachers to work in an environment that supports adequate

delivery of the curriculum CBA, CM, EA, ECB, FA, FE, FEA

1

• Directs consideration of multi-year facilities planning efforts to adequately support the district curriculum and program priorities

1

• Directs facilities planning linked to future curriculum and instructional trends and to the teaching-learning environment incorporated in the documented system mission and vision statements

1

5.4 Support systems focused on curriculum design and delivery• Provides a clear connection between district support services and the achievement of

the district curriculum design and delivery, and evidence of optimization within the system

EA, GA, GCI, GDA, IL, ILB

1

• Requires formative and summative evaluation practices for each support service to provide data for improving these services and documented evidence of improvement over time

0

• Requires periodic reports to the board with recommendations for continuing, revising, and/or developing new support services to enhance fulfillment of the mission, including needs-based data

0

5.5 Data-driven decisions for the purpose of increasing student learning• Directs the development of specific requirements for data analysis that lead to

improved student learning for the core curriculum areas and electivesCBA, DB, DBC-R, DBI, DD, EA, GA, IA, IHA, IL, ILB

0

• Directs the development of specific requirements for data analysis that lead to improved student learning for all curriculum areas and grade levels (including electives)

0

• Directs the development of specific requirements for data analysis that lead to improved student learning for all operations of the district

0



On Audit Standard Five to Determine Quality and Degree of AdequacyPhoenix Elementary School District #1

May 2017Standard Five—Provides for Productivity:

Directs the superintendent or designee to oversee the development of board policy to ensure:Audit Criteria and Characteristics Relevant


Auditors’ Rating

5.6 Change processes for long-term institutionalization of district priority goals• Requires the identification of strategies, grounded in documented assessment

of program success or efficacy, to be used by the district to ensure long-term institutionalization of change

BA, CA, CBA, CH, IA, IHA, IHB, IHBD, IL

0

• Directs the development of school improvement plans that address the use of specific change strategies at the building level to ensure the institutionalization of change and improved results or performance

0

• Directs that all district, department, and program plans incorporate procedures for change strategies to ensure the institutionalization of change for improvement and include procedures with formative and summative practices that provide data about change implementation and effectiveness

0

Standard Five Rating (number of points for the six criteria with a possibility of 18) 4Percentage of Adequacy (points divided by the number of possible points—18) 22%Note: One point was awarded for every characteristic met under each criterion for a maximum of three points. No points are awarded when policies do not meet any characteristics.©2013 CMSi

Auditors found board policies lacked sufficient content, specificity, and direction to meet this audit criterion. At least 70% of the characteristics must be met for the policies to be considered adequate; the auditors rated 4 of 18 (22%) of the criteria as adequate.

The following presents information about the auditors’ ratings on Standard Five:

Criterion 5.1: Program-centered budgeting

Two policies and one administrative regulation include a discussion of budgeting activities, each of which hold opportunities for program-driven budgeting. Board Policy DA: Fiscal Management Goals/Priority Objectives states that the district shall “Encourage short- and long-range planning through the best possible budgeting procedures” and “Guide the expenditure of funds to achieve the greatest educational returns.” There is no mention of how or when the educational returns should be determined.

Administrative Regulation DBC-R: Budget Planning, Preparation, and Schedules presents a timeline for the budgeting process along with needed activities, none of which include a presentation of program data, connections between allocations and program outcomes, or comparisons of program expectations and allocated monies. Board Policy IL: Evaluation of Instructional Programs states that “The use of tests is one indication of the success and quality of the educational program.” However, connections between test usage, program results, and allocation of funds to support effective programming are not mentioned. No points were awarded for this criterion.

Criterion 5.2: Resource allocation tied to curriculum priorities

Board Policy CA: Administrative Goals/Priority Objectives presents several board goals and priority objectives, none of which include budgeting of funds, connections of allocations to curriculum priorities, or a multi-year budget with ongoing support based on data driven results. One objective states an expectation that “planning, organizing, implementing, and evaluating educational programs” shall be utilized, but no expectation is described as to how this process will impact the budgeting process. Board Policy CBA: Qualifications and Duties of the Superintendent describes the duties of the superintendent, including “Assumes responsibility for


the overall financial planning of the District and for the preparation of the annual budget,” and “Establishes and maintains efficient procedures and effective controls for all expenditures of school funds in accordance with the adopted budget.” Neither requirements includes any expectation for program centered budgeting. No points were awarded for this criterion.

Criterion 5.3: Environment to support curriculum delivery

Several board policies mention the physical environment of the district to support curriculum delivery, including the following:

• Board Policy CBA: Qualifications and Duties of the Superintendent requires the superintendent to assume responsibility for the use of buildings and grounds and to recommend “the locations and sizes of new school sites and of additions to existing sites; the locations and sizes of new buildings; the plans for new school buildings; all appropriations for sites and buildings; and improvements, alterations, and changes in the buildings and equipment of the District.”

• Board Policy CM: School District Annual Report requires an annual report to include the actual energy and cost savings pursuant to a guaranteed energy cost savings contract.

• Board Policy EA: Support Services Goals/Priority Objectives requires “a physical environment for teaching and learning that is safe and pleasant for students, staff members, and the public.”

• Board Policy ECB: Building and Grounds Maintenance requires regular evaluation of the condition of facilities as stated, “The Board directs a continuous program of inspection and maintenance of school buildings and equipment. Wherever possible, maintenance shall be preventive and will focus on providing an on-going healthy learning environment for both students and school personnel.”

• Board Policy FA: Facilities Development Goals/Priority Objectives outlines several expectations related to the future facility needs of the district, including:

○ To integrate facilities planning with other aspects of planning in a comprehensive educational program.

○ To base educational specifications for school buildings on identifiable learner needs.

○ To design for sufficient flexibility to permit program modification or the installation of new programs.

○ To design school buildings as economically as feasible, providing that learner needs are effectively and adequately met by the design.

○ To involve the community, school staff members, available experts, and the latest in related current development and research in building plans and specifications.

○ To analyze life-cycle costs as they compare with capital expenditures versus a maintenance and operations expense projection.

• Board Policy FE: Facilities Construction states that long-range planning efforts shall, in part, include “the projection of student enrollments, long-range financial plans, long-range projected use, and need for additional facilities. Staff members, students, and citizens will be used in the planning.”

• Board Policy FEA: Educational Specifications for Construction describes a number of education specification requirements for new and remodeled facilities. Among them are a requirement to include enrollment expectations, proposed curriculum and teaching methods, space requirements and locations of spaces, special areas, mechanical features and special finishes.

All three points were awarded for this criterion.


Criterion 5.4: Support systems focused on curriculum design and delivery

Board Policy EA describes an expectation for support services to be supportive of the educational program as follows:

• To provide a physical environment for teaching and learning that is safe and pleasant for students, staff members, and the public.

• To provide safe transportation for students who ride the school bus to and from school.

• To provide services, resources, and assistance responsive to the needs of the educational programs.

• To establish a thorough, effective, and economical maintenance program that will assure a useful life for school sites, plants, and equipment.

Policies requiring evaluation practices for each support service providing data for program improvement or reporting to the board on a regular basis were not presented to auditors for review. One point was awarded for this criterion.

Criterion 5.5: Data-driven decisions for the purpose of increasing student learning

Board Policy EA describes the budgeting process with an expectation to “formulate the annual budget, considering at all times that resources must be utilized to produce the most positive effect on the student’s opportunity to gain an education.” The superintendent shall also be responsible for “reviewing budgetary requests, providing guidelines and limitations, and presenting the proposed budgets and documentation necessary for Board study, review, and action.” However, policy is silent on how or when this activity is to occur.

Auditors were not presented with policies requiring the use of data analysis that leads to improved student learning or analysis of other operations within the district. No points were awarded for this criterion.

Criterion 5.6: Change processes for long-term institutionalization of district priority goals

Board Policy CA describes the duties of the superintendent in general terms related to institutionalization of change, use of change strategies, and improvement of procedures to affect student outcomes. “The Superintendent is the professional consultant to the Board and, in this capacity, makes recommendations to the Board for changes in Board policies and the educational program.” Further, the superintendent “Administers the development, coordination, maintenance, and evaluation of the educational program; Supervises methods of teaching, supervision, and administration in effect in the schools; Assumes responsibility for the overall financial planning of the District; Establishes and maintains efficient procedures and effective controls.”

While latitude should be granted to allow for particular circumstances within a district, policies should also reflect the desire of the governing board as to the attitudes and processes related to administrative implementation of the policies, none of which are reflected in Policy CA. None of the descriptions satisfy the expectations of the audit principle of adapting to change for long term institutionalization of goals. Zero points were awarded this criterion.


Exhibit 1.2.7 presents the summary ratings of auditors’ analysis of adequacy of board policies to direct curriculum design and delivery in the district.

Exhibit 1.2.7

Summary Ratings of the Auditors’ Analysis of Board Policy To Determine Quality and Degree of Adequacy

Phoenix Elementary School District #1 May 2017

Standard Number of Criteria

Number of Possible

Points

Points Given

Percentage of Points Relative to 70%

Standard for AdequacyOne 6 18 3 17%Two 5 15 1 7

Three 5 15 1 7Four 4 12 0 0Five 6 18 4 22

Overall Rating For all Criteria 26 78 9 12%

©2013 CMSi

The current policies met 9 points (12%) of the 78 possible points for the 26 criteria of strong curriculum management policies. Auditors found the policies inadequate to direct systemic curriculum management and to establish quality control of the educational program. Many policies stopped short of including clear details and provided only basic direction and general language of state guidance. The auditors noted the following specifically related to the five criteria:

1. Control: No Phoenix Elementary School District #1 policy presents the district’s mission statement, which can be found in the district’s strategic plan. Board Policy ADA: School District Goals and Objectives does state the focus of the district (but not limited to):

• Student academic success.

• Creating and maintaining a culture of success.

• Incorporating partners for success into our operations.

Policies regarding curriculum do not address the development or alignment of the written, taught, and tested curriculum but assign the superintendent the responsibility to “formulate procedures for the development and use of curriculum guides.” Board policy does require the professional staff to teach the board adopted curriculum, “Teachers will adhere to the guides.” Regardless of statements in board policy, principals reportedly do not all follow every district-developed curriculum. “Not all principals are on the same page. Principals are making decisions going against what the central office is deciding.” (Teacher)

Description of other roles and responsibilities related to the most critical function of curriculum design and delivery are not included in policy. Expectations and guidelines that assure control over these functions and their coordination were not found in the district’s board policies. Auditors noted that many board policies are vaguely worded, focus on tasks to be completed, and do not tie tasks to student achievement results. Rather, policies include generic statements without specific expectations for implementation. Policies often imply position statements but then provide the latitude for employees to choose to adhere to the policies or, in some instances, provide an avenue for school sites to propose alternative directions.

2. Direction: Board Policy IGA: Curriculum Development requires that “It is essential that the school system continually develop and modify its curriculum to meet changing needs.” Board Policy IGE


establishes that “Curriculum guides shall be developed for the various subject areas.” Policy does direct the superintendent “to develop proposals relating to curriculum modifications and additions that…are essential to the maintenance of a high quality program of education, from prekindergarten (PreK) through grade eight.”

Board Policy CBA: Qualifications and Duties of the Superintendent outlines the responsibility to “evaluate the educational program” but does not specifically refer to curriculum and does not provide specific references to what components of the instructional program are to be evaluated, how they are to be assessed for effectiveness, or the level of rigor expected by the board.

3. Connectivity and Equity: None of the policies addresses the congruence of curriculum design such as components, spiraling predictability of learning objectives, articulation, and coordination. Policies do not explicitly require connectivity—vertically or horizontally. “Equal educational opportunity” is directed in board policy but lacks the specificity describing how access to such opportunities will be assured, and ow it will be supported through aligned delivery of the adopted curriculum. No policy requires site and district staff to monitor the delivery of the curriculum to ensure that equal opportunities are provided to every student.

4. Feedback: Auditors found almost no board policies that reference local formative or summative assessment of student learning aligned to a board adopted district-wide curriculum. Board Policy ILB: Test/Assessment (State-Mandated Testing) directs responsibilities for the administration of state tests only. No guidance is provided by policy for a district assessment plan, the selection of aligned formative assessments and their implementation, the collection and storage of student assessment data, or the use of such data to inform decisions regarding classroom instruction, evaluation of the curriculum, or student learning.

The intent and expectations for yearly reports from the superintendent to the board about program effectiveness are vague and unclear.

5. Productivity: Auditors found no board policies that directed the development of a budgeting process incorporating performance. The district lacks a program-centered, performance-based budget process tied to curriculum priorities and documented needs. Multiple interventions have been initiated but, lacking policy direction, programs are constantly added with no evaluation process to determine of previous programs should be modified, continued, or terminated.

The district has consistently acquired and disseminated programs to schools designed to improve student performance without corresponding data to inform its decisions. Multiple intervention programs have been initiated in the district but have no foundation in board policy.

Policies related to the instructional use of technology are vague and generic regarding the value and potential of such technology use. Board policy does not adequately direct the development, implementation, integration, and evaluation of a district technology plan for the improvement of student achievement or increased efficiency of business and management functions. Board policies are limited to rules and processes for acceptable use of technology and are void of any direction regarding the use and value of instructional technology to enhance and increase student learning.

Policy Use

After reviewing various documents, auditors engaged in interviews with governing board members, school and central office administrators, and staff members to determine the extent to which policies are used to inform ongoing operational and educational decision-making. Examples of relevant comments related to policy use and board policy leadership were:

• “Some of the schools do their own thing despite the fact that this may not fall in line with the district’s vision.” (Teacher Survey Response)

• “There is not a lot of accountability for things. People kind of do their own thing. I see a vast difference between schools.” (Central Office Administrator)


• “There are no common messages and everyone is doing something different.” (Support Staff)

• “No one holding people accountable. Not everyone is doing the same thing.” (Board Member)

• “A lot of what happens in this district is reaction.” (Central Office Administrator)

• “Every year it’s something new, but nothing gets eliminated. No curriculum lasts long enough to see results…and the few curricula that have stood the test of time have not received positive marks in terms of test scores.” (Teacher Survey Response)

• “We’re dropping the ball and wasting a lot of money by making poor choices in our curriculum.” (Central Office Administrator)

• [Something we need to change is…] “Holding principals and schools accountable for following curriculum.” (Teacher Survey Response)

The interview and survey comments indicate a district in which decision making and direction are left to individuals, rather than being guided by policy expectations and district-wide priorities and goals (see also Finding 1.3).

Phoenix #1 Superintendent insists that all decisions enhance student learning.

Summary

The auditors found that board policies were inadequate to direct curriculum management and operational decision-making, and to establish a framework for quality control. Most policies are general in nature and are state requirements. They provide no specificity from the local board to direct student learning. Some are so general in language that direction and expectations can not readily be identified.

A lack of clarity pervades the district as related to the expectations of the board to direct a consistent written curriculum for all students within the district.

The auditors compared governing policies to audit criteria for quality in the areas of control, direction, connectivity and equity, feedback, and productivity. Auditors rated nine (12%) of the characteristics as adequate. Therefore, the board policies of the Phoenix Elementary School District #1 are inadequate to guide a sound curriculum management system. Board policies are inadequate to direct the superintendent and staff for effective management of curriculum and other district functions. Policies fell short of audit expectations in all areas of quality management.


Finding 1.3: Evidence of planning was found in the district, but system planning did not direct key district functions to promote the change needed to close existing gaps in student achievement. Planning is inadequate to direct curriculum management, assessment, professional development, technology, and budgeting to ensure system-wide congruence and drive improvement in student achievement.

The needs of society and students are continually evolving. A characteristic of an effective school system is the ability to consistently engage in long- and short-range planning focused on the attainment of agreed-upon goals and priorities. The planning function in a school system serves to chart the course for progress. Structured planning establishes the vision and mission for all district efforts and affords the district an opportunity to assess and reassess its beliefs, values, commitments, and resources in terms of its vision and mission.

District planning is a process by which district officers envision the district’s future and develop the necessary procedures and operations to achieve that future. Embedded in this planning is the ability to modify and adjust direction based upon student needs, new legislation, or changes in the community as district officers identify, prioritize, and respond to the continually evolving needs of those it serves. The planning process assists a district to anticipate emerging needs, develop a framework for systemic action toward the attainment of organizational goals, and strategically focus activities that create the future. The focus of planning is about how senior officers can adjust to circumstances in planning. Planning is about the ability of leadership to prepare for the future and to carry out that planning. In this process multiple data sources are used. Decisions are made with clear future goals in mind.

Auditors reviewed a number of documents in order to understand how Phoenix Elementary School District #1 carries out the planning process. Exhibit 1.3.1 lists the key planning documents reviewed by auditors.

Exhibit 1.3.1

Key District Planning Documents Reviewed by AuditorsPhoenix Elementary School District #1

May 2017

Phoenix Elementary School District #1 Board Policies, Regulations, Exhibits VariousPhoenix Elementary School District #1 “Our Blueprint for Student Success” Revised July 2013

Working copy of Strategic Plan: “ Blueprint for Student Success Action Plan” Update and revision 2016-17

District Continuous Improvement Plan 2016-17School Title I Continuous Improvement Plans 2015-16School Mission and Vision Statements UndatedPhoenix Elementary School District #1 Technology Plan January 2013English Language Learner (ELL) Program Philosophy UndatedSpecial Education: Guided Implementation Plan Revised March 2017

In addition, interviews were conducted with board members, central office administrators, campus level administrators, parents, and teachers regarding planning in the district. An anonymous survey of district teachers was also conducted by auditors. Overall, the auditors found planning is a new emphasis district-wide with a strategic planning process underway. However, planning has not been effective in coordinating system functions and in improving student achievement. Existing plans are inadequate.

A number of board policies outline goals and priority objectives for various departments. Auditors reviewed those policies as a component of the district’s planning process.

• Board Policy ADA: School District Goals and Objectives requires the creation of a strategic plan that “outlines the long term goals and objectives of the District.”

• Board Policy BA: School Board Operational Goals directs that “the Board accepts the responsibility to identify community attitudes and opinions and District needs and to require short- and long-range strategies that are responsive within the budgetary limitations of the District.”


• Board Policy CA: Administration Goals/Priority Objectives includes an expectation for “the administration to specialize in:

○ the processes of decision making and communication;

○ planning, organizing, implementing, and evaluating educational programs; and

○ the demonstration of educational leadership.”

• Board Policy DA: Fiscal Management Goals/Priority Objectives states, “The Governing Board recognizes that money and its management constitute the foundation of the entire school program. To make that support as effective as possible, the Board intends to: Encourage short- and long-range planning through the best possible budgeting procedures.”

• Board Policy EA: Support Services Goals/Priority Objectives directs a goal, “To provide services, resources, and assistance responsive to the needs of the educational program.”

• Board Policy FA: Facilities Development Goals/Priority Objectives places a “Priority in the development of facilities shall be based on identified educational needs and on programs developed to meet those needs. The Board establishes these broad goals for development: To integrate facilities planning with other aspects of planning in a comprehensive educational program.”

• Board Policy GA: Personnel Goals/Priority Objectives states that “The Board recognizes that dynamic and efficient staff members dedicated to education are necessary to maintain a constantly improving educational program.”

• Board Policy IA: Instructional Goals and Objectives states that “The District will provide basic communication and computational skills and the exploration of different disciplines and decision-making techniques to enable the student to choose between alternatives.” However, the policy makes no mention of curriculum planning direction.

• Board Policy CBA: Qualifications and Duties of the Superintendent, however, did not include any specific expectation that the superintendent engage the district in short- or long-range planning, either district-wide or by discrete departments. The superintendent’s job description written in 1996 did reference that the superintendent was to “develop both short and long range planning” but was absent any reference to planning for distinct department responsibilities in personnel, students, instruction, auxiliary services, properties, or financial and business.

Three different types of analyses were used by the auditors in this finding. The first analysis deals with the district planning process as a whole. This analysis looks at the planning function within the district and how it is carried out at various levels of the system. The second type of analysis looks at what the district considers its current key planning document, the Phoenix Elementary School District #1 Strategic Plan: Our Blueprint for Student Success. The third analysis examines the integration and alignment of department and campus improvement plans with the district improvement plan.

The following details the auditors’ findings on the three types of analysis.

I. The quality of planning design, deployment, and delivery is inadequate to achieve the vision of planning.

The auditors found documented evidence of planning over time. The district’s Strategic Plan (2012) demonstrated extensive planning and widespread involvement of various stakeholders: students, alumni, staff, parents, community members, board members, and organizational leaders. The district is in the process of developing a new strategic plan but is awaiting the results of the curriculum audit.

The working groups completed the planning activities and developed three major goals aligned with the focus outlined in Board Policy ADA:

• Student academic success,

• Creating and maintaining a culture of success, and

• Incorporating partners for success into our operations.


To determine the quality of the planning function, the auditors used the CMIM characteristics of quality planning for design, deployment, and delivery. This analysis approached the planning function across the district, at the central office level, across divisions and departments, and at school sites. In order for the auditors to rate the overall quality as adequate, at least six of the eight characteristics must receive a rating of adequate. Exhibit 1.3.2 lists the audit characteristics for examining a school district’s planning effort and the auditors’ ratings.

Exhibit 1.3.2

Level I: Rating Characteristics of Quality Planning Design, Deployment, and DeliveryPhoenix Elementary School District #1

May 2017

There is evidence that…Auditors’ Rating

Adequate Inadequate1. Policy Expectations: The governing board has placed into policy the

expectation that the superintendent and staff collectively discuss the future and that this thinking should take some tangible form without prescribing a particular template, allowing for flexibility as needed.

Partial*

2. Vision/Direction: Leadership has implicit or explicit vision of the general direction in which the organization is going for improvement purposes. That vision emerges from having considered future changes in the organizational context.

X

3. Data-driven: Data influence the planning and system directions/initiatives. X4. Budget Timing: Budget planning for change is done in concert with other

planning, with goals and actions from those plans driving the budget planning. X

5. Day-to-Day Decisions: Leadership makes day-to-day decisions regarding the implicit or explicit direction of the system and facilitates movement toward the planned direction.

X

6. Emergent/Fluid Planning: Leadership is able to adjust discrepancies between current status and desired status, facilitates movement toward the desired status, and is fluid in planning efforts (emergent in nature).

X

7. Deliberate Articulated Actions: Staff is involved in a purposeful way through such efforts as school/unit improvement planning, professional development councils, and district task forces that are congruent with the articulated direction of the system or system initiatives.

Partial*

8. Aligned Professional Development: Professional development endeavors are aligned to system planning goals and initiatives. X

Total 1 7Percentage of Adequacy 13%

*Partial ratings are tallied as inadequate.©2013 CMSi

Exhibit 1.3.2 reflects that the district’s planning process satisfied one (13%) of the eight audit criteria.

The following discussion provides more information on what auditors found with respect to each of the characteristics above.


Characteristic 1: Policy Expectation

The Governing Board of Phoenix #1 clearly directed and supported the development of a district-wide strategic plan but auditors found no written policies requiring planning as a discrete function. While annual analysis of student performance data is essential to planning for continuous improvement, this is only one aspect of the planning function in a district.

The scope of quality planning includes an analysis of future factors that may impact enrollment, demographics, policy, delivery structures to meet enhanced curriculum requirements or changing student needs, and the development of initiatives to prepare students to meet state and local expectations. Such planning engages the board and district staff in examining emerging needs to develop a framework for systemic action through organizational goals that are strategically focused on guiding growth and improvement for an anticipated future. This characteristic was only partially met.

Characteristic 2: Vision/Direction

The governing board of Phoenix Elementary #1 has established a district mission statement. The board states that its mission is, “Phoenix Elementary School District #1 provides each child we serve with educational experiences that ensure academic and social success. Our mission is aligned with state academic standards that include: Every child performing at grade level or above in reading, writing, mathematics, science and social studies. Every child prepared for high school.” The policy falls short of establishing an expectation that staff continuously analyze future challenges and reflect necessary contextual changes in a revised vision of the educational program. This characteristic was met, although it should be noted that each individual school has its own mission and vision statements that do not all align with the district vision and mission statements.

Characteristic 3: Data-Driven

Board policies do not require that data be analyzed in planning annual goals for program improvement at the district and campus levels, and data use was not seen as influencing system direction and possible initiatives in response to data. Auditors noted that student performance results over the past several years have not prompted the district to make systematic changes based on continuing gaps in achievement for all students compared to state results and lower achievement levels of subpopulations (Hispanic learners, special education, low socioeconomic students). The district’s response has been the proliferation of interventions programs and services rather than the identification of the causes for low performance and a focus on improvement of instruction for students. Data-driven districts use such data to take action to move the system toward improved results. This characteristic was not met.

Characteristic 4: Budget Timing

Auditors found no board policy that directed the development of the district’s budget. Auditors reviewed the district’s budget development process did not require coordination with other planning goals and actions. This characteristic was not met.

Characteristic 5: Day-to-Day Decisions

Auditors found insufficient evidence that board policy, mission statement, or district and campus plans were used as driving forces for day-to-day decisions. Auditors noted that, except for the strategic plan, the planning function in the district was limited. Planning did not extend to other functions within the district such as the design of curriculum to direct delivery to a student population characterized by increasing poverty; comprehensive assessment and use of data to improve student performance; professional development differentiated by identified teacher/program need; and program design to respond to the significant variations among students PreK-8. This characteristic was not met.


Characteristic 6: Emergent Fluid Planning

There were no processes in place to facilitate movement of the system from its current state toward future goals. The district is in the process of developing a new strategic plan. The concept of forecasting in planning practices was not previously part of the district’s culture. This characteristic was not met.

Characteristic 7: Deliberate Articulated Actions

Policy-directed planning was limited. No policy or administrative action established a requirement for professional staff engagement in planning to direct system initiatives other than the development of campus improvement plans. Activities in the district showed limited use of performance data to take deliberate action to close the gaps in student achievement. Personnel in the district acknowledge the existence of the gaps in student achievement, but responses have been anecdotal rather than analytical in determining the causes of low student performance. New planning efforts centered on developing a new strategic plan. This characteristic was partially met.

Characteristic 8: Aligned Professional Development

Providing professional development is part of the district’s strategic plan. Professional development objectives in the plan are generic and lack specificity. These include:

• Implement and cultivate Professional Learning Communities (PLCs),

• Apply Fundamentals of Teaching for Learning,

• Build content expertise and capacity for instructional staff to provide relevant and rigorous instruction, and

• Conduct Action Research to determine effectiveness of instructional practices.

These objectives and strategies do not focus on student learning, are not driven by student performance data, nor address the individual needs of professional staff.

Professional development that is congruent with system planning goals is offered, but not mandatory in all areas. Current practice indicates a lack of focus for professional development across the district. Measures of successful professional development are cited in the strategic plan but do not assess the impact of professional development on student learning. Rather, the measures of success are based on surveys indicating the level of participation. This characteristic was not met.

Summary

The district does not have an articulated planning process in place. Existing planning practices lack focus, are not responsive to existing realities in student achievement, are poorly designed and implemented, and do not address a means of evaluating results.

II. District plans are inadequate to direct change and improvement in the district.

If the auditors find planning in the system, they then proceed to determine if there are plans, and if there are, they examine these plan documents for certain change characteristics. Planning was found in the district, and the district Continuous Improvement Plan 2016-2017 School Year was assessed. Overall, the plan was found to be inadequate to direct system change.

Exhibit 1.3.3 lists the Curriculum Audit™ characteristics of a quality planning document and the auditors’ assessment of adequacy. If the plan meets five of the seven characteristics, the audit criteria for adequacy have been met.


Exhibit 1.3.3

Level II: Rating Characteristics of District-wide Plan Quality For Design, Deployment, and DeliveryPhoenix Elementary School District #1

May 2017

Characteristics of Plan QualityAuditors’ Rating

Adequate Inadequate1. Reasonable and Clear: The plan is reasonable; it has a feasible number of goals

and objectives for the resources (financial, time, people) available. Moreover, the goals and objectives are clear and measurable.

X

2. Emergent/Fluid: The plan allows for emergent thinking, trends, and changes that impact the system both internally and externally. X

3. Change Strategies: The plan incorporates and focuses on those action strategies/interventions that are built around effective change strategies (e.g., capacity building of appropriate staff).

Partial*

4. Deployment Strategies: The plan clearly delineates strategies to be used to support deploying the steps and tasks outlined in the plan (e.g., orientation to the change, staff development on the proficiencies needed to bring about the change, communication regarding planned change).

X

5. Integration of Goals and Actions: All goals and actions in the plan are interrelated and congruent with one another. X

6. Evaluation Plan and Implementation: There is a written plan to evaluate whether the objectives of the plan have been met (not to evaluate whether or not the activities have taken place). Evaluation components of plans are actions to be implemented; plans are evaluated for their effects or results, and they are then modified as needed. There is both frequent formative evaluation and annual summative evaluation, so that plans are revised as needed.

X

7. Monitoring: Systems are in place and are being implemented for assessing the status of activities, analyzing the results, and reporting the outcomes that take place as the plan is designed and implemented.

X



As can be noted in Exhibit 1.3.3, all of the seven district-wide plan quality characteristics were rated as inadequate. One characteristic was rated as partially meeting the criteria. Partial ratings are tallied as inadequate. The following comments provide more information on what the auditors found with respect to each of the characteristics above.

Characteristic 1: Reasonable and Clear

Six goal statements form the basis for the district Continuous Improvement Plan. Goals are organized under the following six standards:

1. Continuous Improvement

2. LEA Leadership

3. Curriculum and Instructional Systems

4. Supplemental Supports & Interventions

5. Data, Assessment and Evaluation

6. Stakeholder Relations


For each goal, a number of strategies and action steps were developed. Auditors found 28 strategies delineated with 155 total action steps. The excessive number of expected strategies and action steps diminishes the clarity of purpose for the stated goals. Auditors concluded that the structure of the plan made it unreasonable for practical use. Expected results and actions steps were too numerous, lacked specificity, and were not measurable in terms of student learning results. This characteristic was not met.

Characteristic 2: Emergent/Fluid

The Phoenix Elementary School District #1 Continuous Improvement Plan does not anticipate future challenges or external influences impacting the management of curriculum. The plan is based on quantifiable data from state tests and district processes. No forecast data were used showing the possible impact of internal or external forces on preparing students for future challenges. The plan does not demonstrate a planning process that will allow for emergent thinking, trends, and changes that impact the system.

Discussion throughout the system centers on how to improve student performance, especially on the AzMERIT. Because of the decline in student population that the district has experienced over the past several years and the projections for continued decline, this issue has dominated much of the efforts in the system. During this time, there has been an effort to make teaching and learning the primary focus within the system. Past practice was to give principals and professional staff an abundance of discretion in making decisions about how best to improve teaching and learning in the schools. Initiatives are being designed and implemented at all levels within the system without controls in place to assure their congruence with district priorities.

Because of the level of student performance, the district must now implement strategies that will result in the desired student growth targets. The improvement plan does not provide the actions to be taken to reach the desired end results. Likewise, the planning process demonstrated in the plan must be replicated for the development of plans in other areas: curriculum management planning, an assessment plan, professional development plan, technology plan, budget plan, etc.

There is discussion about emphasizing teaching and learning as the primary focus of the district. Presently, district efforts focus on holding students accountable for their learning without discussion on improving the delivery of the desired curriculum. The district focuses on intervention efforts rather than steps to take to ensure that learning occurs following initial instruction. Auditors received comments during interviews complaining about new initiatives, many of which have been initiated by the central office. Sometimes the efforts are not well coordinated and communicated to the campuses. Auditors determined that this characteristic was not met.

Characteristic 3: Change Strategies

Specific change strategies are not included in the plan although the initial planning process has developed a process of effective change strategies. Professional development activities are addressed in the plan, but often the process of identifying needed trainings are not congruent with efforts to improve student learning results. This characteristic was partially met.

Characteristic 4: Deployment Strategies

The plan does not address strategies to be used to deploy the steps of the plan or to build staff capacity to carry out plan goals and action steps. There was no indication of how the intent of the plan was communicated. This characteristic was not met.

Characteristic 5: Integration of Goals and Actions

Auditors found a lack of congruence among sections of the plan and among action steps within each section. The design of the plan mainly identifies activities/actions for each of the strategies with minimal implicit or explicit linkage to one another. Most of the activities in each section were not aligned with action steps in other sections. In most cases, each of the sections had a different emphasis. The goals in the district improvement plan do not align with the goals being proposed by the district’s Strategic Planning Team. The district’s planning efforts are not aligned or congruent with each other. This characteristic was not met.


Characteristic 6: Evaluation Plan and Implementation

The system does not have a plan to evaluate the goals and objectives of the district improvement plan. The action steps predominantly reflect activities not result statements. These steps are not strategic, measurable, or results oriented. They do not lead the system to modify and adjust its improvement effort, rather to simply assert whether particular actions have taken place. Many of the action steps are simply tasks to be completed, services provided to staff, and routine activities (e.g., advertise, attend, ensure, purchase, review, implement, etc.), none of which places a focus or priority on improving student learning.

The data collected are not sufficient for the district to assess to determine any specific cause for student performance. There were no provisions in the plan to evaluate whether or not the plan as a whole made a difference or was instrumental in attaining its intended purpose. This characteristic was not met.

Characteristic 7: Monitoring

Auditors found that the Continuous Improvement Plan assigns specific staff members responsibility for each action step but was missing an evaluation component and timeline. No formal measurement tools were cited in the plan and there was no common alignment with student or program assessment data. The plan did not include a process to analyze the results of the CIP annually or provide reports to the board on the design and implementation of the plan. This characteristic was not met.

Summary

Auditors found that the existing Phoenix Elementary School District #1 Continuous Improvement Plan 2016-2017 in its entirety was not comprehensive or specific enough, despite the number of action steps, to impact system-wide change and influence improvement in student performance. The plan completely met none of the audit quality criteria, but one was partially met. The plan lacked connectivity among district goals, student performance data, plan goals, and specific, measurable steps that were feasible and focused on improved learning for all students.

III. Department and campus improvement efforts from district plan documents do not meet audit criteria for bring about change.

The last type of analysis conducted by auditors was to review school continuous improvement plans. The same types of characteristics used for the district-wide plan are used in addition to the tracing of connectivity to the district-wide plan. In order for maximum impact of the planning process to be achieved, there needs to be a tight line of control that provides the necessary structure throughout the district planning efforts and still allows for creativity and flexibility at all levels. When properly structured, this planning process reduces slack within the system. Slack occurs when connections among divisions, departments, and schools are not clearly defined.

It is essential that functions related to curriculum management, professional development, program evaluation, and school improvement plans be guided by board policy and that they adhere to the administrative regulations that provide the backbone for these operations. These connections are especially important when a district is experiencing enrollment change and frequent alterations in administrative roles. Administrators within the system should be able to consult board policies and administrative regulations for guidance in how the planning functions of the system relate to job responsibilities and for gaining an understanding of the parameters within which they should operate. Planning efforts across all standards were reviewed (see Findings 1.3, 2.1, 3.1, 4.3, 5.1, 5.3, and 5.4).

School improvement planning is a rational approach to dealing with problems that require attention over an extended period to prepare for anticipated events and for limiting the negative impact of an uncertain future. A sound school improvement plan, representing the best judgment of stakeholders, provides the necessary blueprint for applying school and district resources to programs designed to attain and maintain high student achievement. When such planning is not conducted, goals may not be attained and resources may be wasted on inappropriate and ineffective programs. Meanwhile, the staff must conduct day-to-day operations without adequate direction.

To assess the quality of school improvement plans, the audit team examined documents that guide planning and a sample of campus improvement plans. They interviewed principals, staff representatives, and central office staff.


School improvement plans in the Phoenix Elementary School District #1 are produced in response to mandated federal requirements for Title I funding. The Title I Schoolwide Plan is developed to meet the Arizona Title I Continuous School Improvement Plan requirement. Auditors determined that there are no board policies that outline the requirements for school improvement plans. Twelve district schools had developed CIPs for their Title I program. (Shaw Montessori did not receive Title I funding and was not required to complete a plan.) Ten of the schools listed only one goal, “Continuous Improvement.” This goal topologically mirrored Goal 1 of the district Continuous Improvement Plan. However, the district CIP listed five additional goals. Two schools cited two additional goals, not aligned to the district plan. To assess the quality of school planning, auditors examined the school improvement plans.

Overall, auditors found school improvement plans and planning processes do not meet audit quality criteria for such plans. All elements of campus plans lacked focus, adequate specificity to direct staff action, interconnections, response to data analyzed, monitoring for implementation, and evaluation of the plan’s impact on student achievement.

Eight characteristics of quality department and campus improvement plans are rated as adequate or inadequate. To meet audit standards, six of the eight criteria must be determined to be adequate. Exhibit 1.3.4 presents the eight criteria and the auditors’ ratings on the various plans as a whole.

Exhibit 1.3.4

Level III: Rating Characteristics of Department and School Improvement Plan Quality For Design, Deployment, and DeliveryPhoenix Elementary School District #1

May 2017

Characteristics of Plan QualityAuditors’ Rating

Adequate Inadequate1. Congruence and Connectivity: Goals and actions are derived from, explicitly linked

to, and congruent with the district plan’s goals, objectives, and priorities. X

2. Reasonable and Clear: The plan is reasonable; it has a feasible number of goals and objectives for the resources available (finances, time, people). The goals and objectives of the plan are clear and measurable.

X

3. Emergent/Fluid: The plan allows for emergent thinking, trends, and changes that impact the system both internally and externally. X

4. Change Strategies: The plan incorporates and focuses on those action strategies/interventions that are built around effective change strategies (e.g., capacity building of appropriate staff).

X

5. Deployment Strategies: The plan clearly delineates strategies to be used to support deploying the steps and tasks outlined in the plan (e.g., orientation to the change, staff development on the proficiencies needed to bring about the change, communication regarding planned change).

X

6. Integration of Goals and Actions: All goals and actions in the plan are interrelated and congruent with one another. X

7. Evaluation Plan and Implementation: There is a written plan to evaluate whether the objectives of the plan have been met (not to evaluate whether or not the activities have taken place). Evaluation components of plans are actions to be implemented; plans are evaluated for their effects or results and modified as needed. There is both frequent formative evaluation and summative evaluation, so that plans are revised as needed.

X

8. Monitoring: Systems are in place and are being implemented for assessing the status of activities, analyzing the results, and reporting outcomes that take place as the plan is designed and implemented.

X


©2013 CMSi


As can be noted in Exhibit 1.3.4, none of the eight characteristics was rated adequate, and the plans collectively are considered inadequate. The following provides more information on what the auditors found with respect to each of the characteristics above.

Characteristic 1: Congruence and Connectivity

Generally, planning documents reviewed by the auditors are explicitly linked to the district goals and objectives. In areas where there are no formalized plans (e.g., curriculum management, assessment, and professional development), there are some implicit efforts to link activities to the district goals, but it was not clear how these efforts took place across departments, divisions, and schools. For instance, in professional development, there is no plan to coordinate the trainings in any systematic manner. The trainings may or may not be connected to the district plan. Pieces of information regarding a curriculum management plan are present, but are not collected into a single comprehensive document outlining process, procedures, expectations, and rules (see Finding 2.1). The most explicit linkage was found with the school improvement plans. This characteristic was not met.

Characteristic 2: Reasonable and Clear

School improvement plans have a variety of stated action steps that address the goals and expected result statements. The structure, however, does not mirror the district improvement plan with the varied actions to be undertaken by the school staff. The cited resources generally note the cost and funding resource for the stated activities. As a group, school plans do not identify these elements in sufficient detail to be useful as guides to action.

Auditors found that, although school plans had a singular goal, the improvement plans had multiple strategies (from five to 15) and relatively excessive action steps (from 27 to 139). The average number of action steps was 64 per site, which may make the plans unmanageable in conjunction with a school’s daily routine.

Overall, school improvement plans did not satisfy this criterion. This characteristic was not met.

Characteristic 3: Emergent/Fluid

There is no evidence of trend analyses impacting schools either internally or externally. The opportunity for emergent fluid planning exists within the planning process. At the campus level, each year the school improvement planning teams are able to review and revise plans when data suggest that modifications need to be made. The use of a needs assessment tool was not designed to be comprehensive in terms of emergent realities to allow for forecasting of future challenges and linkage between this tool and actual planning. This characteristic was not met.

Characteristic 4: Change Strategies

At the campus level, day-to-day decisions are guided by the principal, Professional Learning Communities, the Continuous Improvement Plan, and directions provided by instructional specialists. Plans varied in the level of detail and type of action described to attain improved student achievement. Generally, plans did not detail how PLCs will be incorporated into actions to be taken or how they will interface with the campus improvement committee. Departmental plans and school improvement plans did not focus on change strategies for bringing about desired results.

Campus plans did not include strategies for effective change. Most actions within the plans were not designed to initiate or sustain change over time to target specific needs. Building staff capacity was not clearly evident in many professional development offerings within campus plans. This characteristic was not met.

Characteristic 5: Deployment Strategies

The campus improvement plans do not address deployment strategies. While the plans followed a format that identified start and end dates, action steps, action step descriptions, resources needed, person(s) responsible, estimated costs, funding source(s), and evaluation tools, all of these elements were discrete with no focus on how staff would be prepared and supported during implementation. Specific strategies for communicating the planned changes to the school community were absent. Deployment strategies were not built into any of the


plan analyzed by auditors. The proficiencies needed to bring about change are not specified in all plans. This characteristic was not met.

Characteristic 6: Integration of Goals and Actions

The integration of goals and actions within each plan are not evident. Sections of the plans were disconnected—e.g., professional development is not congruent with the performance goals, summative evaluation, and activities in each section of the plan. In most instances, there are few if any linkages. Within each section of the plan, activities described are isolated actions and showed little or no connection to the goals, expected results, or the evaluation measure. This characteristic was not met.

Characteristic 7: Evaluation Plan and Implementation

Auditors found no written intent to evaluate the goals of the plan. The assessment strategy is identified as the “Evaluation Tool” section of the plans. Inadequate ratings were assigned because the evidence of implementation strategies is not adequate to evaluate attainment of the goals and action/tasks. They simply note that activities have occurred and do not measure the results obtained by the action/tasks listed. No element of the CIPs provided for an evaluation of the district goals within the content of the school campus improvement plans.

The district has the capability of generating an overwhelming amount of data relative to program effectiveness and student achievement. However, there is no systematic approach to program evaluation and no scheduled plan for such evaluations. While schools have access to student achievement data, how a school decides to utilize that access is up to the school. This characteristic was not met.

Characteristic 8: Monitoring

Monitoring methods within the plans do not constitute a system for assessing the status of action steps while in progress or at completion. Given the immeasurability of the activities within the plans and the broad nature of the action steps cited, monitoring poses a challenge. Methods of assessment were presented in isolation, with no linkage to expected results or criteria establishing accomplishment. No procedures describe how monitoring data would be used to adjust the plan’s design during deployment or redesign. This characteristic was not met.

In interviews with district and campus administrators, board members, and teachers, the auditors heard the following comments about the district planning efforts:

• “There doesn’t seem to be any body in charge in Phoenix Elementary holding any one accountable.” (Central Office Administrator)

• “It depends who you talk to—some want to close the achievement gap and do everything possible, others don’t verbalize that. It’s not everyone’s goal.” (Central Office Administrator)

• [Campus Improvement Plan] “We are just getting ready to start talking about the new regulations. In the past years, we’ve been looking at goals, instruction, interventions, parents, and PBIS.” (Campus Administrator)

• “There is not a lot of uniformity in this district. People work very hard, but in isolation.” (Campus Administrator)

• “My sense is that we are in the space right now where we are being reflective. I just hope that we have the courage to be doing what is right.” (Central Office Administrator)

• [Campus Improvement Plan] “I initially work on it by myself, start it off then I include staff members and parents to see what we need, don’t need, and open it up to get feedback.” (Campus Administrator)

• “My goals are in alignment with what I think the strategic plan is going to prove: student achievement, family engagement, extracurricular activities, fiscal responsibilities.” (Central Office Administrator)


Summary

Recent planning in the Phoenix Elementary School District #1 has focused on the district’s strategic plan. Auditors were not provided a curriculum management plan, assessment plan, professional development plan, or budget plan. Some planning focused on meeting legal requirements for compliance, such as the Title I Continuous Improvement Plans.

No plan adequately addresses the current reality of significant gaps in student achievement (see Finding 4.1). Except for the strategic plan, overall, current planning approaches do not produce a viable planned approach to curriculum and program design capable of supporting strategic actions to close existing gaps in student achievement.

Generally, the district and school improvement plans evaluate completion of actions/activities and do not focus on improving student achievement. Continuous Improvement Plans do promote student improvement in specific categories but overall the plan does not outline specific actions to be taken to improve student performance.


STANDARD 2: The School District Has Established Clear and Valid Objectives for Students.A school system meeting this audit standard has established a clear, valid, and measurable set of pupil standards for learning and has set the objectives into a workable framework for their attainment.

Unless objectives are clear and measurable, there cannot be a cohesive effort to improve pupil achievement in the dimensions in which measurement occurs. The lack of clarity and focus denies to a school system’s educators the ability to concentrate scarce resources on priority targets. Instead, resources may be spread too thin and be ineffective in any direction. Objectives are, therefore, essential to attaining local quality control via the school board.


Common indicators the CMSi auditors expected to find are:

• A clearly established, board-adopted system-wide set of goals and objectives for all programs and courses;

• Demonstration that the system is contextual and responsive to national, state, and other expectations as evidenced in local initiatives;

• Operations set within a framework that carries out the system’s goals and objectives;

• Evidence of comprehensive, detailed, short- and long-range curriculum management planning;

• Knowledge, local validation, and use of current best practices and emerging curriculum trends;

• Written curriculum that addresses both current and future needs of students;

• Major programmatic initiatives designed to be cohesive;

• Provision of explicit direction for the superintendent and professional staff; and

• A framework that exists for systemic curricular change.


This section is an overview of the findings that follow in the area of Standard Two. Details follow within separate findings.

The findings within Standard Two are presented in the following order:

• Curriculum management planning;

• Overall scope of the curriculum in the Phoenix Elementary School District #1 (Phoenix #1);

• Overall minimal basic components for quality and specificity of curriculum document design; and

• Internal consistency and cognitive complexity of samples of Galileo assessments when compared with English language arts and mathematics Arizona College and Career Ready Standards (AZCCRS).

Auditors found some direction for curriculum management scattered among Phoenix Elementary School District #1 documents including board policies, job descriptions, and various planning documents. For example, teachers are not required in policies to teach the authorized curriculum but are required to do so in the job descriptions. However, there is no one document or formalized process that guides administrators, staff, and teachers in curriculum planning. As a result, the district does not have a means of assuring cohesion and consistency in curriculum planning. Current documents reviewed do not meet audit criteria for planned management of the design and delivery of curriculum (see Finding 2.1).

Auditors found the scope of the written curriculum provided by the district to teachers to be inadequate to direct instruction for core and non-core subjects PreKindergarten through grade 8 (see Finding 2.2).


Phoenix Elementary School District #1 curriculum documents lacked the quality and specificity of minimal basic components needed to promote a highly focused, consistent educational program in which the written, taught, and tested curriculum is aligned. When samples of Galileo English language arts and mathematics assessment items were compared with the AZCCRS, alignment was found to be inadequate for some items in content and context. No assessment items analyzed expanded or extended student thinking beyond the standards (see Finding 2.3).

Finding 2.1: Phoenix Elementary School District #1 lacks a comprehensive curriculum management plan and a documented process to direct and implement the design, delivery, revision, or evaluation of curriculum consistently across the school district.

A district that has quality control over its educational program has clearly articulated board policies that clarify and direct its expectations for the design, delivery, and evaluation of its curriculum. A curriculum management plan outlines the specific procedures that are expected in order to execute such board policy. A comprehensive curriculum management plan outlines how curriculum is written, adopted, delivered, and supported with instructional resources. This plan details the expectations, processes, tasks, and responsibilities for development, adoption, implementation, monitoring, evaluation, and revision of the curriculum.

A comprehensive curriculum management plan provides direction for staff and results in consistent and coherent practices across content areas, departments, and schools. It provides a structure for the district to communicate clearly those elements tightly held across the district, ensuring consistency where it matters, and those loosely held, allowing principals and teachers to use school-specific and creative approaches to meet individual student needs. Such a plan provides a blueprint of expectations for internal consistency, increasing the opportunity for effective delivery of a curriculum that is horizontally coordinated, vertically articulated, aligned to assessments, and systematically evaluated to determine effectiveness. Without such direction, there is a greater chance for significant variation in curriculum design, inconsistent adherence to the district-approved written curriculum, fragmented instructional focus, and a variety of monitoring practices. The audit expectations regarding those functions of curriculum management that should be tightly held and those that should be loosely held are presented in Exhibit 2.1.1.

Exhibit 2.1.1

Tightly Held vs. Loosely Held Curriculum Management Functions and Components

CMA Beliefs on Curriculum Versus InstructionEnds

(Curriculum and Aligned Assessments)Means

(Instruction and Programs)

Tightly-held (Nonnegotiable)

District Level

Loosely-held (Aligned to the Tightly-held but Negotiable by

Teacher/Faculty) School/Classroom Level

• Mission • Curriculum—Standards/Outcomes/

Student Expectations/Objectives• Priority Standards/Outcomes/Student

Expectations/Objectives (usually a deep alignment backloading)

• Aligned CRT

• Differentiation of when which students get which standards/outcomes/student expectation/objectives

• Processes, procedures, instructional strategies used• Resources, textbooks, etc.• Programs (e.g., ELL Program, Spec. Educ. Program,

Reading Program)• Groupings • Staffing• Informal assessments for diagnostic purposes

©2013 CMSi

Exhibit 2.1.1 emphasizes the need for a curriculum management plan to maintain quality control over how loosely-held components are to function within the system to maintain alignment of the written, taught, and


tested curriculum, and how the tightly-held components are to be developed, monitored, communicated to all stakeholders, and revised.

To determine if there is adequate direction for curriculum management in the district, auditors examined documents provided by the staff, including board policies and administrative regulations, job descriptions, staff memoranda, and plans. Auditors also interviewed board members, administrators, teachers, and parents.

The audit team found that some curriculum management guidance was scattered among several district documents such as board policies, job descriptions, and planning documents. Although these and other information available to the audit team contained some elements of curriculum management planning, collectively, the data sources did not include a comprehensive curriculum management plan (see Finding 1.3).

Key curriculum planning documents reviewed by the audit team are listed in Exhibit 2.1.2.

Exhibit 2.1.2

Key Curriculum Planning Documents Reviewed by AuditorsPhoenix Elementary School District #1

May 2017

Document DatedPhoenix Elementary School District #1 Board Policy, and Policy Exhibits Various

Phoenix Elementary School District #1 Our Blueprint for Student Success Revision approved July 25, 2013

Working copy of Strategic Plan: Our Blueprint for Student Success Action Plan

Pending update and revision 2016-17

District Continuous Improvement Plan 2016-17School Mission and Vision Statements UndatedPhoenix Elementary School District #1 Technology Plan January 2013Job Descriptions VariousBoard Agendas and Minutes VariousLiteracy Instruction in the Phoenix Elementary School District—Goals and Beliefs May 2013

ELA Leadership Team Session 2 February 20, 2013Session Outcomes February 19, 2013Assessing the Quality and Alignment of Literacy Instructional Materials to the Common Core Undated

Criteria for the Adoption of Instructional Material UndatedRequest for English Language Arts Instructional Materials UndatedELA Leadership Team Governing Board Information Session June 13, 2013English Language Arts (ELA) Adoption Committee UndatedPhoenix Elementary School District #1 Writing Position Statement UndatedBook Room Materials 2015-16English Language Learner (ELL) Program Philosophy UndatedPhoenix Elementary School District #1—Assessments UndatedAssessment Letter to staff, parents, and/or guardians July 2016Phoenix Elementary School District No. 1 Organizational Chart 2014-15Special Education: Guided Implementation Plan Revised March 2017Move On When Reading School K-3 Literacy Plan 2016-17Galileo/AIMSweb Test Security Agreement UndatedOnline Survey Results May 2017


Auditors reviewed board policies to determine what direction does exist for curriculum management efforts in the district. A few board policies were found that had rudimentary directives to curriculum management:

• Board Policy IGA: Curriculum Development states, “It is essential that the school system continually develop and modify its curriculum to meet changing needs. The Board authorizes the Superintendent to develop the curriculum for the school system and to organize committees to review the curriculum. All curriculum changes shall be approved by the Governing Board.”

• Board Policy IGE: Curriculum Guides and Course Outlines requires, “Curriculum guides shall be developed for the various subject areas. These guides shall present at least a minimal outline for instruction and a basis for further development of the particular courses.”

No board policy provided specific direction for the development, implementation, and monitoring of district curriculum, nor did the auditors find any policy that requires the development of a plan to direct curriculum management in the district.

Next, the auditors reviewed the Phoenix Elementary School District #1 Strategic Plan: “Our Blueprint for Student Success” and found no specific direction for curriculum management other than a guiding focus on student achievement by focusing on quality instruction and providing educational experiences for high school readiness.

The auditors examined job descriptions for administrators, principals, teachers, and other relevant positions to determine roles and responsibilities for curriculum management and fragmentation of responsibilities contributing to a lack of clarity and consistency (see Exhibit 2.1.4).

Overall, the auditors found that, although various job descriptions contained responsibilities for providing direction and alignment of curriculum initiatives and resources, there is no single policy or plan that coordinates these roles and responsibilities in conjunction with all curriculum management functions district-wide.

Few of the documents provided by the district contain elements of curriculum planning and there was no comprehensive plan per se. Auditors analyzed these elements from these documents using audit criteria that describe the characteristics of comprehensive curriculum management planning. To be considered adequate, planning elements must exhibit 11 of the 15 characteristics (at least 70%). These characteristics and the auditors’ ratings of the district documents are displayed in Exhibit 2.1.3:

Exhibit 2.1.3

Curriculum Management Plan Characteristics And Auditors’ Assessment of District Approach


Characteristics: Auditors’ RatingAdequate Inadequate

1. Describes the philosophical framework for the design of the curriculum, including such directives as standards-based, results-based, or competency-based; the alignment of the written, taught, and tested curriculum; and the approaches used in delivering the curriculum.

X

2. Identifies the timing, scope, and procedures for a periodic cycle of review of curriculum in all subject areas and at all grade levels. X

3. Defines and directs the stages of curriculum development. X4. Specifies the roles and responsibilities of the board, central office staff members,

and school-based staff members in the design and delivery of curriculum. X

5. Presents the format and components of all curriculum, assessments, and instructional guide documents. X


Exhibit 2.1.3 (continued)Curriculum Management Plan Characteristics And Auditors’ Assessment of District Approach


Characteristics: Auditors’ RatingAdequate Inadequate

6. Directs how state and national standards will be considered in the curriculum. This includes whether or not to use a backloaded approach, in which the curriculum is derived from high-stakes tested learnings (topological and/or deep alignment), and/or a frontloaded approach, which derives the curriculum from national, state, or local learnings.

X

7. Requires for every content area a focused set of precise student objectives/student expectations and standards that are reasonable in number so the student has adequate time to master the content.

X

8. Directs that curriculum documents not only specify the content of the student objectives/student expectations, but also include multiple contexts and cognitive types.

X

9. Specifies the overall beliefs and procedures governing the assessment of curriculum effectiveness. This includes curriculum-based diagnostic assessments and rubrics (as needed). Such assessments direct instructional decisions regarding student progress in mastering prerequisite concepts, skills, knowledge, and long-term mastery of the learning.

X

10. Directs curriculum to be designed so that it supports teachers’ differentiation of instructional approaches and selection of student objectives at the right level of difficulty. This ensures that those students who need prerequisite concepts, knowledge, and skills are moved ahead at an accelerated pace, and that students who have already mastered the objectives are also moved ahead at a challenging pace.

X

11. Describes the procedures teachers and administrators will follow in using assessment data to strengthen written curriculum and instructional decision making.

X

12. Outlines procedures for conducting formative and summative evaluations of programs and their corresponding curriculum content. X

13. Requires the design of a comprehensive staff development program linked to curriculum design and its delivery. X

14. Presents procedures for monitoring the delivery of curriculum. X15. Establishes a communication plan for the process of curriculum design and

delivery. X


©2013 CMSi

As indicated in Exhibit 2.1.3, none of the audit characteristics of comprehensive curriculum management planning were met by current documents; therefore, curriculum management planning in Phoenix Elementary School District #1 is inadequate. Comments are provided for each characteristic.

Characteristic 1: Describes the philosophical framework for the design of the curriculum (Inadequate)

For this characteristic auditors look for a philosophical framework for the design of curriculum including any such as standards-based, results-based, alignment of the written, taught, and tested curriculum, and approaches to the delivery of curriculum. While the auditors found direction for the intent and purposes of the educational program, no policy specifically outlined the philosophical approach for teaching and learning across the district.


Elements of a framework for the design of the curriculum were generalized in board policies and exhibits and included the following (see Finding 1.2):

• Board Policy IHA: Basic Instructional Program states, “The various instructional programs will be developed to maintain a balanced, integrated, and sequential curriculum that will serve the educational needs of all school-aged children in the District. The curriculum will be broad in scope and provide for a wide range in rate, readiness, and potential for learning.” This policy also states, “The first priority of the instructional program will be language acquisition through a planned sequence of reading skills and language experiences beginning in the kindergarten program. The second priority of the instructional program will be mastery of the fundamentals of mathematics, beginning in the kindergarten program.” The policy then describes planned sequences of the instructional program covering core and non-core subjects.

• Board Policy IA: Instructional Goals and Objectives maintains, “All parts of the curriculum are interrelated and important to the development of the student. The physical, emotional, social, aesthetic, and cognitive development of the student are all elements of importance within the school program. The District will provide basic communication and computational skills and the exploration of different disciplines and decision-making techniques to enable the student to choose between alternatives.”

• Board Policy IHA-E (Exhibit): Basic Instructional Program requires for students in Kindergarten through grade 3, the “District shall adopt a scientifically based reading curriculum including the essentials of reading instruction.”

• Board Policy IGE: Curriculum Guides and Course Outlines states, “In accordance with requirements of the state of Arizona, the District will develop its own curriculum guides to reflect alignment to the state standards and local needs.” The auditors did not find other policies referencing alignment of the written, taught, and tested curriculum (see Finding 1.2).

District beliefs in the strategic plan affirm that “All students can and will learn. NO EXCEPTIONS.” However, the auditors found no philosophy or description of what this looks like when implemented in the classroom. Overall, this criterion was not met.

Characteristic 2: Identifies the timing, scope, and procedures for a periodic cycle of review of curriculum (Inadequate)

Board Policy IGA notes that the school system should continually develop and modify its curriculum to meet changing needs. Board Policy IGE states, “The Superintendent will formulate procedures for the development and use of curriculum guides.” However, no policy or other district document was presented to the auditors that directs a periodic cycle of review specifically identifying the timing, scope, and procedures of curriculum review of the curriculum for all subjects at all grade levels or how such a review would be conducted or by whom. The auditors did not find written curriculum provided by the district to teachers to use in planning lessons for some grade level core subjects and most non-core courses (see Finding 2.2).

Characteristic 3: Defines and directs the stages of curriculum development (Inadequate)

This characteristic details the work of each stage of curriculum development. Board Policy IGA maintains that curriculum development, review, and evaluation involve students, parents, teachers, and administrators. However, board policy and job descriptions did not address the specific stages of curriculum development. Although central office administrators provided the auditors with access to developed and online curriculum documents that demonstrated the presence of some curriculum, they did not provide any documents defining or directing the stages of curriculum development.

Characteristic 4: Specifies the roles and responsibilities of the board, central office staff members, and school-based staff members (Inadequate)

The following board policies specify roles and responsibilities related to the design and delivery of curriculum:


• Board Policy IGA: Curriculum Development gives the board authority for approval of all curriculum changes and the superintendent responsibility for developing the curriculum for the school system and to organize committees to review the curriculum. Policy IGA also states, “All certificated personnel have professional obligations to the school program beyond regular classroom duties, and these obligations will include work on curriculum committees.”

• Board Policy IGD: Curriculum Adoption declares, “Curricular proposals from the professional staff may be presented to the Superintendent, who will be responsible for making recommendations to the Board on such matters.”

• Board Policy IGE specifies responsibilities for delivery of the curriculum when it states, “The Superintendent will formulate procedures for the development and use of curriculum guides.” Board Regulation IGE-R affirms, “The principal, department heads, or other supervisors shall see that optimum use is made of available curriculum guides. Teachers will adhere to the guides.”

Auditors found that many job descriptions address responsibilities for the design and delivery of curriculum (see Finding 1.1). Exhibit 2.1.4 displays a sampling of such job responsibilities:

Exhibit 2.1.4

Job Descriptions with Responsibilities for the Design and Delivery of CurriculumPhoenix Elementary School District #1

May 2017

Job Title Date Approved Responsibilities for the Design and Delivery of Curriculum

Classroom Teacher October 2001 • Teaches district-approved curriculum.• Instructs pupils in citizenship and basic subject matter specified by District

policy.• Develops lesson plans and instructional material and provides individualized

and small group instruction to adapt the curriculum to the needs of each pupil.

• Task analyzes and translates lessons into developmentally appropriate learning experiences.

Elementary School Principal

November 2003

• Be current in knowledge of curriculum development.• Assume responsibility for appropriate placement and instruction (grade level

and special programs).• Assume responsibility for assisting in the development of the curriculum and

in planning and adapting the courses of study to the needs and interests of the children.

English Language Arts Implementation Specialist

April 2013 • Assist classroom teachers to plan and implement high-quality, rigorous English Language Arts instruction.

• Support curriculum development (Pacing Guides, assessment, instructional materials alignment, etc.)

High School Implementation Specialist

May 2016 • Support collaboration efforts to align curriculum and instruction in core contents.

K-8 Assistant Principal

April 2006 • Assists the principal in curriculum and program development and evaluation. • Assist in the implementation of District instructional objectives.

Assistant Superintendent for Curriculum and Instruction

March 2003 • Give leadership to the instructional program and the major components of this division.

• Translate system-wide philosophy and goals into a coordinated curriculum for students of differing abilities and interests.

• Directs the design and delivery of curriculum in all content areas.• Ensures the alignment of curriculum, instruction and assessment.• Recommends curricular and instructional strategies to assist schools in

improving student achievement.


Exhibit 2.1.4 (continued)Job Descriptions with Responsibilities for the Design and Delivery of Curriculum


Job Title Date Approved Responsibilities for the Design and Delivery of Curriculum

Literacy Coach June 2004 • Assist teachers in the identification and/or development of curriculum to support classroom instruction.

• Recommend modification in curriculum and instructional strategies to improve language acquisition and student achievement for all students.

Math Coach June 2005 • Assists in the development of effective math intervention plans and in the implementation of the District adopted math program.

• Assist teachers in implementing standards-based instruction aligned with Arizona Math Standards for improving program quality.

Director of Special Education

December 2009

• Assist in development and implementation of curriculum and instruction for special education programs.

Director of Assessment and Accountability

July 2009 • Organize student performance data to assist schools and departments in curricular planning.

Director of Educational Technology

August 2016 • Administers the implementation of the District’s K-8 technology curriculum.• Collaborates with instructional staff, including teachers, to develop

curriculum materials and specific lesson plans that integrate technology into content.

• Models the integration of technology in all curriculum areas.Director of General Studies

May 2003 • Responsible for the design, implementation plan and assessment of the districts programs for mathematics, science, health, social studies, art, music and physical education.

• Participate in the design and oversee the implementation of general studies in summer school, after school, and during intersession.

New Teacher Mentor

June 2009 • Develops teacher content knowledge based upon state standards and local curriculum.

• Assists in the planning of instruction and curriculum implementation. Superintendent of Schools

January 1996 • Observe and develop curriculum.• Make headway in instructional objectives as per District and State guidelines.

Source: Job descriptions provided by District

Exhibit 2.1.4 indicates that current designated roles and responsibilities for the design and delivery of curriculum are spread among several positions in Phoenix #1. This fragmentation of responsibilities contributes to a lack of clarity and consistency. Reference to the specificity of the design and delivery of curriculum in all noted job descriptions was vague, although the descriptions do address some functions as they relate to curriculum management. This criterion was not adequately met.

Characteristic 5: Presents the format and components of all curriculum, assessments, and instructional guide documents (Inadequate)

This characteristic looks for documents that present the format and components of all curriculum, assessments, and instructional guide documents. Board Policy IA: Instructional Goals and Objectives and Board Policy IHA list the skills to be taught in planned sequences, but neither policy provides direction for the format or components of the district written curriculum for these skills. Board Policy IGE states, “These guides shall present at least a minimal outline for instruction…”

Central office administrators provided the auditors with access to their online curriculum documents and other documents, which did show a similar format for the components of curriculum documents for most grade levels and subjects. District documentation that describes specifically the format of curriculum, assessments, and instructional guide documents was not provided. Auditors compared PreK-8 curriculum documents provided by the district against audit criteria for minimal basic components and specificity and found them to be inadequate (see Finding 2.3).


Characteristic 6: Directs how state and national standards will be considered in the curriculum (Inadequate)

This characteristic looks for direction as to how state and national standards will be considered in the district curriculum. Board Policy IGA states the importance for the school district to continually develop and modify its curriculum to meet changing needs, but falls short of formally directing the alignment of the development of curriculum to state and national standards. Board Policy IGE does require that “In accordance with requirements of the state of Arizona, the District will develop its own curriculum guides to reflect alignment to the state standards and local needs.”

Auditors were presented with a document entitled “Literacy Instruction in the Phoenix Elementary School District Goals and Beliefs,” drafted by the ELA Leadership Team. This document states, “Literacy instruction in the Phoenix Elementary School District will be driven by the Common Core English Language Arts (ELA) standards which encompass reading, writing, listening/speaking and language. These standards provide a clear, consistent understanding of what students are expected to learn. They are designed to be robust and relevant and reflect the knowledge and skills that young people need for success in college and careers.”

One goal in the district Continuous Improvement Plan (CIP) states, “The LEA will ensure the implementation of AZCCRS by aligning instructional materials supporting documents and by providing job-embedded professional development to all staff as measured by the comprehensive needs assessment.” Action Step 1 for this goal is “Ensure all curriculum (instructional materials) and support documents (Pacing Guides) are aligned to AZCCRS.”

The district curriculum documents for the core subjects of English language arts, mathematics, science, and social studies utilize AZCCRS as the basis for the curriculum in those areas.

A letter sent to all staff and parents from the Accountability and Assessment Department, states, “Our District has created Curriculum and Pacing Guides that are linked to success and mastery on AzMERIT.” Auditors noted that these curriculum documents do not include a clear linkage to the AzMERIT.

Teachers and administrators in Phoenix #1 utilize the district website for curriculum documents. While this system holds district-developed curriculum documents used by teachers, it also houses curriculum documents provided by the Arizona Department of Education. The district Galileo assessments presented to the auditors for review contain alignment information concerning the AZCCRS (see Finding 2.3).

The audit team received no documentation specifying that curriculum development be carried out with either a backloading or a frontloading approach, nor is there any mention of national standards. This criterion was not fully met.

Characteristic 7: Requires for every content area a focused set of precise student objectives/student expectations and standards that are reasonable in number (Inadequate)

Board policy does not exist to provide guidance for the identification of student objectives. District administrators did not provide the auditors with any system requirement that the written curriculum be based on a focused set of objectives that are reasonable in number. Rather, auditors found that the objectives for mathematics and English language arts were directly derived from the expectations found within the state and Arizona Standards for College and Career Readiness, with no refinement so these standards are more specific and measurable.

Characteristic 8: Directs that curriculum documents not only specify the content of the student objectives/student expectations, but also include multiple contexts and cognitive types (Inadequate)

Auditors did not find board policy that provides direction or sets expectation for a rigorous curriculum that includes not only expectations for content mastery, but also describes the contexts in which students practice their learning and demonstrate that mastery and the ways that they should be cognitively engaged in the classroom. The auditors found no documentation requiring that learning objectives be written to include multiple contexts and cognitive types. The audit team examined a sampling of the district Galileo assessment items. Fifty percent of the assessment items are written in such a way as to require understanding (Revised Blooms Taxonomy) in student thinking (see Finding 2.3). In addition, auditors did not observe a wide range of cognitive types during observations in district classrooms (see Finding 3.2).


Characteristic 9: Specifies the overall beliefs and procedures governing the assessment of curriculum effectiveness (Inadequate)

District administrators did not provide the auditors with documents clearly describing the beliefs and procedures for assessing the effectiveness of the district curriculum across all grade levels. Board Policy IHA-E (Exhibit) calls for the district to “Conduct a curricular evaluation” of Kindergarten through grade 3 and then “review reading progress and take corrective action as specified by the State Board of Education whenever more than 20% of the third grade students do not meet the AIMS standards.” The auditors found no evidence of any plan or policy other than Policy Exhibit IHA-E that directs how student progress in mastering the curriculum should be evaluated and the results addressed through classroom instruction.

No assessments were presented that measure prerequisite skills or long-term mastery of content.

A letter sent to all staff, students and parents from the district’s Accountability and Assessment Department in July, 2016 states, “Our District Assessment Plan is designed to provide students, parents and teachers with valid, timely and reliable testing results regarding literacy and the mastery of Arizona’s College and Career Ready Standards to improve teaching and learning in our schools.” This letter further states, “Our District has created Curriculum and Pacing Guides that are linked to success and mastery on AzMERIT. These guides’ help teachers plan instruction that helps students meet the State standards. Selected Performance Objectives are assessed for Kindergarten – grade 8 on Galileo ELA and Math Benchmark tests three times prior to the spring AzMERIT assessment and there is an additional post-test after the state assessments to demonstrate individual student growth and mastery from the pre- assessment. Science is also assessed for 4th & 8th graders to prepare them for the AIMS Science test. The information from these tests is used by classroom teachers to determine whether students are prepared to master the state standards. These data are also used to determine whether students are likely to pass AzMERIT & AIMS Science.” During analyses of district curriculum documents, auditors found no direct linkage to AzMERIT or AIMS (see Finding 2.3).

Strategic Plan: Our Blueprint for Student Success Action Plan (working copy) cites as one objective, “Teachers, students and parents are provided frequent opportunities to assess and guide student learning.” Strategies for accomplishing this objective include:

• Collect and analyze data

○ Formative Assessments

○ Chapter Tests

○ Measurement of Progress toward Standards

○ Portfolios of Student Work

• Summative Assessments

○ End of year State Test.

Documents reviewed do not specify overall beliefs concerning the assessment of curriculum effectiveness. The district’s assessment approaches and procedures are not an integrated component of a comprehensive curriculum management plan or assessment plan (see Finding 4.3). This criterion was not adequately met.

Characteristic 10: Directs curriculum to be designed so that it supports teachers’ differentiation of instructional approaches and selection of student objectives at the right level of difficulty (Inadequate)

District administrators did not provide the auditors with any documents directing the inclusion of differentiated instructional approaches in the written curriculum. Board policy does not reference differentiating instruction to meet the learning needs of all students. The job descriptions for campus administrators stated their responsibility to encourage differentiated instruction for all learners. No instructional approaches indicating differentiation were included in any curriculum documents that were provided to the auditors. Auditors did not find integrated, culturally responsive approaches in the current curriculum documents, nor any mention of suggestions for regrouping, re-teaching, or accelerating content for students.


The teacher job description requires the teacher to provide differentiation as follows:

• Develops lesson plans and instructional material and provides individualized and small group instruction to adapt the curriculum to the needs of each pupil.

• Task analyzes and translates lessons into developmentally appropriate learning experiences.

• Identifies pupil needs and makes appropriate referrals and develops strategies for individual education plans.

• Provides individualized and small group instruction to adapt the curriculum to the needs of the student.

Despite the lack of support in district curriculum for differentiation, there appears to be a culture among school-based personnel that acknowledges the need to use data in planning instruction. Teachers and principals reported using programs that they considered to be differentiated, such as Imagine Learning. Auditors noted that at some schools all students spend time on Imagine Learning, not just those students needing intervention. Auditors did not consistently observe varied groupings and differentiation of curriculum during classrooms visitations (see Finding 3.2).

The audit team noted that few Phoenix #1 core curriculum documents reviewed contained suggested instructional strategies and resources meant to meet the needs of English language learners and struggling students. In some curriculum documents, auditors found reference to the English Language Proficiency Standards (ELPS). While the ELPS were listed no suggested resources or strategies were included. Few resources within the district curriculum documents reviewed were directed toward meeting the needs of gifted students (see Finding 3.1).

Characteristic 11: Describes the procedures teachers and administrators will follow in using assessment data (Inadequate)

The district CIP describes various action steps related to the use of data to assess curriculum effectiveness. Samples of these action steps include:

• Conduct and update the comprehensive needs assessment throughout the year to include data regarding students, families, and teachers.

• Meet ADA mandate to collect data on student progress in GOLD [PreK] to assist in planning and informing instructional decision making.

• Provide and use both Pearson’s AIMSweb and Galileo Assessments throughout the year to identify the tiers for intervention and the students at-risk.

• Use multiple assessments and data analysis to determine that students are making progress in meeting AZCCRS.

• Analyze benchmark data to adjust support as needed.

• Use gathered data to assist district and school leadership teams in completing the district/school comprehensive needs assessments.

• Conduct quarterly “Data Talks” with leadership to review data and inform instructional decisions.

Per job description, the Director of Assessment and Accountability is required to “Provide principals and staff with assistance in analyzing test results for the purpose of improved delivery of instruction.”

When asked how they and their teachers worked with achievement data, often Phoenix #1 campus administrators described Collaborative Team Meetings (CTM) or Professional Learning Communities (PLCs) as the avenue through which they analyzed data and, in some instances, made decisions about instruction based on the data. Some campus improvement plans include strategies including use of CTMs to analyze data. However, auditors were not presented with district expectations for how CTMs or PLCs should be implemented nor procedures in general to be used in using assessment data to strengthen written curriculum and instructional decision-making. This characteristic was not met.


Characteristic 12: Outlines procedures for conducting formative and summative evaluations of programs (Inadequate)

This characteristic was not met. Central office administrators did not provide the auditors with any documents requiring formative and summative evaluation of programs and their corresponding curriculum. Board policies linked to evaluation of programs were found to include the following requirements:

• Board Policy IGA directs the superintendent to organize committees to review the curriculum. There is no directive requiring formative and summative evaluation of programs and their corresponding curriculum.

• Board Policy IL: Evaluation of Instructional Programs (Testing Programs) authorizes participation in a district testing program that “will be subject to regular review and evaluation.”

The district’s strategic action plan (working copy) requires that “All new initiatives align with district goals and be evaluated for effectiveness” that there is a needs assessment to “Identify purpose of the new initiative” and “Provide data supporting the need for the initiative.” The action plan goes on to state that program evaluation will include:

• Analyze outcomes;

• Recommend modifications; and

• Explore opportunities for replication or abandonment.

Job descriptions for campus administrators require them to develop, plan, and evaluate school programs and curriculum, but no policy or documents established the expectation that procedures would be in place for conducting formative and summative evaluations of programs and their corresponding curriculum content (see Finding 4.3).

Characteristic 13: Requires the design of a comprehensive staff development program linked to curriculum design and its delivery (Inadequate).

Auditors were not presented with a district- or campus-level comprehensive professional development plan adopted by the board that outlines the professional development mission of the district nor provides a linkage to curriculum design and delivery (see Finding 3.3).

Auditors were not provided with district-developed professional development standards.

The district CIP includes a number of action steps related to the design of staff development linked to curriculum design and its delivery. A sample of the many action steps focusing on professional development follows:

• Fund district level Math Content Coaches, English Language Arts Specialists, SPED Instructional Coaches and district level Mentor Teachers to support classroom teachers.

• Provide on-going professional development based on benchmark and needs assessment data with an emphasis on content, instruction, literacy and language development.

• Use consultants to provide professional development to teachers of ELL students.

• Provide professional development to district and site coaches to support content and classroom teachers.

• Ensure staff participates in external professional development opportunities, aligned to needs of the district, that support high quality instructional programs.

The district strategic action plan (working copy) lists the following objectives for professional development:

• Build content expertise and capacity of instructional staff to provide relevant and rigorous instruction.

• Implement and cultivate Professional Learning Communities (PLCs).

• Apply Fundaments of Teaching for Learning.

• Conduct Action Research to determine effectiveness of instructional practices.


The audit team found that comprehensive professional development and related planning is not linked to curriculum design and delivery in Phoenix Elementary School District #1 (see Finding 3.2). Therefore, this characteristic is not met.

Characteristic 14: Presents procedures for monitoring the delivery of curriculum (Inadequate)

Board Regulation IGA-RB states, “The principal, department heads, or other supervisors shall see that optimum use is made of available curriculum guides.” No other policy was presented to the auditors specifically describing procedures for monitoring the delivery of curriculum. There are also policies describing the formal evaluation of administrators, teachers, and support personnel.

The Coaching Observation Instrument (COI) is used to evaluate district coaches. The Learning Observation Instrument (LOI) is utilized by principals to formally evaluate teachers. New teacher mentors monitor new teachers using a special walkthrough form in the creation of the new teacher portfolio as directed by the New Teacher Induction Program.

All Phoenix #1 principals are expected to formally evaluate teacher performance using the Learning Observation Instrument (LOI). The district has specific procedures and forms through the LOI for monitoring the delivery of curriculum. Campus and district administrators acknowledge that monitoring of curriculum is inconsistent across campuses as principals struggle to find the time to get into classrooms for the required LOI observation cycles.

During school visits, auditors collected a variety of classroom observation forms utilized by principals. Many of these forms include elements of LOI. Some of the forms focused on single topics such as “Number Talks” and “Critical Thinking.” In an online survey, 90% of the school leaders who responded indicated they made daily or weekly visitations in classrooms. No evidence of a district-wide comprehensive and explicit direction for monitoring classroom delivery of instruction, beyond the formal teacher evaluation process, was presented to the auditors (see Finding 3.2). This characteristic is not met.

Characteristic 15: Establishes a communication plan for the process of curriculum design and delivery (Inadequate)

Auditors were not presented with a written plan exclusively for communications regarding curriculum design and delivery issues. Board policies related generally to district and campus communication include:

• Board Policy CA Administration: Goals/Priority Objectives requires the administration to specialize in “The processes of decision making and communication” and “The development and maintenance of close working relationships and channels of communication within the District and the community.”

• Board Policy BHD: Board Communications with the Public state that official communication between the Board and the community is subject to the following:

○ Any community member who exhausts the opportunity of discussing a matter at the administrative level may communicate with the Board in writing. No anonymous communication will be considered by the Board.

○ A member of the community who wishes to address the Board in person may do so by following the procedures in Policy BEDH.

○ Official communications, policies, Board concerns, and Board action, as appropriate, will be imparted to the community by the Superintendent.

The district strategic plan states as one goal to “Practice and model expert communication and customer service calls.” The strategic action plan (working copy) requires the district to “Engage local business leaders and community members to gain their support and enhance schools” but does not require communication specifically for the processes of design and delivery of curriculum. Communication strategies are listed in the action plan as:

• Inform community members of Governing Board actions.


• Report the financial status of the District to our community.

• Communicate with the community using a variety of formats, such as newsletters, websites, and forums.

• Identify additional communication channels to reach our parents and community members.

The district’s CIP lists as a strategy for increasing stakeholder relations, “Establishes lines of communication among all educational stakeholders.” A sample of actions for the purpose of establishing lines of communication follow in the document. These include:

• Schedule regular, meaningful consultation with stakeholder groups to ensure a sense of urgency is maintained to improve student achievement.

• Develop outreach and communication strategies to include all stakeholders. Communication will include important educational information regarding student learning. LEA and Schools will present and provide materials to inform families/community members about the Arizona College and Career Ready Standards (AZCCRS).

• Disseminate information regarding student progress to families (parent teacher conferences, report cards, progress reports, phone calls, home visits, and technology-enhanced tools).

• The High School Implementation Specialist in collaboration with the Family Engagement and Education Coordinator will work to provide High School Awareness Night (a district-wide event to support students’ transition to H.S.). High School Awareness Night will be used to communicate the importance of high school to families.

• Provide translation and interpretation services for staff and families to increase effective communication.

• Utilize district website, social media and other tools strategically to communicate and/or share policies, programs, reforms, calendar of events, and news.

• Build two-way communication and identify strategies to bring diverse groups of stakeholders together. Develop a system between district, schools and families that will effectively increase student achievement.

Phoenix #1 personnel shared that communications from the district and campuses for parents are often electronic, and not all parents can receive and/or print the information.

Various job descriptions include responsibilities for communication in general. A sample of the job descriptions follows:

• Assistant Principal

○ Participate actively in parent organizations.

○ Assume initiative for keeping the Principal well informed regarding developments in the unit for which the Assistant Principal is responsible or of which the Principal maybe unaware.

○ Assist in the development and dissemination of school newsletters and bulletins.

• Assistant Superintendent for Curriculum and Instruction

○ Maintains effective communication.

• Literacy Coach

○ Research and communicate with staff regarding latest developments and research language acquisition and reading programs.

• Math Coach

○ Research and communicate with staff regarding latest developments and information in the field of mathematic instruction and student learning.


• Elementary School Principal

○ Ability to promote and maintain strong parental and community involvement.

• Parent Engagement Liaison

○ Plans, implements and evaluates trainings/workshops to help parents/families participate more effectively in improving their child’s learning.

○ Serves as outreach liaison between school, family, community and District.

○ Collaborates with teachers, coaches, interventionists, and other staff to provide parent/family education and training relevant to student learning and/or goals.

○ Confers effectively with appropriate school personnel, parents, and community agencies.

○ Confers with parents at meetings and at school to stimulate and increase parental involvement.

• Superintendent of Schools

○ Interpret District guidelines, policies, and procedures to staff members and the public.

○ Provide information to members of the community and staff on a timely basis.

○ Keep the Board informed of various developments throughout the system, including the broad area of personnel.

○ Develop in cooperation with the Public Relations Coordinator improved communication throughout the District.

The district website hosts a variety of information resources for teachers, students, and parents. Teachers, administrators, students, and parents may utilize, in varying degrees of accessibility, online links to Arizona College and Career Readiness Standards, district-developed curriculum documents, Galileo and AzMERIT testing, and online links to the Arizona Department of Education resources.

Auditors noted that, although they did not find a written communication plan specifically related to the process of curriculum design and delivery, listed communication avenues are nevertheless abundant, although not coordinated district-wide. Moreover, when asked in interviews and via the online survey about areas in need of improvement in Phoenix Elementary School District #1, many individuals attested to a lack of communication in the system. Without written direction in a communication plan for the processes related to curriculum design, development, implementation review and revision, this characteristic is not met (see Finding 1.3).

These comments regarding a need for greater communication and connectivity across the district included:

• Challenges? “Timely communication between departments and schools.” (Teacher)

• “There is a disconnect between schools and the district.” (Teacher)

• “We do not communicate our curriculum out to our teachers well enough.” (District Administrator)

• “No clear connection with curriculum across the district…we have adopted programs, but can still choose to do something else potentially leaving gaps if a student goes from one school to another in the district.” (Campus Administrator)

• “A structure exists but is not always clear whom we should contact to accomplish tasks.” (School Leader Survey Response)

• “[We need] systems in place that guarantee the success of students and staff, rather than a culture where help is given but only when asked for.” (School Leader Survey Response)

• “One person makes all the decisions [at the district level] and it’s not a team effort. It would be nice if they want a new pacing guide, have a group of people make a guide instead of one person. We have no connection to the curriculum department.” (Campus Administrator)


• Weakness in the district? “When it comes to relaying information from the district level to the school level. From Testing and Curriculum.” (Campus Administrator)

• “There is no curriculum. We really need to define our curriculum. The teachers don’t think we have a curriculum. I am going to assume that their perception is valid.” (District Administrator)

Overall, the auditors found no clearly defined system and processes for designing, developing, implementing, monitoring, evaluating, and revising curriculum in Phoenix School District #1. District leaders have not created clear connections between the district office and individual schools in the areas related to student learning. Curriculum development is haphazard and relies too much on purchased programs and resources (see Finding 2.3), and there is insufficient written curriculum available to support teachers in planning and delivering instruction (see Finding 2.2).

Summary

While documents and other data presented to the audit team contain some elements of curriculum management planning, the auditors found no single district-level planning document or process that provided the integrated, comprehensive guidance critical to developing a strong system for developing, implementing, monitoring, evaluating, and revising curriculum. The existing curriculum documents reviewed by the audit team do not meet audit criteria for the comprehensive curriculum management planning needed to establish district-wide processes, procedures and timelines for curriculum review, development, and implementation (see Recommendation 3).

Finding 2.2: The scope of the PreKindergarten through grade 8 written curriculum is inadequate for the core and non-core content areas to direct teaching and learning.

A written curriculum is an essential tool for keeping teachers focused on the objectives students need to master. Well-developed curriculum follows the tightly held/loosely held balance discussed in Finding 2.1 and includes clear, translatable objectives for learning, assessments, suggested strategies and approaches, and the resources available to teachers (texts, videos, kits, and other instructional materials). The scope of the written curriculum refers to the percentage of subjects and courses in a district for which written curriculum documents are available, but the scope analysis does not indicate quality. The audit expectation is that written curriculum guides should be present for every subject at every grade level.

When written curriculum is not available for any course or subject area, it can decrease the consistency of subject delivery across grades and schools, particularly when different textbooks are utilized across a subject within the same grade level. Conversely, the presence of a written curriculum helps ensure consistency in student learning (that is, the concepts, skills, and vocabulary that students obtain), while allowing flexibility and professional judgment in how that student learning is obtained.

A written curriculum is a written plan or guide that organizes student learning objectives into a rational sequence within given time frame, ties each objective to a common assessment, and provides a district-wide language of instruction across subjects and grades. Thus, a complete district curriculum defines the continuum of learning from PreKindergarten through grade 8. This allows teachers to accurately meet the individual needs of each student, because teachers can assess where students fall on the continuum and instruction can be planned accordingly. The audit does not consider commercially produced resources and materials to be a curriculum, and these are, therefore, not counted as curriculum documents when determining scope.

Finding 2.2 addresses only the scope of the written curriculum. The quality of the written curriculum documents reviewed by auditors is discussed in Finding 2.3. For Finding 2.2, the auditors reviewed the presence of curriculum relative to the number of subjects being taught. The documents can be traditional hard copy or accessible through online technology services within the district. The key question is whether a centrally defined curriculum for any given subject exists and is made available to all teachers in the system (not just at a single school) to direct and support classroom instruction.


Auditors looked for board policy requiring curriculum for all subjects and courses. Two board policies indirectly suggest this requirement:

• Board Policy IGE: Curriculum Guides and Course Outlines states, “Curriculum guides shall be developed for the various subject areas.”

• Board Policy IHA: Basic Instructional Program refers to planned sequences for the instructional program in:

○ Language arts—reading, spelling, handwriting, English grammar, composition, literature, and study skills;

○ Mathematics experiences;

○ Social studies—history, geography, civics, economics, world cultures, political science, and other social science disciplines;

○ Science experiences;

○ Fine and practical arts experience—art education, vocal and instrumental music, and vocational/business education;

○ Technology skills;

○ Health and safety education (to include skin cancer prevention as required by Senate Bill 1297);

○ Physical education; and

○ Foreign or Native American language.

Auditors did not find language in any policy that specifically requires a written curriculum guide for every single subject taught, although Board Policy IGE does direct the development of guides without specifying for which content areas.

To determine the scope of the written curriculum in the Phoenix Elementary School District #1, auditors looked for a district curriculum document for every subject listed as being offered on the Phoenix Elementary School District #1 2016-2017 Report Card. Central office administrators verified that the subject list was correct.

The audit standard considers the scope of curriculum to be adequate when 100% of core subject areas (English language arts, mathematics, science, and social studies) and 70% of all other subject areas have a corresponding curriculum document. The auditors found in the Phoenix Elementary School District #1 that 86% of core and 9% of non-core subject areas have a curriculum guide. Exhibit 2.2.1 displays the scope of written curriculum by subject area and by grade level for PreK-8.

Exhibit 2.2.1

Scope of the Written Curriculum by Subject Area and Grade Level PreKindergarten through Grade 8


Subject Areas Grade Levels # Subjects Offered

# With Guides

% With Guides

PreK K 1 2 3 4 5 6 7 8Core Subjects

English Language Arts Reading Literature, Reading Information, Reading Foundations, Handwriting, Language, Listening & Speaking

X X X X X X X X X X 10 10 100%


Exhibit 2.2.1 (continued)Scope of the Written Curriculum by Subject Area and Grade Level

PreKindergarten through Grade 8Phoenix Elementary School District #1

May 2017

Subject Areas Grade Levels # Subjects Offered

# With Guides

% With Guides

PreK K 1 2 3 4 5 6 7 8Core Subjects (continued)

Writing Narrative, Opinion/Argument and Informative/Explanatory

X X X X X X X X X X 10 10 100

Mathematics Counting, Operations & Algebraic Thinking, Numbers & Operations in Base Ten, Proportional Relationships, Expressions & Equations, Geometry, Measurement & Data

X X X X X X X X X X 10 10 100

Science Inquiry Process, Life Science, Physical Science, Earth & Space Science

X X X X X X X X X X 10 10 100

Social Studies American History, World History, Civics & Government, Geography, Economics

X X X X X X X X X X 10 3 30

Subtotal—Core Subjects Totals/Percentage 50 43 86%

Non-Core SubjectsArt Demonstrates Knowledge of Art Standards

X X X X X X X X X X 10 1 10

General Music Demonstrates Knowledge of Music Standards

X X X X X X X X X X 10 1 10

Instrumental Music Demonstrates Knowledge of Instrumental Music Standards (only grades 4-8)

X X X X X 5 0 0

Physical Education Demonstrates Knowledge of Physical Education Standards

X X X X X X X X X X 10 1 10

Drama (PreK & Herrera only) X X X X X X X X X X 10 1 10Dance (PreK & Herrera Only) X X X X X X X X X X 10 1 10

Subtotal—Non-Core Subjects Totals/Percentage 55 5 9.1%All Subjects Totals/Percentage 105 48 45.7%

X = Subject offeredSource: Phoenix Elementary School District #1 2017-2018 Report Card


As can be noted in Exhibit 2.2.1:

• The district offers five core subjects and six non-core subjects PreK-8.

• The scope of the written curriculum for core subjects PreKindergarten through grade 8 is inadequate with 86% of the subjects having district curriculum documents. Auditors were not presented with district social studies curriculum for Kindergarten through grade 6. Only PreKindergarten and grades 7 and 8 have district curriculum documents for social studies.

• The scope of non-core subjects is also inadequate, with just 9.1% of the subjects offered having district curriculum documents, therefore, not meeting the audit standard of 70% or more. Auditors found district written curriculum documents for only PreKindergarten non-core subjects. There is a curriculum for Shaw Montessori non-core subjects. Because the curriculum for Montessori is utilized only at the one school, it was not considered in the scope.

• Overall, the scope of the written curriculum in Phoenix #1 is inadequate. Only 48 of the 105 subjects offered had written curriculum. Overall, the district percentage of written curriculum was 45.7%, which is inadequate compared to audit standards.

Teacher posts standards and objectives for daily lessons.

Auditors also interviewed teachers, principals, central office administrators, and instructional support staff regarding the availability of curriculum across the district. The auditors found that there is confusion among district stakeholders regarding what comprises a curriculum. This was evident from interview and survey comments, as illustrated by the following remarks:

• “As a new teacher, I had no idea of what to teach. That’s pretty basic. We have a pacing guide but that is not curriculum.” (Support Staff)

• “We are inconsistent with curriculum. Everyone has different materials but that’s not curriculum.” (Teacher)

• “I feel the district provided curriculum in reading could improve. Currently, what we use only addresses reading strategies, and does not necessarily align to the district given assessment, nor does it align to standards.” (Teacher Survey Response)


More often than not, interviewees and teachers who responded to a district survey expressed recognition of the need for a common curriculum and the lack of curriculum guides for teachers. The following comments were typical:

• “Our biggest academic challenge is that we have no curriculum.” (Central Office Administrator)

• [We need] “A comprehensive curriculum for all subject areas, not just the four ‘core’ areas.” (Teacher Survey Response)

• “There is very little district-adopted curriculum for social studies.” (Teacher Survey Response)

• “They [non-core teachers] do not have a curriculum.” (Campus Administrator)

• “The school could benefit from a curriculum.” (Teacher Survey Response)

• [There is a] “Lack of curriculum in ELA.” (Teacher Survey Response)

• “PESD needs a dedicated ELA curriculum, writing curriculum, and social studies curriculum.” (Teacher Survey Response)

• “Writing, Math, and Reading goals and curriculum alignment K-8 and across sites—[this] seems to be haphazard across the sites and in some cases neglected.” (Teacher Survey Response)

• “Teachers follow the standards. But there is no district curriculum as such. The [curriculum] department provides copies of standards.” (District Administrator)

• “As a new teacher, I had no idea of what to teach. That’s pretty basic. We have a pacing guide but that is not curriculum.” (Teacher)

• “I don’t think there is one. We have a lot of programs but not curriculum.” (District Administrator)

• What’s missing? “The curriculum. [Teachers] know a lot [about strategies], but they [don’t have curriculum].” (District Administrator)

The auditors heard many comments during interviews indicating schools develop their own curriculum, indicative of a lack of coordinated and focused efforts to develop and implement a common, aligned curriculum. Several attested to the development of curriculum happening at the school level:

• “Specials, like music, art, and physical education create their own curriculum for their building.” (Campus Administrator)

• “Here at the campus we create our own curriculum.” (Campus Administrator)

• “We start with the AZ standards, then select out programs, like Full Option Science System™ (FOSS) or Engage NY, then teachers create their curriculum for our building.” (Campus Administrator)

• “We use building content committees to write our curriculum, but it is really too large a job for one school to handle.” (Campus Administrator)

• “My teachers write their own curriculum based on the Arizona standards.” (Campus Administrator)

• “We have a committee that gets together and revise it [curriculum] periodically. We used our standards and divided them up and organized them and correlated to early childhood and goal indicators. We use pieces of the Creative curriculum.” (Campus Administrator)

• “He [one teacher] writes all of his own content. It’s all teacher created.” (Teacher)

Others commented on the lack of consistency resulting from the lack of a focused, district-wide effort to develop curriculum:

• “In certain instances, pacing maps help. There is the map but we don’t have to stick to it. It’s so loose.” (Campus Administrator)

• “Too many curriculum choices are left in the hands of the teachers.” (Campus Administrator)


• [Curriculum across the district is] “Disjointed, not all the same.” (Campus Administrator)

• [We need] “A little more consistency with our curriculum, it’s kind of left to every school.” (Campus Administrator)

Overall, auditors found that personnel at all levels of the district recognize the need for a quality common written curriculum (see Finding 2.3).

Summary

Auditors found that the scope of the written curriculum was inadequate (46%) to direct instruction for all subjects taught. Written guidance is available for 100% of core subject except for social studies, which rendered the overall core percentage to 86% (inadequate compared to audit standards). Written curriculum for non-core subjects is present only for PreKindergarten.

Finding 2.3: The quality of the written curriculum (PreK-8) is inadequate to provide clear guidance for effective teaching and learning. The design of assessment to support effective delivery of the district curriculum is inadequate to support student learning and success on state and local tests.

A clear and comprehensive written curriculum provides the foundation for a school system’s efforts to reach desired levels of student achievement. A quality curriculum provides for consistency and coordination while supporting methodological flexibility in how teachers interact with and instruct students. Quality curriculum guides support alignment of the written, taught, and tested curriculum. They focus instruction on essential learnings and connect the curriculum vertically and horizontally within the system, ensuring equal access to the curriculum for every student. When guides are incomplete or nonexistent, the content taught across district classrooms is less likely to connect in a logical sequence, and instruction is more likely to be inconsistent among teachers and between campuses, which in turn can result in less predictable learning outcomes for students.

Auditors expected to find clear direction in governing district documents for quality and components of written and tested curriculum. Many of the board policies described earlier in Finding 2.1 are referred to again here with a focus on quality and components direction:

• Board Policy IGE: Curriculum Guides and Course Outlines requires that “These guides shall present at least a minimal outline for instruction…” and further states “In accordance with requirements of the state of Arizona, the District will develop its own curriculum guides to reflect alignment to the state standards and local needs. The guides shall be designed to assist users in implementing the District philosophy regarding the teaching of a subject and will, when possible, suggest a variety of possibilities for instruction, patterns of individualization, variations of approaches, and materials.”

• Board Policy IHA: Basic Instructional Program states, “The curriculum will be broad in scope and provide for a wide range in rate, readiness, and potential for learning.” This policy also states, “The first priority of the instructional program will be language acquisition through a planned sequence of reading skills and language experiences beginning in the kindergarten program.”

Specific direction for quality and components of the written curriculum was not found in board policy. No direction for curriculum quality and components was found in job descriptions (see Findings 1.1 and 2.1).

One action referencing components of the written curriculum was found in the Strategic Plan: “Our Blueprint for Student Success” Action Plan (working copy). The action states, “Integrate English Language Acquisition standards in all subject areas.”

The District Continuous Improvement Plan states, “Ensure all curriculum (instructional materials) and support documents (Pacing Guides) are aligned to AZCCRS” and “Purchase supplemental materials as needed to ensure alignment.”

Auditors found no other document outlining the quality and components of the district written curriculum.


The analysis of quality of the taught and tested curriculum will be presented in two ways:

1. Frame one analysis: minimal basic components for curriculum document quality and specificity

2. Analysis of the alignment of district assessments with the Arizona College and Career Readiness Standards (AZCCRSs)

Frame One Analysis: Minimal Basic Components for Curriculum Document Quality and Specificity

To determine the quality of existing curriculum, auditors examined documents provided by the district, including policy and job descriptions, as well as all written curriculum documents approved by the governing board. These curriculum documents were rated against the minimal basic components audit criteria for curriculum quality and specificity. In addition, auditors interviewed central office and campus administrators, support staff, and teachers to determine the quality, availability, and use of curriculum documents, and to ascertain the degree to which the curriculum was articulated and coordinated across grade levels and schools. A survey was provided for completion by teachers, school leaders, and parents.

The auditors found that existing curriculum documents did not meet minimum standards for minimum component quality and specificity. Use of the available curriculum was inconsistent and teachers reported relying on many different resources in planning their instruction. The curriculum as it is taught in district classrooms was found to be inconsistent as well, which has resulted in poor articulation and inadequate coordination across grade levels and among schools (see Finding 3.2).

The auditors were presented with a plethora of curriculum documents for each grade level. Most documents were developed at the state level and many included similar information formatted differently. Montessori curriculum was included in the analysis of quality as there is one PreK-8 school following the Montessori educational approach in the district. Exhibit 2.3.1 displays a sample of the many curriculum documents reviewed for the various subject areas taught in the district.


Exh

ibit

2.3.

1

Dis

tric

t Cur

ricu

lum

Doc

umen

ts R

evie

wed

Pr

eKin

derg

arte

n T

hrou

gh G

rade

8Ph

oeni

x E

lem

enta

ry S

choo

l Dis

tric

t #1

May

201

7

Eng

lish

Lan

guag

e Art

sM

athe

mat

ics

Scie

nce

Soci

al S

tudi

esO

ther

Shar

ed R

eadi

ng a

nd R

eade

r’s

Wor

ksho

p Pa

cing

Gui

de –

Lis

ts

stan

dard

s, ac

adem

ic la

ngua

ge, a

nd

text

s to

use

in in

stru

ctio

n.

Paci

ng G

uide

s for

Inve

stig

atio

ns/

CM

P –

Uni

ts o

f stu

dy w

ith

corr

elat

ions

to th

e Ariz

ona

Stan

dard

s (2

010)

. In

clud

es c

ore

mat

h id

eas,

focu

s que

stio

ns, s

tand

ards

ta

ught

, mat

h pr

actic

es, b

ench

mar

k as

sess

men

t sug

gest

ions

, and

dai

ly

prac

tice

or h

omew

ork

page

num

bers

fo

r eac

h st

anda

rd.

FOSS

Pla

nnin

g an

d Pa

cing

Gui

des K

-8—

Con

tain

s big

idea

s/sc

ienc

e co

ncep

ts, f

ocus

que

stio

ns,

stan

dard

and

per

form

ance

ob

ject

ives

, aca

dem

ic

lang

uage

, em

bedd

ed

asse

ssm

ent s

ugge

stio

ns,

corr

elat

ions

to li

tera

cy

stan

dard

s, an

d a

sam

ple

paci

ng c

alen

dar.

Soci

al S

tudi

es P

acin

g G

uide

s (G

rade

s 7/

8) –

Bas

ed o

n “B

lend

ed L

earn

ing

App

roac

h,”

Uni

ts (C

once

pts)

of s

tudy

in

clud

es c

once

ptua

l und

erst

andi

ng,

esse

ntia

l que

stio

ns, l

earn

ing

over

view

; lis

ts th

e on

line

med

ium

s for

the

pres

enta

tion

of c

onte

nt; l

ists

the

Engl

ish

lang

uage

arts

and

soci

al

stud

ies s

tand

ards

taug

ht in

eac

h un

it;

sugg

ests

“re

al ti

me

asse

ssm

ent;”

list

s su

ppor

ting

stud

ent a

ctiv

ities

; lis

ts

reso

urce

s for

eac

h un

it.

The

Mon

tess

ori

Earl

y Ye

ars

Lear

ning

Pr

ogra

mm

e Th

ree

to S

ix Y

ears

: The

C

hild

ren’

s Hou

se

Smal

l Gro

up In

stru

ctio

n w

ithin

Re

ader

’s W

orks

hop

– Po

wer

Poin

t on

cond

uctin

g sm

all g

roup

inst

ruct

ion.

End

of U

nit

(Inv

estig

atio

ns/C

MP)

As

sess

men

ts –

Lis

ts st

anda

rds t

o be

te

sted

for e

ach

unit

(gra

des 2

-8).

Ariz

ona

Scie

nce

Cor

rela

tions

with

FO

SS (G

rade

s 4-8

) –

Cor

rela

tes s

peci

fic F

OSS

in

vest

igat

ions

with

eac

h A

rizon

a st

anda

rd.

Scop

e an

d Se

quen

ce –

Lis

ts th

e En

glis

h la

ngua

ge a

rts a

nd so

cial

st

udie

s sta

ndar

ds ta

ught

in e

ach

unit

(con

cept

).

The

Mon

tess

ori

Nat

iona

l C

urri

culu

m fo

r the

Se

cond

Pla

ne o

f D

evel

opm

ent f

rom

Si

x to

Tw

elve

Yea

rsPa

cing

Cal

enda

rs -

Scop

e an

d se

quen

ce li

stin

g st

anda

rds c

lust

ered

by

sigh

t voc

abul

ary,

pho

nics

, ph

onem

ic a

war

enes

s, an

d gr

amm

ar

acro

ss g

rade

leve

ls.

Ariz

ona

Mat

h St

anda

rds K

-5 (2

010)

O

verv

iew

- O

rgan

ized

by

focu

s are

as

for e

ach

grad

e.

2016

-201

7 Sc

ienc

e Ro

tatio

n Sc

hedu

le –

Te

ache

rs sh

are

FOSS

m

ater

ials

so th

e sc

hedu

le

indi

cate

s whe

n te

ache

rs

have

the

mat

eria

ls.

MC

ESA

(Mar

icop

a C

ount

y Ed

ucat

ion

Serv

ice

Agen

cy) A

sses

smen

t Blu

epri

nt

– gr

ades

6-8

soci

al st

udie

s dis

tribu

tion

of re

porti

ng c

ateg

ory

by p

erce

nt a

nd

repo

rting

cat

egor

y di

strib

utio

n of

as

sess

men

t per

con

cept

by

perc

ent.

Pre-

Kin

derg

arte

n C

urri

culu

m

3rd –

6th T

oolk

it Te

xt O

verv

iew

– S

cope

an

d se

quen

ce sh

owin

g fo

r eac

h le

sson

in

eac

h gr

ade

the

text

bei

ng u

sed.

Peda

gogi

cal S

hifts

Dem

ande

d by

th

e C

omm

on C

ore

Stan

dard

s –

Dis

cuss

es th

e ch

ange

s to

peda

gogy

in

pre

parin

g st

uden

ts fo

r the

C

omm

on C

ore

Stan

dard

s.

2016

-201

7 Sc

ienc

e M

odul

e Sc

ope

and

Sequ

ence

with

FO

SSw

eb

Acce

ss C

odes

– p

urpo

se

sim

ilar t

o th

e Sc

ienc

e R

otat

ion

Sche

dule

.

A Bl

ende

d Le

arni

ng A

ppro

ach

to 7

/8

Soci

al S

tudi

es (w

ith E

LA In

tegr

atio

n)

– Pp

t. de

scrib

ing

the

plan

to d

evel

op

soci

al st

udie

s les

sons

aro

und

tech

nolo

gy.


Exh

ibit

2.3.

1 (c

ontin

ued)

Dis

tric

t Cur

ricu

lum

Doc

umen

ts R

evie

wed

Pr

eKin

derg

arte

n T

hrou

gh G

rade

8Ph

oeni

x E

lem

enta

ry S

choo

l Dis

tric

t #1

May

201

7E

nglis

h L

angu

age A

rts

Mat

hem

atic

sSc

ienc

eSo

cial

Stu

dies

Oth

erTo

olki

t Pac

ing

Gui

de –

For

eac

h le

sson

, des

crib

es c

once

ptua

l un

ders

tand

ing,

ELA

stan

dard

s, ta

sk

anal

ysis

, aca

dem

ic v

ocab

ular

y, a

nd

teac

hing

sequ

ence

.

Stan

dard

s Unp

acke

d –

For s

elec

t K

-8 m

ath

stan

dard

s, de

scrib

es

conc

eptu

al u

nder

stan

ding

, gui

ding

qu

estio

ns, c

onne

ctin

g st

anda

rds,

exam

ples

and

exp

lana

tions

(ta

sk a

naly

sis)

, pos

sibl

e st

uden

t m

isco

ncep

tions

, aca

dem

ic

voca

bula

ry.

Seve

nth/

Eigh

th G

rade

Yea

r Lon

g Pl

an

– Fo

r eac

h un

it of

stud

y in

clud

es so

cial

st

udie

s per

form

ance

obj

ectiv

es, k

ey

conc

epts

, ess

entia

l que

stio

ns.

Paci

ng G

uide

s – D

istri

ct te

ache

r-de

velo

ped

units

of s

tudy

abo

ut g

iven

pi

eces

of l

itera

ture

; inc

lude

s big

id

eas,

list o

f sta

ndar

ds, a

sses

smen

t su

gges

tions

, que

stio

ns, w

ritin

g id

eas,

voca

bula

ry.

Com

mon

Cor

e St

ate

Stan

dard

s for

M

athe

mat

ics K

-12

Soci

al S

tudi

es S

tand

ard

Artic

ulat

ed

by G

rade

Lev

el S

even

th/E

ight

h G

rade

–

List

s per

form

ance

obj

ectiv

es a

nd

conc

epts

for e

ach

stra

nd o

f his

tory

as

sign

ed to

the

grad

e le

vel.

Gen

re C

hara

cter

istic

s Cha

rt –

D

escr

ibes

the

diffe

rent

gen

res

Inve

stig

atio

ns –

NY

Enga

ge

Cal

enda

r – O

n on

e ca

lend

ar

com

pare

the

two

prog

ram

s.K

inde

rgar

ten

Read

ing

Info

rmat

iona

l Te

xt S

tand

ards

– L

ists

stan

dard

s, id

eas,

activ

ities

, and

que

stio

ns to

ask

.

Mat

hem

atic

s - A

rizo

na A

cade

mic

C

onte

nt S

tand

ards

Art

icul

ated

by

Gra

de L

evel

EL

A Pr

ogre

ssio

n K

-8 –

Dis

trict

-de

velo

ped

scop

e an

d se

quen

ce K

-8

of a

ll EL

A C

omm

on C

ore

Stat

e St

anda

rds.

CC

SSM

Gra

de L

evel

Tra

ject

ory

– D

istri

ct-d

evel

oped

scop

e an

d se

quen

ce K

-8 o

f all

mat

hem

atic

s C

omm

on C

ore

Stat

e St

anda

rds.

Ariz

ona’

s Com

mon

Cor

e St

anda

rds

Engl

ish

Lang

uage

Art

s and

Lite

racy

in

His

tory

/Soc

ial S

tudi

es, S

cien

ce, a

nd

Tech

nica

l Sub

ject

s K

inde

rgar

ten

–12t

h G

rade

Mat

h G

ames

(Gra

de O

ne) –

Stu

dent

m

ath

activ

ities

cor

rela

ted

to a

giv

en

stan

dard

.


Exh

ibit

2.3.

1 (c

ontin

ued)

Dis

tric

t Cur

ricu

lum

Doc

umen

ts R

evie

wed

Pr

eKin

derg

arte

n T

hrou

gh G

rade

8Ph

oeni

x E

lem

enta

ry S

choo

l Dis

tric

t #1

May

201

7E

nglis

h L

angu

age A

rts

Mat

hem

atic

sSc

ienc

eSo

cial

Stu

dies

Oth

erK

inde

rgar

ten

Read

ing

Lite

ratu

re

Stan

dard

sK

-2 S

ampl

e Pr

oble

ms –

Sto

ry

prob

lem

s cor

rela

ted

to th

e 20

08

Ariz

ona

Stan

dard

s.En

glis

h La

ngua

ge D

evel

opm

ent:

Gra

mm

ar S

cope

& S

eque

nce

–

Cor

rela

tes E

LP S

tand

ards

to A

Z EL

A

Com

mon

Cor

e St

ate

Stan

dard

s for

EL

A.

K-2

Pro

blem

Sol

ving

Pro

gres

sion

: U

nwra

ppin

g th

e St

anda

rds –

K-2

sc

ope

and

sequ

ence

of w

ord

prob

lem

stan

dard

.

K-2

, 3-6

Eng

lish

Lang

uage

Art

s Blo

ck

– D

escr

ibes

the

stru

ctur

e an

d tim

e to

spen

d w

ith e

ach

read

ing

stru

ctur

e (s

hare

d re

adin

g, w

ord

stud

y, re

ader

’s

wor

ksho

p, w

riter

’s w

orks

hop,

and

re

ad a

loud

) in

a 15

0-m

inut

e bl

ock

of ti

me.

Lis

ts st

anda

rds a

nd g

ives

ex

ampl

es o

f wha

t to

look

.

Stor

y Pr

oble

m P

roto

col (

Gra

des

1-7)

– In

stru

ctio

n fo

r stu

dent

s in

add

ress

ing

stor

y pr

oble

ms

indi

vidu

ally

and

as a

gro

up.

Read

ing

Stra

tegi

es –

des

crib

es

stra

tegi

es to

incl

ude,

pre

dict

/infe

r, ph

onic

s/de

codi

ng, m

onito

r/cla

rify,

qu

estio

n, e

valu

ate,

and

sum

mar

ize.

Dai

ly S

tory

Pro

blem

Sol

ving

Rou

tine

– st

ory

prob

lem

s to

solv

e. I

nclu

des

sugg

estio

ns fo

r tea

chin

g st

rate

gies

.

6-8

Engl

ish

Lang

uage

Art

s Blo

ck

(Dep

artm

enta

lized

) – D

escr

ibes

the

stru

ctur

e, st

anda

rds,

purp

ose,

and

ex

ampl

es o

f act

iviti

es d

urin

g a

five-

day

wee

k.

Hig

h Sc

hool

Res

ourc

es –

Lis

t of

mat

h vo

cabu

lary

.

Com

mon

Cor

e Te

achi

ng a

nd L

earn

ing

Stra

tegi

es E

nglis

h &

Lan

guag

e Ar

ts R

eadi

ng In

form

atio

nal T

ext

Gra

des 6

-12

(Illi

nois

Sta

te B

oard

of

Educ

atio

n) –

Lis

ts fo

r eac

h st

anda

rd

seve

ral s

ugge

stio

ns fo

r stra

tegi

es/

less

ons a

nd a

sses

smen

ts.

Inve

stig

atio

ns O

verv

iew

(K-5

) –

For e

ach

grad

e le

vel a

nd u

nit,

sum

mar

izes

the

mat

h to

pics

.


Exh

ibit

2.3.

1 (c

ontin

ued)

Dis

tric

t Cur

ricu

lum

Doc

umen

ts R

evie

wed

Pr

eKin

derg

arte

n T

hrou

gh G

rade

8Ph

oeni

x E

lem

enta

ry S

choo

l Dis

tric

t #1

May

201

7E

nglis

h L

angu

age A

rts

Mat

hem

atic

sSc

ienc

eSo

cial

Stu

dies

Oth

erSc

ope

and

Sequ

ence

Gra

mm

ar –

M

arks

with

“x”

gra

mm

ar st

anda

rds t

o be

taug

ht e

ach

mon

th.

Stan

dard

s Sco

pe a

nd S

eque

nce

- ex

cel s

prea

d sh

eet s

how

ing

for e

ach

K-8

Inve

stig

atio

n un

it th

e st

anda

rds

taug

ht; c

olor

-cod

ed to

indi

cate

w

hen

a st

anda

rd is

add

ress

ed fo

r th

e la

st ti

me

and

whe

n a

stan

dard

is

addr

esse

d in

six

sess

ions

.Sc

ope

and

Sequ

ence

of T

ext –

Lis

ts

the

stan

dard

s to

be a

ddre

ssed

in e

ach

less

on u

tiliz

ing

a gi

ven

text

.

Com

mon

Cor

e D

omai

n Pr

ogre

ssio

n –

Cha

rt sh

owin

g pr

ogre

ssio

n of

eac

h m

ath

dom

ain

acro

ss K

-8.

Pre-

Scho

ol P

acin

g G

uide

s (20

14-

2015

) – D

istri

ct-d

evel

oped

list

of

lear

ning

s by

mon

th fo

r cor

e an

d no

n-co

re a

reas

. D

ocum

ent i

nclu

des

conc

ept,

stat

e in

dica

tor,

and

gold

in

dica

tor.

Frac

tion

Path

way

Acr

oss A

rizo

na’s

Com

mon

Cor

e St

anda

rds –

K-8

sc

ope

and

sequ

ence

of f

ract

ion

stan

dard

s.

Prog

ress

ions

for C

omm

on C

ore

Stat

e St

anda

rds i

n M

athe

mat

ics

(dra

ft) -

Det

aile

d de

scrip

tion

of th

e pr

ogre

ssio

n of

all

the

mat

h st

anda

rds

K-8

.


The auditors reviewed all curriculum documents that teachers have access to on the district website, including those listed in Exhibit 2.3.1. Auditors found that navigating through all of these documents was difficult, time consuming, and confusing. In addition, it was determined that many of the documents included similar information.

The curriculum document components were analyzed for quality and specificity using the audit criteria listed in Exhibit 2.3.2.

Exhibit 2.3.2

Curriculum Management Improvement Model Frame One Analysis: Minimal Basic Components for

Curriculum Document Quality and Specificity

Point Value Criteria

Criterion One: Clarity and Specificity of Objectives0 No goals/objectives present1 Vague delineation of goals/learner outcomes2 States tasks to be performed or skills to be learned

3 States for each objective the what, when (sequence within course/grade), how actual standard is performed, and amount of time to be spent learning

Criterion Two: Congruity of the Curriculum to the Assessment Process0 No assessment approach1 Some approach of assessment stated2 States skills, knowledge, and concepts that will be assessed3 Keys each objective to district and/or state performance assessments

Criterion Three: Delineation of the Prerequisite Essential Skills, Knowledge, and Attitudes0 No mention of required skill1 States prior general experience needed2 States prior general experience needed in specified grade level

3 States specific documented prerequisite or description of discrete skills/concepts required prior to this learning (may be a scope and sequence across grades/courses if PreK-8)

Criterion Four: Delineation of the Major Instructional Tools0 No mention of textbook or instructional tools/resources1 Names the basic text/instructional resource(s)2 Names the basic text/instructional resource(s) and supplementary materials to be used

3 States for each objective the “match” between the basic text/instructional resource(s) and the curriculum objective

Criterion Five: Clear Approaches for Classroom Use0 No approaches cited for classroom use1 Overall, vague statement on approaching the subject2 Provides general suggestions on approaches3 Provides specific examples of how to approach key concepts/skills in the classroom

The criteria in Exhibit 2.3.2 represent the tightly held/loosely held components of quality curriculum discussed in Finding 2.1. Criteria One, Two, and Three represent the curriculum components that must be tightly controlled by the district. If teachers are not uniformly working toward the same objectives and assessing mastery in the same way, consistency of instruction and achievement will be lost. Criterion Three—delineation of the prerequisite essential skills, knowledge, and attitudes—must be tightly held by the district in order to ensure that instruction is efficient and moves students through learning pathways smoothly. Criteria Four and


Five represent the loosely held components of quality curriculum, allowing teachers to choose from a broad menu of resources and strategies that will target their particular students’ interest and academic strengths and weaknesses. Without these components, curriculum may become merely rote drill and recitation of facts, leading to loss of creativity, excitement, and passion for lifelong learning.

Auditors rated the curriculum documents for each subject as a whole from zero to three on each of the five criteria, with three representing the highest rating. To receive a three for the first criterion would require that each objective state what students will do to meet the objective, when within the course the objective is met, how/ under what conditions and to what degree the actual standard is to be performed, and the amount of time to be spent learning material related to the objective.

To receive a three for the second criterion would require that each objective be keyed to district and/or state performance evaluations, linking the objective to sample questions from the district and/or state assessments. To receive a three for the third criterion would require identification of specific prerequisite skills and concepts that should have been mastered prior to this objective (such as a detailed PreK-8 scope and sequence delineating discrete objectives).

A three rating for the fourth criterion would require a page-specific match between the basic text/instructional resources and each objective. To receive a three rating for the fifth criterion would require provision of specific examples on how to approach key concepts/skills in the classroom for each objective.

A total score for each curriculum is obtained by adding the five separate criterion scores. The highest score a guide can receive is 15. A rating of 12 points is considered the minimum rating for adequate quality of design of a given curriculum. To obtain an overall picture of curriculum quality, a mean is calculated for each criterion and for total ratings.

Auditors were advised that the district elected not to adopt new published textbooks when new English language arts standards were adopted by the state. Instead, teachers developed units of study around literature texts. These teacher-developed units were analyzed along with all other English language arts curriculum documents presented. Auditors’ ratings of the English language arts curriculum documents analyzed are presented in Exhibit 2.3.3.

Exhibit 2.3.3

Auditors’ Ratings of English Language Arts Curriculum Documents PreKindergarten Through Grade 8


Curriculum Documents1 2 3 4 5 Total

RatingObj. Asmt. Prereq. Res. Strats.Preschool Language and Literacy 2 0 2 0 0 4Montessori Language – First Plane 1 1 1 1 2 6English Language Arts Curriculum Grade K 2 1 2 1 1 7English Language Arts Curriculum Grade 1 2 1 2 1 1 7Montessori Language – Second Plan 1 1 1 1 2 6English Language Arts Curriculum Grade 2 2 1 2 1 1 7English Language Arts Curriculum Grade 3 2 1 2 1 1 7English Language Arts Curriculum Grade 4 2 1 2 1 1 7English Language Arts Curriculum Grade 5 2 1 2 1 1 7English Language Arts Curriculum Grade 6 2 1 2 1 1 7English Language Arts Curriculum Grade 7 2 1 2 1 1 7English Language Arts Curriculum Grade 8 2 1 2 1 1 7

Mean Rating for Each Criterion 1.8 0.9 1.8 0.9 1.1 6.6


The following observations may be made about Exhibit 2.3.3:

• The overall mean rating for all ELA curricula was 6.6 points. This did not meet the audit expectation of a minimum score of 12 points.

• The scores per grade level ranged from four to seven. No curriculum documents for a given grade level in English language arts were rated adequate for quality of design when taken as a whole.

Overall, the auditors rated the district’s English language arts curriculum as inadequate in design. The auditors’ comments for each criterion in Exhibit 2.3.3 follow:

Criterion 1: Clarity and Specificity of Objectives – Mean Rating 1.8

The Arizona College and Career Readiness Standards (AZCCRS) are the foundation of the district’s English language arts curriculum. Most English language arts curriculum documents where the standards were listed contained conditions under which students should perform. Though some curriculum documents indicated which cluster of standards were to be addressed in a given lesson or day, none included a specific time frame for learning each of the standards. None were noted to have a specific performance target such as “with at least 85% accuracy.”

Criterion 2: Congruity of the Curriculum to the Assessment Process – Mean Rating 0.9

Connections to the district benchmarks, pre- and post-assessments, and state assessment processes were noticeably missing from all ELA curricula including the teacher-developed units of study. Grade level district-developed and state curricula contained references to general approaches to assessment, though the documents did not specify what would be tested, how it would be tested, or when it would be tested.

Criterion 3: Delineation of the Prerequisite Essential Skills, Knowledge, and Attitudes – Mean Rating 1.8

K-8 teachers have district website access to an ELA scope and sequence published by the Arizona Department of Education entitled “Arizona’s Common Core Standards English Language Arts and Literacy in History/Social Studies, Science, and Technical Subjects Kindergarten–12th Grade.” In addition, the district-developed “ELA Progression K-8” is a scope and sequence of all K-8 ELA standards. Because the district supports PreKindergarten in most of its schools, the audit expects that PreK be included in the scope and sequence. Therefore, most grade levels received a rating of two for this criterion.

Criterion 4: Delineation of the Major Instructional Tools – Mean Rating 0.9

Teacher-developed units of study for grades 1-8 list the literary text to be used, the state standards to be taught, and include a recommended vocabulary list. No basic text nor additional resources are referenced in any grade. Teachers at grades 4-8 are able to access a Grammar Handbook for the grammar standards, though each standard is not linked to a specific page or section in the grammar book.

Criterion 5: Clear Approaches for Classroom Use – Mean Rating 1.1

English language arts curriculum documents developed by the district for grades K-8 contained few strategies to help teachers deliver instruction effectively other than some general suggestions such as instruction on how to implement small group instruction. Documents were reviewed that map out the English language arts block to include shared reading, word study, reader’s workshop, writer’s workshop, and read aloud. However, these documents do not describe what each of these elements is nor give suggestions for implementation. Some state Common Core Standards documents and those for Montessori reference general approaches. However, none of the documents contained specific examples of how to approach key concepts/skills in the classroom.

Overall, curriculum documents for English language did not meet minimal audit standards for quality in grades PreK through 8, although elements of quality were present at each grade level.

The auditors analyzed documents related to mathematics instruction from the district website using the same criteria for quality and specificity. The Phoenix Elementary School District #1 board has adopted Investigations and Connected Mathematics as their core mathematics program. Central office administrators shared that the district is in the process of transitioning its pacing guides to the newest version of these materials. In addition,


the state just adopted new standards so central office administrators state the district must also revise the new pacing guides to match the new standards. Auditors’ quality ratings for the math curriculum documents are presented in Exhibit 2.3.4.

Paired students work together on reading assignment. One students points to the words as the other reads aloud.

Exhibit 2.3.4

Auditors’ Ratings of Mathematics Curriculum Documents PreKindergarten Through Grade 8


Curriculum Document Title1 2 3 4 5 Total

RatingObj. Asmt. Prereq. Res. Strat.Preschool Mathematics 2 0 0 0 0 2Montessori Mathematics – First Plane 1 1 1 1 2 6Mathematics Curriculum Grade K 2 2 2 3 1 10Mathematics Curriculum Grade 1 2 2 2 3 1 10Montessori Mathematics – Second Plane 1 1 1 1 2 6Mathematics Curriculum Grade 2 2 2 2 3 1 10Mathematics Curriculum Grade 3 2 2 2 3 1 10Mathematics Curriculum Grade 4 2 2 2 3 1 10Mathematics Curriculum Grade 5 2 2 2 3 1 10Mathematics Curriculum Grade 6 2 2 2 3 1 10Mathematics Curriculum Grade 7 2 2 2 3 1 10Mathematics Curriculum Grade 8 2 2 2 3 1 10


The following can be noted from Exhibit 2.3.4:

• The overall mean for math curriculum documents was 8.7 points. This did not meet the audit minimum of 12 points considered adequate in design.

• The area receiving the highest rating (2.4 points) was Criterion 4—Delineation of the Major Instructional Tools—where linkages to the district adopted programs, Investigations and Connections Mathematics, were presented for teachers to use in the curriculum documents of most grade levels reviewed.


• Criterion 5—Clear Approaches for Classroom Use—received the lowest rating of 1.1 points.

Overall, the math curriculum documents did not meet minimum audit standards for quality. Auditors’ comments related to each criterion in Exhibit 2.3.4 follow:


The AZCCRS are the foundation of the district curriculum for mathematics. As was the case for English language arts, most mathematics curriculum documents where the standards are listed contained conditions under which students should perform. Though some curriculum documents indicate which cluster of standards was to be addressed in a given unit, lesson, day or month, none included a specific time frame for learning each of the standards. None are noted to have a specific performance target such as “with at least 85% accuracy.”


Connections to the district benchmarks, pre- and post-assessments, and state assessment processes were not found in all mathematics curricula. In order to receive a rating of two, curriculum documents must state skills, knowledge, and concepts that will be assessed. The Investigations pacing guides for each unit describe “benchmarks” or “unit assessments” for assessing student mastery of specific AZCCRS, which gives this criterion a rating of two for most grade level curriculum documents.


K-8 teachers have district website access to K-8 mathematics scope and sequence documents published by the Arizona Department of Education. In addition, the district-developed “CCSSM Grade Level Trajectory” is a scope and sequence of all K-8 mathematics standards. Because the district supports PreKindergarten in most of its schools, the audit expects that PreK be included in the mathematics scope and sequence. Therefore, most grade level curricula received a rating of two for this criterion.


To receive a rating of three for this criterion, mathematics curriculum documents must indicate for each objective or standard the “match” between the basic text/instructional resource(s) and the curriculum objective. The mathematics document entitled “Pacing Guides for Investigations/CMP,” units of study with correlations to the AZCCR Standards, lists daily practice or homework page numbers found in Investigations for each standard. Most grade level curricula were rated a three for this criterion.


Just as was the case for the English language arts documents, mathematics curriculum documents developed by the district or by the publisher of Investigations/CMP for grades K-8 contained few strategies to help teachers deliver instruction effectively. State Common Core Standards documents and documents for Montessori reference general approaches. However, none of the documents contained specific examples of how to approach key concepts/skills in the classroom.

Overall, the adopted mathematics curriculum was rated as inadequate in design to direct teaching.


Pairs of students work on math problem at the whiteboard.

Students use manipulatives in their mathematics lesson to calculate the perimeter of shapes.

The auditors also analyzed curriculum documents related to science from the district website using the same criteria for quality and specificity. The Full Option Science System™ (FOSS) materials were adopted as the approved science program for the district. Central office administrators shared that teachers use these materials primarily when planning science instruction. Auditors reviewed district-developed pacing guides correlated to the FOSS materials as part of the science curriculum documents for grades K-8. Auditors’ quality ratings for the science curriculum are presented in Exhibit 2.3.5.

Exhibit 2.3.5

Auditors’ Ratings of Science Curriculum Documents PreKindergarten Through Grade 8


Curriculum Document Title1 2 3 4 5

Total RatingObj. Asmt. Prereq. Res. Strat.

Preschool Science 2 0 0 0 0 2Montessori Science – First Plane 1 1 1 1 2 6Science Curriculum Grade K 2 1 0 2 0 5Science Curriculum Grade 1 2 1 0 2 0 5Montessori Science – Second Plane 1 1 1 1 2 6Science Curriculum Grade 2 2 1 0 2 0 5Science Curriculum Grade 3 2 2 0 2 0 6Science Curriculum Grade 4 2 2 0 2 0 6Science Curriculum Grade 5 2 2 0 2 0 6Science Curriculum Grade 6 2 2 0 2 0 6Science Curriculum Grade 7 2 2 0 2 0 6Science Curriculum Grade 8 2 2 0 2 0 6



• The overall mean for all adopted science curriculum was 5.4 points. This did not meet the audit minimum of 12 points needed to be adequate in components and specificity.

• Criterion 1—Clarity and Specificity of Objectives—received the highest rating of 1.8 points.


• Criterion 3—Delineation of the Prerequisite Essential Skills, Knowledge, and Attitudes—was rated the lowest with a mean of 0.2 points. With the exception of Montessori documents, auditors found no mention of prerequisite skills in the science documents reviewed.

Overall, the adopted science curriculum did not meet minimum audit standards for quality. Auditors’ comments related to each criterion in Exhibit 2.3.5 follow:


The science pacing guide documents in grades K-8 list the correlation of the Big Ideas and Focus Questions in the FOSS materials to the 2005 State science concepts and performance objectives (PO). There is also a correlation between the FOSS concepts with the English language arts standards. The degree of mastery required and time to be spent learning each performance objective are not described.


At grades 4 and 8, teachers have access through the district website to an AIMS Student Guide for Science that provides examples of the format and types of questions that will appear on the State AIMS Science. However, not every concept in each strand of the science standards has a corresponding sample question. An answer key for the sample questions is provided at the end of the guide. In addition, teachers have access to AIMS Science blueprints listing the performance objectives tested and number of items on the test. Beginning with grade 3, teachers have access to the Benchmarks 1/2 and Pre- and Post-assessment blueprints. These blueprints list the science standard tested. Because teachers are not given information that clarifies which of the science standards or performance objectives are not tested on either the AIMS Science test or the district assessments, auditors rated this criterion with a two for grades 3 through 8. Grades K-2 documents do not include assessment blueprints but embedded assessments and other assessment suggestions are listed.


In the curriculum documents reviewed, references to skills that students were expected to have prior to this course were not found at all levels. The exception was found in Montessori documents. Montessori curriculum documents state general reference to experiences with which students come to each plane (level of the program).


To receive a rating of three for this criterion, science curriculum documents must indicate for each objective or standard the “match” between the basic text/instructional resource(s) and the curriculum objective. The science pacing guides are correlated to the 2005 State science concepts and performance objectives and include links for every general concept taught to the FOSS material. However, there is not a precise match between the science performance objectives and the FOSS resources. Most grade level curricula were rated a two for this criterion.


Auditors were unable to find evidence of clear approaches for classroom use in most science curriculum documents analyzed. The exception was in Montessori curriculum documents, where general suggestions to approaches for delivering the curriculum based on the program’s philosophy are presented.

Overall, the science curriculum documents were rated as inadequate in design to direct teaching. As discussed in Finding 2.1, without clear system direction, these efforts are not fully coordinated. The science documents posted on the district website are based on dated 2005 state science performance objectives.


Microscopes are available for students to use in science class.

The auditors analyzed social studies guides next. The state is currently undertaking a major revision of the standards for social studies. Nevertheless, the auditors analyzed the existing curriculum documents provided. No K-6 social studies curriculum was presented to the auditors for review. The ratings are presented in Exhibit 2.3.6.

Exhibit 2.3.6

Auditors’ Ratings of Social Studies Curriculum Documents PreKindergarten, Montessori, Grades 6 Through 8


Curriculum Document Title1 2 3 4 5

Total RatingObj. Asmt. Prereq. Res. Strat.

Preschool Social Studies 2 0 0 0 0 2Montessori History – First Plane 1 1 1 1 2 6Montessori History – Second Plane 1 1 1 1 2 6Social Studies Curriculum Grade 7 2 2 0 2 0 6Social Studies Curriculum Grade 8 2 2 0 2 0 6



• Since auditors were not presented with guides for grades K-6, the overall rating of social studies curriculum documents are seriously inflated. If the missing seven grade levels were included, the overall mean score would receive only 2.2 points.

• The mean for all adopted social studies curriculum presented to auditors was 5.2 points. This did not meet the audit minimum of 12 points needed for adequacy in design and specificity.

• Criterion 3—Delineation of the Prerequisite Essential Skills, Knowledge, and Attitudes—was rated the lowest with a mean of 0.4 points, with only Montessori curriculum documents describing expectations for students’ prior learnings.

• The pacing guides for grades 7 and 8 do not mention district or state assessments.


• Criterion 1—Clarity and Specificity of Objectives—was rated the highest with a mean score of 1.6 points.

Overall, the adopted social studies curriculum analyzed did not meet minimum audit standards for quality. Auditors’ comments related to each criterion in Exhibit 2.3.6 follow:


In the adopted social studies curriculum, the Common Core and Arizona state standards were utilized as learning objectives. These state the skill to be performed but are frequently lacking information on the conditions under which the skill is to be performed and the degree of mastery required.


The pacing guides for grades 7 and 8 social studies do not mention district or state assessments. Teachers are given access on the website to the MCESA assessment blueprint and post-assessment item specification but these documents are not linked to the pacing guides. Because these two assessment documents list the concepts tested but do not link to the pacing guides, the auditors rated grades 7 and 8 social studies curriculum documents with two points.


As noted above, with the exception of the Montessori curriculum documents, social studies documents do not include prerequisite skill information. The website contained no PreK-8 scope and sequence of skills.


To receive a three for this criterion, auditors must find the match between resources and objectives. Grades 7 and 8 documents list links to online resources but do not match them to each of the objectives. For that reason the grades 7 and 8 documents were rated two points. No resources were listed in the preschool pacing guide and Montessori documents contained only general reference to resources.


As was the case for other core subject areas, auditors found general approaches to instruction in the Montessori social studies curriculum documents. No clear approaches for classroom use were found in either the preschool or the grades 7 and 8 social studies documents.

Overall, the social studies curriculum documents were inadequate in design and specificity to direct teaching.

Students enjoy music education class on a regular basis.


Exhibit 2.3.7 presents the quality ratings for all non-core curriculum documents for PreKindergarten and Montessori provided by district personnel. Curriculum documents for K-8 non-core subjects were not presented to the auditors.

Exhibit 2.3.7

Auditors’ Ratings of Non-Core Subject Curriculum Documents PreKindergarten Through Grade 8


Curriculum Document Title1 2 3 4 5 Total

RatingObj. Asmt. Prereq. Res. Strat.Art CurriculumPreschool Visual Art 2 0 0 0 0 2Montessori Visual Arts – First Plane 1 1 1 1 1 5Montessori Visual Arts – Second Plane 1 1 1 1 1 5MusicPreschool Music and Creative Movement 2 0 0 0 0 2Montessori Music – First Plane 1 1 1 1 1 5Montessori Music – Second Plane 1 1 1 1 1 5Physical EducationPreschool Physical Development, Health, and Safety 2 0 0 0 0 2Montessori Physical Education/Health – First Plane 1 1 1 1 1 5Montessori Physical Education/Health – Second Plane 1 1 1 1 1 5Dance and DramaPreschool Drama 2 0 0 0 0 2Montessori Dance/Movement – First Plane 1 1 1 1 1 5Montessori Dance/Movement – Second Plane 1 1 1 1 1 5



• Twelve non-core subject curriculum documents were reviewed for quality and specificity of components.

• Non-core curriculum documents received a mean rating of 4.0.

• The preschool non-core curriculum documents state the skills to be performed for each subject but lack information on the degree of mastery required and time to be spent mastering each skill. No information on assessment, prerequisite skills, resources, and suggestions for approaches to instruction was found in the preschool curriculum documents. The preschool documents received a rating of two points. The mean score for Criterion 1—Clarity and Specificity of Objectives—is 1.3 points.

• Montessori non-core curriculum documents reviewed provide vague delineation of goals/learner outcomes, some general approaches to assessment, description of prior experience expected, general information about materials to be used, and vague reference to approaches to instruction. These documents received a rating of five points.


Exhibit 2.3.8 displays a summary of the mean ratings of all the curriculum documents analyzed for minimal basic components quality and specificity.

Exhibit 2.3.8

Summary of Auditors’ Mean Ratings of District Curriculum Documents by Content AreaPhoenix Elementary School District #1

May 2017

Mean Ratings by Auditors

Content Area1 2 3 4 5 Total

RatingObj. Asmt. Prereq. Res. Strats.English Language Arts 1.8 0.9 1.8 0.9 1.1 6.6Mathematics 1.8 1.7 1.7 2.4 1.1 8.7Science 1.8 1.4 0.2 1.7 0.3 5.4Social Studies 1.6 1.2 0.4 1.2 0.8 5.2Non-Core 1.3 0.7 0.7 0.7 0.7 4.0

Mean Ratings n=53 1.7 1.2 1.0 1.4 0.8 6.0Data Source: district website and hard copies provided by district administrators

The following observations can be made about Exhibit 2.3.8:

• Mathematics curriculum documents had the highest total rating, with a mean of 8.7 points.

• Non-core curriculum documents scored the lowest with a mean of 4.0 points.

• Criterion 1—Clarity and Specificity of Objectives—had the highest mean score of 1.7 points out of a possible score of 3 points.

• The lowest mean score (0.8 points) was for Criterion 5—Clear Approaches for Classroom Use.

• None of the subject area curriculum documents met the minimum audit score of 12 points.

Overall, core and non-core curriculum documents had a mean total rating of six out of a possible 15 points. This did not meet minimum audit criteria of 12 points for quality and specificity in minimum components. Standards or performance objectives were always included in one or more curriculum documents for a given subject and grade level. No curriculum documents linked standards or performance objectives to district or state assessments. Only mathematics curriculum documents directly matched resources to each standard. None of the curriculum documents reviewed provided specific examples of how to approach key concepts/skills in the classroom.

Having a comprehensive set of quality curriculum documents is a prerequisite for consistency across grades and between schools. An adequate scope of curriculum that is well designed and supports teaching increases the likelihood of uniform access to success and of high levels of student achievement across the system. However, if teachers do not utilize the curriculum, these benefits are not realized. In an online survey, auditors asked teachers about the quality of the district curriculum and whether it was user friendly. The responses to this survey question are presented in Exhibit 2.3.9:


Exhibit 2.3.9

All Teacher Responses: Quality and User-Friendliness of District CurriculumPhoenix Elementary School District #1

May 2017

16%5% 7% 6% 6% 3% 4% 4%

43%

48% 48% 46%37%

20% 18%

46%

21%

19% 20% 28%

32%

44% 47%

28%

13%20% 16%

13% 17%25% 23%

14%

7% 7% 9% 7% 7% 7% 7% 7%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

Easily accessible User friendly Useful inplanning

Effective inproviding

suggestions forstrategies andapproaches

Helpful inidentifying

aligned materialsand resources for

my lessons

Effective insuggesting waysto differentiatemy instruction

Effective inproviding

suggestions forreteaching or

intervention ideas

Helpful insuggestingmeaningful

student activities

Strongly agree Agree Disagree Strongly Disagree Does not apply; I do not use curriculum for my job

The following is noted about Exhibit 2.3.9:

• When questioned about the accessibility of the district curriculum, more than half (59%) of respondents to the teacher survey agreed or strongly agreed that the online district-provided curriculum is easily accessible, while 34% disagreed or strongly disagreed.

• Fifty-five percent said the district curriculum was useful for planning, while 36% said it was not useful.

• Twenty-two percent said the district curriculum is effective in suggesting ways to differentiate instruction while 69% of respondents disagreed.

• Twenty-two percent of teacher respondents to this question agreed or strongly agreed that the district curriculum is effective in providing suggestions for reteaching or intervention ideas, while 70% disagreed or strongly disagreed.

Respondents to the online teacher and school leader surveys and teachers and support staff interviewed by the auditors commented on the quality and accessibility of the district curriculum. The following is a sample of those comments:

• “I feel the curriculum should be improved to be better aligned to the state expectations for standards, and so it will match up with the districts provided assessments.” (Teacher Survey Response)

• [Needs improvement] “Curriculum/quality of curriculum, academic rigor….” (Teacher Survey Response)

• The district needs “Aligned curriculum.” (School Leader Survey Response)

• “There are pacing guides that are difficult to navigate—new teachers need something that is more scripted.” (Support Staff)


• “Curriculum is extremely disjointed. It’s confusing to follow for teachers and students alike. Teachers should be provided with a comprehensive curriculum so that the focus turns from pulling resources to analyzing and implementing the given curriculum appropriately.” (Teacher Survey Response)

• “We need a better social studies curriculum.” (Teacher Survey Response)

• [There is a] “Lack of a clear and user-friendly curriculum framework.” (School Leader Survey Response)

• [Curriculum] “Design does not address the language levels of my students.” (Teacher Survey Response)

• “It’s [district provided curriculum] outdated and better used as a resource.” (Teacher Survey Response)

• “There is better curriculum [referring to the district-provided pacing guides] out there in many cases (Envisions instead of Investigations for example). Some of the curriculum (FOSS) is not feasible when students are crammed into rooms that are not equipped for Science. Example: Trying to use 35 microscopes in a room with 2 electrical outlets.” (Teacher Survey Response)

• “The pacing guide doesn’t account for testing format and opportunities to practice. [The] district curriculum also doesn’t match all of the Common Core Standards.” (Teacher Survey Response)

• “The curriculum is not effective. In primary they are pulling whatever.” (Campus Personnel)

As discussed earlier, auditors found that the design of curriculum documents reviewed contained little linkage to district adopted instructional resources. During school and central office visits, auditors noted many quality instructional resources, most district-provided and some school specific, especially in the area of English language arts. When principals were asked what their teachers use to plan their lessons, many responded saying that teachers used programs such as Investigations and FOSS materials. However, it needs to be pointed out that commercial instructional resource materials or programs are not considered curriculum; these are resources. Auditors received comments in survey and interview responses directed toward concerns about availability and quality of instructional resources designed to support delivery of the district curriculum. A sample of comments focusing on instructional resources follows:

• “The Comprehension Toolkit is easier but not standards-based.” (Support Staff)

• “It is so hard to work with teachers at schools that are using different materials than those the district has adopted.” (Support Staff)

• “We really don’t have an adopted resource for social studies in the lower grades.” (Central Office Administrator)

• “I have to search the Internet for materials. There’s [a] limited amount of time in the planning day. I find it as very inefficient. In the past, we had a lot of pre-aligned materials.” (Teacher)

• “We have a lot of resources and it is hard to decide what to use. It is hard to pull them all together into one cohesive block.” (Campus Administrator)

• “We have a tool kit for comprehensive strategies and we find that teachers use other things. From school to school they do different things.” (Campus Administrator)

• “A little more consistency [is needed] and having a designated program for phonics K-3. My fear is that a kid might leave my school and go to another school within the district and they are doing something different and now we have created a gap.” (Campus Administrator)

• [How do teachers know what to teach?] “Investigations, FOSS, standards-based and if we really don’t like what our programs give, then we see how to supplement.” (Campus Administrator)

• “Consistency. Every year it’s something new, but nothing gets eliminated. No curriculum lasts long enough to see results...and the few curricula that have stood the test of time have not received positive marks in terms of test scores.” (Teacher Survey Response)


• [We need] “Updated curriculum materials. We are encouraged to use the workbooks that the school has with our students, but the workbooks do not match the online resource for the same text. The workbooks are older and the online resources are being updated. Provide the special education team with a copy of the curriculum being used as we are now having to hunt out materials down from the general education teacher or having to make our own books from the content website.” (Teacher Survey Response)

• “We’re dropping the ball and wasting a lot of money by making poor choices in our [instructional resources]. Millions spent on unusable resources is not showing wise stewardship of money.” (Teacher Survey Response)

• “English language arts [curriculum] is a hodge-podge of materials.” (Campus administrator)

• “[District curriculum leaders] just do programs programs programs. Instead of addressing the standards and addressing what our students need to learn they just give us programs.” (Support Personnel)

• “We are inconsistent with curriculum. Everyone has different materials but that’s not curriculum.” (Teacher)

Summary

Overall, district stakeholders report that the curriculum that is available is not high quality nor is it user friendly. Many report following resources rather than using a district-developed guide, but without a clearly defined curriculum, others attested to a lack of alignment that has resulted district-wide from the lack of defined objectives in a written curriculum. While variety in resources is a critical component of meeting individual students’ needs, when the variety of resources is added to a lack of defined learning targets, inconsistencies in student learning can result.

Most curriculum documents list the standards to be learned. However, what mastery looks like, specifically, is not clearly defined. Linkages to district and state assessments are missing from most curriculum documents. Some subjects indicate prerequisite skills through a scope and sequence. Only mathematics, in some grade levels, indicates for each objective or standard the “match” to the basic text/instructional resource(s). Few documents reference instructional approaches. The core and non-core curriculum documents reviewed are inadequate to provide clear guidance for effective teaching and learning.

Alignment of Galileo® Pre-Assessments and Benchmarks with Arizona Standards for College and Career Readiness

A critical premise of curriculum alignment is that the instruction in the classroom is aligned to expectations for student mastery found in both the curriculum standards and in the assessments used to measure mastery of those standards. Likewise, student work should align with district-level assessments; those district assessments should also align to high stakes assessments to ensure that students’ performance on the local assessments is a valid predictor of their performance on high stakes assessments. The most critical role of written curriculum is providing teachers with objectives, resources, and materials to guide their instruction so that it is aligned to all assessments in use. This alignment is assured in the design of the curriculum and increases the likelihood that students will be prepared for the content, contexts, and cognitive demand of any assessments.

The auditors examined a sample of the Galileo® pre-assessments and benchmark assessments to see if these tools are adequately aligned in the dimensions of content, context, and cognition to the Arizona College and Career Readiness Standards (AZCCRS). Content refers to the skills, processes, knowledge, concepts, or vocabulary students must learn. Context refers to how they are to practice or demonstrate that learning, such as in a real-life situation or with pencil and paper. Cognitive type refers to the nature of the cognitive engagement the learning demands. Looking at alignment along all three dimensions gives teachers and administrators a more specific picture of the extent, nature, and degree of alignment.

Overall, the auditors determined that the district Galileo® pre-assessment and benchmark assessment items analyzed were not always congruent with the state standards in content and context but were found to be topographically aligned with the state standards in their cognitive demand.


To perform the analyses of cognitive type, auditors used the framework based on the revised Bloom’s Taxonomy of Cognitive Domains as presented in Exhibit 2.3.10. Webb’s Depth-of-Knowledge (DOK) is compared to Bloom’s Cognitive Process Dimensions in Appendix C.

Exhibit 2.3.10

Description of Cognitive Types in the Revised Bloom’s TaxonomyPhoenix Elementary School District #1

May 2017

Cognitive Pro-cess Dimension Definition of Type Additional Clarification Comments

Remembering Finding or remembering information.

Answers questions that stem from prompts such as list, find, name, identify, locate, describe, memorize, or define.

Understanding Understanding and making sense out of information.

Answers questions that stem from prompts such as interpret, summarize, explain, infer, paraphrase, or discuss.

Applying Using information in a new (but similar) situation.

Answers questions that stem from prompts such as use, diagram, make a chart, draw, apply, solve, or calculate.

Analyzing Taking information apart and exploring relationships.

Answers questions that stem from prompts such as categorize, examine, compare and contrast, or organize.

EvaluatingCritically examining information and making judgments.

Answers questions that stem from prompts such as judge, critique, defend, or criticize.

Creating Using information to create something new.

Answers questions that stem from prompts such as design, build, construct, plan, produce, devise, or invent.

The auditors used the classifications presented in Exhibit 2.3.10 for the analyses found in subsequent exhibits.

Auditors were informed that the district’s pre-post assessments and benchmark assessments are housed in the Galileo® Online Instructional Improvement and Effectiveness System from Assessment Technology Incorporated (ATI). The system provides an array of curriculum, assessment, instructional effectiveness, and reporting tools to the district. Auditors examined how well Galileo® pre-assessments and benchmark assessments align with the AZCCRS using the lens of the dimensions of content, context, and cognition. The state-tested core content areas of English language arts and mathematics at grades 3, 5, and 8 were reviewed.

The auditors selected Galileo® pre-assessment and benchmark assessment items linked to a given AZCCRS indicated in the detailed item analysis of the test. If the content, context, or cognitive type of the item selected for analysis fully matched the content, context, or cognitive type of the AZCCRS, it was considered topologically aligned. If the content, context, or cognitive type of an assessment item did not fully match the AZCCRS, it was classified as inadequately aligned. When content is inadequately aligned auditors go no further with analysis of context and cognition. If the item exceeded the content, context, or cognitive type of the standard, it was considered deeply aligned.

Exhibit 2.3.11 provides an analysis of sample English language arts Galileo® pre-assessment and Benchmark 1 assessment items to Arizona College and Career Readiness Standards (2010) for grades 3, 5, and 8. For purposes of efficiency in the alignment analyses write-up, the Arizona College and Career Readiness Standards will be referred to as the AZCCRS.


Exhibit 2.3.11

Internal Consistency of District Galileo® Pre-Assessment and Benchmark Assessment Items To Arizona College and Career Readiness Standards in English Language Arts

Grades 3, 5, and 8Phoenix Elementary School District #1

May 2017

AZCCRS Assessment Items Alignment Analysis

Grade 3 Pre-Assessment: 2016-17 ATI AzMERIT ELA 03 Gr. _TE Pretest-IE

AZ-L.3.2b Conventions of Standard English: Use commas in addresses.

12. Click on all the spaces where commas should be added.

My friend lives at 1635 Montebella Road Tucson Arizona.

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Topologically AlignedThe assessment item and the AZCCRS are matched in context. The assessment item does not expand or exceed the AZCCRS in context.Cognition: Topologically AlignedThe AZCCRS and assessment item generate understanding in student thinking.

AZ-RL.3.7 Integration of Knowledge and Ideas: Explain how specific aspects of a text’s illustrations contribute to what is conveyed by the words in a story (e.g., create mood, emphasize aspects of a character or setting).

37. From “Sam Needs to Work”

Why does Sam look unhappy?

A. He cannot play outside in the rain.B. He is not done with his book report.C. He does not have the book he needs.D. There is no one around for him to play.

Content: Inadequately AlignedThe AZCCRS requires the student to explain how specific aspects of a text’s illustrations contribute to what is conveyed by the words in a story. The assessment item requires the student to simply select the reason why Sam looks unhappy. Because content is inadequately aligned auditors go no further with analysis of context and cognition.

Z-L.3.4a Vocabulary Acquisition and Use: Use sentence-level context as a clue to the meaning of a word or phrase.

40. Read the sentence.

We wanted to ship the box to our grandmother. What does “ship” mean as it is used in this sentence?

A. a large boatB. a spacecraftC. to sendD. to hold

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Topologically AlignedThe assessment item is multiple choice and does not expand or exceed the AZCCRS in context.Cognition: Inadequately AlignedThe assessment item generates understanding in student thinking, but not application as suggested by the AZCCRS.


Exhibit 2.3.11 (continued)Internal Consistency of District Galileo® Pre-Assessment and Benchmark Assessment Items

To Arizona College and Career Readiness Standards in English Language Arts Grades 3, 5, and 8



Grade 3 Benchmark #1: 2016-17 ATI AzMERIT ELA 03 Gr. TE CBAS #1

AZ-RL.3.2 Key Ideas and Details: Recount stories, including fables, folktales, and myths from diverse cultures; determine the central message, lesson, or moral and explain how it is conveyed through key details in the text.

1. From “Brad’s Report”

(Student reads text independently)

Read the list of events. Click and drag them into the order they happen in the story, starting with the first event at the top.

Brad plays with his dog after school.

Brad and Rob ride bikes after school.

Brad’s teacher reminds the class about their reports.

Brad gets to stay up late.

Brad and his dad dig a garden.

Brad’s mom says he cannot go to the zoo.

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Inadequately AlignedThe AZCCRS requires the student to recount stories, determine the central message, and explain how it is conveyed through key details in the text. The assessment item has the student rearrange statements in a story so they are in the same sequence as the story. The item does not have the student determine the central message or explain how it is conveyed.Cognition: Inadequately AlignedThe assessment item generates only understanding in student thinking, while the AZCCRS requires application.

AZ-RI.3.3 Key Ideas and Details: Describe the relationship between a series of historical events, scientific ideas or concepts, or steps in technical procedures in a text, using language that pertains to time, sequence, and cause/effect.

13. From “The Children’s Museum”


What is the first thing you see in the museum?

1. a fire truck2. an ocean3. a dinosaur4. a shark

Content: Inadequately AlignedThe AZCCRS requires the student to describe relationships between events, scientific ideas or concepts, or steps in technical procedures in a text. The assessment item requires the student to simply select the answer to a story detail question. Because content is inadequately aligned auditors go no further with analysis of context and cognition.

AZ-RI.3.6 Craft and Structure: Distinguish their own point of view from that of the author of a text.

30. Read the paragraph.

Bird-watching is easy to get started with. You do not need to buy anything. If you like, you can check out a book on birds from your library. It will help you know the names of the birds you see. You can have a lot of fun spotting interesting birds, and you do not even need to leave your backyard to do so.

What does the author think about bird-watching?

A. It is a lot of fun to do.B. Bird-watchers must know a lot about birds.C. It is more fun when you have a book about

birds.D. It is a good way to get exercise.

Content: Topologically AlignedThe assessment item matches the TEKS/SE in content.Context: Inadequately AlignedThe AZCCRS requires the student to distinguish his/her own point of few from that of the narrator. The assessment requires the student to decide what the author is thinking about bird-watching. The mode of response for the assessment item is multiple choice, and the items do not expand or exceed the AZCCRS in context.Cognition: Inadequately AlignedThe assessment item generates understanding in student thinking, while the AZCCRS requires application.







AZ-RL.5.9 Integration of Knowledge and Ideas: Compare and contrast stories in the same genre (e.g., mysteries and adventure stories) on their approaches to similar themes and topics.

4. From “Eagle Boy” and “The Lake of Lost Spirits”

(Student reads two stories independently)

How are these stories different?

A. “Eagle Boy” takes place over one day, while “The Lake of Lost Spirits” takes place over many days.

B. The setting of “Eagle Boy” changes, while “The Lake of Lost Spirits takes place in only one location.

C. Eagle Boy apologizes for his misdeeds, but Plain Feather does not.

D. Eagle Boy survives, but Plain Feather does not.

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Inadequately AlignedThe mode of response for the assessment item is multiple choice, and the item does not meet the AZCCRS in context, since students are not fulfilling the intent of the standard in its entirety.Cognition: Inadequately AlignedThe assessment item generates understanding in student thinking, but the standard requires applying. Cognition was not aligned.

AZ-RI.5.1 Key Ideas and Details: Quote accurately from a text when explaining what the text says explicitly and when drawing inferences from the text.

31. From “John Pennekamp Coral Reef State Park”


Which two are requirements for joining a snorkeling tour?

A. Knowing which sharks are dangerousB. Being a good swimmerC. Knowing how to snorkelD. Attending a snorkeling classE. Wearing a life vest

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Inadequately AlignedThe mode of response for the assessment item is multiple choice, and the item does not meet the AZCCRS in context.Cognition: Inadequately AlignedThe assessment item only generates remembering in student thinking, while the standard requires applying the skill with new and unknown text in an open-ended setting.

AZ-RL.5.5 Craft and Structure: Explain how a series of chapters, scenes, or stanzas fits together to provide the overall structure of a particular story, drama, or poem.

2. From “Eagle Boy”


Which quotation is from the climax of the story?

A. “The eagle launched himself into the air and the two began to fly away.

B. “The eagle gave Eagle Boy a coat of feathers and taught him how to fly.”

C. “Oh my eagle,’ cried Eagle Boy, ‘if you can hear me, please help me return to Sky Land.’”

D. “So, Eagle Boy resumed his work in the fields.”

Content: Inadequately AlignedThe AZCCRS requires the student to explain how a series of chapters, scenes, or stanzas fits together to provide the overall structure of a story, drama, or poem. The assessment item requires the student to select the quotation that is from the climax of the story. Because content is inadequately aligned auditors go no further with analysis of context and cognition.







AZ-RI.5.8 Integration of Knowledge and Ideas: Explain how an author uses reasons and evidence to support particular points in a text, identifying which reasons and evidence support which point(s).

16. from “My First Yoga Class”

Which quotation supports the author’s claim that yoga has improved her gymnastics?

A. “Being on the balance beam used to scare me a little, but I am not as afraid of falling anymore.”

B. “The changes I’ve noticed in the last month tell me that I will keep doing yoga, maybe for my whole life.”

C. “Since I am a gymnast, I thought that yoga might help me win more competitions and be able to perform harder moves.”

D. “There is always very calming music playing, which makes it easy to forget about everything that might be bothering me.”

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Inadequately AlignedThe mode of response for the assessment item is multiple choice, and the item does not meet the AZCCRS in context, since students are given only closed response opportunities and the standard requires open-ended responses.Cognition: Topologically AlignedThe standard and assessment item generate understanding in student thinking.

AZ-RI.5.4 Craft and Structure: Determine the meaning of general academic and domain-specific words and phrases in a text relevant to a grade 5 topic or subject area.

13. From “Getting Started with Yoga”


Click on the phrase that best defines “asana.”

Yoga is a form of exercise that began in India 26,000 years ago. It is a system of movement and breathing designed to make a person healthy and peaceful. People who study yoga are called “yogis.” Yoga is complicated. There are many different exercises for the body, types of breathing, and even meditations, which are ways of concentrating to bring peace to the mind. One of the great things about yoga, however, is that a person doesn’t have to learn the entire system. A few simple exercises done regularly will bring health benefits to anyone. The simple body movements in yoga are called “asanas.” One of the simplest asanas is the sitting forward bend.

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Topologically AlignedThe assessment item is a match with the AZCCRS but does not does not expand or exceed the Standard in context.Cognition: Topologically AlignedThe standard and assessment item generate understanding in student thinking.

AZ-L.5.2b Conventions of Standard English: Use a comma to separate an introductory element from the rest of the sentence.

25. Click on the word that should have a comma after it.

If you like you can ask Gabby to come with us to the mall.

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Topologically AlignedThe assessment item is a match with the AZCCRS but does not does not expand or exceed the Standard in context.Cognition: Topologically AlignedThe standard and assessment item generate application in student thinking.







AZ-RL.8.3 Key Ideas and Details: Analyze how particular lines of dialogue or incidents in a story or drama propel the action, reveal aspects of a character, or provoke a decision.

4. From “The Open Window”


Why does Framton run out of the house when he sees the men coming across the lawn?

A. He is afraid they will not like that he has been visiting the women.

B. He does not want to meet any more new people.

C. The tea has made him feel ill.D. He thinks he is seeing ghosts.

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Inadequately AlignedThe assessment item is multiple choice and does not meet the AZCCRS in context, since the standard requires analysis, an open-ended type of response.Cognition: Inadequately AlignedThe standard specifies analyzing in student thinking, but the test only engages students in understanding.

AZ-RI.8.6 Craft and Structure: Determine an author’s point of view or purpose in a text and analyze how the author acknowledges and responds to conflicting evidence or viewpoints.

26. from “Should Wigs Stay or Go?”

Which two quotations reveal the author’s bias?

A. “A small part of British society, though, is still required to wear those scratchy and itchy horsehair wigs!”

B. “In that time, the more elaborate the wig, the more fashionable the person wearing it was considered.”

C. “This gave them protection in public against criminals they may have previously met in court.”

D. “They did not seem to consider the discomfort and risk of heat stroke to which lawyers and judges are subjected as important.”

E. “To do away with the outfit would be the start of a ‘downward slide.’”

Content: Inadequately AlignedThe AZCCRS requires the student to determine an author’s point of view or purpose AND analyze how the author acknowledges and responds to conflicting evidence or viewpoints. The assessment item requires the student to select quotations that reveal author bias. Because content is inadequately aligned auditors go no further with analysis of context and cognition.

AZ-L.8.4a Vocabulary Acquisition and Use: Use context (e.g., the overall meaning of a sentence or paragraph; a word’s position or function in a sentence) as a clue to the meaning of a word or phrase.

32. Read the sentence.

He grinned sheepishly, recovered himself, and rode away on his horse.

The word “recovered” has more than one meaning. What is the meaning of the word “recovered” in this sentence?

A. to reclaim a useful substance from wasteB. to regain a state of control or balanceC. to get back something stolen or lostD. to wrap an object with new material

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Topologically AlignedThe mode of response for the assessment item is multiple choice, and the item does not expand or exceed the AZCCRS in context.Cognition: Topologically AlignedThe standard and assessment item generate understanding in student thinking.







AZ-RI.8.9 Integration of Knowledge and Ideas: Analyze a case in which two or more texts provide conflicting information on the same topic and identify where the texts disagree on matters of fact or interpretation.

10. From “Repeal Seatbelt Laws” and “Seatbelt Laws Save Lives”


On which point do the authors of “Repeal Seatbelt Laws” and “Seatbelt Laws Save Lives” disagree?

A. Seatbelts help keep the cost of healthcare down.

B. Seatbelt laws are an acceptable part of a free society.

C. High fines encourage people to buckle up.D. Seatbelts can trap people in their cars.

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Inadequately AlignedThe mode of response for the assessment item is multiple choice, and the item does not meet the AZCCRS in context. The standard requires the student to analyze two separate texts for differences in fact and interpretation. How the student is to demonstrate this is not specified, but the standard requires a full analysis of the texts. The item only asks about one area where authors disagree.Cognition: Inadequately AlignedThe assessment item does not generate analyzing in student thinking; students are only required to use understanding in their thinking.

AZ-RL.8.5 Craft and Structure: Compare and contrast the structure of two or more texts and analyze how the differing structure of each text contributes to its meaning and style.

18. From “29”


Which poem reflects the structure of “29”?

A. “Wind”Wind through the tall trees;How amazing it can beTo see and to hear.

B. “Longing” Yesterday, we met. Today, we sing! Tomorrow…we part.

C. “At Home” When the call of the wild Enters his soul, A howl escapes.

D. “Rain” A drip, a drop, A sprinkle on top: Rain.

Content: Inadequately AlignedThe AZCCRS requires the student to compare and contrast the structure of two or more texts AND analyze how the differing structure of each text contributes to its meaning and style. The assessment item requires the student to select the poem that reflects the structure of “29.” The assessment item does not require the student to analyze how the differing structure of each text contributes to its meaning and style. Because content is inadequately aligned auditors go no further with analysis of context and cognition.






Grade 8 (continued) Benchmark #1: 2016-17 ATI AzMERIT ELA 08 Gr. TE CBAS #1

AZ-RL.8.3 Key Ideas and Details: Analyze how particular lines of dialogue or incidents in a story or drama propel the action, reveal aspects of a character, or provoke a decision.

27. From “An Ordinary Romance”


Which statement from Melisande signals that she is resolving her conflict?

A. “MELISANDE. I want romance, I want beauty. I don’t want jokes.”

B. “MELISANDE [smiling and shaking her head at him]. What nonsense you talk!”

C. MELISANDE. Are they? Oh, if I could believe they were!”

D. ‘MELISANDE [gently]. What do you want me to do?”

Content: Inadequately AlignedThe assessment item only matches part of the AZCCRS in content. It fails to address all aspects of the standard.Context: Inadequately AlignedThe mode of response for the assessment item is multiple choice, and the item does not meet the AZCCRS in context.Cognition: Inadequately AlignedThe standard requires analyzing, but the item only engages the student in understanding.

Source: Grades 3, 5, and 8 Galileo® Pre-Assessment and Galileo® Benchmark 1 Assessments

Exhibit 2.3.11 indicates the following:

• Nine Galileo® English language arts pre-assessments and nine Galileo® benchmark 1 assessments, a total of 18 assessment items, were analyzed.

• Four (22%) of the items analyzed were topologically aligned in all three dimensions (content, context, cognition) with the AZCCRS.

• Six (33%) of the items were inadequately aligned in content. When content is inadequately aligned auditors typically go no further with analysis of context and cognition.

• Eight (61%) of the items analyzed for all three of the dimensions (n=13) were inadequately aligned in context.

• No English language arts assessment items were considered deeply aligned with the AZCCRS.

• Five (38%) of the assessment items analyzed for all three of the dimensions were topologically aligned in cognition with the AZCCRS.

Overall, after examining the Galileo assessment, the auditors found that alignment of this assessment with the AZCCRS is inadequate. No items met and exceeded the context and cognitive type expressed in the standards, and only four of the items (less than one fourth) were topologically aligned in all three dimensions. Content alignment only occurred in one third of all the ELA items reviewed.

Of even greater concern was the lack of context or cognitive type alignment. Of the 13 total items that were analyzed for context and cognitive type alignment, only five (39%) were topologically aligned in context and cognitive type. The multiple choice nature of the assessment renders context and cognitive type alignment unlikely, even at the topological level. Deep alignment, which requires the assessment to minimally meet and exceed the contexts and types of cognition required by the standard, was not found in any item.

Exhibit 2.3.12 displays an analysis of sample mathematics Galileo® pre-assessment and benchmark assessment items to Arizona College and Career Readiness Standards (AZCCRS) for grades 3, 5, and 8.


Exhibit 2.3.12

Internal Consistency of District® Pre-Assessment and Benchmark Assessment Items To Arizona College and Career Readiness Standards in Mathematics

Grades 3, 5, and 8Phoenix Elementary School District #1

May 2017


Grade 3 Pre-Assessment: 2016-17 ATI AzMERIT Math 03 Gr. _Pretest TE-IE

AZ-3.MD.C.7c Use tiling to show in a concrete case that the area of a rectangle with whole-number side lengths a and b + c is the sum of a × b and a × c. Use area models to represent the distributive property in mathematical reasoning.

A. Which expression gives the area of the entire rectangle in square units?

A. (4 x 4) x 1B. (4 x 4) x 4C. (4 x 4) x (4 x 1)D. (4 x 4) + (4 x 1)

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Inadequately AlignedThe mode of response for the assessment item is multiple choice, and the item does not meet the AZCCRS in context, as students are not using manipulatives and using models (that they create) to represent the distributive property.Cognition: Inadequately AlignedThe assessment item only generates understanding in student thinking. The standard requires applying.

AZ-3.G.A.2 Partition shapes into parts with equal areas. Express the area of each part as a unit fraction of the whole. For example, partition a shape into 4 parts with equal area, and describe the area of each part as 1/4 of the area of the shape.

8. What part of the area is shaded?

A. ¾B. ½C. ⅓D. ¼

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Inadequately AlignedThe mode of response for the assessment item is multiple choice, .and the item does not meet the AZCCRS in context since the student does not apply the skill in an open-ended context. The item requires the student to select the right answer from the four possibilities given.Cognition: Inadequately AlignedThe standard requires applying in student thinking, but the test item only requires understanding.

AZ-3.MD.A.2 Measure and estimate liquid volumes and masses of objects using standard units of grams (g), kilograms (kg), and liters (l). Add, subtract, multiply, or divide to solve one-step word problems involving masses or volumes that are given in the same units, e.g., by using drawings (such as a beaker with a measurement scale) to represent the problem. Excludes multiplicative comparison problems (problems involving notions of “times as much).

14. A pile of 12 bricks weighs a total of 24 kilograms. If all of the bricks weigh the same, how much does one brick weigh?

A. 2 kilogramsB. 3 kilogramsC. 6 kilogramsD. 12 kilograms

Content: Inadequately AlignedThe assessment item only matches part of the AZCCRS in content.


Exhibit 2.3.12 (continued)Internal Consistency of District® Pre-Assessment and Benchmark Assessment Items

To Arizona College and Career Readiness Standards in Mathematics Grades 3, 5, and 8



Grade 3 Benchmark #1: 2016-17 ATI AzMERIT Math 03 Gr. CBAS #1 TE

AZ-3.MD.B.3 Draw a scaled picture graph and a scaled bar graph to represent a data set with several categories. Solve one- and two-step “how many more” and “how many less” problems using information presented in scaled bar graphs. For example, draw a bar graph in which each square in the bar graph might represent 5 pets.

7. How many fewer students picked fall than spring as their favorite season?

A. 2B. 3C. 5D. 7

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Inadequately AlignedThe mode of response for the assessment item is multiple choice, .and the item does not meet the AZCCRS in context since the student does not apply the skill in an open-ended context. The item requires the student to select the right answer from the four possibilities given.Cognition: Inadequately AlignedThe standard requires applying in student thinking while the test item only requires understanding.

AZ-3.OA.D.9 Identify arithmetic patterns (including patterns in the addition table or multiplication table), and explain them using properties of operations. For example, observe that 4 times a number is always even, and explain why 4 times a number can be decomposed into two equal addends.

17. Which is true of the multiplication table below?

A. The numbers under 2 are all odd.B. The numbers under 3 are all even.C. The numbers under 1 are sometimes

even and sometimes odd.D. The numbers under 4 are sometimes

even and sometimes odd.

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Inadequately AlignedThe mode of response for the assessment item is multiple choice and the item does not meet the AZCCRS in context, as students are not required to explain identified patterns (open-ended).Cognition: Inadequately AlignedThe standard requires applying in student thinking while the test item only requires understanding.

AZ-3.NBT.A.1 Use place value understanding to round whole numbers to the nearest 10 or 100.

27. Which numbers equal 300 when rounded to the nearest ten?

A. 303B. 306C. 297D. 287E. 383

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Topologically AlignedThe mode of response for the assessment item is multiple choice and the item does not expand or exceed the AZCCRS in context.Cognition: Topologically AlignedThe standard and assessment item generate understanding in student thinking.






Grade 5 Pre-Assessment: 2016-17 ATI AzMERIT Math 05 Gr. _Pretest TE-IE

Z-5.MD.C.5c Recognize volume as additive. Find volumes of solid figures composed of two non-overlapping right rectangular prisms by adding the volumes of the non-overlapping parts, applying this technique to solve real world problems.

2. Which two expressions will find the total volume of the figure below?

A. (1 x 1 x 4) + (1 x 1 x 7)B. (1 x 1 x 5) + (1 x 1 x 7)C. (1 x 1 x 6) + 1 x 1 x 4)D. (1 x 1 x 6) + (1 x 1 x 5)

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Inadequately AlignedThe assessment item does not have students apply finding volumes of solid figures to solve real world problems. The mode of response for the assessment item is multiple choice and the item does not expand or exceed the AZCCRS in context.Cognition: Inadequately AlignedThe standard requires applying in student thinking, but the item only requires understanding.

AZ-5.NF.B.5a Compare the size of a product to the size of one factor on the basis of the size of the other factor, without performing the indicated multiplication.

8.

A. Bar AB. Bar BC. Bar CD. Bar D

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Inadequately AlignedThe mode of response for the assessment item is multiple choice and the item does not meet the AZCCRS in context.Cognition: Inadequately AlignedThe standard requires applying in student thinking, but the item only requires understanding.

AZ-5.MD.C.4 Measure volumes by counting unit cubes, using cubic cm, cubic in, cubic ft., and improvised units.

23. What is the volume of this prism in cubic inches?

Use the digits 0-9 to type your answer.

(student types answer here)

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Topologically AlignedThe assessment item does not expand or exceed the AZCCRS in context.Cognition: Topologically AlignedThe standard and assessment item generate applying in student thinking.






Grade 5 Benchmark #1: 2016-17 ATI AzMERIT Math 05 Gr. CBAS #1 TE

AZ-5.NBT.A.3a Read and write decimals to thousandths using base-ten numerals, number names, and expanded form, e.g., 347.392 = 3 × 100 + 4 × 10 + 7 × 1 + 3 × (1/10) + 9 × (1/100) + 2 × (1/1000).

42. Which of the following is 72.368 written in expanded form?

A. 7 + 2 + 0.3 + 0.6 + 0.8B. 7 + 2 + 0.3 + 0.06 + 0.008C. 70 + 2 + 0.3 + 0.06 + 0.008D. 70 + 2 + 0.3 + 0.6 + 0.8

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Inadequately AlignedThe AZCCRS requires the student to read AND write decimals in expanded form. The assessment item only has students reading decimals in expanded form.Cognition: Inadequately AlignedThe standard requires applying in student thinking; the item only requires understanding.

AZ-5.NBT.B.5 Fluently multiply multi-digit whole numbers using the standard algorithm.

24. What is the product?

53x 37

A. 1,861B. 1,961C. 2,061D. 2,161

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Inadequately AlignedThe mode of response for the assessment item is multiple choice AZCCRS requires open-ended problem solving.Cognition: Inadequately AlignedThe standard requires applying in student thinking; the item only requires understanding.

AZ-5.NBT.A.1 Recognize that in a multi-digit number, a digit in one place represents 10 times as much as it represents in the place to its right and 1/10 of what it represents in the place to its left.

30. If the digit in the tens place of 36,784,921 is switched with the digit in the ten thousands place, what is the new number?

A. 32,784,961B. 36,284,971C. 36,724,981D. 36,782,941

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Topologically AlignedThe mode of response for the assessment item is multiple choice and the item does not expand or exceed the AZCCRS in context.Cognition: Topologically AlignedThe standard and assessment item generate understanding in student thinking.

Grade 8 Pre-Assessment: 2016-17 ATI AzMERIT Math 08 Gr._Pretest TE-IE Alignment Analysis

AZ-8.EE.B.6 Use similar triangles to explain why the slope m is the same between any two distinct points on a non-vertical line in the coordinate plane; derive the equation y = mx for a line through the origin and the equation y = mx + b for a line intercepting the vertical axis at b.

12. Order the linear equations according to their y-intercept. Place the equation with the greatest y-intercept on top.

y = 3 – 6xy = 6x – 3y = 3(x – 6)

Content: Inadequately AlignedThe AZCCRS requires the student to use similar triangles to explain why the slope m is the same between any two distinct points on a non-vertical line in the coordinate plane. The assessment item does not require the student to use similar triangles to explain. Because content is inadequately aligned auditors go no further with analysis of context and cognition.






Grade 8 (continued) Pre-Assessment: 2016-17 ATI AzMERIT Math 08 Gr._Pretest TE-IE

AZ-8.G.A.1: Verify experimentally the properties of rotations, reflections, and translations. Properties include: lines are taken to lines, line segments are taken to line segments of the same length, angles are taken to angles of the same measure, parallel lines are taken to parallel lines.

31. A translation takes line l to m. Which is true about line n, the image of line m under the same translation?

A. n is parallel to both l and m.B. n is parallel to l only.C. n is not parallel to either l or m.

Content: Topologically AlignedThe assessment item matches part of the AZCCRS in content.Context: Topologically AlignedThe mode of response for the assessment item is multiple choice and the item does not match the standard’s requirement to “verify experimentally.”Cognition: Inadequately AlignedThe assessment item generates understanding in student thinking, but the standard requires applying.

AZ-8.G.B.6 Explain a proof of the Pythagorean Theorem and its converse.

35. Which set of numbers could represent the side lengths of a right triangle?

A. 0.1, 0.2, 0.3B. 1.8, 2.4, 3C. 3, 8, 10D. 5, 7, 12

Content: Inadequately AlignedThe AZCCRS requires the student to explain a proof of the Pythagorean Theorem and its converse. The assessment item requires the student to select the set of numbers that represents the side lengths of a right triangle. Because content is inadequately aligned auditors go no further with analysis of context and cognition.

Grade 8 Benchmark #1: 2016-17 ATI AzMERIT Math 08 Gr. CBAS #1 TE Alignment Analysis

AZ-8.EE.A.4 Perform operations with numbers expressed in scientific notation, including problems where both decimal and scientific notation are used. Use scientific notation and choose units of appropriate size for measurements of very large or very small quantities (e.g., use millimeters per year for seafloor spreading). Interpret scientific notation that has been generated by technology.

12. The number in scientific notation is written as a decimal. Click on the position of the decimal point.

7.51 x 104

0 0 0 0 0 0 7 5 1 0 0 0 0 0 0

Content: Inadequately AlignedThe AZCCRS requires the student perform operations with numbers expressed in scientific notation. The assessment item requires the student to decide where the decimal should be positioned in a scientific notation. Because content is inadequately aligned auditors go no further with analysis of context and cognition.






Grade 8 (continued) Benchmark #1: 2016-17 ATI AzMERIT Math 08 Gr. CBAS #1 TE

AZ-8.F.A.1 Understand that a function is a rule that assigns to each input exactly one output. The graph of a function is the set of ordered pairs consisting of an input and the corresponding output.

24. Which ordered pair (x, y) can be added to the table so that y is still a function of x?

A. (6, 4)B. (3, -2)C. (0, 4)D. (-2, 5)

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Topologically AlignedThe mode of response for the assessment item is multiple choice and the item does not expand or exceed the AZCCRS in context.Cognition: Topologically AlignedThe AZCCRS and assessment item generate understanding in student thinking.

AZ-8.G.C.9 Know the formulas for the volumes of cones, cylinders, and spheres and use them to solve real-world and mathematical problems.

28. Select two figures that have the same volume.

A. cone with base radius 2 and height 1B. cone with base diameter 2 and height 2C. cone with base radius 4 and height 2D. sphere with radius 1E. sphere with diameter 1

Content: Topologically AlignedThe assessment item matches the AZCCRS in content.Context: Inadequately AlignedThe mode of response for the assessment item is multiple choice and the item does not meet the AZCCRS in context. The standard requires open-ended student responses and problem solving.Cognition: Inadequately AlignedThe assessment item generates understanding in student thinking, but the standard requires application.

Source: Grades 3, 5, and 8 Galileo® Mathematics Pre-Assessment and Galileo® Benchmark 1 Assessments


• Nine Galileo® mathematics pre-assessments and nine Galileo® mathematics benchmark 1 assessments, a total of 18 assessment items, were analyzed.

• Four (22%) of the mathematics items analyzed were topologically aligned in all three dimensions (content, context, cognition) with the AZCCRS.

• Four (22%) of the mathematics items were inadequately aligned in content. When content is inadequately aligned auditors go no further with analysis of context and cognition.

• Four (28%) of the mathematics items analyzed for all three of the dimensions (n=14) were adequately aligned in context, and 10 were not (72%).

• Four (28%) of the mathematics items analyzed for all three dimensions (n=14) were adequately aligned in cognitive type.

• No mathematics assessment items were considered deeply aligned with the AZCCRS.


Exhibit 2.3.13 summarizes the analyses of alignment and cognitive type of a sample of Galileo® English language arts and mathematics pre-assessments and benchmark assessments with Arizona Standards for College and Career Readiness (AZCCRS).

Exhibit 2.3.13

Summary of the Analyses of Topological Alignment and Cognitive Type of Galileo® Assessments With the Arizona College and Career Readiness Standards

English Language Arts and MathematicsPhoenix Elementary School District #1

May 2017

Dimensions Cognitive TypesContent Context Cognition Understanding Applying Analyzing

# It

ems

Ana

lyze

d

% A

ligne

d

# It

ems

Ana

lyze

d

% A

ligne

d

# It

ems

Ana

lyze

d

% A

ligne

d

# It

ems

%

Und

erst

and

# It

ems

% A

pply

# It

ems

% A

naly

ze

ELA Total 18 66 13 38 13 38 13 85 13 7 13 0Math Total 18 78 14 28 14 28 14 93 14 8 14 0

All Total 36 72 27 33 27 33 27 89 27 7 27 0Note: Variation in number of items analyzed is due to audit requirement that the content of an item must be aligned if the item is to be analyzed for context and cognition.


• Of the 36 items analyzed, 72% were topologically aligned in content.

• Nine of the 27 items (33%) analyzed for all three dimensions (content, context, cognition) were topologically aligned in context.

• Nine (33%) of the 27 items analyzed for all three dimensions were topologically aligned in cognition.

• Only one assessment item exceeded understanding in the type of cognition required. Most only required understanding

Overall, the auditors found the alignment of the Galileo assessments to the AZCCR low, and confined to topological alignment, only. The rigor of the assessment does not exceed understanding, while the standards expect applying and analysis. This level of alignment impedes the value and validity of the assessment data in predicting student performance on the state tests, which are more demanding than the Galileo and better aligned to the standards.

In an online survey and in interviews, teachers and administrators expressed concerns over the quality of the Galileo® assessments. A sample of these comments follows:

• “We are adopting tests that have been abandoned in other states. Galileo is NOT CC-aligned and looks nothing like the AzMERIT.” (Teacher Survey Response)

• “I feel that the Galileo benchmark assessments are not directly linked to the standards.” (Teacher Survey Response)

• “The district Galileo assessment is AWFUL. There are questions on the test that aren’t even standards for that grade level. Additionally, there are trick questions that don’t actually measure the standards. Some of the questions are like brain teasers.” (Teacher Survey Response)

• “Galileo as an assessment does not do the job of giving clear information. The questions are not aligned assessment questions. They are not high quality questions.” (Teacher Survey Response)


• “AzMERIT is going to test the standard but we have no idea how they are going to test the standard. Just because a student got it right on the Galileo doesn’t mean they will get it right on the AzMERIT because of the different components and ways that exist in measuring the standard.” (Campus Administrator)

• “The Galileo is not effective at all to determine whether students are making progress. Furthermore, many of the questions asked on the test are vague and not helpful in giving kids the feedback they need (reading and math in particular).” (Teacher Survey Response)

• “I do not find data from Galileo to be reliable.” (Teacher Survey Response)

Auditors found that some English language arts and mathematics Galileo® pre-assessment and benchmark 1 assessment items in the samples analyzed were inadequately aligned in content and context with the AZCCRS being tested. While some items analyzed were aligned topographically in content, context, and cognition, no items analyzed were considered deeply aligned by expanding or extending student thinking. Teacher comments reflect concerns about the quality of the Galileo® assessment items.

Summary

Overall, the auditors found that existing curriculum documents for core and non-core subjects did not meet minimum standards for component quality and specificity. Use of the available curriculum was inconsistent and teachers reported relying on many different resources in planning their instruction. The curriculum as it is taught in district classrooms was found to be inconsistent as well, which has resulted in poor articulation and inadequate coordination across grade levels and among schools (see Finding 3.2).

The district Galileo® pre-assessment and benchmark assessment items analyzed were not always congruent with the state standards in content and context. The items analyzed were topologically aligned with the state standards in their cognitive demand. None of the items reviewed expanded or extended student thinking in keeping with district expectations for increasing rigor in student learning.


STANDARD 3: The School District Demonstrates Internal Consistency and Rational Equity in Its Program Development and Implementation.A school system meeting this Curriculum Audit™ standard is able to show how its program has been created as the result of a systematic identification of deficiencies in the achievement and growth of its students compared to measurable standards of pupil learning.

In addition, a school system meeting this standard is able to demonstrate that it possesses a focused and coherent approach toward defining curriculum and that, as a whole, it is more effective than the sum of its parts, i.e., any arbitrary combinations of programs or schools do not equate to the larger school system entity.

The purpose of having a school system is to obtain the educational and economic benefits of a coordinated and focused program for students, both to enhance learning, which is complex and multi-year in its dimensions, and to employ economies of scale where applicable.


The CMSi auditors expected to find a highly-developed, articulated, and coordinated curriculum in the school system that was effectively monitored by the administrative and supervisory staffs at the central and site levels. Common indicators are:

• Documents/sources that reveal internal connections at different levels in the system;

• Predictable consistency through a coherent rationale for content delineation within the curriculum;

• Equity of curriculum/course access and opportunity;

• Allocation of resource flow to areas of greatest need;

• A curriculum that is clearly explained to members of the teaching staff and building-level administrators and other supervisory personnel;

• Specific professional development programs to enhance curricular design and delivery;

• A curriculum that is monitored by central office and site supervisory personnel; and

• Teacher and administrator responsiveness to school board policies, currently and over time.


This section is an overview of the findings that follow in the area of Standard Three. Details follow within separate findings.

Finding 3.1: All students in the district do not have equal access to all programs. Certain subgroups are disproportionately represented in the accelerated learning and special education programs, and other subgroups are more likely to be retained or suspended. Although there is direction for ELL programming and MTSS, these programs are not consistently implemented in schools. Support services for students who are behind are not based on aligned assessment data and are not designed in response to specific student need.

Students come to school with different experiences and prior understandings, which encompass language, habits, understandings, and states of mind. Embracing the unique differences among students, effective school districts make equity a central focus in learning. Through board policies, institutional practices, and beliefs, effective school districts create an equity infrastructure and leadership capacity to advance strategies that virtually ensure every student achieves at high levels regardless of an individual student’s background, race, socioeconomic status, or gender. To attain outcome equity, effective school districts alter instruction, time, and resources increasing learning opportunities and outcomes for those students who have historically been low achievers.


In school systems, the concept of equity refers to treating students according to need, such as allocating additional resources to schools with high percentages of students living in poverty. Equality, however, refers to treating students the same, which is expected when dealing with access to programs and content. The audit expects both equity and equal access in school districts since, without the judicious application of each in different circumstances, diminished student learning will result. Equity is a hallmark of effective school districts, since the effects of poverty are well-known and its impact on student preparedness to learn and on their success in school is profound. Equity is necessary to ameliorate the effects of poverty and to provide those students with a level playing field. Additionally, effective districts work to ensure that each student has equal access to programs and services and is not prevented from participating in certain programs due to cultural or linguistic background or income level. These districts assiduously train teachers and building leaders in the special needs and circumstances related to poverty so that truly student-centered instruction may result.

Related to equal access is assuring that students who are not performing on grade level have a system of supports in place to accelerate their learning, when and where possible, or to assist them in accessing on-level content despite gaps in prerequisites. ELL programs are an answer to providing students whose native language is not English with support in accessing the content and assuring their success despite language barriers. The auditors, therefore, examined the programs and supports in place to ensure Phoenix Elementary School District #1 students experience academic success.

To determine the extent of the supports available to students with special needs and to evaluate the equality of program access and overall equity in decision making, proportionality of discipline actions and retention, and proportionality of program enrollment, the auditors reviewed documents and policies presented to them by district leaders. They interviewed stakeholders concerning program access and implementation, administered online surveys to teachers, school leaders, and parents, and visited school sites to observe instructional delivery.

Overall, the auditors found that certain subgroups do not have equal access to all programs in the system, particularly the Montessori program and the Accelerated Learning Procedures program. Other student subgroups are disproportionately represented in the students who are suspended or retained. Low-income and African American male students are more likely to be suspended, retained, and referred for special education. The ELL program, although specific in its format and design, is not implemented consistently and teachers report feeling inadequately prepared to effectively deal with diversity in the classroom. The Multi-Tiered System of Supports (MTSS) process is not clearly defined and is not linked to high quality, reliable, and aligned assessments.

Auditors found direction in district documents and policy concerning access to programs. In the Student Handbook, 2016-17, it states under the section non-discrimination, “No person…shall, on the basis of gender, creed, color, race, sexual orientation, homelessness or disabling condition, be excluded from participation in, be denied the benefits of or be subjected to discrimination under any educational program or activity.” Board Policy JB: Equal Educational Opportunities, states students’ rights to access the educational program. This policy states, “The right of a student to participate fully in classroom instruction shall not be abridged or impaired because of race, color, religion, sex, age, national origin, and disability, or any other reason not related to the student’s individual capabilities.” Board Policy AC: Nondiscrimination/Equal Opportunity requires non-discrimination related to equal opportunity. “The Board is committed to a policy of nondiscrimination in relation to race, color, religion, sex, age, national origin, and disability. This policy will prevail in all matters concerning staff members, students, the public, educational programs and services, and individuals with whom the Board does business.”

Program Access

Despite the assurances in policy, the auditors did not find that all students have equal access to all programs in the district. The Phoenix Elementary School District #1 serves the urban core of Phoenix, and, as such, has a high percentage of students who receive free or reduced-price (F/R) lunches. Over 80% of the students in the district are considered to be children in poverty. Certain schools have over 90% of their students living in poverty. However, two schools in the district have low percentages of children who receive F/R lunch, each with an application process. The Shaw Montessori program is a program funded by desegregation monies. Unlike the rest of the district, its student population is only 35% low income. The Magnet Traditional School


also has a below-average percentage of children who receive F/R priced lunch. Both schools have unusually low percentages of ELL students (Shaw Montessori has no ELL students) when compared to other schools in the district. In both cases, the data suggest that students of poverty and for whom English is not their first language do not have equal access to these programs. Enrollment data for these two schools, compared to other district schools, are presented in Exhibit 3.1.1.

Exhibit 3.1.1

English Language Learner and Free/Reduced lunch Enrollments by SchoolPhoenix Elementary School District #1

May 2017

0%10%20%30%40%50%60%70%80%90%

100%

School and Program Enrollment: F/R Lunch and ELL

F/R Lunch ELL

As can be seen in Exhibit 3.1.1:

• Most schools have at least 80% of their population eligible to receive free or reduced-price lunch.

• The Magnet school’s population is 70% low income, while Shaw Montessori reports only 35% of its students to be low-income.

• These two schools have the lowest percentages of ELL students, when compared to the district average and to other schools. District-wide, around 18% of the students are English learners, while at Magnet, only 7.23% are ELL and at Shaw, no students are ELL.

The auditors heard and received many comments regarding perceived inequities from stakeholders during interviews and via the online surveys.

• “Shaw Montessori has a field trip a week. Bethune is in a depressed neighborhood so they barely get a field trip a year. You have a very active community at the Montessori school; they’re throwing money at that school all the time.” (Central Office Administrator)

• “[We need] increased diversity at Shaw Montessori. More advertising to students in district about this excellent school. Many of the students are out of district. Would love to see more minority and low income families have the opportunity to benefit from this school.” (Parent Survey Response)

• “A challenge is the inequity in the district. I noticed, for example, that there is an aide in every classroom at the Magnet but not all schools. That’s an issue of equity.” (Teacher)

• “Extreme inequity exists between schools, with respect to resources, staffing, and quality of education.” (School Leader Survey Response)


• “Shaw has a lot of resources that others don’t. I would like to see it where all the students have equal access to those resources.” (Teacher Survey Response)

• “We definitely have some extreme needs at the school. The needs are different. We need to see schools as individual. There is a bit of favoritism—some have more power. Distribution of time [of district support] is not equal.” (Teacher Survey Response)

Auditors found unequal access to the Accelerated Learning Procedures (ALPS) when examined by ethnic group and gender. These data are presented in Exhibits 3.1.2 and 3.1.3.

Exhibit 3.1.2

Accelerated Learning Procedures Enrollment by GenderPhoenix Elementary School District #1

May 2017

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

GT Enrollment

District Enrollment

Gifted Enrollment by Gender

Females Males


• Just under one half of the district’s total student population is female (49%), but females comprise only 42.5% of the district’s Accelerated Learning Procedures. Representation by gender is disproportional with overall district enrollment.


Exhibit 3.1.3 presents the data on ALPS enrollment proportionality by ethnicity.

Exhibit 3.1.3

Accelerated Learning Procedures Enrollment by EthnicityPhoenix Elementary School District #1

May 2017

2% 2%6%

80%

2%8%

3% 1%

8%

81%

2%5%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

AmericanIndian

Asian AfricanAmerican

Hispanic Two or more White

ALPS Enrollment by Ethnicity

GT Enrollment District Enrollment


• Although Asian students comprise only 1% of district enrollment, they represent twice that rate in the ALPS program (2%).

• American Indian students represent 3% of the district enrollment but only 2% of the ALPS enrollment.

• Black students represent over 8% of district enrollment but only 6% of the ALPS program.

• Hispanic students are more proportionally represented with 81% of the population and 80% of the ALPS program.

• White students are over-represented. They represent 5% of the district’s total enrollment, but 8% of the ALPS enrollment.

Auditors then examined enrollment in special education. Certain disabilities are physiological and are, therefore, not expected to be proportional. However, other categories of special education services are subject to criteria that can be more objectively applied. The auditors reviewed if overall identification of special education students is proportional by gender and ethnicity. The results are presented in Exhibits 3.1.4 and 3.1.5.


Exhibit 3.1.4

Special Education Enrollment by GenderPhoenix Elementary School District #1

May 2017

0% 20% 40% 60% 80% 100%

Special Education

District Enrollment

Special Education Enrollment by Gender

Males Females

As can be seen in Exhibit 3.1.4, males are over-represented in special education. Males represent just over half the district enrollment but almost 70% of the special education program enrollment.

Exhibit 3.1.5 presents special education enrollment data by ethnic group.

Exhibit 3.1.5

Special Education Enrollment by EthnicityPhoenix Elementary School District #1

May 2017

3% 1%8%

81%

2%5%4%

1%

11%

77%

1%6%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

AmericanIndian


Hispanic Two or more White

2016-17 SPED Enrollment

District Enrollment SPED Enrollment


The following is evident in Exhibit 3.1.5:

• American Indian, African American, and White students are over-represented in the special education program. These subgroups represent 16% of the total district enrollment but 21% of the total special education population.

• African American students, in particular, are over-represented in the program by almost 3%.

• Hispanic students are under-represented. They comprise over 81% of the student population, but only 77% of the special education student enrollment.

Disproportionality in program enrollment may be indicative of several factors, some of which may be based on equity issues related to cultural proficiencies. Hispanic students’ under-representation may indicate a lack of referral for these students attributing their performance to language proficiency issues, when actual disabilities may require improved or more comprehensive accommodations. The auditors did not speculate as to why such disproportionalities exist.

Overall, the auditors found disproportionalities in program enrollment and identification by gender and ethnicity. Free/reduced lunch data were not available to the auditors by specific program to see if disproportionalities exist for this subgroup.

Disciplinary Actions and Retention

Another measure of equity is the extent to which certain subgroups receive disciplinary actions or are retained. The auditors first examined the gender and ethnicity of all students who were placed in out-of-school and in-school suspension for the 2015-16 school year. This school year was the most recent for which the data were available. These data are presented in Exhibit 3.1.6.

Exhibit 3.1.6

Disciplinary Actions by GenderPhoenix Elementary School District #1

2015-16 School Year

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

TOTAL Displine

District Enrollment

Displinary Actions by Gender

Males Females

As can be seen in Exhibit 3.1.6, males were overrepresented in suspensions for the 2015-16 school year, by over 50% of their enrollment percentage. They accounted for over three/fourths of all suspensions.


Exhibit 3.1.7 presents similar data by ethnic subgroups.

Exhibit 3.1.7

Disciplinary Actions by EthnicityPhoenix Elementary School District #1

2015-16 School Year

5%0%

22%

67%

2% 4%3% 1%

8%

81%

2%5%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

AmericanIndian


Hispanic Two or More White

TOTAL Displinary Actions

TOTAL Displinary Actions District Enrollment

Exhibit 3.1.7 displays the following:

• African American students represented just over 8% of the total district enrollment but 22% of the total suspensions. They were suspended at almost three times their enrollment percentage rate.

• American Indian students were suspended at 1.7 times their enrollment. They were 3% of the total district enrollment in 2015-16 but received 5% of the suspensions.

• Hispanic students were under-represented. They accounted for over 81% of the student enrollment but just over two thirds (67%) of the total suspensions.

• White students represented 5% of the district enrollment but were only 4% of the suspensions.


The auditors then examined disciplinary actions by program. These data are presented in Exhibits 3.1.8, 3.1.9, and 3.1.10.

Exhibit 3.1.8

Disciplinary Actions by Program English Language Learner SubgroupPhoenix Elementary School District #1

2015-16 School Year

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%

% of Disciplinary Actions

% of District Enrollment

Total Disciplinary Actions (ISS/OSS)Special Education Students

Special Education Students Non-Special Education Students

Exhibit 3.1.8 shows that ELL students are suspended at a rate below their district enrollment. They represent over 18% of the district population but only 15% of suspensions.

Exhibit 3.1.9 presents the data for special education students.

Exhibit 3.1.9

Disciplinary Actions by Special Education Program SubgroupPhoenix Elementary School District #1

2015-16 School Year

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%



Total Disciplinary Actions (ISS/OSS)Special Education Students

Special Education Students Non-Special Education Students

Exhibit 3.1.9 shows that special education students represent 11% of the total district enrollment, but 18% of all disciplinary actions in 2015-16.


Exhibit 3.1.10 displays the data for free and reduced lunch students.

Exhibit 3.1.10

Disciplinary Actions by Free/Reduced Lunch Program SubgroupPhoenix Elementary School District #1

2015-16 School Year

0% 10% 20% 30% 40% 50% 60% 70% 80% 90% 100%



Total Disciplinary Actions (ISS/OSS)F/R Price Lunch Students

Students Receiving Free/Reduced Price Lunch

Students Not Receiving Free/Reduced Price Lunch

As can be seen in Exhibit 3.1.10, Free/Reduced Lunch students represent 83% of the total population but 91% of all the suspensions in 2015-16.

The auditors also examined retention data by subgroup. Exhibit 3.1.11 presents district retention data by program subgroup.

Exhibit 3.1.11

Retention Data by Program SubgroupPhoenix Elementary School District #1

2015-16 School Year

44%

56%

16%

84%

19%

82%

11%

89%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

ELL Non-ELL SPED Non-SPED

Retention Data by Subgroup

% Retained % District Enrollment



• ELL students are retained at more than double their enrollment. They represent 44% of all students retained but only 19% of the students in the district.

• Special Education students represent 16% of the students retained but 11% of the total district students.

Overall, the auditors found disproportionality in program enrollments and representation, when examined by gender and ethnicity.

Program Quality and Effectiveness

Auditors examined the design and delivery of certain programs in the district. Teachers were asked to rate the effectiveness of a variety of programs offered in the district that are intended to meet certain subgroups’ specific needs. These data are presented in Exhibit 3.1.12.

Exhibit 3.1.12

Teachers’ Ratings of Quality of District Programs On Online Survey (N=93)


0% 20% 40% 60% 80% 100%

Special Education/IEP

504 Plan

ESL/Bilingual

ALPS

Title I

Interventions

Other (specify below)

The overall quality of programs at my campus designed to support students with special learning needs is indicated below:

Excellent Good Mediocre Poor Not available at my campus

Exhibit 3.1.12 shows the following:

• Teachers’ ratings for Title I programs are the highest, with over two thirds rating these programs good or excellent.

• Just around half of the teachers rated the services provided with the ALPS program, 504 plans, and ESL/Bilingual program at good or excellent. Remaining teachers rated these services mediocre or poor or they were not provided in their school.

• Over half of the teachers responding to the survey (50 out of 93) rated the special education services as good or excellent.


Principals were also asked to rate the quality of these programs. These results are presented in Exhibit 3.1.13.

Exhibit 3.1.13

Principals’ Perceptions Regarding Special Programs On Online Survey (N=9)


0% 20% 40% 60% 80% 100%

Special Education/IEP

504 Plans

ELL/Bilingual

ALPS

Title I

Interventions

Quality of Programs: Principals' Ratings

Poor Mediocre Good Excellent


• Principals were least positive concerning the ALPS program. Less than one fourth rated it Good while the remainder rated it as Mediocre or Poor. None of the respondents rated it as Excellent.

• The Special Education and Interventions programs were considered good or excellent by two thirds of principals.

• ELL/Bilingual and 504 Plans were rated Good or Excellent by most of the principals.

Overall, ratings for these programs, particularly ALPS, interventions, ELL/bilingual (by teachers) and special education, were not overwhelmingly positive. The auditors also examined more specifically the quality of the ELL program, the effectiveness of tiered interventions, and special education. The information is presented in the sections that follow.

ESL/ELL/Bilingual Programming

The State of Arizona has state-mandated structured English Immersion that is specifically delineated in state legislative code. Providing services to ELL students is required by law; when enough students are in a class, the teacher is required to deliver a four-hour block of structured English Immersion with specific components. Although the program itself is specifically defined by the state, the auditors found inadequate direction at the district level for its implementation, and program delivery was found to be inconsistent.

The auditors reviewed district documents regarding the ELL program and found some written direction in place. Developed in the 2015-16 school year, there is a type of plan although no title is given. In 2016, a second document was created titled, “English Language Learner Program.” This document includes all the components of the original document but with added sections from the Office of English Language Acquisition Services at the Department of Education.

While these documents outline a philosophy for the program and note the requirements by the district and state for how services are expected to be delivered and how students are to be assessed and with what instruments, direction for instructional delivery is limited. The plan lacks a specific protocol or resource that lists high quality strategies with specific descriptions for how they are to be implemented in the classroom, and no clear


distinction in the strategies to be used based on a student’s proficiency level beyond the minutes of instruction required by the state. Specific examples of how to approach content with sample language objectives and language development strategies are not provided.

The district has offered training to teachers on Structured English Immersion (SEI) delivery and on English Language Proficiency Standards (ELPS)-driven instruction, as well as on differentiating for ELLs, but no training was longer than three hours in duration, most were 1-2 hours long. When asked about professional development offerings, only 15% of teachers mentioned any training on ELL instruction. One teacher stated on the online survey, “In my time at the district, we have only had 2 ELL trainings.”

Despite the written directions for the ELL program, no teacher or principal mentioned the document at any time during interviews or on the online survey. Many questions regarding direction for the program continue to exist in the district. Teachers were asked specifically about the ELL program in the district. Their responses are presented in Exhibit 3.1.14.

Exhibit 3.1.14

Teachers’ Perceptions Regarding ELL Program On Online Survey (N=91)


0% 20% 40% 60% 80% 100%

Our district has a well-designed plan to support students whoseprimary language in not English.

There is an explicit instructional model teachers use for Englishlanguage development.

All students have full access to the core curriculum through shelteredlanguage instruction or primary language support.

My school has fully implemented the district plan for Englishlanguage learners.

I have been trained in effective strategies for working with Englishlanguage learners.

My teaching is very effective with my English language learnerstudents.

Please respond to the following questions about the district's program for English language learners.

Don't Know Strongly Disagree Disagree Agree Strongly Agree


• Teachers had the highest agreement with the statements that their teaching is very effective with ELL students and that they have been trained in effective strategies to work with these students.

• Almost one half of teachers disagreed or didn’t know if their school has fully implemented the district plan for ELL students.

• Over one third of teachers disagreed or didn’t know if students have access to the core content through sheltered instruction.

• Teachers had the highest disagreement, almost one half, that there is an explicit instructional model teachers use for English Language Development and that there is a well-designed plan to support students whose language is not English.


Principals were also asked regarding teachers’ preparedness for ELL students and for diversity in their classrooms. These data are presented in Exhibit 3.1.15.

Exhibit 3.1.15

Principals’ Perceptions Regarding Teachers’ Preparedness for Diverse Students On Online Survey (N=11)


0% 20% 40% 60% 80% 100%

Sensitive to the linguistic, cultural, and economicdiversity among our students.

Are effective in meeting the needs of Englishlanguage learners and are successful in improving

their test performance.

Teachers have been trained in strategies forunderstanding and addressing the various cultural

needs of students.

Principals' Perceptions Regarding Teachers' Preparedness for Diverse Students

Strongly Agree Agree Disagree Strongly Disagree


• Over 60% of principals disagreed that teachers have been trained in strategies for understanding and addressing the various cultural needs of students.

• Over two thirds of the principals agreed that teachers are effective at meeting the needs of ELL students and at improving their test performance.

• Approximately 30% disagreed that teachers are sensitive to the linguistic, cultural, and economic diversity among their students.

When asked about ELL programming, several commented on the lack of consistency in its implementation at the building level. These comments included:

• “Sometimes walk-through’s are done just to focus on ELL, but that was earlier in the year—just to get ready for the (ELL Program) audit and we haven’t really done it since.” (Central Office Administrator)

• “[We] still meet with the teachers and make sure that they are teaching [the ELPS]. If you just let it go they pay no attention to the ELL students. They just sit there.” (Central Office Administrator)

• “People had to be on board. Principals were not on board. They weren’t really on board—I can’t tell the teachers ‘get on board’—only the principals can tell the teachers what to do. So they weren’t doing [the SEI block].” (Central Office Administrator)

• “Principals were not assuring the four-hour block, not having the teachers teach to the ELPS during that block time.” (Central Office Administrator)

• “They are making the AZELA more accountable. It is part of the new accountability system. I think it’s going to be a good thing—they can’t ignore the ELL kids anymore.” (Central Office Administrator)


On the online survey, teachers commented on their concerns over meeting these students’ needs in district classrooms:

• “Large class sizes limit practical reach for fully effective instruction.”

• “In the last two years, we have had three monolingual students enroll with absolutely no support. They are thrown in the classroom to sink or swim.”

• “We are on our own to modify instruction, create leveled materials, use best teaching strategies to scaffold learning for English language learners.”

• “Clicked agree, however we FAIL our middle school students (new to the school) who do not speak English. There is no support beyond the ILLP. No age appropriate resources, no support outside of the school day. Kids are on their own or at the mercy of their teachers to find English lessons.”

One support person at a school mentioned, “We all wear different hats. I do the ELL at the school but I don’t receive any PD from the district.” Others commented on the need for additional resources and support for ELL, citing insufficient resources and curriculum (see also Finding 3.2). These comments included:

• “But there is nothing for different levels of ELL students.” (Central Office Administrator)

• [Improve/change one thing?] “Probably more attention to curriculum for ELL students-the curriculum is supposed to be “mainstream,” but there really is no…curriculum.” (Central Office Administrator)

• “There also needs to be updated curriculum sources for ELL students.” (Teacher Survey Response)

• “I’m unsure of what the district does outside of the use of ILLP for ELLs—I’m not aware of any focus of curriculum or instructional practices for ELLs that the district has.” (Teacher Survey Response)

The auditors found some direction for ELL programming at the district level, but these expectations and requirements were not widely communicated nor cited by personnel at the school sites. Teachers and principals reported a need for better support for ELL instruction, and several commented on the lack of success for these students in district classrooms and a lack of consistency in its implementation.

Special Needs Students/RTI

The auditors then looked at programming for special needs students and the Multi-Tiered System of Supports (MTSS) in the district. They found no documentation concerning the functioning of the MTSS program although there was some direction from the Special Education Department concerning the philosophy of the district in how students’ needs are to be served and the importance of providing “sound academic opportunities to facilitate their physical, emotional, and social potential. It is our task to provide challenging programs appropriate for all levels of ability.”

The district has recently established a strong focus on co-teaching in the district as a means to meet most special education students’ needs in the classroom. There has also been an emphasis on MTSS being a support system in place for those students not performing on level in an effort to minimize the over-referral of students to special education who are not experiencing success in Tier I instruction. The auditors did not find expectations for these two models, co-teaching and MTSS, in any document although they were mentioned by principals and teachers at school sites.

Overall, the auditors found a common understanding across the district that co-teaching is the desired model of delivery of special education services and an MTSS team at every building. However, the implementation of these was found to be varied and inconsistently effective.


Teachers were asked about the quality and effectiveness of special education services in district schools. These responses are presented in Exhibit 3.1.16.

Exhibit 3.1.16

Teachers’ Perceptions Regarding Special Education Services On Online Survey (N=93)


0% 20% 40% 60% 80% 100%

Our district has a well-designed plan to support students whohave a learning disability.

I have been trained in the effective strategies for working withspecial education learners.

I feel the RtI process is effective for learners who arestruggling.

I am kept up-to-date regarding the students in my class with504 plans and receive support in providing them with

accommodations.

Please respond to the following questions about the district's program for special needs students.

Don't Know Strongly Disagree Disagree Agree Strongly Agree

Exhibit 3.1.16 displays the following information:

• Teachers had the highest level of agreement, with just over 50% agreeing, that their district has a well-designed plan to support students who have a learning disability.

• Teachers are split regarding whether or not they are kept up-to-date on their students’ 504 plans and on whether they have been trained in effective strategies for working with special education learners. Forty-five percent agreed or strongly agreed and 47% disagreed or strongly disagreed with the former; 43% agreed or strongly agreed that they have been trainer in effective strategies but 55% disagreed or strongly disagreed.

• The highest level of disagreement was regarding the RtI process (MTSS), with over two thirds (67%) disagreeing, strongly disagreeing, or reporting they don’t know that RtI is effective for learners who are struggling.

The MTSS program, an extension of RtI that includes behavioral considerations as well as academic, was consistently understood by school leaders. Comments on the program included:

• “The MTSS team looks both at the academic and behavioral needs of the child. The goal of MTSS is to be sure students’ needs are met—even before they get to a referral for SPED services.” (Central Office Administrator)

• “RTI/MTSS is one of the better ones I’ve had at any of my schools. We have a good referral process. We look at our data 25% bottom students. We look at our interventions; if they don’t make progress we look at referring [them].” (Central Office Administrator)


• “Ideally there is an MTSS team and a Leadership Team. The bottom 25% of students to come up with interventions. [We are] concerned that some school teams do better because there is no accountability, some principals do not come to the meetings.” (Central Office Administrator)

• “Each school site has an accelerated/MTSS team, a facilitator at each campus, all receive a stipend they develop interventions to support students who are struggling. I coordinate a training for the team, keynote speaker who pertain to the needs.” (Central Office Administrator)

• “I think the MTSS process is strong. I think there need to be more intervention for upper grades. We have very good interventions for reading, but very little in math for 5th and up.” (Support Staff)

• “We put a lot of time and money into staffing to prevent SPED placement and meet student needs. We do a fantastic job of keeping SPED numbers down on placement.” (Central Office Administrator)

• MTSS (Multi-Tier Support System) each school has an MTSS team There are regular trainings for members of the MTSS – are responsible for monitoring Tier II+. The goal of MTSS is to be sure students’ needs are met before they even get to referral to SPED. (Central Office Administrator)

• “Some of my campuses really rock [MTSS]. I have really tried to work on changing the paradigm to, it’s not a process to get them into SPED, it’s intended to keep them out unless they need it.” (Central Office Administrator)

There was a clear understanding of the MTSS program and several comments regarding its positive effects. However, in the implementation of the interventions, the auditors found a distinct lack of consistency and effectiveness. Most attested to using Imagine Learning as the main intervention, for SPED students as well as for MTSS, with concerns regarding its effectiveness. These comments included:

• “We use Imagine Learning for language intervention. It is a self-paced program. But we do not have records in the building as to how the kids are doing.” (Teacher)

• “A lot of [interventions] last year was just put them on Imagine learning. But half the time Imagine learning doesn’t work that well. It’s not real life. It talks to them and there is no interaction.” (Support Staff)

• “In my opinion, kids who are in intervention need interaction.” (Support Staff)

• “For low-level service, cat[egories] A and B SPED there are no other provided interventions. [Teachers] are not aware of anything instructional.” (Support Staff)

• “That is the answer in the older grades in SPED world. It is a minimum of 90 minutes a week on Imagine Learning for children. It is what it is—It doesn’t teach fluency. It does phonemic awareness.” (Support Staff)

Others discussed the way interventions are delivered, most often with instructional assistants. Several commented on the need for more interventionists or the lack of training for these individuals.

• “We can’t have Instructional Assistants (IAs) pulling kids to the corner when they do not know what to do and haven’t been trained. Let’s do something that has been tested and has a scope and sequence.” (Support Staff)

• “Title I funds are used to purchase interventionists at some schools. What is most important is that first taught instruction.” (Central Office Administrator)

• “We need more reading interventionists at our school.” (Teacher Survey Response)

• I know that we had [reading interventionists] and it was almost like a weed-out position. ‘Well, if you can’t be effective in the classroom we are going to try you in a small group.” (Support Staff)

The auditors concluded that MTSS, although a clear expectation, is not consistent in its implementation nor effective in yielding improved student achievement on high stakes measures.


The auditors heard of inconsistencies in how co-teaching is implemented and a lack of sufficient support for its implementation. These comments included:

• “Co-teaching—it’s a slow paradigm shift—really new on this journey, working on it teacher by teacher. We have done big staff development on it but people just don’t get it.” (Central Office Administrator)

• “What instruction (co-teaching) looks like and what is possible are two completely different things. Interruptions, lack of resources, majority of students achieving far below grade level, minimal classroom support for SPED and ELL.” (Teacher Survey Response)

• “Within our special education groups, hours, and resources have been cut and we are not meeting those student needs in the classroom as well as we have in the past.” (Teacher Survey Response)

• “We are told that district wants inclusion rather than pull-out; however, we are not shown what that truly looks like or how to make it possible, especially when we are going into multiple classes in multiple grade levels. We are also told that co-teaching is a priority; however, we are unable to accomplish this due to scheduling and the amount of time in each classroom.” (Teacher Survey Response)

• “On our campus, SPED students are not serviced. Instead, SPED teachers are allowed to assume other duties and responsibilities. Following walk-throughs, no feedback is given unless specifically asked for.” (Teacher Survey Response)

• “But it’s like some schools make [co-teaching] a priority, some schools don’t.” (Central Office Administrator)

Principals and teachers both expressed concern over meeting special education students’ needs, and others expressed frustration over not having sufficient staffing for special education. As one administrator commented, “I have a hard time retaining some of the [SpEd] teachers. Sometimes they are not treated like one of the staff [at the buildings].” Others noted a lack of respect for teachers from the Special Education Department. “People who work at district office are very disrespectful to and dismissive of teachers. When I sit in on my committee meeting with people from the SPED department, they speak about teachers and students in a very disrespectful manner.” (Teacher Survey Response)

Although there are clear expectations for MTSS and for inclusion with a co-teaching model, the auditors found no clear direction in any documents regarding their implementation nor guidelines for measuring their effectiveness.

Accelerated Learning

The auditors also examined the Accelerated Learning Procedures in the district. They learned from district leaders that there has been a change in program implementation in recent months and a desire to be more focused on delivering services in a child’s regular classroom through a clustering model. However, identification of students is still disproportional and certain subgroups are over/under-represented in the program. Additionally, the number of students enrolled in ALPS is quite low compared to other districts nationally. Gifted and Talented identification (for participation in the ALPS) is at less than 3% in the district; nationally, 6 to 10% of the student population are identified as gifted.

During interviews and on the online survey, several personnel expressed the desire to improve the ALPS services, but that change has been slow. Identification of students is not inclusive of all subgroups. As one teacher commented, “If a student comes with only Spanish we don’t have anyone in the gifted program to give that test. We have the test.” Others mentioned only using certain tests to identify students:

• “Instead of taking 98% with verbal in a Title I school they take them at 83%. We lowered and now will take them at 93%.” (Central Office Administrator)

• “[We have] a mass screening with the Naglieri every year, self, parent and teacher referral. If someone is nominated they use the Cog AT 6; any new student gets the Naglieri.” (Central Office Administrator)


There were many comments regarding the inadequacy of current services and the desire to change the district focus, but there is still resistance:

• “Piggybacking on that we go in to support them with differentiation, but we don’t see it. That happens because the teachers are unaware of what gifted really is.” (Teacher)

• “[We want to start] sharing the education of those kids between the GT teacher and the classroom education. So gifted ed isn’t going over to Monterey Park to do some special activity, but doing some activity in the regular classroom.” (Central Office Administrator)

• “There was no continuity between what happens at Monterey Park and what happens in their regular classrooms. So we tiptoed into bringing gifted ed back into the school campus.” (Central Office Administrator)

• “There are some people who think we need to take all 180 of our [GT] kids in our district and put them all over on one campus.” (Central Office Administrator)

• “Some are resistant to cluster grouping.” (Central Office Administrator)

• “We need to do cluster grouping in schools. [We are] starting to figure out how to do that. We all don’t fully understand what gifted education really means and what that looks like.” (Central Office Administrator)

• “We have been talking about gifted ed way too long. Shifting some top people has helped.” (Central Office Administrator)

• “[Gifted Education] doesn’t run well at every school.” (Central Office Administrator)

• “Our gifted kid’s needs are not adequately met by a once a week program at ALPS—the program is stand alone/separate from their gen ed classroom experiences.” (Teacher Survey Response)

Overall, identification of giftedness is low and not proportional. Changing to a research-based model is met with resistance, and several expressed concerns the district is currently not meeting students’ needs.

Equity

The auditors also asked district stakeholders what needed improvement in the district. Several mentioned equity issues, or concerns over equity-related issues, such as parental engagement and involvement. Several mentioned the need to be proactive in considering equity issues in the system. Comments from the Teacher Online Survey included:

• “Those of us who are considered to be professional need to use an equity lens to guide our instruction.”

• “I think that we may be moving in that direction and will no doubt improve our outcomes in terms of student learning.”

• “The major issue is equity: gender, socioeconomic status, race, languages, immigration, refugees, homeless. I want our students to have what any other students have.”

• “Instead of from a strength perspective they are seen as a deficit. Perceptions about what they don’t have and they are assigned a certain value or judgement and that ends up being an impediment.”

• “The main thing is the equity part and conversations about culture. Getting people out of the poverty mind, people think that [students] can’t do well because they are poor.”

• “It’s great to have good teachers, but if the district cannot connect to students. Look at discipline data. Our biggest equity issue comes [down to] school to school, classroom to classroom, and who the person is.”

Others commented on the survey regarding the level of connection and coordination between some schools and parents. These comments included:

• “Lack of integration of family engagement instead of being at the core; it’s relegated to the peripheral.”


• “I’m looking forward to a new shared vision that encompasses improved instructional practices, student and family engagement and equity in educational practices and policies.”

• “We fall way short to communicating with our families in a language other than English.”

• “A lot of them {parents} don’t speak English and they don’t feel that they can express what they want, need.”

• “There are no parents or community members on the budget advisory committee. It is made up of central office, a couple of principals, and some teachers.”

• “We need to find a way to engage parents in a different way.”

• “We need more parental involvement. We need teachers, staff members, parents, community members, business partners, and student representatives involved in the decision making process for curriculum, resources, instruction, professional development, cafeteria breakfast and lunch choices, and funding to help make the school building safer and healthier for each child. We do not have high expectations of our parents to participate in their child’s education.”

• “In some schools, parents are all involved. It is not equitable.”

The comments indicate inconsistency in how certain schools manage to engage and reach out to parents while others have not built up these important relationships. Stakeholders commented on other inequities in the system:

• “The inequity in materials, etc, is caused in part by principals being able to use their Title money any way they want.” (Teacher Survey Response)

• “Inequity comes with so much choice on the part of the principals. Student achievement should not depend on the school you are in.” (Teacher Survey Response)

• “We have widespread efforts without any special plan. We do it if it feels good. It’s intended to be helpful for students but it’s just a shotgun approach. Maybe something will stick, like MTTS, academic interventionist, coaches, LOI, desegregation, mentors, “Let’s do it,” “How about trying….” (Teacher)

• “The Arizona Tax Credit creates an imbalance between buildings. Some parents can contribute more, others, not at all.” (School Leader Survey Response)

• ‘What needs improvement? “Teacher’s ability to transfer to other schools including 1st Year teachers; equity of resources among schools; Extreme inequity exists between schools, with respect to resources, staffing, and quality of education.” (Teacher Survey Response)

There were many comments in the Parent Online Survey from parents regarding specific circumstances, although most were positive in nature. The biggest concern shared by parents on the online survey was regarding the lack of communication from the school or classroom teachers. These comments included:

• “[There is] no communication from teachers.”

• “Teachers and staff do not communicate with parents. It like pulling teeth for the most part.”

• [Communication?] “NEVER—teacher can’t even be bothered to write comments on the report card.”

• [Communication?] “Never, teacher says she is doing great in everything, however child is unable to do simple multiplication at the end of her 3rd grade year.”

• “I wasn’t aware of my daughter getting extra help until I questioned her progress. I think the school should make it know to parents when child is getting extra assistance.”

• “I was told during my child conference my child was behind and then I see the reports and it shows all A’s and B’s. How is this behind?”


Other stakeholders expressed concern over how students or they themselves are treated on campuses. These comments included:

• “[One of my child’s teachers] is always yelling at them. He yells at them if they can’t do things the way he wants them to. It is important that they make a change. That is not a teacher. I watched him, it is not right the way he treats children.” (Parent Survey Response)

• “I got here on March 6. My son was no allowed to start school until his records from California came. It took weeks, I called the school in California and they faxed the records. I went to school to ask if he could start school and they still did not want to take him. They gave me a paper, an affidavit, to have it notarized because I live with my mother-in-law. I did the paper and when I went back they sent me back to get proof of an APS [Arizona Public Service] bill. So, I went to get it and they finally let him in. Four weeks later.” (Parent Survey Response)

• “[the front office and welcome to parents] is awful. Schools—I don’t know if they are that overworked or that busy, but it isn’t that uncommon to not be greeted if you walk into the office. Now there is a piece of plexiglass between you and the counter [at some schools].” (Central Office Administration)

There were parent comments on the need for a parent resource center to welcome new families and to handle registrations, particularly with new students. One central office administrator commented, “I would start a parent resource center. Set up some kind of transportation for the parents to and from…have food, uniforms—all in one spot. Not that big.”

Overall, access to programs is not consistent district-wide. The Montessori and Magnet programs have very low representation of low-income students. Some subgroups are over-represented in certain programs and under-represented in others. Support services for students needing ELL and special education programming are defined to an extent in district documents, but expectations are not specific and these programs and supports are not consistently implemented. MTSS is present at each building, but the interventions used are of limited scope and quality.

The need for greater cultural sensitivity and an equity lens was mentioned by many stakeholders on the online survey, and many mentioned the need to be more proactive and positive in engaging parents and soliciting their support. Parents have concerns over safety issues and food, as well as, a few isolated incidents where respect for students or parents was lacking. The biggest concern from parents on the online survey (113 respondents) was over the lack of communication from the classroom or school.

Summary

Auditors examined Gifted and Talented (called ALPS in the district) and Special Education programs to determine the extent of equality and equity issues. Auditors expected to find participation data in these programs that reflected the general population demographics of the district. Conversely, auditors uncovered a system that has unintentionally given rise to issues of equity and equality.

Male students tend to constitute the majority of students in the Accelerated Learning Procedures program. Low socioeconomic students are under-enrolled in accelerated classes, as well when compared to their percentage of the general school population. Auditors found a slightly disproportionate number of White and Asian students enrolled in the ALPS program.

The Special Education Program reflects an overrepresentation of White, Asian, and American Indian students but underrepresentation of Hispanic students. Males and Black students represent a disproportionate number of disciplinary actions, while Special Education students represent an excessive percentage of disciplinary actions taken.

Auditors examined many aspects of the district’s programs and approaches to determine the impact on student learning. Although many interventions and programs are in place in Phoenix Elementary School District #1, auditors found staff efforts to be ineffective in supporting subgroups of students to achieve their potential. The district must focus on ensuring that initial instruction is more effective to thus reduce the district’s dependency on interventions and special programs. A focus on level achievement for ELL, Special Education, minority, and low SES students will solidify the district’s current instructional efforts.


Finding 3.2: The district has no clearly defined instructional model or expectations for delivery. Curriculum delivery is typified by whole-group approaches or seatwork, basic types of cognition, limited use of differentiation and technology, and minimal use of powerful instructional strategies. Monitoring the delivery of curriculum is inadequately defined and inconsistently implemented to ensure the improvement of instructional practices leads to increased student achievement.

Effective educational programs are best achieved when a clear model of what effective instructional delivery looks like is provided to teachers, along with the tools and professional development needed to deliver instruction according to those expectations. The instructional model should address how teachers are expected to plan instruction to meet individual student needs, using data from formative assessments. In addition to this model, district leaders should provide guidance concerning the expected strategies and approaches teachers should rely on when delivering instruction, especially with populations that are diverse and perhaps learning English as an additional language. Such guidance provides teachers with direction concerning the nature of cognitive engagement that should be evident in all district classrooms.

Specificity concerning what effective instruction looks like is essential in assisting principals in their primary function, monitoring the delivery of the educational program and its curriculum. Without clear and specific expectations for teaching and learning, principals lack clarity regarding what these core activities should look like and how to support them. Visiting classrooms is an effective building leadership tool, but without direction concerning what should be taught and clearly defined instructional expectations these classroom visits may be less effective in improving student learning. Additionally, both teachers and principals require focused support from a comprehensive and coordinated professional development program that gives teachers the tools they need to plan instruction focused on individual student needs (using assessment data) and deliver it in a way that reflects district priorities and expectations.

Auditors reviewed district documents, interviewed key stakeholders, and surveyed principals, teachers, and parents concerning the adequacy of an instructional and student engagement model for the all classrooms in the Phoenix Elementary School District #1. Overall, the auditors found no clear model and direction for what effective instruction and student engagement should look like except criteria found in the teachers’ evaluation instrument. Observed instructional practices were predominantly whole-group, direct instruction, or involved students working on seatwork or individual assignments. There were only limited examples of differentiation observed in a few classrooms; those that were observed reflected instructional differentiation.

The auditors found no clear expectations concerning the purpose and performance of classroom monitoring. Although many principals reported visiting classrooms frequently, the auditors found no district-defined direction for monitoring and the forms principals used to collect data during classroom visits were different across the system, indicating different purposes and goals for monitoring.

Instructional Practices

The auditors looked at instructional practices in Phoenix Elementary School District #1 classrooms to determine what kinds of practices were most evident, how students were engaged, what kinds of student learning arrangements were being used, what were the observed objective of instruction, and whether or not there was differentiation in that content, or in how students were practicing or demonstrating their content.

The intent of these observations is not to evaluate teachers, nor to report out on specific teacher activity for any one building. The intent is to provide the district with a picture from a single point in time of what instruction looked like in Phoenix Elementary School District #1 classrooms and compare these observational data with district expectations for instructional practice. The auditors present the classroom observational data with the caveat that no firm conclusions may be drawn, due to the brevity and nature of the classroom visits. The auditors visited classrooms for anywhere from 1 to 10 minutes, depending on the activities occurring in each classroom, for a single visit on a single day. Therefore, using a single observation to draw conclusion regarding the nature of instruction throughout the year would be inappropriate. However, the data do provide a sampling and district leaders can continue to observe in classrooms to monitor for similar information.


The auditors collected the following information from district documents regarding instructional expectations. The former district vision and mission communicates an expectation for rigor. Policy IA: Instructional Goals and Objectives includes expectations for an instructional program that provides for developing, in addition to other skills:

• Research and problem-solving skills.

• Ability to think analytically, critically, and independently.

• Understanding and respect for our cultural heritage.

• Appreciation for the intrinsic value of education.

The Classroom Teacher job description included minimal expectations for instructional delivery, including:

• Task analyzes (sic) and translates lessons into developmentally appropriate learning experiences.

• Models lessons for students.

• Provides individualized and small group instruction to adapt the curriculum to the needs of the student.

The auditors found extensive expectations for instructional delivery in the teacher evaluation instrument, the Learning Observation Instrument developed by REIL (Rewarding Excellence in Instruction and Leadership). This instrument includes expectations for teachers classified under six domains, each with a five-point rubric. These domains include: content, instructional strategies, formative assessment, learner engagement, learning community, and professional responsibilities. The auditors found key expectations for student engagement and instructional delivery in three of the six domains that were considered. These are presented below.

Content—Task Analysis, Connections to Content, Content Accessibility:

• Utilizes-representations and explanations of concepts (e.g., comparisons, analogies, examples, TPR, regalia, manipulatives, anchor charts, graphic organizers that capture key ideas-and-details that build conceptual understanding in the discipline.

• Utilizes questions at essential sub objectives that support student understanding in the discipline and stimulate discussion for a specific purpose.

• Prompts student reflection of prior content knowledge; linking new concepts to familiar concepts; makes connections to students’ experiences.

Instructional Strategies—Instructional Approach, Practice/Aligned Activity, Feedback (During the Lesson), Monitor and Adjust:

• Varies teacher role (e.g., teacher led, facilitation of student learning) effectively during the instructional process in-relation-to-content-and purpose-of-instruction.

• Utilizes student responses to appropriately move forward with or adjust one-or-more of the following:

○ Instructional approach (e.g.; pacing, modeling, questioning, guided practice, feedback, etc.);

○ Content sub skills or complexity (e.g. number in problem or Lexile);

○ Depth of knowledge;

○ Grouping; and

○ Task/Product requirements.

Learner Engagement—Student to Student, Student to Teacher, Authentic Engagement, Critical Thinking:

• Students engage in structured, scaffold student to student academic dialogue aligned to the lesson objective in order to develop expressive-language-proficiency and solidify-learning.

• Students demonstrate individual accountability, equal-participation, application-of-content-vocabulary, and justification of ideas.


• Authentic Engagement: by assigning tasks that most students associate with a result or outcome that has clear meaning and personal relevance.

• Students use complex reasoning to make new meaning not provided by the teacher.

From the various documents the auditors reviewed, there are clear expectations for classrooms where students are engaged in discussion and dialogue, with the teacher and with one another, where they are challenged cognitively, and involved in activities that they see as personally relevant. There are expectations that the teacher is not solely a dispenser of information; that students are given multiple opportunities to connect with the content and see connections within and across disciplines and asked to reflect upon their prior learning in connection with new learning. There is the expectation for differentiation—teachers are using student responses and observed performance to plan and deliver instruction that is at the proper level or in accordance with their academic needs.

To determine whether or not the instructional expectations found in Phoenix Elementary #1 documents were met, the auditors observed in over 138 classrooms across the district and collected data concerning the rigor or cognitive challenge of student activities, instructional differentiation in content and context, student engagement, dominant teacher activity, and dominant student activity and student learning arrangements. The auditors noted when examples of effective teaching practices were noted, based on research, and when effective ELL strategies were observed in setting up, delivering the lesson, or when engaging students in practicing their learning.

Overall, the auditors found that the classroom activities and learning observed in Phoenix Elementary #1 did not consistently reflect district expectations, particularly expectations for student-centered activities that are cognitively demanding and engaging. They found that specific expectations for what instruction should look like in the district are almost entirely limited to the teacher evaluation instrument in use, and are not specified in district policies, plans, or documents beyond the few cited in this finding.

On the surveys, the auditors asked teachers and principals if there are clear expectations for instructional delivery communicated across the district. Their responses are presented in Exhibits 3.2.1 and 3.2.2.

Exhibit 3.2.1

Principals’ Perceptions Regarding District Direction for Instructional Delivery On Online Survey (N=9)


0% 20% 40% 60% 80% 100%

Teachers have a clearly defined model for deliveringinstruction to students.

There is clear direction from the district regarding whatclassroom instruction should look like.

Principals' Perceptions Concerning Direction for Instructional Delivery


As can be seen from Exhibit 3.2.1:

• One third of principals disagreed that there is clear direction from the district regarding what classroom instruction should look like.

• Eleven percent disagreed that teachers have a clearly defined model for delivering instruction to students.


Exhibit 3.2.2

Teachers’ Perceptions Regarding District Direction for Instructional Delivery On Online Survey (N=95)


0% 20% 40% 60% 80% 100%

We have a clearly defined model for deliveringinstruction to students in the district.

There is clear direction from the district regardingwhat classroom instruction should look like.

Teachers' Perceptions Regarding District Instructional Expectations


As can be seen from Exhibit 3.2.2:

• The majority of teachers report disagreeing with both statements. Sixty percent of all teachers who responded disagreed or strongly disagreed that there is clear direction from the district regarding what classroom instruction should look like.

• Just under two thirds (63%) disagreed or strongly disagreed that they have a clearly defined model for delivering instruction to students.


Exhibit 3.2.3 describes teachers’ responses to the level of written communications of expectations.

Exhibit 3.2.3

Teachers’ Perceptions Regarding District Communication of Expectations for Instructional Delivery (N=87)


0% 20% 40% 60% 80% 100%

Board policy and communications

Central office communications

Building administrator communications

Written curriculum

Professional development activities

Monitoring of classroom instruction by administrators orinstructional coaches

How clearly are expectations for classroom delivery of the curriculum and classroom instruction communicated through the following:

N/A Not clearly Somewhat clearly Clearly Extremely clearly


• About one fourth of teachers reported that expectations for instruction are reported extremely clearly or clearly in board policy or central office communications, while over half disagreed or called the statement not applicable to their position.

• Around 42-44% of teachers agreed that expectations for instruction are communicated clearly or extremely clearly in professional development activities and building administrator communications. An almost equal percentage stated they were not clear or only somewhat clear.

• Regarding the written curriculum, 42% of the teachers reported expectations for instruction to be communicated clearly or extremely clearly in the curriculum; about one fourth reported them to be communicated somewhat clearly; and just under one third reported them to be unclear or not applicable.

• Teachers reported the greatest clarity regarding expectations for monitoring of instruction. Almost 60% of teachers reported these expectations to be communicated clearly or very clearly.

Classroom Observations

The auditors observed in 138 classrooms across the district in an effort to gather data at a single point in time concerning what instruction looks like in Phoenix Elementary #1 classrooms. During these classroom visits, the auditors collected data regarding the activities in which students were engaged, what type of learning arrangements were observed, as well as the dominant teacher activity. In the 138 classrooms visited, no data were collected in classrooms where there was a substitute teacher. It is important to note that these observations only reflect data from a specific point in time and are not intended to comprehensively reflect the nature of all instruction that occurs in Phoenix Elementary #1 over the course of the year.


Exhibit 3.2.4 presents and describes the characteristics auditors typically observed in classrooms.

Exhibit 3.2.4

Classroom Observation Characteristics and DefinitionsPhoenix Elementary School District #1

May 2017

Dominant Teacher Activity Descriptions

Large group/direct instruction, teacher-

centered

Teacher is presenting lesson or engaging with students on the same concept/task in a large group format, with the teacher as the focal point. Students are receiving information, not actively participating.

Large group/direct instruction, student-

centered

Teacher is presenting lesson or engaging with students on the same concept/task in a large group format, but in a manner that engages students in answering questions or solving problems.

Assisting Teacher is moving about the classroom helping students with tasks.

Small group/pairs Teacher is engaged in direct instruction with a small group while other students.

Monitoring student work Teacher is observing students as they work, but not interacting with them.At desk Teacher is seated at desk.

Teacher using technology (Active/passive)

Teacher is using technology as part of the instruction. Technology can be active when the teacher is interacting with the technology and students or passive, when technology is used as a tool only.

Giving a test The teacher is administering an assessment.Giving directions Teacher is providing directions to students on a specific task.

Other“Other” can be noted when the teacher is engaged in an activity other than the classifications previously listed (e.g., distributing papers, grading papers, preparing students for transitions).

Dominant Student Learning Arrangements Descriptions

Large/whole group: student-centered

Students are participating in whole group lesson formats, actively answering questions or taking turns solving problems.

Large/Whole group: Teacher Centered

Students are listening to the teacher provide direct instruction or model a skill or activity.

Seatwork Students are working individually on cognitively undemanding work, such as worksheets or copying activities.

Individual work Students are working individually on projects that are cognitively engaging, such as open-ended writing assignments or research projects.

Small group Students are working in small groups, to complete an activity, solve problems, or discuss issues.

Centers Students are working at centers on assigned tasks. Typically used during reading instruction time.

Pair work Students are working with a partner on practice activities or projects.

Other Students are completing some activity in a learning arrangement different from those described here.


Exhibit 3.2.4 (continued)Classroom Observation Characteristics and Definitions


Dominant Student Activity Descriptions

Reading Students are reading quietly, independently or to the class.Listening Students are listening to either the teacher or another student.Writing Students are engaged in a writing task or assignment.

Speaking Students are engaged in a speaking activity (discussion with other students, presentations, etc.).

Lab/hands-on Students are doing an experiment or creating/building a project.Word work/phonological

awarenessStudents are working on sound/letter correspondence, word recognition, word building, etc. (early literacy skills)

Worksheet/textbook Students are completing a worksheet or textbook assignment.

Computer work Students are working on a computer based activity either in groups or independently.

Practice activity Students are working on an activity for reinforcement of a concept (e.g., math problems) either independently or in groups or pairs.

Warm-up/review Students are reviewing previously learned content in preparation for a new lesson or are remembering things they already know in preparation.

Modeling/simulation Students are participating in a simulated real-life activity (such as launching rockets, testing the effectiveness of a hypothesis, etc.).

Interaction with others Students are engaged in an activity with another student(s).

Transitioning Students are transitioning from one class to another or one activity to another; no academic engagement observable.

Taking assessment Students are taking an assessment.Student presentations Students are giving presentations, orally, with/without technology.

Other“Other” can be noted when students are engaged in an activity other than the classifications listed above (e.g., handing out papers for the teacher, moving to the carpet, putting away material).


Exhibits 3.2.5 to 3.2.7 present the data collected by auditors during the site visits regarding the dominant teacher and student activities and the observed learning arrangements.

Exhibit 3.2.5

Dominant Teacher Activity Observed (N=131)Phoenix Elementary School District #1

May 2017

Large group instruction,

teacher-centered28.2%

Whole-group instruction, student-

centered17.6%

Small group10.7%

Individual work7.6%

Monitoring student work

27.5%

Other8.4%

Dominant Teacher Activity


• Teachers were most frequently observed engaged in whole-group, teacher-centered instruction in 28.2% of all classrooms visited.

• The second-most frequently observed teacher activity was teachers monitoring student work (27.5%), followed by student-centered whole group instruction (17.6%).

• In just over 10% of classrooms, teachers were working with small groups; in the remaining classrooms they were working with individual students (7.6%) or doing something other than the activities noted (8.4%).


Exhibit 3.2.6 presents the dominant student activity learning arrangements observed.

Exhibit 3.2.6

Dominant Student Activity Learning Arrangements (N=137)Phoenix Elementary School District #1

May 2017

Large group/whole group - Student

centered25.5%

Large group/whole group - Teacher

centered26.3%

Seatwork (low-level, worksheets, copying,

tracing letters)14.6%

Individual work (project, writing,

reading, meaningful work on their own)

10.2%

Small group8.0%

Centers1.5%

Pair work2.2%

Other (please specify)11.7%

Dominant Student Activity Learning Arrangements


• The dominant student learning arrangement observed was teacher-centered, large group, or whole group, followed by student-centered, whole-group arrangement.

• The next most frequently observed learning arrangement was seatwork, seen in almost 15% of classrooms, followed by “other” (11.7%), individual work (10.2%), and small groups (8%).

• Centers and pair work were observed in less than 4% in combination of all classrooms visited.


Exhibit 3.2.7 presents the data collected regarding in what activities students were engaged.

Exhibit 3.2.7

Dominant Student Activity Observed (N=136)Phoenix Elementary School District #1

May 2017

7.4%22.1%

8.1%2.9%

16.2%0.0%

12.5%0.7%

10.3%0.0%0.0%

2.2%2.9%

9.6%0.0%

5.1%

0% 5% 10% 15% 20% 25%

Reading* (see questions on reading analysis)Listening

WritingSpeaking

Lab/hands-on activityWord work/phonological awareness

Worksheet/answering questionsComputer work

Practice activity (problems, algorithms)Warm-up, review

Modeling/simulationInteracting with other students

TransitioningTaking assessment

Student presentationsOther (please specify)

Dominant Student Activity Observed

Exhibit 3.2.7 shows:

• Students were most observed listening to instruction, 22.1% overall.

• Students were observed completing a lab or hands-on activity in 16.2% of all classrooms visited, followed by completing worksheets in 12.5%. Students were completing a practice activity in 10.3%, taking assessments in almost 10%, writing in 8.1%, and reading in 7.4% of classrooms.

• Students were observed engaged in some other type of activity in 5% of classrooms, and in other classrooms, transitioning, speaking, or interacting with other students in 2.9, 2.9, and 2.2%, respectively.

• Observed student activities reflected a high degree of independent work, rather than interacting with other students.

• Students were listening or working on worksheets in 34.6% of the classrooms observed, which is consistent to the nature of activity learning arrangements observed (see Exhibit 3.2.6).


When visiting classrooms, auditors observed for student cognition. The auditors looked for the type of knowledge of student engagement, as well as the type of cognitive process dimension. These data are presented in Exhibits 3.2.8 and 3.2.9.

Exhibit 3.2.8

Type of Knowledge Observed (N=127)Phoenix Elementary School District #1

May 2017

0%

10%

20%

30%

40%

50%

60%

70%

80%

Factual Procedural Conceptual Metacognitive

Cognitive Type: Type of Knowledge

Exhibit 3.2.8 shows the types of knowledge auditors observed in classrooms. The number of observations exceeds the number of classrooms since the auditors noted multiple types of knowledge being practiced or learned in certain classrooms.

• Factual knowledge was the focus of learning in almost 70% of all classrooms. This was almost twice that of other types.

• Procedural knowledge was observed in over one third of all classrooms (36.2%), while conceptual and metacognitive types of knowledge, considered the most challenging knowledge types, were observed in less than 15% of classrooms.

• Metacognitive knowledge was observed in less than 1% of classrooms, although being engaged in metacognition is associated with the greatest impact on learning.


Exhibit 3.2.9 presents the cognitive process dimensions observed by auditors.

Exhibit 3.2.9

Cognitive Process Dimension Observed On Online Survey (N=128)


0%

10%

20%

30%

40%

50%

60%

Remembering Understanding Applying Analyzing Evaluating Creating

Cognitive Type: Cognitive Process Dimension


• The most commonly observed types of cognition observed in classrooms were remembering and understanding, each observed in about half of the classrooms where cognitive type was discernible.

• Applying was observed in less than one third of classrooms.

• The more challenging cognitive types of analyzing, evaluating, and creating were noted in less than 10% of all observations.

Overall, the auditors did not observe students engaged in discussion or in cognitively demanding learning activities in a majority of classrooms, as suggested by district expectations; rather, these were only observed in few classrooms. In addition to the expectations for engaging instruction, there was an additional expectation for instructional differentiation. Data on differentiation are presented in the following section.


Instructional Differentiation

The auditors looked for differentiation, to determine the extent of differentiation in district classrooms and what support exists for it. The auditors posed questions in the online surveys to teachers and to principals regarding their perceptions of the overall need for differentiation, their effectiveness in differentiating for students, and the nature of support, training, and resources they have received to differentiate effectively. Teacher responses are presented in Exhibit 3.2.10.

Exhibit 3.2.10

Teachers’ Responses Regarding Differentiation On Online Survey (N=96)


0% 20% 40% 60% 80% 100%

There is a wide range of academic ability in myclassroom.

My classroom instruction meets the needs of all mystudents.

Differentiation is necessary for my students' needs tobe met.

I have the knowledge, tools, and support to effectivelydifferentiate instruction for my students.

I have received adequate training in how tosuccessfully differentiate instruction for my students.

I have the resources and materials I need to supporteach student's needs in the classroom.

Teacher's Perspectives on Differentiation



• Teachers overwhelmingly agreed with the statement: “There is a wide range of academic ability in their classrooms.” Two percent of teachers disagreed with the statement while 98% agreed or strongly agreed.

• All teachers agreed or strongly agreed that differentiation is necessary for their students’ needs to be met, but just under two thirds disagreed that they have the resources or materials they need to support each student’s needs in the classroom.

• Just under one half of teachers disagreed that they have received adequate training to differentiate effectively, or that they have the knowledge, tools, and support to effectively differentiate instruction.

• The majority of teachers, over three fourths, agreed that their classroom instruction meets the needs of their students.


Principals were asked, via the online survey, how often their teachers use strategies for differentiating instruction. These responses are presented in Exhibit 3.2.11.

Exhibit 3.2.11

Frequency of Differentiation On Online Survey (N=10)


Daily70%

At least weekly

20%

Several times a month

10% Several times a quarter

0%

Rarely0%

In general, how often do teachers in your school use strategies for differentiating instruction to meet the individual needs of students?


• Seventy percent of the principals who responded reported that their teachers differentiate instruction daily.

• Twenty percent said at least weekly, while 10% said several times a month.


Exhibit 3.2.12

Principals’ Perceptions of Instructional Effectiveness On Online Survey (N=9)


0% 20% 40% 60% 80% 100%

There is a wide range of academic ability in my classrooms.

Classroom instruction meets the needs of each student.

Differentiation is necessary to meet students' needs.

Teachers have the knowledge, tools, and support toeffectively differentiate instruction for students.

Teachers have received adequate training in how tosuccessfully differentiate instruction.

Teachers have the resources and materials to support eachstudent's needs in classrooms.

Principals' Perception Regarding Teacher Effectiveness



• Principals were unanimous in agreeing with the statements that there is a wide range of academic ability in classrooms and that differentiation is necessary to meet students’ needs.

• One third disagreed that teachers have the resources and materials they need to effectively differentiate, and 45% disagreed that they have had adequate training.

• Over half (55%) of principals disagreed with the statement that teachers have the knowledge, tools, and support to effectively differentiate their instruction.

• One third again disagreed that classroom instruction meets the needs of all students, with two thirds agreeing with that statement.


Exhibit 3.2.13 presents data concerning additional instructional practices, based on principal responses.

Exhibit 3.2.13

Principals’ Perceptions of Instructional Practices On Online Survey (N=9)


0% 20% 40% 60% 80% 100%

Effective in improving student learning and theirsubsequent test scores.

Effective at differentiating instruction to meet individualstudents' needs.

Consistently use student data in planning their dailyinstruction.

Consistently select instructional interventions based onformative student achievement data.

Principals' Perceptions of Instructional Effectiveness



• One third of principals disagreed that teachers are effective at differentiating instruction to meet individual student needs.

• There was higher agreement, almost 90%, with the statements that teachers consistently use data in planning daily instruction and that they are effective in improving student learning and their subsequent test scores.

• Just under one fourth disagreed that teachers consistently select instructional interventions based on formative student achievement data (see Finding 4.4).


The auditors specifically looked for evidence of differentiation in two main formats: in content, which occurs when teachers teach students different objectives based on their need; and, in context, which occurs when teachers allow students to practice their learning in different ways (process) or demonstrate it in different ways (product). The latter type of differentiation is referred to as instructional differentiation, while the former is referred to as curricular. The results of the auditors’ observations are presented in Exhibit 3.2.14.

Exhibit 3.2.14

Evidence of Differentiation During Classroom Visits On Online Survey (N=*)


0% 20% 40% 60% 80% 100%

Evidence of DIFFERENTIATION in content?Are there different content objectives for

different students/student groups?

Evidence of DIFFERENTIATION in process orproduct (How students are practicing or

demonstrating their learning?

Evidence of Differentiation Observed by Auditors

Yes No

* N=146 classrooms where auditors were able to determine if content was being differentiated. Auditors were only able to determine if there was process or product differentiation in 32 classes.


• Auditors noted the possibility of instructional differentiation (in process or product) in only 32 classrooms. Of these classrooms, less than half had students who were practicing or demonstrating their learning in different ways.

• Auditors were able to record data on content differentiation in 146 instances. In some classrooms, because of student learning arrangements, multiple observations of curricular differentiation were possible. However, in these classrooms, curricular differentiation was observed in only 11% of the classrooms.

Despite the use of data in planning instruction, the auditors observed students working on almost identical assignments or participating in whole-group lessons where the content was the same for every student. These observations support what the auditors recorded for dominant student activity and learning arrangements.

There were comments made during interviews and on the online survey regarding differentiation. One teacher commented on the lack of differentiation for gifted students, “We go in to support them with differentiation, but we don’t see it [in the classroom]. That happens because the teachers are unaware of what gifted really is.” A principal commented, “You probably won’t see differentiation in the classroom. We hope for more small group instruction; [but] sometimes technology is the differentiation mode.”


Other teachers attested to the challenges they face trying to differentiate due to the lack of resources, materials, and curriculum support from the district. These comments (from the online survey) included:

• “I absolutely have the knowledge to effectively differentiate student instruction, but I don’t have the tools.” (Teacher Survey Response)

• “While the evaluation instrument provides a rubric for what instruction SHOULD look like—the reality, day-to-day is not supported by that rubric, nor are space, resources, and curriculum provided for a full differentiation of student needs.” (Teacher Survey Response)

• “I have not received any resources on how to differentiate within my room.” (Teacher Survey Response)

• “I meet the needs of my different learners by buying my own materials. The district has not provided sufficient materials for differentiation.” (Teacher Survey Response)

• “There is very little if any training, collaboration, or resources available for differentiation.” (Teacher Survey Response)

• “I am a special needs teacher and I provide differentiated lessons. I have figured out how to differentiate lessons, but did not get quality training on differentiating. Also, we are lacking materials. It is very time consuming to ‘create’ various assessments, and that is why differentiation is not happening effectively. If we had more planning time, teachers could do a better job of meeting subgroup needs.” (Teacher Survey Response)

District curriculum administrators commented on the desire to support differentiation with the most recent curriculum revision, but noted a lack of success. They commented, “Our goal was to take on the constructivist approach (maybe five years ago) in order to give teachers the freedom to make decisions about instruction. The problem is, teachers don’t know how to do that.”

Finally, auditors collected data concerning effective teaching practices. In each classroom, the auditors noted any examples of these practices when they were observed. There was no limit regarding how many could be recorded for a single classrooms, so the data presented in Exhibit 3.2.15 shows the frequency of instances recorded for each strategy, based on the number of times that strategy was observed divided by the number of classrooms in which strategies were noted. There were a total of 256 instances of effective strategies noted in 81 classrooms. These are presented in Exhibit 3.2.15 to show the frequency of each strategy in the 81 classrooms where they were noted.


Exhibit 3.2.15

Effective Strategies Observed On Online Survey (N=81 classrooms out of 138; 256 strategies noted overall)


0%

10%

20%

30%

40%

50%

60%

Effective Strategies Observed


• The most frequently observed strategy, seen in 57% of the 81 classrooms where effective strategies were noted, was reinforcement of effort and recognition.

• The next most-observed strategy was that of teachers providing cues and prompts, seen in 47% of classrooms, followed by teachers giving corrective feedback with support in 41% of classrooms.

• Auditors observed teachers using kinesthetic activities to increase student understanding in just over one fourth of the 81 classrooms.

• Non-linguistic representations; physical models being used; time for practice, review, application; and ample wait time were effective strategies noted in about one fifth of the classrooms each.

• Using graphic organizers or concept maps, or having students take notes were observed in about 15% of the classrooms, and the remaining strategies (having specific learning objectives, cooperative learning, summarizing, mental pictures, comparing/classifying, and pacing) were observed in less than 10% of the classrooms.

When administrators were asked about the instructional strategies the auditors would observe, the auditors heard the following comments:

• “I think you are going to see actions that are all over the page. Some classrooms where there are high levels of engagement, and some classrooms where there is little or no engagement. I think you are going to see nervous principals and teachers.” (Central Office Administrator)


• “The primary level classrooms have more small group instruction. Upper grades [are] whole group; the quality of the instruction is at issue. We’ll see teachers reading to kids; kids doing individual seatwork.” (Central Office Administrator)

• “There is no joy. There is just none of that.” (Campus Administrator)

One student commented, “Some teachers need to go back to school and learn different teaching styles.”

Teachers also communication frustration over the instructional strategies expected. These comments included:

• “What instruction looks like and what is possible are two completely different things. Interruptions, lack of resources, majority of students achieving far below grade level, minimal classroom support for SPED and ELL.” (Teacher Survey Response)

• “On a schoolwide level, we have a strong principal and we’re doing the best we can. On a district level, there is virtually no support for teachers, and what support is given is useless or faulty.” (Teacher Survey Response)

• “While I do believe there is a strong idea of what instruction should look like, it seems to solely be centered around the LOI. That is not bad within itself, however it does not always suffice as understanding what that looks like for our specific content area and grade level.’ (Teacher Survey Response)

Teachers’ comments showed frustration over a perceived lack of support from district leaders for delivering the most effective instruction to their students.

Overall, the auditors found that the cognitive engagement of students observed in district classrooms was not reflective of the types expected in district documents. The most commonly observed activities were whole-group in nature, followed by students working on worksheets or seatwork at their desks. Interaction with other students was observed in a small number of classrooms and most student-centered instruction was still whole-group in nature. Teachers were most often seen engaged in direct instruction, followed by monitoring student work. Effective strategies were observable in district classroom, but were not reflective of cognitively demanding strategies and were more teacher-dependent than dependent on the nature of the student activity.

The auditors next examined the expectations for monitoring in the district and its frequency and effectiveness.

Monitoring

Monitoring classroom instruction is an important part of supporting alignment of the written, taught, and tested curriculum. Monitoring is intended to be a non-threatening, supportive process whereby principals visit classrooms frequently for brief periods of time to be aware of the content being delivered and in what manner. The audit expectation for monitoring is that the district has clearly defined expectations for its purpose and implementation and for its expected impact on teaching practices and student achievement. Monitoring should first focus on what is being taught, to assure the proper content at the right level is being delivered to students, and, secondly, on the strategies and approaches used to engage students with the content so their learning (and cognitive engagement) is maximized.

The auditors examined district documents to determine what expectations exist concerning monitoring the curriculum in Phoenix Elementary School District #1. They interviewed district administrators, principals, teachers, and instructional coaches about monitoring and posed questions concerning the process on the teacher and principal surveys. Overall, auditors found almost no written direction for monitoring in district documents, although almost all principals communicated an awareness of the importance to be in classrooms frequently and reported using a specific tool when doing so. There is a lack of clarity concerning how the instructional leadership role of the principal works in concert with the coaches in each building, and there was inconsistency noted in how frequently principals are actually in classrooms and the benefits of the feedback they offer to teachers. Forms for collecting data during walk-throughs were entirely school-based.


Despite the lack of written direction for monitoring, or for performing walk-throughs beyond the scope of teacher evaluation, principals reported being in classrooms frequently. Their responses to this question on the online survey are presented in Exhibit 3.2.16.

Exhibit 3.2.16

Frequency of Classroom Visits by Principals On Online Survey (N=10)


Daily or almost daily40%

At least weekly50%

At least monthly

10%

At least twice a year

0% Rarely0%

I visit each classroom in my building:


• Forty percent of principals reported being in each classroom daily or almost daily.

• About half of principals reported being in classrooms at least weekly.

• Ten percent reported visiting each classroom on their school campus at least monthly.

When asked how often they are in classrooms, principals responded with the following comments:

• “Twice a week in each class.” (School Leader Survey Response)

• “Every morning I do a walk-about. Occasionally I’m invited to watch an activity. My goal for the next year is to get in classrooms more.” (School Leader Survey Response)

• [How often are you in classrooms?] “Depends—August through February it’s almost daily. I always want to be there. Then when we hit testing, spring break, end of the year reports. Now it’s a brief walk through and not do a sit-down.” (Campus Administrator)

• [How often in classrooms?] “Every day. I try to go in and sit a while.” (Campus Administrator)

• “I get in the classrooms a lot because of the nature of our evaluation system.” (School Leader Survey Response)

• “For a given classroom—in grades 6-8 daily.” (School Leader Survey Response)

• “K-5—we have instructional specialists so [I] try to get in there once a week.” (School Leader Survey Response)


The auditors asked teachers regarding the frequency of classroom visits by principals, as well as by other building administrators or coaches. Their responses are presented in Exhibit 3.2.17.

Exhibit 3.2.17

Teacher Responses to the Frequency of Classroom Visits By Principals and Other School Administrators (N=89)


0% 20% 40% 60% 80% 100%

Principal

Assistant principal

Specialist/coach

Other (Specify below)

Frequency of Classroom Visits Reported by Teachers

Daily or almost daily At least weeklyAt least monthly At least twice a yearI rarely see this person in my classroom


• Fewer than 5% of teachers reported seeing their principals daily or almost daily, and 31% reported seeing them at least weekly.

• By contrast, 8% of teachers reported seeing their instructional coach daily or almost daily, and 36% said at least weekly.

• While only 9% of teachers said they almost never see their principals in their classroom, over one fourth of teachers said they almost never see their instructional coach in their classroom.

• The largest percentages of teachers reported seeing their principal or assistant principal monthly.

On the survey, the auditors also asked principals to describe what they look for when visiting classrooms. The following answers were provided:

What do you look for?

• Instructional rigor; Learner community; Learner engagement.

• Objectives posted/referenced, Number Talks, science, and critical thinking opportunities.

• Student engagement and completion of assignments. Students taking ownership of their learning. Teachers assisting to help students successful engage in the assignments and hold students accountable.

• Objectives, student-to-student interaction, lesson alignment, culture/routines and procedures, learning.

• Standard objectives posted, instruction matches state standards posted, LOI components/elements for instruction, active learner engagement, teacher/ student engagement, resources used to promote and enhance learning.


• Lesson objective that is aligned to a grade-level content standard; Percentage of students physically/mentally engaged in the lesson; Percentage of students who can articulate what they are learning and accurately work through/explain the target skill; Feedback that is aligned to the teacher’s individual area of focus (This is typically an element from our LOI. The focus area changes as teachers show progress in their target area).

• Student/staff interactions. Differentiated instruction. Classroom arrangement that supports learning. Evidence of planning of lessons and collaborative planning with team. Student engagement. Smiles on people’s faces. Happy chatter.

• Objectives aligned to grade level standards

○ Activities aligned to objectives/standards;

○ Student interaction;

○ Rigor/critical thinking (questioning, student responses);

○ Use of academic vocabulary;

○ Teacher role vs. student roles;

○ Are students able to articulate learning goal/objective;

○ Checks for understanding; and

○ Teacher talk vs. student talk.

• Student work, engagement and joyous learning. I also observe the lessons that teachers are giving.

• When I visit classrooms I look for student engagement and differentiation.

• Looking for alignment to the objective, critical thinking, student-to-student interaction.

• Instructional strategies aligned to our PD offerings; example: rigor and questions; student engagement. Is the engagement at a higher level? Every day is there an objective? Can the student articulate what they are learning?

• Looking for evidence of the focus this year: number talk, science implementation, site specific objectives.

• Post standards and reference them. Do an alignment with the standard.

• I count the number of math and English lessons going on and how students are engaged.

As is evident from the school leader survey responses, a few principals are looking for alignment of the taught curriculum to the state standards/curriculum. Most noted looking for student engagement. Almost none mentioned looking at what is being delivered and its alignment to the curriculum—the only mention was alignment to the state standards.


The principals were then asked about the protocol they use to collect classroom data. Their responses are presented in Exhibit 3.2.18.

Exhibit 3.2.18

Principal Walk-Through Protocols (N=10)Phoenix Elementary School District #1

May 2017

I use a district walk-through

protocol.0%

I use a protocol selected by myself

for my school.80%

I do not use a formal walk-

through protocol.20%

Do you use a walk-through protocol to monitor instructional delivery?


• Eighty percent of the building administrators who responded to the survey reported using their own protocol for monitoring instructional delivery.

• No principal reported using any tool provided by the district.

• Twenty percent of principals reported using no tool.


Teachers were also asked about the usefulness of the feedback they receive from the instructional leaders in their building or from central office administrators. Exhibit 3.2.19 presents these data.

Exhibit 3.2.19

Usefulness and Frequency of Feedback to Teachers On Online Survey (N=88)


0% 20% 40% 60% 80% 100%

Principal

Assistant Principal

District Administrator

Coach/Strategist

Please check the responses which describe the degree to which your principal, assistant principal, other administrator or coach/strategist

provide you with useful feedback on informally observed lessons.

Feedback is not useful Feedback is somewhat useful

Feedback is always useful No feedback given


• Teachers were most favorable concerning the feedback they receive from their principals or coaches, with almost half reporting that this feedback is always useful. Almost one third of teachers said the feedback from their principals is somewhat useful.

• One fifth (20%) reported the feedback from their coaches is somewhat useful. An equal percentage (20 percent) reported not getting feedback from their coaches.

• Sixteen percent reported not getting feedback from their principals. Over one third reported not receiving any feedback from their assistant principals. District administrators were reported as not providing feedback by almost 60% of the teachers who responded.


Teacher responses regarding the quality of instructional leadership in their building were mixed; these responses are presented in Exhibit 3.2.20.

Exhibit 3.2.20

Teacher Perception of Campus Instructional Leadership On Online Survey (N=87)


Highly effective17%

Effective34%

No opinion17%

Somewhat ineffective

20%

Not effective12%

Teachers' Perception of Instructional Leadership


• Seventeen percent of teachers reported the instructional leadership in their buildings to be highly effective.

• The greatest percentage of teachers (34%) reported the instructional leadership to be effective, while remaining teachers were divided among no opinion (17%), somewhat ineffective (20%), and not effective (12%).

There were many comments made on the leadership in schools:

• “Guidance on what to teach varies by site. You get a sense from teachers that it is oversight in supervision rather than support.” (Central Office Administrator)

• “Riding teachers is not leadership.” (Teacher Survey Response)

• “The left hand doesn’t know what the right hand is doing.” (Teacher Survey Response)

• “The lack of communication and support makes our leadership team not effective.” (Teacher Survey Response)

• “Administrators need to be visible at all school events, classrooms and transitional areas throughout the school day. Administrators need to be disciplinarians and display a very strong sense of authority for students to tale their education and respect seriously. (Teacher Survey Response)

• [Instructional leadership?] “Absolutely terrible.” (Teacher Survey Response)

• “Our [building administrator] does not follow through on anything, and we are continuously expected to solve issues on our own that she should be supporting with. The expectations are not universally employed across our campus, and people get away with things they shouldn’t.” (Teacher Survey Response)


• “Leadership does things last minute—so quality is not where it should be. Support on Principal’s end is for families, not teachers. This divides the campus.” (Teacher Survey Response)

Others had positive comments:

• “Our leadership team is doing an awesome job this year. They are available to help teachers, staff and our community.” (Teacher Survey Response)

• “The principal communicates strong and effective goals for the entire school.” (Teacher Survey Response)

The auditors found that not all teachers feel supported equally. Instructional leadership was reportedly provided by positions other than the principal, such as a coach or assistant principal. These data are presented in Exhibit 3.2.21.

Exhibit 3.2.21

Effectiveness of Instructional Leadership (N=87)Phoenix Elementary School District #1

May 2017

Principal18%

Assistant principal

5%

District-based instructional

coach18%

Campus-based instructional

coach30%

Formally assigned mentor

0%

Department head or lead teacher

2%

Another teacher

14%

I do not receive instructional

support/coaching13%

From whom do you get the most instructional support/coaching (including monitoring and feedback, modeling, and feedback)?


• The largest percentage of teachers, 30%, reported that their campus-based instructional coaches are the persons from whom they receive the most instructional support.

• The next largest percentage of teachers (18%) reported both their principals and a district-based instructional coach as the persons from whom they receive the most instructional support.

• Also mentioned was another teacher, with 14% of teachers reporting this position, followed by an assistant principal (5%) and a department head or lead teacher (2%).

Summary

The auditors found that teachers had mixed responses regarding the source and extent of instructional leadership in their buildings. Monitoring is taking place in the form of classroom visits, but the focus of that monitoring is not consistently on curriculum delivery. The focus of site principals is decided at individual campuses, with inadequate direction provided in district documents. Evidence that the data from classroom walk-throughs and the process to collect those data are having impact on classroom instruction was not found.


The auditors observed that classroom-based instruction did not consistently reflect district criteria and expectations found regarding classroom instruction. This instrument was the most comprehensive in communicating what those expectations are; direction in other district documents was very limited. Classroom instruction was mostly whole group, with few instances of differentiation noted. Types of cognition were almost all lower, less demanding, and reflected mostly factual and procedural types of knowledge.

The auditors found that teachers had mixed responses regarding the source and extent of instructional leadership in their buildings. Monitoring is taking place in the form of classroom visits, but the focus of that monitoring is not consistently on the curriculum that is being delivered. The focus of site principals is decided at individual campuses, with inadequate direction provided in district documents. Evidence that the data from classroom walk-throughs and the process to collect those data are impacting classroom instruction was not found.

The auditors observed that classroom-based instruction did not consistently reflect district criteria and expectations found regarding classroom instruction. The teacher evaluation instrument was the most comprehensive in communicating what classroom expectations are; direction in other district documents was very limited. Classroom instruction was mostly whole group, with few instances of differentiation noted. Types of cognition were almost all lower, less demanding and reflected mostly factual and procedural types of knowledge.

Finding 3.3: Professional development is mostly site-based, and is neither coordinated nor planned at the district level in response to curriculum delivery needs. Teachers report different levels of support from leaders depending on the campus at which they work and are dissatisfied with the quality of the training provided at the district-level.

Professional development for teachers is a critical part of assuring improvement in student learning, and thereby improved student achievement scores. Focused and differentiated professional development provides teachers with the training in those skills and processes they need to more effectively deliver the content students must master for success. This training is an essential part of aligning instruction with both the written and assessed curriculum. Such training is further improved when the district has a defined instructional model in place that directs how teachers are to use assessment data and plan instruction accordingly, as well as a framework for strategies that communicates to teachers what effective instruction looks like, in terms of the approaches and activities used with students, in the classroom.. These models then serve as the foundation for any and all trainings aimed at improving teacher practice, and maintains the focus on improving student learning.

Auditors examined the design and delivery of professional development in Phoenix Elementary School District #1 to determine the extent to which trainings improve teacher practice and, ultimately, increase student learning. The auditors first reviewed documents related to professional development goals and implementation. They interviewed central office and building administrators, and surveyed school leaders and teachers to determine their perceptions regarding the quality and effectiveness of professional development offerings across the system.

Auditors found that professional development is not coordinated at the district level and is largely left to school sites to deliver. There is no database that tracks which teachers have had which trainings, nor any apparent monitoring of the trainings provided at school sites to assure that observed weaknesses in the educational program, based on student achievement data, are targeted with a planned and coordinated system of training.

The auditors were not presented with a plan or calendar of professional development offerings planned at the district level beyond those associated with the instructional coaches at each building. They did not receive any records of offerings in the district beyond coaching information and, therefore, relied on policy and interview and survey responses to evaluate the design and delivery of professional development.

Typically, auditors evaluate professional development policies and plans against a set of criteria. However, since documentation was so limited, the auditors found that no criteria were met. Only a single policy was found that directs professional development and this policy, Policy GCI: Professional Staff Development, states that the board “recognizes its particular responsibility to provide opportunity for the continual professional growth of its certificated staff. Such opportunities include, within budgetary limitation, special in-service training courses, workshops, school or District visitations, conferences, professional library, and assistance from supervisors and


consultants.” Auditors did not find policy that directs teacher training to be linked to curriculum delivery, nor any expectation that professional development focus on areas of weakness to improve student achievement.

The criteria used to evaluate professional development programming are presented in Exhibit 3.3.1:

Exhibit 3.3.1

Curriculum Management Improvement Model Staff Development Criteria Auditors’ Assessment of Staff Development Program


CharacteristicsAuditors’ Rating

Adequate InadequatePolicy X1. Has policy that directs staff development efforts. X2. Fosters an expectation for professional growth. X3. Is for all employees. XPlanning and Design4. Is based on a careful analysis of data and is data-driven. X5. Provides for system-wide coordination and has a clearinghouse function in

place. X

6. Provides the necessary funding to carry out professional development goals. X7. Has a current plan that provides a framework for integrating innovations

related to mission. X

8. Has a professional development mission in place. X9. Is built using a long-range planning approach. X10. Provides for organizational, unit, and individual development in a systemic

manner. X

11. Focuses on organizational change—staff development efforts are aligned to district goals. X

Delivery12. Is based on proven research-based approaches that have been shown to

increase productivity. X

13. Provides for three phases of the change process: initiation, implementation, and institutionalization. X

14. Is based on human learning and development and adult learning X15. Uses a variety of professional development approaches. X16. Provides for follow-up and on-the-job application necessary to ensure

improvement. X

17. Expects each supervisor to be a staff developer of staff supervised. XEvaluation18. Requires an evaluation of process that is ongoing, includes multiple sources

of information, focuses on all levels of the organization, and is based on actual change in behavior.

X

Total 0 18Percentage 0%

©2013 CMSi

For a district’s professional development program to be determined as adequate, 70% of the 18 audit criteria must be met. As noted above, the auditors rated Phoenix Elementary School District’s professional development


program as inadequate on all audit characteristics. Therefore, auditors rated the professional development program presently as inadequate in design.

The auditors asked teachers and principals several questions related to training and support provided to them by various entities in the district. Teachers were asked whether or not they receive the training they need most to improve their teaching. These responses are presented in Exhibit 3.3.2:

Exhibit 3.3.2

Teachers’ Perceptions Concerning Adequacy of Professional Development On Online Survey (N=88)


Strongly Agree10%

Agree35%

Disagree30%

Strongly Disagree16%

Other (please specify)

9%

I receive the trainings and support I need most to improve my teaching.


• Thirty-five percent of teachers agreed and 30% disagreed that they receive the trainings and support they need most to improve their teaching.

• Ten percent strongly agreed while 16% strongly disagreed.

Sample comments related to this question, provided on the survey, as well as during interviews, included:

• “It’s a waste of money and resources to be chasing after the newest, shiniest thing, but then refuse to train teachers or to provide ongoing support and resources.” (Teacher Survey Response)

• “To me, the only district structures that supported my teaching or effectiveness has (sic) been the New Teacher Mentor which has been a valuable resource in my effectiveness in the classroom.” (Teacher Survey Response)

• “Teacher retention is the biggest challenge. We probably aren’t doing enough for teacher morale. People don’t feel that they’re getting ongoing training.” (Central Office Administrator)

• “We have those computers, but there is no training. That is a concern because we have people that don’t have any knowledge. So how is the district going to support us? And that’s just in technology; there are other [areas].” (Teacher)

• “In the last couple of years we have had a lack of PD on anything.” (Teacher)

• “So now the PD is nothing. It’s not consistent, it’s not equitable, and it’s not provided to new teachers.” (Central Office Administrator)


• [What needs to improve?] “When we get a new curriculum, continue professional development for all new staff. We have initial PD and then no one gets PD again. Also, continuing PD beyond the initial ‘getting to know the curriculum.’” (Teacher Survey Response)

• “PESD needs improvement in the areas of staff professional development, a dedicated ELA curriculum, writing curriculum, and social studies curriculum.” (Teacher Survey Response)

• “There needs to be an improvement in professional development opportunities for middle school math teachers. Since starting in the middle school, there have been no professional development opportunities offered by the district for the curriculum.” (Teacher Survey Response)

• “In the past the district offered professional development on new curriculum that was purchased.…In the last year or so this has gotten less.” (Teacher Survey Response)

Auditors asked teachers to rank the overall quality of the trainings they receive. These data are presented in Exhibit 3.3.3.

Exhibit 3.3.3

Teachers’ Perceptions Concerning Quality and Relevance Of Professional Development Offerings (N=89)

Phoenix Elementary School DistrictMay 2017

0% 20% 40% 60% 80% 100%

District-provided training (Outside consultant orspecialist.)

District-provided training with district personnelconducting (curriculum personnel, central office…

School site-provided (principal, department head,etc.)

State-provided training or workshop

Out-of-district professional development(conference, workshop)

Quality and Relevance of Professional Development

Excellent Above Average Average Poor


• Teachers rated out-of-district professional development as the highest quality, with 20% reporting it as excellent and one third as above average.

• State-provided and school site-provided trainings were next in the degree of quality and relevance. Just over 40% of teachers reported these trainings to be excellent or above average.

• District-provided training had the lowest rating and the greatest percentage of teachers, almost one fourth, reporting it as poor. Only one fourth of teachers reported district-provided training by district personnel to be excellent or above average.

• District-provided training by outside consultants or specialists was reported by just over 30% to be excellent or above average, with 20% reporting it as poor.

• School site-provided training had the least number of teachers (15%) reporting it as poor. Overall, of the trainings provided in the district, school-based trainings were seen as the most relevant.


While providing professional development at the site level is good practice, the auditors found no connection between district-level professional development planning and what is planned and delivered at the sites. Several building administrators commented on the positive nature of the professional development planning process at their school site:

• “I look at my observations in the classroom, our LOI teacher evaluations, the class, the teachers and I come up with a list of things that look like PD for next year and we vote on those. Then I put them all that together and come up with a PD plan with support from Quality First.” (Campus Administrator)

• “Everything that happens, happens at the sites. District personnel do not come into the classrooms. No professional development from the district is at the campus.” (Campus Administrator)

• “We have our own PD, as needed. We have data analysis training with an outside consultant.” (Campus Administrator)

• “I find [our] PD approach very effective as we have a strong leadership team and we know the needs of our teachers. Our PD is targeted to the needs of our teachers. Oftentimes centralized district PD is not tailored to the needs of our teachers and does not have the same impact on instructional practice.” (School Leader Survey Response)

• “On a schoolwide level, we have a strong principal and we’re doing the best we can. On a district level, there is virtually no support for teachers, and what support is given is useless or faulty.” (Teacher Survey Response)

Principals were also asked about the quality of training in the system. These data are presented in Exhibit 3.3.4:

Exhibit 3.3.4

Principals’ Perceptions Concerning Professional Development (N=9)Phoenix Elementary School District #1

May 2017

0% 20% 40% 60% 80% 100%

Effective implementation of professional development isevident as teachers deliver instruction to meet diverse

student needs.

Adequate professional development time and opportunitiesare available to support teachers in improving instruction to

meet diverse student needs.

Adequately funded professional development is available tosupport teachers in improving instruction to meet diverse

student needs.

Principals' Perceptions of Professional Development


Principal responses to the questions presented in Exhibit 3.3.4 showed that most principals agreed with each statement.

• Twenty-two percent disagreed that adequately funded professional development is available to support teachers in improving instruction to meet students’ needs.

• Eighty-nine percent agreed or strongly agreed with the statements that adequate professional development time and opportunities are available to support teachers in improving their instruction to meet diverse student needs.

• Most (88%) agreed that effective implementation of professional development is evident.


Auditors noted that principals’ agreement with these statements stands in contrast with their level of agreement with other questions concerning whether or not teachers have had adequate training in certain areas. These survey data are presented in Exhibit 3.3.5:

Exhibit 3.3.5

Principals’ Perceptions Concerning Quality and Relevance Of Professional Development Offerings (N=8)

Phoenix Elementary School DistrictMay 2017

0% 20% 40% 60% 80% 100%

Teachers have adequate training in the use of the curriculumdocuments.

Teachers have adequate training in the use of instructionalresources.

Teachers have been trained in strategies for understandingand addressing the various cultural needs of students.



• The majority (75%) of principals disagreed that teachers have been trained in strategies for understanding and addressing the cultural needs of students.

• Just under two thirds disagreed that teachers have had adequate training in the use of instructional resources, and three fourths again disagreed that teachers have had adequate training in the use of the curriculum documents.

When asked what needed improvement in the district, many comments indicated a need for additional training for teachers. Comments were made by campus administrators as well as teachers. Comments made in response to the question, “What is something that needs improvement?” included:

• “Worthwhile professional development based on teacher need and teacher input without backlash.” (Teacher Survey Response)

• “Professional development that is district wide, sincerity to one program for more than a couple years.” (Teacher Survey Response)

• “Consistent district content professional development opportunities, consistent and targeted new teacher support/mentoring.” (School Leader Survey Response)

• “When a school gets technology, PD needs to be provided by the district.” (Teacher Survey Response)

Others commented on the low quality of trainings or lack of adequate training:

• “Most of the schools do not want the district coaches on their campuses. The district professional development just confuses my teachers.” (Campus Administrator)

• “PD—it’s only been optional. Some schools do it and some schools don’t want anything to do with it.” (Support Staff)


• “The principals have to arrange the trainings from the district office. There is no plan for the district to come regularly to train teachers.” (Campus Administrator)

• “Some teachers have said that nothing is going to change in my classroom because of the PD they have been receiving. There have been no new ideas for them to make them change their instruction through PD.” (Campus Administrator)

• “Nothing was going on with the technology because there was no training for teachers. It still hasn’t happened.” (Board Member)

• “So our teachers are not well-prepared. We have to do all of the training.” (Central Office Administrator)

Some responses commented on a propensity to offer training only one time or to deliver resources to schools with no training on their use. Several individuals mentioned a lack of consistency or a tendency to keep moving to something new. These comments included:

• “The district does a lot of that—training one time and then we are expected to go in and help and do it.” (Campus Administrator)

• “It doesn’t feel like it’s cohesive. They are already launching new things.” (Support Staff)

• “The curriculum should be district-wide, but that’s not what I hear. If we did, we could have consistent professional development. We’re always doing the new thing.” (Board Member)

Similar comments were submitted by teachers on the Teacher Online Survey and the district’s new teacher survey about a perceived lack of support for teachers in the district. These comments included:

• “Overall, I feel like they’re out of touch with what teachers want and need. They’re wasting money. We’re out here writing our own curriculum in every subject area because we don’t even have textbooks. Teachers are the boots on the ground, and deserve to be supported at every turn. It’s their job to make my life easier so I can focus on my students…” (Teacher Survey Response)

• “I will not be returning due to lack of support for first year special education teachers and a top down decision making process from administration.” (New Teacher Survey)

• “I feel that new hires should receive more support, especially with the LOI, which are very difficult to execute. There seems to be a lot of things that we do not know & we find out a little too late. I really do not like surprises & wish there is more communication.” (New Teacher Survey)

Finally, auditors found that board members and central office administrators communicated the need to help principals in supporting teachers with good training. These included:

• “At schools, they don’t have time on the agenda to provide some professional development for general education teachers. They’re always pulled in separate directions.” (Central Office Administrator)

• “We have some really smart leadership, some good principals, and some good directors. I would like to use them better and more. I don’t want to lose them” (Central Office Administrator)

• [What is needed?] “Finding and keeping teachers so we don’t bleed professional development out the door. Good teachers make the difference.” (Board Member)

• “Provide resources, training and hold accountable. Create higher standards to serve our kids just like other students.” (Central Office Administrator)

Summary

Auditors found that Phoenix Elementary #1 does not have a system for professional development. Most training is relegated to the schools. Support for effective curriculum delivery is not coordinated at the district level. Teachers repeatedly reported not having adequate training to support their instructional delivery or orient them to a district wide curriculum. Several new teachers indicated that they experienced a lack of support at both the building or district levels.


STANDARD 4: The School District Uses the Results from System-Designed and/or -Adopted Assessments to Adjust, Improve, or Terminate Ineffective Practices or Programs.A school system meeting this audit standard has designed a comprehensive system of assessment/testing and uses valid measurement tools that indicate how well its students are achieving designated priority learning goals and objectives. Common indicators are:

• A formative and summative assessment system linked to a clear rationale in board policy;

• Knowledge, local validation, and use of current curricular and program assessment best practices;

• Use of a student and program assessment plan that provides for diverse assessment strategies for varied purposes at all levels—district, school, and classroom;

• A way to provide feedback to the teaching and administrative staffs regarding how classroom instruction may be evaluated and subsequently improved;

• A timely and relevant data base upon which to analyze important trends in student achievement;

• A vehicle to examine how well specific programs are actually producing desired learner outcomes or results;

• A data base to compare the strengths and weaknesses of various programs and program alternatives, as well as to engage in equity analysis;

• A data base to modify or terminate ineffective educational programs;

• A method/means to relate to a programmatic budget and enable the school system to engage in cost-benefit analysis; and

• Organizational data gathered and used to continually improve system functions.

A school district meeting this audit standard has a full range of formal and informal assessment tools that provide program information relevant to decision making at classroom, building (principals and school-site councils), system, and board levels.

A school system meeting this audit standard has taken steps to ensure that the full range of its programs is systematically and regularly examined. Assessment data have been matched to program objectives and are used in decision making.


The auditors expected to find a comprehensive assessment program for all aspects of the curriculum, PreK through grade 12, which:

• Was keyed to a valid, officially adopted, and comprehensive set of goals/objectives of the school district;

• Was used extensively at the site level to engage in program review, analysis, evaluation, and improvement;

• Was used by the policy-making groups in the system and the community to engage in specific policy review for validity and accuracy;

• Was the foci and basis of formulating short- and long-range plans for continual improvement;

• Was used to establish costs and select needed curriculum alternatives; and

• Was publicly reported on a regular basis in terms that were understood by key stakeholders in the community.



This section is an overview of the findings that follow in the area of Standard Four. Details follow within separate findings.

Finding 4.1: District-wide student assessment data are below state averages in English language arts and mathematics and show significant gaps in various subgroups.

In effective school districts, leaders monitor student achievement trends closely from a variety of assessment instruments to determine how well students are mastering the intended curriculum and to evaluate the effectiveness of programs. Student achievement data provide a district with feedback that allows personnel to determine the effectiveness of a district’s delivered curriculum.

Regular review of data trends and disaggregated assessment data provide leaders insight in monitoring program and district effectiveness over time, and in measuring performance in comparison to other subgroups, districts, or even states. Analysis of data enables district leaders to see how their students perform from one grade level to the next, as well as make informed modifications and adjustments to curriculum and instruction that will generally result in improved student achievement.

Comparative data trends provide information on how assessment results change over time. An effective system with quality control shows improved performance for all students while achievement gaps among student groups are reduced. Analysis of disaggregated achievement results and trends from the same assessments allow the district to identify groups of students in need of greater levels of support. Identifying gaps in student achievement between subgroups allows for resources to be directed to the areas of greatest need.

To determine how students in Phoenix Elementary School District #1 are performing on assessment measures currently and over time, auditors examined assessment data prepared by district personnel in order to determine achievement trends that exist in the district. Overall, auditors found district student achievement trends to be below state averages in English language arts and mathematics as measured by the AzMERIT assessment. Furthermore, the trend data showed the district to have significant achievement gaps between various student subgroups.

Well performing school systems show improved student achievement relative to state averages over time. The following exhibits show the percentage of students in grades 3-8 meeting or exceeding state standards in English language arts and mathematics as compared to the average state percentage of students meeting or exceeding the state standard.

Exhibit 4.1.1 compares district and state performance in English language arts for grades 3-8 on the AzMERIT for 2015 and 2016.

Exhibit 4.1.1

Comparison of State and District English Language Arts Performance By Percentage of Students Scoring Proficient and Above in Grades 3-8 on the AzMERIT

Phoenix Elementary School District #12015 and 2016

2015 2016Grade State District Difference State District Difference

3 41% 24% -17 41% 24% -174 42 28 -14 46 30 -165 32 14 -18 45 27 -186 36 20 -16 38 18 -207 34 17 -17 41 29 -128 35 21 -14 34 22 -12

Average Difference -16.0 Average Difference -15.8Source: District Documents



• The percentage of the student population in each grade level (3-8) that met the English language arts standard (proficient or above) was lower than the average state percentage.

• Overall, district students were an average of 16 percentage points below their state peers in 2015 and 15.8 percentage points below in 2016 in grades 3-8.

• The smallest gap was in grades 7 and 8 with a district gap of 12 percentage points below the state average in 2016.

• The largest gap was in grade 6 with a district gap of 20 percentage points below the state average in 2016.

• In 2015, the range of difference between the district and state performance was from -14 to -18 percentage points.

• In 2016, the range of difference between the district and state performance was from -12 to -20 percentage points.

Auditors then examined state and district performance in mathematics. Exhibit 4.1.2 compares district and state performance in mathematics for grades 3-8 on the AzMERIT for 2015 and 2016.

Exhibit 4.1.2

Comparison of State and District Mathematics Performance By Percentage of Students Scoring Proficient and Above in Grades 3-8 on the AzMERIT


2015 2016Grade State District Difference State District Difference

3 42% 28% -14 46% 29% -174 42 25 -17 44 26 -185 40 24 -16 46 29 -176 33 16 -17 39 24 -157 31 17 -14 32 18 -148 34 17 -17 26 23 -3

Average Difference -15.8 Average Difference -14.0Source: District Documents


• The percentage of the district student population in each grade levels (3-8) that met the mathematics standard was lower than the average state percentage across all grades by an average of almost 16 percentage points in 2015 and 14 points in 2016.

• The smallest gap was in grade 8 with a gap of 3 percentage points below the state mathematics average in 2016.

• The largest gap was in grade 4 with a gap of 18 percentage points below the state mathematics average in 2016.

• In 2015, the range of differences between the district and state performance was from -14 to -17 percentage points.

• In 2016, the range of differences between the district and state performance was from -3 to -18 percentage points.

Overall, the district shows student achievement data in grades 3-8 as lower than the state average in both English language arts and mathematics for 2015 and 2016.


Well performing schools systems show a narrowing of student achievement gaps over time. The following exhibits show the difference in AzMERIT achievement gaps between various subgroups in English language arts and mathematics.

Exhibit 4.1.3 and Exhibit 4.1.4 compare the performance between district White students and Hispanic students for grades 3-8 on the 2015 and 2016 AzMERIT English language arts and mathematics assessments.

Exhibit 4.1.3

Comparison of White and Hispanic Students Performance by Percentage of Students Scoring Proficient and Above in Grades 3-8 on the AzMERIT in English Language Arts


2015 2016Grade White Hispanic Difference White Hispanic Difference

3 43% 21% -22 37% 23% -144 55 27 -28 34 30 -45 31 13 -18 30 28 -26 38 19 -19 46 18 -287 35 17 -18 64 28 -368 30 21 -9 40 22 -18

Average Difference -19.0 Average Difference -17.0Source: District documents


• The smallest gap of 2 percentage points existed in grade 5 with White students outperforming Hispanic students on the 2016 state English language arts assessment.

• The largest gap of 36 percentage points existed in grade 7 with White students outperforming Hispanic students on the 2016 state English language arts assessment.

• White students consistently outperformed Hispanic students in 2015 and 2016 on the English language arts state assessment.

• Across grade levels in both years of data (2015 and 2016), district Hispanic students were on average 19 and 17 percentage points respectively below their White peers on AzMERIT assessments.

• In 2015, the differences between the performances of Hispanic and White students ranged from -9 to -28 percentage points.

• In 2016, the differences between the performances of Hispanic and White students ranged from -2 to -36 percentage points.


Exhibit 4.1.4

Comparison of White and Hispanic Students Performance by Percentage of Students Scoring Proficient and Above in Grades 3-8 on the AzMERIT in Mathematics


2015 2016Grade White Hispanic Difference White Hispanic Difference

3 40% 26% -14 32% 29% -34 55 23 -32 38 24 -145 47 23 -24 33 28 -56 15 16 +1 42 24 -187 25 17 -8 36 17 -198 35 16 -19 32 24 -8



• The smallest gap of 1 percentage point existed in grade 6 with Hispanic students outperforming White students on the 2015 state mathematics assessment. The second smallest gap of 3 occurred with White students outperforming Hispanic students in grade 3 in 2016.

• The largest gap of 32 percentage points existed in grade 4 with White students outperforming Hispanic students on the 2015 state assessment in mathematics.

• Although Hispanic students outperformed White students in 2015 by 1 percentage point in grade 6, the following year the cohort of Hispanic students fell below White students by 19 percentage points in grade 7.

• Overall, across grades 3-8, White students were an average of 16 percentage points above their Hispanic peers in 2015 and more than 11.2 points in 2016.

• In 2015, the difference between the performance of Hispanic and White students ranged from -32 to 1 percentage points.

• In 2016, the difference between the performance of Hispanic and White students ranged from -3 to -19 percentage points.

Overall, auditors found achievement gaps existed between White and Hispanic students on the 2015 and 2016 AzMERIT. White students outperformed Hispanic students in all but one case.


Auditors then examined the assessment of White and African American students. Exhibit 4.1.5 and Exhibit 4.1.6 compare district performance between White and African American students for grades 3-8 on the 2015 and 2016 AzMERIT English language arts and mathematics assessments.

Exhibit 4.1.5

Comparison of White and African American Students Performance By Percentage of Students Scoring Proficient and Above in Grades 3-8

On the AzMERIT in English Language ArtsPhoenix Elementary School District #1

2015 and 2016

2015 2016

Grade White African American Difference White African

American Difference

3 43% 27% -16 37% 29% -84 55 26 -29 34 27 -75 31 16 -15 30 16 -146 38 20 -18 46 14 -327 35 16 -19 64 26 -388 30 37 +7 40 21 -19



• Grade 8 African American students outperformed White students by 7 percentage points on the state assessment in 2015, but in 2016 White students outperformed African American students by 19 percentage points in grade 8.

• The largest gap of 38 points exists in grade 7 with White students outperforming African American students on the state assessment in 2016.

• Across grade levels, an significant average of 15 percentage points in 2015 and almost 20 points in 2016 existed between the performance of White students and African American students.

• In 2015, the difference in African American students and White students ranged from -29 to 7 percentage points.

• In 2016, the difference in African American students and White students ranged from -7 to -38 percentage points.


Exhibit 4.1.6

Comparison of White and African American Students Performance By Percentage of Students Scoring Proficient and Above in Grades 3-8

On the AzMERIT in MathematicsPhoenix Elementary School District #1

2015 and 2016

2015 2016

Grade White African American Difference White African

American Difference

3 40% 35% -5 32% 21% -114 55 24 -31 38 24 -145 47 22 -25 33 21 -126 15 18 +3 42 20 -227 25 7 -18 36 18 -188 35 26 -9 32 10 -22



• African American students outperformed White students only one time grade 6 by 3 percentage points on the state assessment in 2015. However, in 2016 White students outperformed African American students by 22 percentage points in grade 6.

• The largest gap of 31 points existed in grade 4 with White students outperforming African American students on the state assessment in 2015.

• On average across grades 3-8, African American students scored below White students more than 14 and 16 percentage points, respectively, on the 2015 and 2016 AzMERIT mathematics state assessment.

• In 2015, the difference in African American students and White students ranged from -31 to 3 percentage points.

• In 2016, the difference in African American students and White students ranged from -11 to -22 percentage points.

Overall, auditors found achievement gaps exist between White students and African American students on the 2015 and 2016 AzMERIT. White students outperformed African American students in all but one case.


Exhibit 4.1.7 and Exhibit 4.1.8 compares district performance between Hispanic students and African American students for grades 3-8 on the AzMERIT for 2015 and 2016 English language arts and mathematics assessments.

Exhibit 4.1.7

Comparison of Hispanic and African American Students Performance By Percentage of Students Scoring Proficient and Above in Grades 3-8


2015 and 2016

2015 2016

Grade Hispanic African American Difference Hispanic African

American Difference

3 21 27 +6 23 29 +64 27 26 -1 30 27 -35 13 16 +3 28 16 -126 19 20 +1 18 14 -47 17 16 -1 28 26 -28 21 30 +9 22 21 -1

Average Difference +2.8 Average Difference -2.7Source: District documents


• Overall, African American and Hispanic students performed relatively similar on the 2015 and 2016 AzMERIT English language arts assessments. On average, African American students outperformed Hispanic students in 2015, but were outperformed in 2016.

• African American students outperformed Hispanic students in grades 3, 5, 6, and 8 in 2015.

• African American students outperformed Hispanic students only in grade 3 in 2016.

• The largest gap of 12 points existed in grade 5 with Hispanic students outperforming African American students on the state assessment in 2016.

• In 2015, the difference in African American students and Hispanic students ranged from -1 to 9 percentage points.



Exhibit 4.1.8

Comparison of Hispanic and African American Students Performance By Percentage of Students Scoring Proficient and Above in Grades 3-8


2015 and 2016

2015 2016

Grade Hispanic African American Difference Hispanic African

American Difference

3 26% 35% +9 29% 21% -84 23 24 +1 24 24 05 23 22 -1 28 21 -76 16 18 +2 24 20 -47 17 7 -10 17 18 +18 16 26 +10 24 10 -14

Average Difference +1.8 Average Difference -5.3Source: District document


• African American students outperformed Hispanic students in grades 3, 4, 6, and 8 on the 2015 AzMERIT mathematics assessment, but performed below Hispanic students in all but two grades in 2016.

• African American students outperformed Hispanic students only in grade 7 only in 2016.

• The largest gap of 14 points existed in grade 8 with Hispanic students outperforming African American students on the state assessment in 2016.

• In 2015, the difference in African American students and Hispanic students ranged from -10 to 10 percentage points in grades 7 and 8.


Overall, auditors found achievement gaps exist between Hispanic students and African American students on the 2015 and 2016 AzMERIT. African American students outperformed Hispanic students in most grade levels in 2015 for both English Language Arts and mathematics. However, Hispanic students outperformed African American students 2016 in English language arts and mathematics.


The district’s White student population is a small percentage (5.8%) of the total student population; however, the district’s White student population is outperforming other district student groups on the state assessment. Auditors felt the necessity to compare their respective results to the state average proficiency. Exhibit 4.1.9 and Exhibit 4.1.10 compares district performance of White students to the average state proficiency for White students for 2015 and 2016 on the AzMERIT English language arts and mathematics assessments.

Exhibit 4.1.9

Comparison of District White Students and State White Students Performance By Percentage of Students Scoring Proficient and Above in Grades 3-8


2015 and 2016

2015 2016

Grade White State

White District Difference White

StateWhite

District Difference

3 56% 43% -13 57% 37% -204 56 55 -1 63 34 -295 47 31 -16 61 30 -316 50 38 -12 54 46 -87 47 35 -12 56 64 +88 47 30 -17 47 40 -7

Average Difference -12 Average Difference -15Source: District Documents


• The percentage of the district’s White student population in each grade level (3-8) that met standards was lower than the state White student percentage across all grades by an average of 12 percentage points in 2015 and 15 percentage points in 2016.

• The only case in which district students outperformed state peers was in grade 6 with a gap of 8 percentage points above the 2016 state results.

• The largest gap was in grade 5 with a gap of 31 percentage points below the state average in 2016.

• In 2015, the difference between the district and state performance was from -1 to -17 percentage points.

• In 2016, the difference between the district and state performance was from -31 to -8 percentage points.


Exhibit 4.1.10

Comparison of District White Students and State White Students Performance By Percentage of Students Scoring Proficient and Above in Grades 3-8


2015 and 2016

2015 2016

Grade White State

White District Difference White

StateWhite

District Difference

3 55% 40% -15 61% 32% -294 56 55 -1 60 38 -225 54 47 -7 60 33 -276 46 15 -31 55 42 -137 44 25 -19 45 36 -98 46 35 -11 37 32 -5

Average Difference -14 Average Difference -18Source: District Documents


• The percentage of the district’s White student population in grade levels (3-8) that met standards was lower than the state White student percentage on average across all grades by an average of 14 percentage points in 2015 and 18 percentage points in 2016.

• The smallest gap was in grade 4 with a gap of 1 percentage point below the state average in 2015.

• The largest gap was in grade 6 with a gap of 31 percentage points below the state average 2015.

• In 2015, the difference between the district and state White students ranged from -1 to -19 percentage points. In 2016, the differences between the district and state White students ranged from -5 to -29 points.

Overall, auditors found the highest achieving ethnic subgroup in the district falls below its peers when compared to the state performance levels. The district’s White student subgroup was outperformed on both the English language arts and mathematics AzMERIT assessment by their respective state White student subgroup in both 2015 and 2016.

With the district’s low SES population exceeding 80% of the student population, auditors would typically compare student achievement of the low SES group to the non-SES group. With the absence of the non-SES data, auditors noted the trend data for the district’s low SES group on the AzMERIT 2015 and 2016 in English language arts and mathematics.


Exhibit 4.1.11 and Exhibit 4.1.12 graphically depict the performance of the district’s low SES students for grades 3-8 on the 2015 and 2016 AzMERIT English language arts assessment and mathematics assessment.

Exhibit 4.1.11

Performance by Percentage of Low SES Students Scoring Proficient and Above In Grades 3-8 on the AzMERIT in English Language Arts


Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 82015 22 26 12 18 16 192016 32 38 33 42 36 32

0

5

10

15

20

25

30

35

40

45


• Performance of low SES students increased across grades 3-8 from 2015 to 2016.

• In 2015, the highest performance was in grade 4 with 26% of the low SES group scoring proficient or higher.



Exhibit 4.1.12

Performance by Percentage of Low SES Students Scoring Proficient and Above In Grades 3-8 on the AzMERIT in Mathematics


Grade 3 Grade 4 Grade 5 Grade 6 Grade 7 Grade 82015 27 23 24 15 17 152016 29 25 28 24 17 23

0

5

10

15

20

25

30

35


• Performance of low SES students increased slightly across grades 3-8 from 2015 to 2016 with the exception of grade 7, which remained at 17%.


• In 2016, the highest performance was also in grade 3 with 29% of the low SES group scoring proficient or higher.

Overall, auditors found the district’s low SES students’ performance increased from 2015 to 2016 on both the English language arts and mathematics AzMERIT.

Years to Parity

With the gaps in scores found from the exhibits above, auditors determined that a “Years to Parity” analysis was warranted. A typical “Years to Parity” is a numerical estimate of the predicted trend of measured achievement differences between two or more groups with three or more years of testing data. Assuming that poverty, race, gender, or other ethnic or demographic differences should not predict differences in achievement levels, conventional wisdom is that group differences in achievement are the result of disparate, inadequate, or ineffective educational experiences rather than ethnic or demographic characteristics. Moreover, the expectation in curriculum management auditing is that all such groups should achieve at comparable levels—demonstrating parity or equivalency in achievement if not at the time of measurement, then with intervention at some demonstrable and reasonable future point in time.

If comparisons of student groups were made with data obtained from dissimilar groups (not cohorts over time), it is understood that the calculation is an estimate or an educated guess only. Years to parity calculations assist districts in identifying gaps in student achievement and determining whether the district should continue current interventions or identify and implement more efficient and effective strategies to address the identified cause of the achievement gap.


To more fully understand those gaps, auditors can calculate the “years to parity.” The rate of change of each lower performing group must be higher than the rate of change of the comparison group in order for the gap to be closed at some point in the future. If the rate of change of the lower performing group is equal to or less than the rate of change of the comparison group, then the gap will not be closed.

To calculate the change for both groups, auditors divide the net change between the first year and last year by the number of years in order to derive the average rate of change. If the rate of change were higher for the lower performing group, the auditors subtract the rate of change of the higher performing group from the lower performing group to determine the average annual gain being made by the lower performing group. Then the auditors divide the last year’s difference in passing rates between the lower performing group and the comparison group by the difference between the average rates of change to determine how many years it would take to close the achievement gap at the present rate of change. The lower performing group must be progressing at a faster rate of change than the comparison group to achieve parity and to close the achievement gap.

Exhibit 4.1.13 is the template for calculating “Years to Parity” between the performance of two or more subgroups of students (e.g., Free/Reduced Lunch students versus Non-Free/Reduced Lunch students; White students versus Hispanic students, etc.).

Exhibit 4.1.13

“Years to Parity” Calculation Template

Student Group Subject Grade 2014 2015 2016 2017 CalculationNon Free/Reduced Lunch ELA 4 67 73 77 90

Free/Reduced Lunch ELA 4 25 28 30 56Difference* 42 45 47 34

Change in Difference First year difference minus final year difference 8Gain by Year Change in difference ÷ gaps (number of years minus one**) 2.67Years to Parity Final year difference ÷ gain by year 12.73*If the mean gap grows larger or stays the same, Years to Parity will equal “Never” (heading the wrong way).**The number of years is numerically 1 greater than the intervals between years. The number of years is four (4), but there are three (3) intervals. Consequently, the divisor needs to be the number of years – 1 (n-1).NOTE: COHORT groups must be used. If cohort data is not available, the calculations will be rough estimates of the years needed to achieve parity. The GRAY cells are for data entry.

Given the disparity presented in Exhibits 4.1.1 through 4.1.12 above and the district-wide desire to use data to assess student needs and improve achievement, auditors questioned why students have not performed better on the state assessments in all subgroups. Auditors provided a rationale that related to three basic assumptions underlying the Curriculum Audit™:

• The written, taught, and tested curricula must be aligned for the best teaching and learning results;

• Alignment of the written, taught, and tested curricula optimizes a student’s chance for success on high-stakes tests by preparing the student to deal effectively with the content, contextual, and cognitive demands of those tests; and professional development must equip teachers to deliver the aligned written curriculum effectively.

During interviews, the auditors heard several comments from district stakeholders regarding their concerns over student achievement. These individuals commented on the low achievement across the system and a need to improve performance, while others shared the opinion that student achievement is not a district priority for all:

• “It seems at times that focus on student performance is not highly valued here.” (District Administrator)

• “But honestly we need to improve our achievement. Our families have said, we don’t want to send our kids to a C school.” (District Administrator)


• “I have great concern about the level of academic achievement of our students. I feel that it requires a sense of urgency and the success of our students is critical.” (Board Member)

• “It depends who you talk to—some want to close the achievement gap and do everything possible; others don’t verbalize that. It’s not everyone’s goal.” (Campus Administrator)

• “Student achievement should not depend on the school you are in.” (Parent)

• Concerns? “Low and stagnant student achievement.” (Teacher)

• “Test scores have to come up some way. A challenge in equity in the district.” (Board Member)

Overall, interview comments reveal a concern over student achievement and the lack of improvement in recent years.

Summary

In systems with effective quality control, performance for all students should improve over time, and performance gaps between student subgroup populations should reduce in size. Systems and processes in place in effective school districts function in a coordinated fashion to maximize student learning and achievement. Since the new state assessment (AzMERIT) was implemented in the 2014-15 school year, the district fell significantly below the state averages in both English Language Arts and Mathematics in grades 3-8. Furthermore, assessment data show significant achievement gaps between student subgroups. Existing trends show that without targeted intervention and different instructional approaches and strategies, achievement gaps are expected to continue without reduction or elimination.

Finding 4.2: The scope of the district’s student assessment program is inadequate to provide sufficient data for curriculum and instructional changes to improve student achievement.

In highly effective school districts, there are quality, aligned assessment tools for every course and subject area taught. These assessment tools are mostly formative so they provide teachers with immediate feedback on students’ progress in mastering the curriculum objectives. Other tools may be diagnostic in nature, while yet others are summative and/or norm-referenced and useful to compare the district’s students with those of other districts and states.

Formative and summative student assessment data provide the foundation for decisions regarding instructional decision making and planning for curriculum design and delivery through evidence of student achievement across grade levels and content areas. Ongoing and timely student achievement data verify the alignment between the written, taught, and tested curriculum. A comprehensive, well-articulated student assessment program allows the district to measure the effectiveness of the taught curriculum in attaining desired levels of student achievement. Student assessment data also provides feedback that can be used to improve curriculum content and classroom practices to maximize academic achievement for individuals and groups of students.

Effective assessment programs require student achievement to be formally evaluated in every taught course at every grade level. When the scope of the assessment program does not meet the standard, the school system lacks the evidence needed to determine progress in student learning and the status of educational programs. The existence of assessment tools does not guarantee their validity and their alignment with the standards and objectives they are intended to measure. This finding addresses only the existence of the various assessment tools and their type.

For this finding, a formal assessment is defined as an administratively mandated standardized assessment for all Phoenix Elementary School District #1 students in a grade level or course that is collected and retained at the district level. Results for the formal assessments are available at the district and utilized for district decision making. The following information is also valid in relation to formal assessments:

• Both formative and summative assessments are considered to be formal if they meet the criteria for standardized and administrative mandate.


• State and national tests are considered formal assessments if they are mandated for all Phoenix #1 students in a grade level.

• Teacher-made assessments are not counted as formal assessments unless they are standardized district wide and mandated by administrative regulation for all district students in a course or grade level.

• Assessments that are mandated for all students in a particular subgroup are considered formal assessments if their results are collected at the district level. An example is the Arizona English Language Learner Assessment.

• Assessments that are given by specialists to diagnose individual student learning difficulties are not considered formal assessments for the purpose of this finding.

While auditors found that formal assessment was prevalent across grade levels in some core content areas, the overall scope of student assessment is inadequate to provide sufficient data to monitor instruction and guide instructional decision making. The lack of adequate assessment data hinders teachers in their efforts to make informed decisions regarding curriculum and instructional changes to improve student achievement.

To determine the scope of formal assessment in Phoenix Elementary School District #1, auditors reviewed board policies, job descriptions, course offerings, curriculum assessment documents, assessment reports, and assessment calendars. Auditors interviewed board members, administrative staff, teachers, parents, and students, and visited classrooms throughout the district to gain further information regarding the scope of the district’s assessment system. An online survey was provided to teachers, school leaders, and parents.

In reviewing board policies, auditors found policies lack specific requirements for all courses taught at all grade levels to be assessed. The following board policies make reference to the scope of student program assessment:

• Board Policy IKE: Promotion and Retention of Students notes that pertinent data be utilized to help determine grade level promotion.

• Board Policy IL: Evaluation of Instructional Programs refers to the use of tests for student achievement and further requires the district to participate in a testing program but lacks the qualifications of covering all content areas at all grade levels.

Auditors reviewed job descriptions in addition to board policies and identified the following with roles and responsibilities in the area of assessment:

• Director of Assessment and Accountability job description states the following, “Manage student data relevant to student assessment.”

• Director of Educational Effectiveness job description includes as part of the duties to “implement with REIL and District staff a data management system enabling leaders and teachers to use data to inform decisions.”

• Instructional Coach’s job description states a responsibility to include “Assisting teachers in identifying students’ instructional needs using lesson plans, formative and summative assessments.”

• The Director of Coaching Effectiveness must provide “On-going support with a variety of student assessment tools and analysis of student achievement data.”

Overall, policies and job descriptions lack a requirement for a formal evaluation of all content areas across all grade levels. Job descriptions do not designate one leadership position or combination of leadership positions with the responsibility of ensuring that the taught curriculum is meeting the goals of the districts instructional program through the use of assessment analysis.


Auditors examined district provided assessment information to identify formal tests administered in the district. Exhibit 4.2.1 describes the major assessments administered in Phoenix #1.

Exhibit 4.2.1

Formal Assessments of Student PerformancePhoenix Elementary School District #1

May 2017

Assessment Months of Testing Grades Description

Galileo August, October January April to

May K-8

Criterion referenced District Benchmark assessment based on the AzMERIT blueprint administered for English Language Arts, mathematics and science – primary to pay for performance system

AIMSweb

August, (Ongoing Progress

Monitoring)

November to

December, January

April to May K-8

Criterion referenced Reading assessment for special education students and some intervention students. Benchmark assessment administered on-line.

AZELLA February to March K-8

The Arizona English Language Learner Assessment (AZELLA) is a standards-based assessment that meets both state and federal requirements to measure students’ English language proficiency. AZELLA is a criterion referenced test.

AIMS Science

March to April 4 and 8 Arizona Instrument to Measure Science

standards criterion referenced test.

AzMERIT March to May 3-8

Arizona’s Measurement of Educational Readiness to Inform Teaching is a statewide achievement test in English Language Arts and mathematics. AzMERIT is a criterion referenced test.

MCESA/PCG

August to September

April to May 2-8

Maricopa County Educational Service Agency’s Assessment for specials (e.g. Arts and PE), and Social Studies – primary to pay for performance system.

Source: District Assessment Calendar 2016-2017, District Document List of Assessments

Exhibit 4.2.1 shows that six required categories of formal assessments are administered in Phoenix #1 with five of the six assessments having an administration window spanning from March to May.


In order to assess the scope of student assessments in Phoenix #1, auditors examined district provided documents related to assessments. The assessments listed in Exhibit 4.2.2 represent formal assessments administered in PreK through grade 8 by content area that are required by the state or district.

Exhibit 4.2.2

Formal Assessments Administered in PreK through Grade 8 By Content Area


Assessment/SubjectGrade Level

PreK K 1 2 3 4 5 6 7 8English Language ArtsAzMERIT S S S S S SAZELLA S S S S S S S S SAIMSweb D D D D D D D D DGalileo D D D D D D D D DMathematicsAzMERIT S S S S S SGalileo D D D D D D D D DScienceGalileo D D D D D DAIMS S SSocial StudiesMCESA/PCG D D DArtMCESA/PCG D D DGeneral MusicMCESA/PCG D D D ChoirMCESA/PCG D DTheaterMCESA/PCG D DDanceMCESA/PCG D DEnsembleMCESA/PCG D DPhysical EducationMCESA/PCG D D

Totals 0 4 4 5 10 8 10 8 11 15Note: S=State Mandated; D=District Mandated Source: District Assessment Calendar 2016-2017


• A total of 75 state and district tests are administered to students from grades K-8.

• No formal testing is administered for PreK.

• Eighth grade students are administered 15 assessments each year. Grades 3, 5, 7, and 8 are administered a total of 46 assessments; 61% of the tests are mandated by the district.


• All students in grades K-8 have required assessments in reading and mathematics by the state and district.

• Science is formally assessed in grades 4 and 8 by the state and by the district in grades 3 through 8 (total of eight tests).

• Social Studies is formally assessed in grades 6, 7, and 8 by the district.

• “Specials” (Art, Choir etc.) are assessed as required by the district in various grade levels.

Exhibit 4.2.3 shows the scope of assessment for core and non-core content areas for PreK through grade 8. To be considered adequate, the scope of the taught curriculum that is assessed must be 100% for the core content areas (English language arts, mathematics, science, and social studies) and 70% for non-core content areas.

Exhibit 4.2.3

Scope of Formal Assessment for Grades PreK-8Phoenix Elementary School District #1

May 2017

Content Area Number of Course Offerings

Courses with Formal Assessment

Percent Courses with Formal Assessment

Core CoursesEnglish Language Arts 10 9 90%Mathematics 10 9 90Science 10 6 60Social Studies 10 3 30

Total Core Courses 40 27 68%Non-Core Courses

Art 10 3 30General Music 10 3 30Choir 10 2 20Physical Education 10 2 20Theater 10 2 20Dance 10 2 20Ensemble 10 2 20

Total Non-Core Courses 70 16 23%Total Courses Assessed 110 43 39%

Source: District Assessment Calendar 2016-17


• The total scope of assessment for core courses was 68%, which is below the audit standard of 100% for adequacy in core subject areas.

• English language arts and mathematics had the highest percentage of assessed core courses with 90%.

• Social studies had the lowest percentage of core courses formally assessed as adequate with 30%.

• The total scope of assessment for non-core courses was 23%, which does not meet the audit standard of 70%.

• The total district percentage of core and non-core courses assessed was 39%.


Students are regularly assessed with district and state test instruments.

Auditors interviewed district personnel, students, and conducted online surveys of teachers and school leaders to learn more about the scope of assessed curriculum in Phoenix #1. Although the scope of the assessments was rated as inadequate, several comments express an opinion about the excessive amount and purpose(s) of testing.

• “The amount of assessment that we do is a weakness and the total reliance on student assessment relies on the decisions that we make.” (Campus Administrator)

• “Our testing window this year spread for a month and a half. I think it was tougher on the staff than the students.” (Support Staff)

• “The pay for performance system is driving the assessments system.” (Central Office Administrator)

• “This Galileo test that kindergartners are doing today, they have been prepping for two weeks. [The culture] is so test focused. [LOI] The teachers get quite a bit—teachers who are highly effective get $6000 back. It’s significant money. And so…it’s not good.” (Campus Administrator)

• “There’s too much testing. We just finished the AzMERIT and now we have to take the Galileo. The tests are too close together.” (Student)

In addition to the lack of sufficient assessments available to teachers and concerns about excessive testing district-wide, the auditors also found that the formative assessment instruments themselves are inadequately aligned to the standards to ensure the validity of resulting data. The Galileo test is the most widely used formative tool, but it is not a cognitively rigorous tool and is not aligned to the AZCCRS (see Finding 2.3).

Summary

The scope of the assessed curriculum in Phoenix Elementary School District #1 is inadequate to provide data for curriculum and instructional changes to improve student achievement. Only 68% of the core curriculum and 23% of the non-core curriculum is formally assessed. Despite the prevalence of assessment in the district, 32% of the core courses taught lack formal assessment that would yield student achievement data to guide instructional decisions and inform the evaluation of instructional strategies and programs across the district.


Finding 4.3: The district lacks a comprehensive student assessment and program evaluation plan to guide instructional decision making for the improvement of student achievement.

Effective student assessment and program evaluation systems ensure that a school system utilizes appropriate assessment data to make informed decisions that have a positive effect on student learning. A purposeful assessment system provides useful data when making decisions related to budgeting, allocation of resources, and making modifications and adjustments to classroom teaching methodologies. Effective assessment systems provide a school district with the means to determine how well programs and instructional practices are producing desired results.

When a district lacks an effective student assessment and program evaluation plan, decision makers from the classroom level to the campus level to the district level lack the data they need to make informed decisions and instead must rely on instinct or prior practice. When desired student achievement levels are not met, districts often implement new programs without knowing what is working and thus end up with an array of programs that are not coherently planned, implemented, monitored, and evaluated.

A well designed assessment and program evaluation system provides teachers, campus administrators, and district administrators with the means to determine how effective programs and instructional practices are producing desired student achievement results over time. A comprehensive student assessment plan provides for reliable student achievement data to measure student progress and thus ensures district administrators have the means to evaluate and make decisions regarding district programs.

An assessment plan serves to guide all aspects of student assessment and program evaluation, and it transcends the changes in staff and administration that can impact work that is already underway. It provides a clear plan for how students are assessed and how the information will be used. The plan expects that students will be assessed in all content areas, in both a formative manner that provides teachers with the diagnostic information needed to adapt and improve instruction immediately and a summative manner at the end of instruction. Additionally, it provides procedures and information for evaluating academic programs to determine their effectiveness so that they can be continued, modified, or terminated. The ultimate impact of an effective student assessment and program evaluation plan is the ongoing improvement of instructional approaches to increase student achievement over time.

To determine the scope and quality of the district assessment and program evaluation plan, auditors reviewed board policies, job descriptions, curriculum assessment documents, assessment reports, and assessment calendars. Auditors interviewed board members, administrative staff, teachers, parents, and students, and visited classrooms throughout the district to gain further information regarding the district’s assessment and program evaluation system.

Auditors found no formal comprehensive student assessment and program evaluation plan. While elements for student assessments were found across various district provided documents, these elements lacked the purposeful intent in utilizing the assessment results to make informed decisions to keep, modify, or eliminate a program. Board policies require student state assessments, but are inadequate in providing direction for producing desired student outcomes.

The auditors found no board policies providing direction for comprehensive student and program assessment planning. The following board policies were found to have some connections to student assessment or program evaluation.

• Board Policy ADA: School District Goals and Objectives states that the district’s strategic plan should address student academic success.

• Board Policy IKE: Promotion and Retention of Students refers to proficiency of standards in core content areas. In addition to the standards, other pertinent data will be used to determine promotion.

• Board Policy IKE-E: Exhibit for Promotion and Retention of Students states that as an ongoing responsibility of the teacher is to test and maintain academic performance data.


• Board Policy IL: Evaluation of Instructional Programs states the following: “The use of tests is one indication of the success and quality of the educational program. In the case of an individual student, tests, in combination with other criteria, can provide an indication of student achievement.” Further, the board authorizes participation in:

○ A District testing program where all students shall be included in state and District assessments with appropriate accommodations and alternative assessments where necessary for the students with disabilities per each student’s individual education program.

○ A District testing program that will be subject to regular review and evaluation.

○ Evaluation of all proposed testing instruments and periodic evaluation.

○ In-service education of teachers and other staff members in the use of tests and interpretation of test results.

• Policy ILB: Test/Assessment Administration refers to security of state mandated testing and required reporting of results to the parents of students.

Board policies describe some expectations for assessments and evaluation of instructional programs with a regular review and evaluation of the districts testing program. However, the policies lack guidance in how and who shall carry out the process and the effective use of data as feedback to make decisions for instructional changes.

Auditors reviewed job descriptions to determine assigned responsibilities for assessment planning and program evaluation. The following district job descriptions contained references to areas of assessment responsibility:

• Director of Assessment and Accountability:

○ Manage student data relevant to student assessment, including SPED and ELL data.

○ Organize student performance data to assist schools and departments in curricular planning.

○ Organize student performance data to assist schools and departments in professional development planning.

○ Provide principals and staff with assistance in analyzing test results for the purpose of improved delivery of instruction.

• Senior Software Support Technician / Assessment and Accountability:

○ Works closely with the Curriculum and Instruction departments to determine what assessment software programs are required and helps determine what programs should comprise the base computer image.

○ Maintains a working relationship with teachers and ensures that support is provided for assessment programs.

○ Ensures the accuracy of student data, classes and test scores within the software application.

• Director of Educator Effectiveness:

○ Cooperatively implement with REIL and District staff a data management system enabling leaders and teachers to use data to inform decisions.

○ Conduct training on data management system.

○ Analyze data to plan professional development and communicate instructional effectiveness.

• Instructional Coach:

○ Provide specific instructional feedback.

○ Target specific areas of need based on data collected from student assessments, walk-throughs and observation.


○ Assist teachers in identifying students’ instructional needs using lesson plans, formative and summative assessments.

○ In cooperation with school administrator(s), disaggregate (analyze various types of data) and assist teachers in its interpretation to measure and improve student achievement and program effectiveness.

• Assessment Data Technician II:

○ Works with all persons responsible for testing and evaluating students placed in Needs programs, such as ELL, SPED, Refugee, Title I, Immigrant, etc.

• Director of Coaching Effectiveness:

○ On-going, constructive and data-based feedback about their performance to increase their effectiveness based on observation settings of the Coaching Observation Instrument (COI).

○ On-going support with a variety of student assessment tools and analysis of student achievement data.

Auditors found that district job descriptions do not assign duties for planning key components of an assessment program that would be used to make effective instructional decisions. Assessment duties have been assigned across a variety of job descriptions with a focus on assessment data for compliance. Job descriptions lack adequate direction to guide stakeholders in making decisions regarding program effectiveness.

District documents were reviewed by auditors to determine the presence of an effective assessment and program evaluation plan. The district’s Strategic Plan: Our Blueprint for Student Success Action Plan (2012) contains objectives and strategies related to assessment, feedback, and program evaluation. The following objectives were noted as relevant:

• Objective 1.2 Teachers, students and parents are provided frequent opportunities to assess and guide student learning.

○ 1.2.1 Assessment

▪ Collect and analyze data

▫ Formative Assessments;

▫ Chapter Tests;

▫ Measurement of Progress toward Standards;

▫ Portfolios of Student Work;

▫ Summative Assessments; and

▫ End of year State test.

○ 1.2.2 Assessment

▪ Communicate with parents

▫ Informative Student/Parent/Teacher Conferences;

▫ End of Term report cards;

▫ Student’s academic records accessible online; and

▫ Short and long-term student learning plans designed collaboratively with parents.

• Objective 2.2 All new initiatives align with district goals and be evaluated for effectiveness.

○ 2.2.1 Needs Assessment

▪ Identify purpose of the new initiative; and


▪ Provide data supporting the need for the initiative.

○ 2.2.2 Implementation Plan

▪ Identify goals and objectives for the initiative;

▪ Develop timelines for implementation; and

▪ Identify professional development needs.

○ 2.2.3 Program Evaluation

▪ Analyze outcomes;

▪ Recommend modifications; and

▪ Explore opportunities for replication or abandonment.

Although some objectives were found concerning assessment data usage, no specificity was present to guide the district in making informed instructional decisions. Broad terms are present regarding expectations for new initiatives in the district, but again no level of specificity was present in guiding the district to keep, alter, or abandon specific programs.

Overall, board policies, job descriptions, and district documents lacked specific direction for an effective student assessment and program evaluation plan. All documents reviewed were inadequate in assigning responsibility for designing and carrying out a comprehensive system that includes formative and summative assessment results to guide instructional practices and evaluate programs.

Auditors expected to find a comprehensive system of utilizing assessment data to measure the effectiveness of the curriculum design, delivery and program evaluation to meet student needs. Audit criteria in Exhibit 4.3.1 describe the elements of a quality student assessment and program evaluation plan.

Exhibit 4.3.1

Characteristics of a Comprehensive Student Assessment and Program Evaluation PlanPhoenix Elementary School District #1

May 2017

Characteristic (The plan…)Auditors’ Rating

Adequate Inadequate1. Describes the philosophical framework for the design of the student

assessment plan and directs both formative and summative assessment of the curriculum by course and grade in congruence with board policy. Expects ongoing formative and summative program evaluation; directs use of data to analyze group, school, program, and system student trends.

X

2. Includes an explicit set of formative and summative assessment procedures to carry out the expectations outlined in the plan and in board policy. Provides for regular formative and summative assessment at all levels of the system (organization, program, student).

X

3. Requires that formative, diagnostic assessment instruments that align to the district curriculum be administered to students frequently to give teachers information for instructional decision making. This includes information regarding which students need which learner objectives to be at the appropriate level of difficulty (e.g., provides data for differentiated instruction).

X

4. Provides a list of student assessment and program evaluation tools, purposes, subjects, type of student tested, timelines, etc. Partial*


Exhibit 4.3.1 (continued)Characteristics of a Comprehensive Student Assessment and Program Evaluation Plan


Characteristic (The plan…)Auditors’ Rating

Adequate Inadequate5. Identifies and provides direction on the use of diverse assessment

strategies for multiple purposes at all levels—district, program, school, and classroom—that are both formative and summative.

X

6. Specifies the roles and responsibilities of the central office staff and school-based staff for assessing all students using designated assessment measures, and for analyzing test data.

X

7. Specifies the connection(s) among district, state, and national assessments. X8. Specifies the overall assessment and analysis procedures used to determine

curriculum effectiveness. X

9. Requires aligned student assessment examples and tools to be placed in curriculum and assessment documents. X

10. Specifies how equity issues will be identified and addressed using data sources; controls for possible bias. X

11. Identifies the components of the student assessment system that will be included in program evaluation efforts and specifies how these data will be used to determine continuation, modification, or termination of a given program.

Partial*

12. Provides for appropriate trainings for various audiences on assessment and the instructional use of assessment results. X

13. Delineates responsibilities and procedures for monitoring the administration of the comprehensive student assessment and program evaluation plan and/or procedures.

X

14. Establishes a process for communicating and training staff in the interpretation of results, changes in state and local student achievement tests, and new trends in the student assessment field.

X

15. Specifies creation of an assessment data system that allows for the attribution of costs by program, permitting program evaluations to support program-based cost-benefit analyses.

X



Auditors found little evidence of a formalized comprehensive student assessment and evaluation plan in the Phoenix Elementary School District #1. In order to be considered adequate, 70% of the characteristics need to be present. As shown in Exhibit 4.3.1, none of the 15 characteristics of a comprehensive student assessment plan were met. While evidence shows the district’s Strategic Plan: Our Blueprint for Student Success Action Plan refers to new initiatives requiring some of the characteristics, there is no specificity as to how to carry out the process of a student assessment and program evaluation plan.

The following discussion provides more information on the evidence found by the auditors during this analysis:

Characteristic 1: Describes the philosophical framework

No board policy or district document exists describing a philosophical framework or expectation of assessment usage for ongoing formative and summative program evaluation. This characteristic was rated inadequate.


Characteristic 2: Includes an explicit set of assessment procedures

No policy or district document provides direction on the design and use of assessment data by appropriate personnel to support an assessment and program evaluation plan. This characteristic was rated inadequate.

Characteristic 3: Requires assessments aligned to the district’s curriculum

Auditors found assessments used in the district to be inadequately aligned to the Arizona state standards (see Finding 2.3). This characteristic was rated inadequate.

Characteristic 4: Provides a list of student assessment and program evaluation tools

While a list of assessment tools was included in the districts assessment calendar, the purpose and use for the assessments were not purposefully delineated for use in the evaluation of programs or instructional practices. This characteristic was rated partially adequate. (Partial ratings are calculated as inadequate.)

Characteristic 5: Identifies and provides direction for diverse assessment strategies

Auditors found no written expectation for the use of diverse formative or summative assessment strategies at the district, program, school, or classroom level. This characteristic was rated inadequate.

Characteristic 6: Specifies roles and responsibilities

Although some job descriptions indicate responsibility for assessment support and management of data, no formal roles are assigned for the appropriate dissemination and use of the data to improve student learning. This characteristic was rated inadequate.

Characteristic 7: Specifies connections among assessments

Auditors found no specificity in connections among district, state, and national assessments. This characteristic was rated partially adequate. (Partial ratings are calculated as inadequate.)

Characteristic 8: Specifies assessment and analysis procedures

Auditors found no formal written procedures for using assessment data to measure the effectiveness of the curriculum. No documents provided to auditors show a connection between the scores on the AzMERIT and instructional practices in the classroom. This characteristic was rated inadequate.

Characteristic 9: Requires aligned assessment examples

Documents provided to auditors contained no expectation that aligned student assessment examples be included in curriculum guides and assessment documents. This characteristic was rated inadequate.

Characteristic 10: Specifies how equity issues will be addressed

Auditors found no evidence of planning to address inequities in assessment data results. This characteristic was rated inadequate.

Characteristic 11: Identifies components of a student assessment system

Auditors found evidence identifying components of a program evaluation plan in objective 2.2 as part of the district’s Strategic Plan: Our Blueprint for Student Success Action Plan (2012). However, no details were present to provide direction on how the data will be utilized in decision making to modify, adjust or selectively abandon a program. This characteristic was rated inadequate.

Characteristic 12: Provides appropriate training

Auditors were not provided with evidence of a plan to provide appropriate training for staff members at all levels on assessment results and connected use in classroom instruction. This characteristic was rated inadequate.


Characteristic 13: Delineates responsibilities for monitoring the assessment program

Job descriptions provided no evidence to auditors delineating the responsibilities for monitoring the administration of the assessment program. Job descriptions focused more on the administration of assessments rather than the use of data in decision making. This characteristic was rated inadequate.

Characteristic 14: Establishes a communication process

Auditors found no evidence for communicating assessment changes and opportunities for training staff in the interpretation of assessment results, changes in the state achievement test, or new trends in the assessment field. This characteristic was rated inadequate.

Characteristic 15: Specifies creation of assessment data system

Auditors found no evidence specifying the creation of an assessment data system that allows for the attribution of costs by program, permitting program evaluations to support program-based cost-benefit analyses. This characteristic was rated inadequate.

Interviews and Surveys

Auditors interviewed board members, administrative staff, teachers, parents, and students directly or through online surveys to gain further information regarding the district’s assessment and program evaluation system. The following statements reflect concerns regarding the current system for assessing student learning and evaluating programs:

• “The system is so loose and student achievement should not depend on what classroom you go to or school. I just question that, I just look at the numbers and we have places where year after year the kids regress.” (Teacher)

• “They leave me alone. They give me data support [run our data for us]. But the data doesn’t get down to the root causes. For example, what does the central office think of our data? They don’t say anything.” (Campus Administrator)

• “Accountability through assessment is site-based. Some sites use the item banks for Galileo.” (Teacher)

• “There isn’t an understanding of what the Benchmark 1/2 and pre-post assessments test.” (Teacher)

• “We do not do a lot of program evaluation using data. We have a lot of data just not really for the purpose of evaluating programs.” (Central Office Administrator)

• “Galileo as an assessment does not do the job of giving clear information. The questions are not aligned assessment questions. They are not high quality questions.” (Teacher Survey Response)

• “I feel that the Galileo benchmark assessments are not directly linked to the standards.” (Teacher Survey Response)

Summary

Overall, Phoenix Elementary School District #1 is assessing students and data are being made available to schools for instructional decision making. However, auditors found planning for student assessment and program evaluation to be inadequate. Board policies, job descriptions, and other district documents, combined with interviews and campus observations, indicate a lack of sufficient evidence of a comprehensive student assessment and program evaluation plan. While some district documents contain elements of a comprehensive plan, clear direction for the design of a student assessment and program evaluation system was not found, as are procedures that direct the use of data to guide instructional decision making for the improvement of student achievement. Data received are not used to analyze and identify the cause(s) for the results of student achievement, nor directed at altering teacher instructional practices and strategies.


Finding 4.4: The district lacks a systemic process for selecting formative assessment instruments and using the data from those instruments for improving instruction and continuing or terminating programs.

Data usage from a variety of sources is essential for sound curriculum management and responsible decision making in planning for various district functions as well as for classroom instructional planning. Critical assessment resources include formative, benchmark, and summative student test data; survey and follow-up studies; audits and reviews; and teacher/administrator evaluations.

Two main types of assessment instruments yield assessment data: formative and summative. Formative assessments are given periodically over the course of instruction. Their use is intended to yield immediate information regarding students’ progress in mastering the objectives so teachers can use the data to adjust and plan instruction and respond to individual students’ needs and gaps in learning. Summative assessments are administered at the end of a course or grade level and provide comprehensive information regarding students’ learning in a course. Summative data are more useful for comparing student performance across classrooms, schools, districts, or even states, but are of limited use when planning instruction, since most students have progressed to the next grade level or course before the data are available to teachers.

In effective school districts, the greatest attention is paid by designated personnel to formative assessments that serve to give teachers, principals, and instructional support staff real-time information on how students are progressing in mastering the intended learning objectives. The more specific and diagnostic the assessment, the more meaningful the data are for teachers who plan instruction and learning activities based on individual student need. Diagnostic, formative tools inform teachers exactly what skills, concepts, knowledge, or vocabulary a student lacks so instruction can address that gap immediately. This results in more efficient instruction since a teacher does not spend time teaching content a student has mastered and focuses instead on the content a student still must learn.

Diagnostic, formative instruments play a critical role in delivering differentiated instruction. Differentiation occurs in two ways: first, instructional differentiation that involves using a variety of strategies and approaches with students and allowing for multiple modes of student practice and demonstrations of learning; second, curricular differentiation that involves teaching students only those objectives and/or prerequisites that they need or require based on assessment data. This type of differentiation is impossible without diagnostic tools.

A final role for formative assessments in effective school districts is providing district leaders with predictive information concerning student performance on high stakes tests. When formative assessments are high quality and tightly aligned with high stakes assessment in all three dimensions of alignment—content, context, and cognitive type—the results yield valuable information on how students will likely perform on the high stakes, external measurement. The intent is that students are not surprised or caught off-guard by the high stakes tests.

The alignment of formative assessments with adopted district curriculum and high stakes assessments cannot be assumed or taken lightly. This alignment must be carefully analyzed and modification made where necessary. It is particularly important to determine the quality and alignment of any purchased assessments or item banks, since publishers’ claims often exceed reality.

Teachers and districts that do not have or do not use formative assessment data are left to rely on the results of summative assessment to identify student weaknesses and respond reactively to gaps in students’ learning, possibly months after the initial instruction occurred. Reacting to summative assessments usually results in designing reteaching and remediation plans to help ensure students master the curriculum. Such efforts often come too late or lack sufficient specific data to ensure a student does not fall behind in mastering grade-level content, as summative assessments are administered too infrequently to inform daily instruction. The resulting cycle becomes difficult to overcome and ultimately does not adequately accelerate students’ learning to ensure they are prepared for high stakes assessments.

An immediate response to a student’s gap in learning is a critical part of preventing students from falling behind. Immediate, instructional responses require timely and specific information—information that is relevant to individual, discrete skills. This type of information is best gleaned from frequently administered formative and diagnostic tests that can be used whenever a teacher deems it appropriate or necessary so he/she can use the information to redesign lessons and create different fluid student groupings.


Auditors found informal expectations for formative data use to support them in instructional decisions. However, auditors found the availability of formative assessment instruments and the overall approach to formative data use to be inadequate. Auditors found that general assessment data are regularly being used at the individual student, classroom, and the campus level. Current assessment instruments are not aligned to the AZCCRS (see Finding 2.3).

Auditors found limited evidence of tools and practices for frequent, needs-based diagnostic and formative assessment. They found that most data-use practices largely results in a remediation and reteaching cycle where students “catch-up” after instruction has occurred without adequately accelerating their learning or exempting them from lessons on objectives they already know or can do.

In the use of formative assessment results, the district and schools in Phoenix ESD #1 have defaulted to expanding interventions and remedial services to students rather than focusing on improving initial instruction. Confusion exists within the district due to the duplicative support services provided to students. Conflicts have arisen between site-based and district-based support staff and their focuses on remediation rather than acceleration. Creation of new support positions endorses the inclination of referring struggling students to support staff rather than providing needed instruction in the regular classroom.

Formative types of school-based feedback such as classroom visit data and information gleaned from collaborative team analysis of student and staff work can also inform decision making at important junctures. The resulting data from these various sources serve as a basis for improving instruction to facilitate student achievement, as well as to inform such work as appropriate comprehensive strategic planning, staff development and program evaluation planning, and developing data driven budget prioritization.

Effective districts have an assessment process that is ongoing and systematic. Administrators and teachers demonstrate a clear understanding of how students are assessed on required testing instruments, including the standards, types of questions, and level of the concepts, skills, and knowledge students must master to be successful. In those school systems, test results are well understood so that all administrators and teachers know how to analyze important trends in the instructional program, as well as areas of strength and weakness by classroom, groups of students, and individual students. Each teacher and school leader makes frequent use of assessment data to redesign classroom instruction aimed at improving student achievement. Surveys and program evaluations, where they are used, provide additional information regarding needs identification that can significantly impact decisions at the district and school levels.

To determine if Phoenix #1’s set of formative assessments, the resulting data, and the data use are adequate to inform instruction and effective in improving student achievement, auditors reviewed board policies, improvement plans, job descriptions, assessment data reports, and other district documents. Auditors also interviewed board members, district and campus administrators, instructional specialists, teachers, parents and students, and visited classrooms throughout the district to gain further information regarding the district’s use of data for effective programmatic decision making practices.

Overall, the auditors found student performance data are available to district leaders to make informed decisions about program effectiveness. However, the district lacks a systemic process for the use of data for proper implementation, continuation, and termination of programs.

Auditors examined board policies to determine the expectations for data usage for proper program implementation, monitoring, and termination. The following policies provide some information related to evaluation of programs:

• Board Policy CCB-RB: Line and Staff Relations (Shared Decision Making) states that “The process will include monitoring and evaluating implemented plans on a scheduled basis, and will include a written evaluation plan. The evaluation plan must include some demonstrably valid, quantifiable measures of progress.”

• Board Policy IGA: Curriculum Development states “The need for and value of a systematic, ongoing program of curriculum development and evaluation involving students, parents, teachers, and administrators are recognized.”


Board policy provides little guidance and expectations for the districts instructional programs to be evaluated on a regular basis. No board policy has an expectation that instructional programs will be implemented, modified or terminated based on the analysis of expected program results.

Additionally auditors examined job descriptions to determine roles and responsibilities related to program evaluation. The following represent the job description with some responsibility:

• Director of Assessment and Accountability need to possess the ability to “Conduct analyses of student assessment data for each school [to] provide information for development of the Arizona School Improvement Plan.”

• Director of Coaching Effectiveness states “While the role of the coach can vary, it can include: on-going data review to include the relationships between teaching decisions, content and student learning.”

• Assistant Superintendent for Curriculum and Instruction is responsible to “Inform the Superintendent of all major teaching and learning activities, programs and progress of the District.”

• Elementary School Principal includes a responsibility to “Evaluate the implementation of programs within the building and provide in-service as needed.”

District job descriptions include some generic references to analysis of programs. However, job descriptions lack the detail responsibilities regarding what programs are to be evaluated, requirements for data use in program evaluation, reporting requirements, and the purpose for conducting program evaluations.

Auditors searched for examples of how data were being utilized to make effective instructional decisions. Auditors reviewed teacher responses from online surveys that included questions related to the use of formative and summative assessment data.

Exhibit 4.4.1 shows the results from 92 teacher responses to a question about the frequency of teacher use of assessment data.

Exhibit 4.4.1

Teacher Response to the Statement: “How frequently do you use the results of assessments to plan instruction?”


26%

23%

28%

20%

2%1%

0%

5%

10%

15%

20%

25%

30%

Daily Several times aweek

Weekly Monthly Rarely or not atall

N/A



• Forty-nine percent of the teacher’s responded to using assessment results daily or several times per week.

• Forty-eight percent of the teachers responded as using assessment results either weekly or monthly.

• Only 2% responded that they rarely or do not use assessment results.

In the subsequent online survey question, teachers were asked to respond regarding how they use student data. The results from responses from 93 teachers are displayed in Exhibit 4.4.2.

Exhibit 4.4.2

Teacher Response to the Statement: “I use student assessment data for the following…”


80%

96%

63%

84%

54%

11%

0%

10%

20%

30%

40%

50%

60%

70%

80%

90%

100%

To give grades To planreteaching

To referstudents forintervention

To placestudents in

small groupsfor targetedinstruction

To placestudents in thecorrect course

or level

Other (pleasespecify)


• The highest percentage of the teachers (96%) indicated that they use assessment data “To plan reteaching.”

• Eighty-four percent of the teachers use the data “To place students in small groups for targeted instruction.”

• Eighty percent of the teachers use data “To give grades.”

Auditors gathered comments related to systemic use of data in the district for effective decision making. The following comments are noted below:

• “I wish I could tell you better…that we are making good use of our data. I do not know whether to believe the data or not in terms of student achievement and standard testing.” (Central Office Administrator)

• “[Galileo] is not predictive; it’s not predictive of AzMERIT.” (Campus Administrator)

• “Pay for performance system is driving the assessments system.” (Central Office Administrator)

• “The problem with using Galileo as we use it now is that it has become a high stakes testing because it’s tied to the TIF grant evaluation piece.” (Campus Administrator)


• “I do not find data from Galileo to be reliable.” (Teacher Survey Response)

Overall, the auditors did not find any systemic process for the utilization of data for programmatic evaluation purposes. No system of feedback is present that supports the increase in student achievement results over time.

To further understand the use of district data, auditors examined a document titled Strategic Plan: Our Blueprint for Student Success Action Plan (2012). The document contains a number of objectives and strategies, one of which contains a reference to program evaluation. Bullet points under program evaluation state “to analyze outcomes, recommend modifications and to explore opportunities for replication or abandonment.” With the lack of a formal program evaluation plan in place, auditors concluded the need to provide the district with the characteristics of a quality program evaluation plan or process to enable the district leaders to develop a plan. Exhibit 4.4.3 provides a listing of the audit characteristics. Audit standards require that at least nine (75%) of the characteristics are met to determine adequacy.

Exhibit 4.4.3

Characteristics of a Quality Program Evaluation Plan or Process And Auditors’ Assessment of the District’s Approach

Characteristics of a Quality Program Evaluation Plan or Process1. Describes board or administrative directives to have program evaluation procedures in place2. Specifies procedures for program evaluation, including needs assessment and formative evaluation

and summative evaluation methods3. Specifies the proficiencies of persons responsible for conducting the evaluation, enhancing

likelihood that findings achieve maximum credibility and acceptance4. Expects multiple measures designed to obtain quality data about the goals and objectives of the

program and to be accurate and reliable measures5. Provides for multiple measures of data collection to be used, including both quantitative and

qualitative data 6. Directs ongoing formative assessments for the first two years for any new program implementation

and summative evaluation at the end of the third year7. Directs that all existing programs undergo a program evaluation at least every three years8. Expects procedures used in the evaluation process to be clearly described9. Specifies that program evaluation reports clearly describe the program, including its context,

purposes, and procedures 10. Expects program evaluation reports to be utilized to support timely decisions regarding program

effectiveness, identify both strengths and weaknesses of the program, and include findings and recommendations for continuation as is, modification, or termination

11. Directs program evaluation designs to be practical, ethical, and cost effective, and to adequately address relevant political issues

12. Expects all proposals for the initiation of new program to include needs assessment data, a description of formative and summative evaluations, and data collection procedures

©2017 CMSi

Summary

Auditors found that Phoenix Elementary School District #1 lacks a systemic process for the use of data for proper implementation, continuation, and termination of programs. No program assessment or program evaluation plan was provided to the auditors. While data is available to the district to help make informed decisions, no formal process exists to aid in improvement of programs and instruction based on student achievement data. Auditors did not find any board policies that specifically address the use of data to improve the design and delivery of curriculum or any job descriptions assigning roles and responsibilities for how data are to be used for the formal implementation, continuation, and termination of programs.


STANDARD 5: The School District Has Improved Productivity.Productivity refers to the relationship between system input and output. A school system meeting this standard of the CMSi Curriculum Audit™ is able to demonstrate consistently improved pupil outcomes, even in the face of diminishing resources. Improved productivity results when a school system is able to create a consistent level of congruence between major variables in achieving enhanced results and in controlling costs.


While the attainment of improved productivity in a school system is a complex process, caused in part by the lack of a tight organizational structure (referred to as “loosely coupled”), common indicators of a school system meeting this audit standard are:

• Planned and actual congruence among curricular objectives, results, and financial allocations;

• A financial data base and network that can track costs to results, provide sufficient fiduciary control, and be used as a viable data base in making policy and operational decisions;

• Specific means that have been selected or modified and implemented to attain better results in the schools over a specified time period;

• A planned series of interventions that have raised pupil performance levels over time and maintained those levels within the same cost parameters as in the past;

• School facilities that are well-kept, sufficient, safe, orderly, and conducive to effective delivery of the instructional program; and

• Support systems that function in systemic ways.


This section is an overview of the findings that follow in the area of Standard Five. Details follow within separate findings.

Standard Five addresses the ability of a district to improve its operations and outputs over time. Current budget development and decision-making processes of the Phoenix Elementary School District #1 were not fully prepared to assure system-wide cohesion and productivity. Without the benefits of formal evaluation to verify program efficacy, there is no systematic linkage between funding and district priorities.

Budget planning followed a traditional model of formula allocation with no linkage to student achievement data or program initiatives. The audit team found that budgeting procedures favor the status quo over flexibility to meet emerging student needs. Also, there are no clear linkages between budgetary decisions, on the one hand, and the district’s mission, curricular goals, and strategic priorities, on the other.

Budget development is not supported by analyses of educational program effectiveness. The district lacks adequate guidance and a rational system for managing interventions to improve student achievement.

The auditors found that the design for management of productivity-enhancing interventions to improve student achievement does not meet audit standards of adequacy. The district lacks policy and regulatory guidance and a rational system for managing interventions. Many programs were in use across school campuses, most geared toward intervention. While there was evidence of some evaluation, approaches were superficial and used inconsistently. The lack of program evaluation meant that decision making about the productivity of investment was limited. Auditors found that district schools defaulted to obtaining intervention programs to address inadequate student performance or using supplemental support staff to work with students, rather than focusing on improving initial instruction efficiency.


Facilities were clean and provided an adequate learning environment for curriculum delivery. Although the district has implemented facilities renovations with funding from a 2015 bond levy, facilities continue to require ongoing maintenance and modernization. The district’s master facilities plan is not adequate to guide present and future facility operations.

The board has expended significant funds to procure instructional technology. However, the district lacks an adequate technology plan and has not provided adequate professional development for teachers to integrate the use of technology into their daily instruction. Presently, technology use is primarily a substitute for prior teaching methodologies.

The auditors found that the district lacks policy, regulatory guidance, and a defined plan for a rational system of managing and promoting the implementation of instructional technology to improve student achievement. Overall, the management of budgeting, facilities, technology, and productivity-improvement interventions is inadequate.

Finding 5.1: The budget development and decision making processes are not tightly aligned to the district’s curricular goals and priorities, nor are there adequate cost-benefit analyses within the financial network to assure maximum productivity.

A school system’s productivity is enhanced by budgetary decisions based on program needs, goals, and priorities. Productivity is improved when clear linkages exist between the curriculum and the budget. These increases in productivity are achieved through cost-benefit analyses and require a clear delineation of costs compared to documented system gains, or results obtained from allocations. Such linkages provide for a budgetary process that is driven by curriculum needs, priorities, and goals. Linkages between the budget and curriculum are critical and document how the district allocates fiscal resources to support and implement its programs.

The budget is the numerical expression of the curriculum and should mirror program priorities. A budget development process focused on supporting the school system’s highest priorities is always critical, and is especially important when that system is faced with fiscal constraints requiring programs and services to be reduced. When the budget does not reflect curricular goals and priorities, it is less likely that students will receive the educational benefits intended by the organization’s leaders.

To determine the extent of the connection between curriculum and budget in the Phoenix Elementary School District #1, auditors interviewed board members and district employees, including the superintendent, assistant superintendents, Director of Finance, and principals. They also reviewed district documents including fiscal audit reports, board policies, and procedures used by the district to prepare, implement, monitor, and evaluate the budget. Documents reviewed are displaying in Exhibit 5.1.1.


Exhibit 5.1.1

Fiscal Planning Documents Reviewed by AuditorsPhoenix Elementary School District #1

May 2017

Document DateVarious Board Policies VariedJob Descriptions VariedFinancial Reports, Arizona Office of the Auditor General 2013-2016Budgeting process document 2017Phoenix Elementary Mobility Study 2017Arizona School Tax Credit Video 2017Annual Financial Report for the Arizona Department of Education 2013-2016PESD #1 Single Audit Reporting Package 2016Comprehensive Annual Financial Report (CAFR) 2016Title I District Budget 2014-2016Building Title I Budgets 2016Support Staff Building Assignments by Number and Role 2017Curriculum & Instruction PowerPoint Presentation 2017

Auditors determined that financial decision making and budget development processes lack cost-benefit analyses and are not adequately linked to curricular goals and priorities. While a budgetary planning process is in place, the auditors found an absence of direct linkages between department goals and budget priorities. No formal, routine effort has been made to link student achievement or program performance feedback to budgetary decisions. Additionally, participation in the budget planning processes at the district lacks full inclusion of stakeholders, and at the campus level lacks consistency across the district.

The following board policy excerpts guide budget development:

• Board Policy DA: Fiscal Management Goals/Priority Objectives states that the district shall “Encourage short- and long-range planning through the best possible budgeting procedures” and “Guide the expenditure of funds to achieve the greatest educational returns.” There is no mention of how or when the educational returns should be determined.

• Administrative Regulation DBC-R: Budget Planning, Preparation, and Schedules presents a timeline for the budgeting process along with needed activities, none of which include a presentation of program data, connections between allocations and program outcomes, or comparisons of program expectations and allocated monies.

• Board Policy IL: Evaluation of Instructional Programs states that “The use of tests is one indication of the success and quality of the educational program.” However, linkages between a program’s success and the budgeting process are not mentioned.

• Board Policy EA: Support Services Goals/Priority Objectives describes the budgeting process with an expectation to “formulate the annual budget, considering at all times that resources must be utilized to produce the most positive effect on the student’s opportunity to gain an education.” The superintendent shall be also be responsible for “reviewing budgetary requests, providing guidelines and limitations, and presenting the proposed budgets and documentation necessary for Board study, review, and action.”

• Board Policy CBA: Qualifications and Duties of the Superintendent describes the duties of the superintendent, including, “Assumes responsibility for the overall financial planning of the District and for the preparation of the annual budget,” and “Establishes and maintains efficient procedures and effective controls for all expenditures of school funds in accordance with the adopted budget…”


Job descriptions related to budget planning are summarized below:

• In addition to the duties cited above in Board Policy CBA, fiscal duties for the superintendent are addressed through the job description as follows:

○ To observe financial records and study the audits and prepare recommendations.

○ To participate in the budget building and encourage a broad range of participation in the development process.

○ Estimate budget needs in relationship to the board and long range needs of the District.

○ To evaluate the Assistant Superintendent.

• Assistant Superintendent for Business Services—oversees the business services department, and administers district finances as follows: “Coordinates with the Superintendent the development and implementation of a five-year financial and facilities plan which supports the District’s Strategic Plan, Vision, Mission and Goals.”

• The Director of Budget and Finance assists in administering the business affairs of the district as follows:

○ Supervises the management of the financial affairs of the schools.

○ Shares responsibility for budget development and long-range financial planning.

○ Establishes and supervises a program of accounting adequate to record in detail all money and credit transactions.

○ Supervises the financial accounting of the District’s student activities program serving as student activities treasurer.

○ Assists directors of federal and special programs with budget development and financial reporting.

○ Supervises the District’s payroll, accounts payable and accounting functions.

○ Administers a budget control system for the District.

○ Acts as advisor to the Assistant Superintendent for Business Services on all questions relating to the business and financial affairs of the District.

○ Assists in recruiting, hiring, training, supervising, and evaluating all clerical, financial, and support staff personnel in Business Services.

○ Arranges for the internal and external auditing of school accounts.

○ Shares in the interpretation of the financial concerns of the District to the community.

• The positions of Assistant Superintendent of Curriculum and Instruction, Elementary Principal, and Director of Special Education each require administration of budgets within their department/buildings, but there is no requirement for coordination across the district or between programs/buildings.

Taken together, these policies and job descriptions partially constitute a statement of desirable budget planning processes and procedures, but they do not provide adequate policy guidance for curriculum-based budget planning (see Finding 1.2).


Exhibit 5.1.2 displays the district overall finances for the past four years, including revenues and expenditures for Maintenance and Operations.

Exhibit 5.1.2

Statement of Revenues and Expenditures Maintenance and Operations Fund


M&O Revenues (Line Number) 2013 2014 2015 2016

All Local Sources (19) $33,753,060 $31,110,993 $28,161,450 $29,410,251County Sources (24) 1,585,660 1,424,382 1,757,026 1,787,925State Sources (28) 14,426,126 13,176,814 16,795,635 17,845,323

Total M&O Revenues (37) 49,715,142 45,883,567 46,714,111 49,043,499Total Funds Available (41) 49,711,231 52,379,056 51,986,014 52,468,553

M&O ExpendituresTotal Expenditures (44) $46,247,405 $47,107,152 $48,560,959 $47,157,204Fund Balance (45) $3,463,826 $5,271,904 $3,425,055 $5,311,349Source: Annual Financial Report for the Arizona Department of Education, FY 2013 through 2016

Auditors noted the following from Exhibit 5.1.2:

• Total M&O Revenue has decreased by 1.3% from FY 2013 to 2016, while Total Funds Available has increased by 5.5%.

• Revenue from state sources has increased by 23.7%.

• Total M&O Expenditures increased steadily from 2013-2015, then decreased in 2016.

• Total Fund Balance has fluctuated over four years, increasing by 53.3% ($1.8 million) from 2013 until 2016.

NOTE: Ending fund balances, as reported, are cash and not budget allocation balances. Arizona’s school district budgets are not revenue based, but rather budgetary based. Cash fund balances are held at the Maricopa County Treasurer’s Office to be used when adjusting for equalization from state aid, mitigate reductions in taxes collected, based on the timing of when taxes are paid, and legislative changes related to taxes.


Auditors examined spending patterns over the past four years. To do so, auditors reviewed financial reports provided by the Arizona Auditor General. Trends in spending were compared from year to year, as displayed in Exhibit 5.1.3.

Exhibit 5.1.3

Budget Spending Percentages by Operational Category Percentage of Maintenance and Operations

Phoenix Elementary School District #12013-2016

CategoryFiscal Year

2013 2014 2015 2016Classroom Instruction 48.0% 48.3% 49.2% 49.5%Administration 9.0 8.9 8.6 9.1Plant Operations 12.6 12.2 12.1 12.3Food Service 6.5 6.0 5.8 5.8Transportation 3.4 3.5 3.1 3.0Student Support 9.9 10.6 10.8 10.7Instructional Support 10.6 10.5 10.4 9.6Source: Arizona Office of the Auditor General, FY 2013-2016 (all figures in percentages)

As displayed in Exhibit 5.1.3, the following can be discerned:

• Classroom Instruction spending has increased by 1.5% from FY 2013 to 2016.

• Student Support spending has increased by 0.8%, while Instructional Support has decreased by 1%.

• Spending for Plant Operations, Food Service, and Transportation, as a percent of the overall budget, has decreased by 1.4% in the time period.

• Administration spending, while fluctuating slightly, has remained essentially unchanged.

Additionally, The Arizona Auditor General provides a summary of the district financial health, referred to as the Financial Stress Assessment. Using six criteria, including student enrollment changes, spending for operations, election results, operating reserves, capital reserves, and internal control, districts are assigned an overall rating of either Low, Moderate, or High Stress. The Phoenix Elementary School District #1 has received a “Low Stress” rating in each of the last four reports (2013-2016). Of the six criteria used for the overall rating, five received a Low Stress rating, with “Change in Number of District Students” receiving a Moderate Stress rating on the 2016 report.


The auditors used six criteria for a curriculum-driven budget to assess the quality of the PESD #1 budget processes. These criteria and the auditors’ assessments are shown below in Exhibit 5.1.4.

Exhibit 5.1.4

Components of a Performance-based Budget And Adequacy of Use in the Budget Development Process


Performance-based Budget CriteriaAuditors’ Rating

Adequate Inadequate1. Tangible, demonstrable connections are evident between assessment of

operational curriculum effectiveness and allocations of resources. X

2. Rank ordering of program components is provided to permit flexibility in budget expansion, reduction, or stabilization based on changing needs or priorities.

X

3. Each budget request or submittal shall be described so as to permit evaluation of consequences of funding or non-funding in terms of performance or results.

X

4. Cost benefits of components in curriculum programming are delineated in budget decision making. X

5. Budget requests compete for funding based upon evaluation of criticality of need and relationship to achievement of curriculum effectiveness. X

6. Priorities in the budget are set by participation of key educational staff in the allocation and decision-making process. Teacher and principal suggestions and ideas for budget priorities are reflected and incorporated in budgeting decisions.

X


©2013 CMSi

Exhibit 5.1.4 reveals that all components of a curriculum-driven budget are inadequate. As a result, the PESD #1 budget planning process, with 0% adequacy, does not meet audit standards that require a minimum of 70% to be considered adequate. A further discussion of the six criteria follows:

Criterion 1: Connections (Inadequate)

Budget allocations are not connected to assessments of program effectiveness required by Criterion 1 primarily because a comprehensive system of program evaluation does not exist in the district (see Finding 4.3). Budget planning documents provided to auditors do not include any descriptions of connections to program effectiveness and budgeting priorities.

Criterion 2: Rank ordering (Inadequate)

Board Policy DBC-R was presented to auditors for review. This policy is basically a timeline of events in the budget process, relating to requests, submissions, and committee deadlines, and not a process that describes preferences and guidelines for determining budget priorities. Although rank ordering of program components is not prohibited by policy, neither is it an expected practice nor are processes provided to achieve this goal.

Criterion 3: Description or evaluation of funding consequences (Inadequate)

The absence of a comprehensive program evaluation system in the district prevents a full description of the consequences of funding or not funding individual budget requests and packages.


Criterion 4: Cost-benefit analysis (Inadequate)

The department lacks the processes to conduct meaningful cost-benefit analyses. While data systems are in place, there is no evidence of cost-benefit analysis occurring. In discussion with district staff, auditors learned that program planning and district fiscal planning essentially occur separately, as noted below:

• “Budgeting is based on the prior year budget, and then a blend of teacher requests and legislative mandates.” (Central Office Administrator)

• “Our budget is based on the year prior, then we look at the enrollment at each school.” (Central Office Administrator)

• “If there is any cost-benefit analysis discussion, it does not include fiscal.” (Central Office Administrator)

Criterion 5: Competition on the basis of need and effectiveness (Inadequate)

Cruciality of need does not occur as part of the budget planning process at PESD #1. Neither is it a required step in policy or an expectation in process. Auditors heard the following comments related to the funding processes:

• “We have no process in place to connect the funding to high need programs.” (Central Office Administrator)

• “All boats rise when we work on the budget.” (Central Office Administrator)

Criterion 6: Staff participation in budget decisions (Inadequate)

Policies and documents were not presented to auditors describing the expectation of participation by stakeholders in the budget planning process. In discussions with district staff, auditors learned that, while stakeholder groups are sometimes included, there is not an expectation that this occurs. “There are no parents or community members on the budget advisory committee. It is made up of central office, a couple of principals, and some teachers.” (Central Office Administrator)

Auditors conducted an online survey of campus principals to determine who creates the budget for both the district and campuses. Exhibits 5.1.5 and 5.1.6 display the results of the survey questions related to budget development.

Exhibit 5.1.5

Results of Campus Leader Survey Question on District Budget DevelopmentPhoenix Elementary School District #1

May 2017

Mostly developed or determined by

personnel at the central office

75%

Mostly developed or determined by a team comprised of district and

building personnel12%

Mostly developed or determined by a team representing various district stakeholders (e.g.,

board members, district and building personnel, parents/community members)

13%

"What is the best description of how your DISTRICT budget is developed or determined each year?"


Exhibit 5.1.6

Results of Campus Leader Survey Question on Building Budget DevelopmentPhoenix Elementary School District #1

May 2017

Mostly developed or determined by

personnel at the central office

40%

Mostly developed or determined by the

principal20%

Mostly developed or determined by a

building leadership team40%

Mostly developed or

determined by a building

personnel as a whole

0%

Mostly developed or determined by team representing various stakeholder groups (e.g., administrators,

teachers, staff members,

parents/community members)

0%

"What is the best description of how your BUILDING budget is developed or determined each year?"

As displayed in Exhibits 5.1.5 and 5.1.6, auditors noted the following:

• Seventy-five percent of campus leaders report that the district budget is developed exclusively by personnel at the district level.

• Forty percent of campus leaders report that their building budgets are developed by the central office.

• Another 40% of campus leaders report site budget development by a leadership team at the campus level.

The lack of clarity in budget planning participation is a reflection of the indistinct direction provided by board policies and expectations as expressed through district planning documents (see Findings 1.2 and 1.3). While participation across district and campus stakeholder groups is present in some cases, consistent participation and the authority of the decision-making groups is unclear or not present.

Summary

The auditors concluded that district policies to direct the budget development processes are inadequate by audit standards. Financial decision making and budget development lack any type of cost-benefit analysis and are not adequately linked to program effectiveness to provide maximum educational productivity.

Declining Enrollment and the Budget

During interviews and a review of district documents, auditors learned that student enrollment in Phoenix Elementary School District #1 has declined over the past years. Since budgets are developed in combination with revenues received versus necessary expenditures and both variable characteristics are determined in part by enrollment, it becomes necessary for districts to have constant conversation about the state of finances.

As stated earlier, the Arizona Auditor General has determined the financial stress level for the district to be “Low.” Notwithstanding, student enrollment is a variable that requires close scrutiny by the district.


Phoenix Elementary School District #1 conducted a mobility study to examine its resident student enrollment versus out-of district enrollment over the past five years. Exhibit 5.1.7 provides a summary of that study.

Exhibit 5.1.7

Enrollment Trends: Resident vs. Non-Resident StudentsPhoenix Elementary School District #1

2013-2017

2013 2014 2015 2016 2017 Number Change

Percent Change

PESD #1 Resident Students 5,940 5,881 5,854 5,453 4,692 -1,248 -21.0%

Non-Resident Students 3,028 3,085 3,016 2,889 2,452 -567 -19.0%

Total 8,968 8,966 8,870 8,342 7,144 -1,824 -20.3%Source: Phoenix Elementary Mobility Study, 2017


• PESD #1 resident students have declined by 21% over the past five years.

• Non-resident students, while making up 34% of the 2017 student population, have decreased by 19% over the past five years

• Total student enrollment has declined by 20.3% over the past five years.

In conversations with district personnel, several factors were presented that may have contributed to the decrease in student enrollment, including more PESD #1 students enrolling outside of the district, fewer non-resident students enrolling in the district, and lack of affordable housing in the district causing families with school-aged children to move from the district. Comments to this effect are below:

• “Families are being pushed out, replaced by high rise buildings and expensive rents.” (Central Office Administrator)

• “Neighborhoods are changing and families are being pushed out of the community.” (Central Office Administrator)

• “People in my community do not attend our schools.” (Board Member)

One ramification of the declining student enrollment is a reduction in revenues. During interviews with district personnel, Federal Title funding was discussed as an area of funding decline, based on the census conducted in 2010. “Federal Title funds are based on a declining enrollment from our census. Funds are going down about 5% per year.” (Central Office Administrator)


Exhibit 5.1.8 displays the trend in Title funding for the district in both tabular and graphic form:

Exhibit 5.1.8

Federal Title Grant FundsPhoenix Elementary School District #1

2012-2017

2012 2013 2014 2015 2016 2017Title I $8,607,236 $8,024,260 $7,724,379 $7,807,211 $7,416,850 $7,046,007Title II 650,888 531,521 527,294 535,308 532,819 519,620Title III 672,357 531,521 457,066 270,618 246,880 240,658

Total Funds $9,930,481 $9,087,301 $8,708,739 $8,613,137 $8,196,549 $7,806,285Source: Curriculum & Instruction PowerPoint presentation, 2017

$0

$2,000,000

$4,000,000

$6,000,000

$8,000,000

$10,000,000

$12,000,000

2012 2013 2014 2015 2016 2017

Total Funds

As displayed in Exhibit 5.1.8:

• Title funds in all categories have steadily decreased over the past six years.

• Total funds have decreased over $2 million (21.4%) since 2012.

The trend displays a continuation of enrollment decline, which will equate to a continued reduction in revenues. Next, auditors would normally examine enrollment projections for the district to determine if the enrollment decline was a trend, or just an anomaly. However, such enrollment projections were not made available to auditors for examination (see Finding 5.3).

It is imperative that the district, as part of its ongoing financial planning discussion, examine the impact of the declining enrollment on both district finances and services to children. Without a fiscal plan that includes cost-benefit analyses of program effectiveness versus costs to the district, dollars that could be used to provide vital services to students could be wasted.

Arizona Tax Credit

During interviews and a review of district documents, auditors learned that an active program of donations to the Phoenix Elementary School District #1 takes place under the rule of the Arizona School Tax Credit. The Arizona School Tax Credit permits individuals residing in Arizona to donate funds to an individual school and receive a tax credit up to $200 per year per individual/$400 per couple. According to the Arizona Department of Education, funds are to be directed to extra-curricular based programs and/or character education. However, schools have discretion as to the exact programs eligible and commonly include curriculum-based field trips,


athletics, drama, music, art, and most clubs. Since donations are directed to individual schools and not the district, monies remain with and are expended at the sole discretion of the individual school.

Auditors examined the donations to PESD #1 schools over the past four years. Exhibit 5.1.9 displays the donations and the per pupil revenue based on current enrollment.

Exhibit 5.1.9

Arizona Tax Credit Revenue by School and Per PupilPhoenix Elementary School District #1

2013-2016

School Total Donations 2016-17 Enrollment Per Pupil RevenueBethune $19,100 502 $38.05Capitol 52,350 659 79.44Dunbar 26,308 373 70.53Edison 28,310 687 41.21Emerson 34,025 548 62.09Garfield 53,712 683 78.64Heard 21,489 623 34.49Herrera 32,300 626 51.60Kenilworth 87,828 559 157.12Lowell 48,755 563 86.60Magnet 23,170 487 47.58Shaw/Shaw Montessori* 111,677 448 249.28Whittier 14,581 450 32.40

Total $553,615 $7,208 $76.81*Shaw ES for 2013-14; Shaw Montessori for 2015-16Source: District Reported Data

As can be seen in Exhibit 5.1.9 above,

• Over $553,000 have been donated to schools in the PESD #1 from 2013-2016.

• Shaw/Shaw Montessori received the largest per pupil amount of $249.28/pupil.

• Whittier received the smallest per pupil donations of $32.40/pupil.

• The average per pupil amount is $76.81, however, this number can be misleading since all funds remain with the school building and funds are not averaged across the district.


Exhibit 5.1.10 displays the Arizona School Tax Credit expenditures and programs by categories at each school from 2013 through 2016.

Exhibit 5.1.10

Arizona Tax Credit Expenditures by Program and BuildingPhoenix Elementary School District #1

2013-2016

BuildingA

thle

tics

Fiel

d Tr

ips

Clu

bs,

Aca

dem

ic

Com

petit

ions

Mus

ic,

Perf

orm

ing

Art

s

Aft

er S

choo

l Tu

tori

ng

Cha

ract

er

Edu

catio

n

Non

-cre

dit

Sum

mer

Pr

ogra

ms

Shor

t Ter

m

Cap

ital

Tota

l

Bethune $753.81 $3,557.09 $,693.25 $802.44 $7,806.59Capitol 2,911.47 25,396.53 2,890.78 17,182.77 48,381.55Dunbar 1,323.62 10,115.30 172.00 709.53 4,625.99 526.34 792.42 18,265.20Edison 4,258.81 12,365.75 3,552.82 531.19 7,101.07 3,615.24 31,424,88Emerson 15,620.04 63.14 2,490.75 1,363.13 377.55 2,235.37 22,109.98Faith (1 yr.) 193.68 26,608,06 14,182.00 2031.44 43,015.18Garfield 7,649.96 81,231.85 1,541.24 1,015.55 298.91 991.15 92,728.66Heard 3,537.75 8,124.18 2,459.37 3,153.38 11,049.57 28,324.25Herrera 2,341.69 55,553.93 350.00 10,965.41 977.28 200.00 77,388.31Kenilworth 8,800.35 57,471.43 2,954.18 1,146.66 18,648.36 3,152.29 594.50 92,768.17Lowell 14,703.39 9,371.40 2,353.40 2,036.85 14,936.38 377.80 43,778.85Magnet 2,250.77 14,699.30 1,596.74 798.91 329.63 19,675.35Shaw ES (2 yrs.) 1,010.98 29,485.73 360.71 6,503.71 424.01 7,448.00 45,233.14Shaw (2 yrs.) Montessori 1,333.58 47,304.49 837.50 751.45 14,979.93 11,316.00 3,591.60 80,114.55

Whittier 1,978.09 4,411.55 149.65 1,114.86 7,654.15Auditor’s Note: Expenditures for some schools in Exhibit 5.1.10 may exceed revenues for that same school in Exhibit 5.1.9. This is due to a prior balance from previous year’s donations for that school.Source: District Reported Data


• The amount of total expenditures range from a low of $7,654.15 (Whittier) and $7,806.59 (Bethune) to a high of $92,768.17 (Kenilworth) and $92,728.66 (Garfield).

• The single largest expense category for one school is field trips at Garfield Elementary, with a total of $81,231.85.

• All schools use some of the funds to support their field trips.

• All schools, except Emerson, use funds to support athletics.

• Eight schools use funds to support character education.

Auditors recognize that donations and expenditures fall within the legal definition of Arizona state law. Auditors also recognize the great disparity of some schools to generate donations compared to other schools. While unlimited donations fall within Arizona state law, the governing board must realize that the disparity of donations may lead to unequal and inequitable educational opportunities for children of less affluent schools. According to district personnel, donations can only be used for extra-curricular or non-graded programs. “Tax credit is supposed to be used for extra-curricular, non-graded activities, and things that are not part of the regular curriculum.” (Central Office Administrator).

Even if the programs were after school hours (extra-curricular) and children were not graded on their involvement, their very participation in the activities creates opportunities for educational enhancement that


would not be afforded children from schools with fewer donations. This creates an inequity that, according to district personnel, has not been addressed by the governing board, as evidenced by comments below:

• “As far as I know there is nothing in place to equalize money from the tax credit.” (Central Office Administrator)

• “The Arizona Tax Credit creates an imbalance between buildings. Some parents can contribute more, others, not at all.” (Central Office Administrator)

This financial inequity must be addressed by the governing board and action taken to alleviate this condition (see Finding 3.1).

Summary

The Phoenix Elementary School District #1 remains a financially stable district, with cash reserves and a Low Stress rating from the Arizona Auditor General. Financial planning, however, lacks any cost/benefit analysis and does not meet any of the criteria for curriculum driven budgeting. Declining enrollment is beginning to place pressure on the district as finances dwindle, especially from federal grant funds. Tax credit donations have created an imbalance between schools’ ability to generate funds that support programming for children. The tax credit imbalance has created inequities among the schools and must be addressed by the governing board (see Recommendation 7).

Finding 5.2: Academic intervention programs are offered on school campuses to improve student achievement; however, programs lack district wide coordination and are not systemically selected, monitored, or evaluated for effectiveness.

An intervention is an overall action taken by school district personnel based on identified needs in order to reach desired results. Program interventions are often used in school systems to improve a selected aspect of the system’s operations, instruction, equity, economy, or general organizational activities. Interventions or innovations can be valuable tools in responding to changing circumstances, identified needs, deficiencies, or less than optimal performance. Auditors expect to find a district that is able to obtain desired results within a defined set of resource parameters connected to a well-designed intervention plan.

There are several essential characteristics necessary for effective interventions to contribute to improved student achievement. Intervention programs are initiated to address identified weaknesses, to serve students with special needs, and/or enrich student experiences. There should be clear linkages between the core curriculum and supporting programs to create a coherent and focused approach to program development and implementation. Standardized procedures should guide the design, delivery, monitoring, and evaluation of all interventions. When these qualities are absent from program management, the likelihood increases that interventions will not achieve their intended goals and that decision-makers will not have the necessary data to make sound decisions about whether programs should be continued, modified, or terminated.

Effective intervention includes the following steps:

• Assess the current situation.

• Diagnose and analyze data collected.

• Identify the problem.

• Propose and examine alternatives.

• Select one of the better alternatives to address the problem.

• Develop a formal plan for the design, deployment, and implementation of the alternative that includes goals and measurable objectives to address the problem.

• Identify the staff proficiencies needed to implement the interventions, appropriate staff development around the proficiencies, and a clear communication plan.

• Provide the fiscal and human resources needed to sustain the intervention.


• Establish a formative and summative feedback evaluation and a plan for monitoring the ongoing deployment and ongoing implementation of the intervention.

• Implement the plans with well-defined mechanisms for monitoring progress.

• Evaluate the program with sound and appropriate techniques.

• Modify or adjust the program as needed, based on data gathered during the evaluation process.

• Implement, based on adjustment needed.

• Reassess and continue monitoring performance results.

The auditors reviewed available district and site documents including board policies, the Phoenix Elementary School District #1 District and Building Improvement Plans, and other district documents. Interviews were conducted with governing board members and administrators, and classroom teachers were surveyed. In addition, the auditors conducted site visits at each campus then used the information from all sources to compare the district’s intervention planning and implementation against the quality criteria of the curriculum audit. Data collected were used to assess the adequacy of program interventions developed to improve student productivity.

Auditors determined that while intervention programs are being utilized by district staff, the district has not developed and utilized a clear process to determine program or intervention effectiveness in Phoenix Elementary School District #1.

The governing board established expectations for program intervention through the following policies:

• Board Policy CA: Administrative Goals/Priority Objectives states that the administration is responsible for “planning, organizing, implementing, and evaluation education programs.”

• Board Policy CBA: Qualifications and Duties of the Superintendent describes one of the duties of the superintendent to “Administer the development, coordination, maintenance, and evaluation of the educational program, including the special education program.”

• Board Policy IHB: Special Instructional Programs deals with special education programming, and is not relevant to the discussion of interventions for academic purposes, other than those children identified with an Individualized Education Program (IEP).

Although the policies listed above provide some expectations for the development, implementation, and evaluation of overall academic programming in PESD #1, they do not provide direction for creating an intervention program to provide instruction to children who require academic intervention to master learning goals. Furthermore, when direction is provided in board policy, as in Board Policy IHB, it is provided in terms of specialized student populations and circumstances and not for the overall program of instruction. Current board policies do not provide clear expectations regarding the design or implementation of intervention programs (see Finding 1.2).

Several job descriptions were presented to auditors that detail specific role responsibility for the implementation of intervention services within the district. Relevant job descriptions are as follows:

• Assistant Superintendent for Curriculum and Instruction “Recommends curricular and instructional strategies to assist schools in improving student achievement.”

• Director of Special Education should “Be familiar with appropriate programs of remediation for children in core academic areas.”

• Elementary Principal’s duties include the following: “Coordinate and supervise all activities and programs conducted at the building level” and “Evaluate the implementation of programs within the building…”

• Instruction coaches have several responsibilities related to intervention and academic programming, including:


○ Assist teachers in identifying students’ instructional needs using lesson plans, formative and summative assessments.

○ In cooperation with school administrator(s), disaggregate (analyze various types of data) and assist teachers in its interpretation to measure and improve student achievement and program effectiveness.

○ Facilitates the use of research-based teaching strategies and best practices to address the needs of teachers and school goals.

• Classroom teachers, as one of their essential duties, must “Develop lesson plans and instructional material and provide individualized and small group instruction to adapt the curriculum to the needs of each pupil.”

• Literacy coach is expected to “Recommend modification in curriculum and instructional strategies to improve language acquisition and student achievement for all students.”

The job descriptions detailed above provide some guidance into the various roles of district staff into service delivery for interventions; however, no position is assigned ultimate responsibility for the overall implementation and oversight of the intervention program. Notably absent were intervention responsibility for the positions of Director of Coaching Effectiveness and English Language Arts Implementation Specialist. Also, not presented to auditors was a job description for the Director of Curriculum (see Finding 1.1).

In order to determine the degree to which the school district has designed and implemented a formalized process for selecting, coordinating, implementing, monitoring, and evaluating intervention programs, the auditors reviewed the district improvement plan, all building improvement plans, and a list provided by school campuses outlining the programs and interventions utilized in PESD #1.

Exhibit 5.2.1 contains a list of the instructional interventions/programs utilized throughout the district obtained from several documents and lists.

Exhibit 5.2.1

Intervention Programs: District and Campus Level Offerings Phoenix Elementary School District #1

May 2017

Program Name Number of Schools Using Description Evaluation

ALEKS 2 Math Test scoresBalanced Literacy 4 Reading Test scores

Study Island 4 Reading, science, social studies

Test scores, Benchmark assessments

Gizmos 1 Math and science Test scores

Imagine Learning 8 Reading and phonics Growth, AIMSWeb, Galileo, AZ Merit, Usage reports

Learning.com 6 Technical skills for AZ Merit AZ MeritST Math 1 Math Not reportedMoby Max 3 Math Usage reportsMobius Math 3 Math Usage reportsBrainPop 1 Not reported Not reportedAccelerated Reader 1 Reading comprehension Not reportedHeart Math 2 Test Anxiety reduction AnecdotalAARP Reading 1 Reading Not reportedEngage New York 2 Math Galileo


Exhibit 5.2.1 (continued)Intervention Programs: District and Campus Level Offerings


Program Name Number of Schools Using Description Evaluation

AZ Merit Coach Books 1 Reading and math AZ MeritBig Brainz Math 4 Math AZ Merit, Usage reportsACT Now 2 Literacy and writing Benchmark AssessmentsGo Phonics 2 Fluency, Decoding AIMSWeb

Sonday 1 Grammar, fluency, comprehension Student summaries

West Virginia Phonics 1 Fluency, decoding AIMSWebSource: District provided data and principal survey

Based on data in Exhibit 5.2.1, twenty intervention programs are implemented in the district. The intervention that is most commonly used, Imagine Learning, is a district-wide program and selected for further examination later in this finding.

Additional observations regarding the data in Exhibit 5.2.1 include:

• There are 20 programs identified as intervention programs implemented across the district.

• Eight programs are implemented on a single campus.

• Most programs/interventions did provide information on a method of evaluation; however, auditors did not find evidence indicating how that information would be used to evaluate the success or impact of the program intervention.

• Usage reports are reported as an evaluation method in four programs; however, this monitors student use and not student achievement.

During interviews with principals, auditors learned of additional methods identified by principals as interventions, including, tutoring, pull-out time, and one-on-one instruction. However, these are not actually intervention programs but time within the school day for intervention to occur. As a result, they are not included in the above exhibit.

Academic intervention is a well-designed, deliberate approach focused on helping students develop new skills and knowledge or build fluency in acquired skills. Interventions are focused, explicit, and structured in a way that engages students. The effectiveness of an academic intervention is based on how well the intervention is tailored to individual student learning needs, how quickly the intervention is provided, how effective the intervention is in meeting students’ immediate needs, and at what costs.

To determine the adequacy of intervention design in PESD #1, auditors selected “Imagine Learning” for review to determine if the intervention design was sufficient for improving student performance. Imagine Learning was selected because it was mentioned as an identified program in place in many campuses. Auditors learned from district personnel that it is required as an intervention across the district. Imagine Learning is a web-based language and literacy program that is centered on an adaptive testing system, the results of which are used to place a student within a language curriculum. As students complete activities, they are moved to the next level of content based on their successes on assessments designed into the system. In this way, it is a self-paced program where students complete and move through the activities based on individual success.

Imagine Learning was first selected in 2014, as part of a pilot program recommended by the state of Arizona. It was originally selected as an intervention program in a few schools to address language skills of struggling students. After the pilot year the district entered into a three year, $1.1 M contract to provide Imagine Learning across all schools. At this time, it ceased to be an intervention and became a required program for all students in


all K-3 classrooms. One requirement of the contract dictates usage, as follows: “at least 20 minutes per student, per school day for prekindergarten and kindergarten students; at least 30 minutes per student, per school day for students in first grade and above” (Imagine Learning purchase contract, 2015). Auditors learned from district administrators that for struggling students, the district recommends increased time using Imagine Learning.

Auditors use seven criteria to determine whether an intervention is designed in such a way that it has a likelihood of successful implementation. For an intervention to receive an adequate design rating, at least five of the seven criteria (more than 70%) must be met fully. Exhibit 5.2.2 lists the criteria and the auditors’ rating of the district’s intervention approach.

Exhibit 5.2.2

Comparison of Imagine Learning to Audit Intervention Design CriteriaPhoenix Elementary School District #1

May 2017

Intervention Design Audit CriteriaAuditors’ Rating

Adequate Inadequate1. The intervention relates to a documented district need – current situation had

been assessed, diagnosed, and analysis data collected and considered in the selection of the intervention.

X

2. There is evidence that a problem has been identified from data analyses, several alternatives proposed and examined, and one of the better alternatives to address the problem selected.

X

3. A formal plan, with goals, measurable objectives, and processes, is in place and is being implemented. Documentation exists to define the purpose of the intervention, why it addresses the system need/problem, and how it will impact student achievement. A plan for design, deployment, and implementation of the intervention is in place.

X

4. Evidence exists that a strong deployment approach was designed, including identification of staff proficiencies needed to implement the intervention, appropriate staff development around the proficiencies, and a clear communication plan for appropriate audiences.

X

5. Human, material, and fiscal resources needed to initiative the intervention (short-term) and to sustain the intervention (long-term) are identified and in place.

X

6. Formative feedback and summative evaluation criteria are identified and are tied to intervention goals, objectives, and explanations. X

7. A plan for monitoring the ongoing deployment and implementation of the intervention is in place and involves appropriate individuals to carry out this plan.

X

Total 3 4Percentage Evident 42.8%

©2013 CMSi

Exhibit 5.2.2 shows that the district’s implementation and evaluation of the Imagine Learning program as a district intervention did not meet the audit criteria for sound intervention design. The district was rated inadequate since only three of seven criteria (42.8%) were rated as evident.

The following was noted regarding the criteria:

Criterion 1: Establishment of need (Adequate)

The use of Imagine Learning was initially supported through state funding and Title I funds. Phoenix #1 elected to utilize Imagine Learning as a program across the district based on test results for the piloted schools and


purchased additional site licenses for all buildings. Imagine Learning had been purposefully selected based upon evidence the program was suitable for addressing a specific identified learning need in the district.

Criterion 2: Selection of alternatives (Inadequate)

The auditors were not presented with an analysis that indicated a search for alternative ways to address a specific, identified problem. Rather, the district supports a number of reading interventions, and campuses have the option of selection. Since Imagine Learning is a mandated program, the likelihood of pursuing an alternative program is diminished.

Criterion 3: Formal plan for design, deployment, and implementation (Inadequate)

While no formal plan with goals and measurable objectives was in place within the district improvement plan to address an identified problem or need, some measurable goals do exist in campus improvement plans. However, a district strategy or plan does not exist for the design, deployment, or implementation of Imagine Learning other than to use the program for the specified number of minutes each day.

The main issue is that, with the dictate that the use of Imagine Learning be required in all classrooms to all children, it ceased being an intervention to becoming a required language and literacy program. In the absence of a district wide written curriculum for language arts, Imagine Learning becomes the de facto curriculum, or at least one component of the curriculum (see Finding 2.2).

The Imagine Learning website indicates that the program is aligned to Common Core standards, but this does not satisfy audit expectations of a district curriculum (see Finding 2.3). Further, the preponderance of programs across the district in place of a written curriculum hinders the district’s ability to determine what program, if any, is having a positive impact on student learning (see Finding 2.1). Imagine Learning has become just another of the many programs and not an intervention to address a specified learning need, as originally intended.

Criterion 4: Staff development and communication (Adequate)

Imagine Learning has been utilized in Phoenix #1 from 2014-2017. Annual training is provided by the district and supported by representatives from Imagine Learning. Imagine Learning building liaisons (typically instructional coaches) attend monthly district wide meetings led by district coaches. Local campus training is available and led by building liaisons. New staff are trained by building liaisons.

Criterion 5: Appropriate resources (Adequate)

Originally, Imagine Learning was funded by grant and Title I funds. Today it is supported by district funds. Staff is in place on a district and building level to monitor the program. Technologies are in place to allow students to access the program.

Criterion 6: Feedback and evaluation (Inadequate)

No formative feedback or summative evaluation criteria were presented to the auditors to determine the success of the program. District personnel indicate that summative reports are available at the end of the year but without a means to correlate results to standardized tests. Without program goals clearly described (see Criterion 2 above), it was impossible for auditors to determine program effectiveness.

Criterion 7: Monitoring (Inadequate)

The district extracts usage reports for each building and classroom to monitor program use. Student data is also available and can be used by teachers to monitor student progress. There was no indication that the teacher evaluation process incorporates the effective use and deployment of Imagine Learning as an intervention.

Summary

The approach to intervention design in Phoenix Elementary School District #1 did not meet audit review criteria and was, therefore, considered inadequate. No documentation was provided that Imagine Learning has goals and objectives that have been selected against which the program effectiveness could be measured. This inadequacy is exacerbated by the fact that Imagine Learning is not being used as an intervention, but rather as


an academic program required for all students in grades K-3, regardless of their academic learning needs. Even if Imagine Learning were a successful program, the current district design makes it impossible to determine its effectiveness.

Auditors then examined the delivery of interventions. Auditors selected the Imagine Learning program to examine in terms of six specific deployment and implementation criteria. For an intervention to receive an adequate delivery rating, at least five of the six criteria must be met with full evidence. Exhibit 5.2.3 lists the criteria and auditors’ rating of the district’s approach. A detailed discussion of the ratings follows the exhibit.

Exhibit 5.2.3

Comparison of the Imagine Learning to Intervention Implementation CriteriaPhoenix Elementary School District #1

May 2017

Audit Criteria for Intervention ImplementationAuditors’ Rating

Evident Not Evident

1. A formal plan, with goals, measurable objectives, and processes, is in place and is being implemented. X

2. Implementation of the intervention is both strategic and purposeful. The staff proficiencies needed to implement the intervention are clearly defined. Appropriate staff development based on these proficiencies takes place every year as new personnel are hired and as additional needs are identified. Continued goals for implementing the intervention and frequent progress reports are clearly communicated to all appropriate personnel.

X

3. The human, material, and fiscal resources needed to initiate and sustain the intervention are identified and allocated. X

4. Feedback from formative and summative evaluations that are tied to intervention goals, objectives, and expectations are systematically administered. X

5. Monitoring implementation of the intervention is taking place; responsibilities and procedures for monitoring are clearly defined and assigned to the appropriate individuals to carry out this plan.

X

6. The intervention is being modified and adjusted as needed, based upon monitoring of formative and summative evaluation data, to ensure continued quality control. X

Total 2 4Percentage Evident 33%

©2013 CMSi

As can be noted in Exhibit 5.2.3, implementation of Imagine Learning as an intervention program met two (33%) of the six criteria for implementation. Therefore, the intervention delivery rating is inadequate.

Following is a discussion of what the auditors found regarding each of the delivery criteria as it relates to the Imagine Learning intervention program:

Criterion 1: Plan implementation (Inadequate)

Auditors found no evidence of a formal plan with goals, measurable objectives, and processes in place for the implementation of Imagine Learning. There was no specific district strategy associated with the implementation other than program usage. The Imagine Learning website states that the program is aligned with Common Core standards. Auditors rated the company’s statement insufficient as a program goal.


Criterion 2: Staff development and communication (Adequate)

The implementation of Imagine Learning at campuses is required by the district. District wide professional development training has been required for building liaisons and new teachers and coordinated through district personnel. Campus principals are contacted regularly regarding the level of usage in their buildings.

Criterion 3: Resource adequacy (Adequate)

The adoption of Imagine Learning was initially funded with money from the state of Arizona. Annually since 2015, costs have been funded through district funds. The district budgets the annual cost each year.

Criterion 4: Assessment data availability (Inadequate)

Imagine Learning is capable of providing district and campus administrators and teachers with a significant amount of student performance data aligned with Common Core standards. Use of the Imagine Learning assessment system generates targeted lessons that are prescriptive to a student’s learning level. District administrators indicated that Imagine Learning reports are used to monitor student progress through the guidance of building liaisons. However, the lack of alignment with district learning expectations, a district-wide curriculum, or standardized tests precludes the assessment’s effectiveness.

Criteria 5: Monitoring (Inadequate)

In reviewing job descriptions, auditors found no position in the district clearly assigned the responsibility for monitoring the implementation of Imagine Learning. Direct day-to-day supervision is the province of the ELA Specialists. Campus implementation is supported by building liaisons, instructional coaches, along with classroom teachers. Procedures for monitoring Imagine Learning are not clearly defined, other than a brief list of guidelines that do not address goals, objectives, or assessment.

Criteria 6: Program modifications based on data (Inadequate)

District administrators reported that no modification or changes are permitted in the program. Implementation and modifications to the program are determined by the vendor and no such modifications were reported to auditors.

Overall, the deployment and implementation of the Imagine Learning program are inadequate.

Interviews with district staff elicited many comments about concerns and frustrations with the intervention programming available to students in the district.

• “We use Imagine Learning for language intervention. It is a self-paced program but we do not have records in the building as to how the kids are doing.” (Campus Administrator)

• “Interventions are selected on a building basis.” (Central Office Administrator)

• “The bottom 25% of students [are identified] to come up with interventions. I’m concerned that some school teams don’t do better because there is no accountability. Some principals do not come to the meetings.” (Central Office Administrator)

• I’m concerned about the students who go through the RTI process. Interventions are limited to more time on computers working independently. Students need a staff member to work with them to ask them questions and guide them not a computer program where the sit by themselves and no support.” (Teacher Survey Response)

• “Teachers in grades 4-8 are not required to provide proper intervention.” (Teacher Survey Response)

Summary

Phoenix Elementary School District #1 does not have a process for the establishment, implementation, or evaluation of program interventions. The auditors found some job descriptions that described components needed in a comprehensive plan for the establishment of interventions. However, no one person is assigned responsibility for oversight of the program. Many of the required audit criteria were absent from planning


documents. Auditors received no documentation of a district wide review process as a result of the expectation of campus implementation of intervention programming.

Imagine Learning is a required program in the district. Originally selected as an intervention program with an initial pilot, it has emerged as a required reading program for all students. Therefore, auditors determined that the program no longer satisfies the definition of an intervention. Interviews with key stakeholders confirmed the lack of established district processes to determine intervention effectiveness, create consistency in the selection and adoption of programs, effective implementation, and regular evaluation of selected programs (see Recommendation 10).

Finding 5.3: A comprehensive facility plan does not exist to guide major renovation, remodeling, and new construction to address facility needs. Some features of aging facilities diminish the quality of the learning environment for students.

The physical climate of a school is an important indicator of its ability to deliver the curriculum effectively. Providing adequate educational facilities is a major responsibility of the governing board and district administration. The learning environment of a school district must be well maintained, clean, safe, and pleasant to support the effective delivery of the instructional program. The design of the school facility, adequacy of space, and flexibility of use should support and enhance the instructional program. Facilities need to be designed and maintained in a manner that conveys to students, parents, staff, and community members that the educational setting is a high priority.

Long-range facilities planning is imperative for effective use of funds and real estate to meet both current and future student needs. Planning should be based on the careful analysis of all factors that impact the learning environment such as enrollment trends, curriculum needs, demographic changes, instructional practices, special educational requirements, technology advancements, and the support services needed to maintain the system. Long-range planning ensures that a district is prepared financially for the task of maintaining the quality of the existing facilities and the possibility of future construction or renovation.

Auditors reviewed board policies, facilities planning documents, and other documents related to school buildings and grounds. The audit team visited each of the district’s schools and many classrooms where instruction was taking place to gather information on the learning environment and any special problems or impediments that may exist in facilities. The auditors attended particularly to overall maintenance, physical atmosphere, accessibility, safety, and use of the buildings. Interviews were conducted with board members, administrators, teachers, students, and community members.

The auditors concluded that Phoenix Elementary School District #1 is presently experiencing many building renovations as a result of the passage of a bond levy in 2015 to help address the problem of aging facilities. The master facilities plan, however, is inadequate by audit standards to guide the process of building utilization.

Several board policies and two job descriptions were presented to auditors relevant to the area of facilities:

• Board Policy CBA: Qualifications and Duties of the Superintendent describes several duties of the Superintendent, including:

○ Assumes responsibility for the use of buildings and grounds.

○ Recommends the locations and sizes of new school sites and of additions to existing sites; the locations and sizes of new buildings; the plans for new school buildings; all appropriations for sites and buildings; and improvements, alterations, and changes in the buildings and equipment of the District.

• Board Policy EA: Support Services Goals/Priority Objectives establishes the following goals in regard to facilities, “to provide a physical environment for teaching and learning that is safe and pleasant for students, staff members, and the public.” And “To establish a thorough, effective, and economical maintenance program that will assure a useful life for school sites, plants, and equipment. Operations and maintenance of the school plant, equipment, and services should set high standards of safety,


promote the health of pupils and staff members, reflect the aspirations of the community, and support environmentally the efforts of the staff to provide instruction.”

• Board Policy ECB: Buildings and Grounds Maintenance addresses building maintenance as follows:

○ Adequate maintenance of buildings, grounds and property is essential to efficient management of the District.

○ The Board directs a continuous program of inspection and maintenance of school buildings and equipment. Wherever possible, maintenance shall be preventive and will focus on providing an on-going healthy learning environment for both students and school personnel.

• Board Policy FA: Facilities Development Goals/Priority Objectives establishes several goals and priority objectives regarding the development of facilities, as follows:

○ Priority in the development of facilities shall be based on identified educational needs and on programs developed to meet those needs. The Board establishes these broad goals for development:

▪ To integrate facilities planning with other aspects of planning in a comprehensive educational program.

▪ To base educational specifications for school buildings on identifiable learner needs.

▪ To design for sufficient flexibility to permit program modification or the installation of new programs.

▪ To design school buildings as economically as feasible, providing that learner needs are effectively and adequately met by the design.

▪ To involve the community, school staff members, available experts, and the latest in related current development and research in building plans and specifications.

▪ To analyze life-cycle costs as they compare with capital expenditures versus a maintenance and operations expense projection.

▪ To analyze the core facility as it relates to future expansion.

▪ To design school buildings for community use when feasible.

• Board Policy FE: Facilities Construction states requirements related to facility construction under several categories:

○ Facility Planning: The planning of major alterations of facilities and/or the planning of new facilities should be the joint effort of those who have the responsibility of the educational program (teachers), those who will use the facilities (students), those who have great concern for the products coming from the use of facilities (parents), and those charged with the legal responsibilities of education in the District (Board and administration). Other outside people will be used in facility planning for their expertise (architects, educational consultants, fiscal agents, and recreational personnel). The executive director for administrative services is responsible for leading a coordinated effort in planning facilities.

○ Objectives and Goals: Facility planning is the process by which objectives are established and resources are allocated in a manner that optimizes the procurement of needed facilities. Planning for school facilities is only one aspect of the total program aimed at meeting the needs of society through an instructional program. The Superintendent and staff will develop answers to questions listed for planning and will present them to the Board for consideration.

○ Long-Range Planning: Long-range planning includes the projection of student enrollments, long-range financial plans, long-range projected use, and need for additional facilities. Staff members, students, and citizens will be used in the planning. The Superintendent is responsible to the Board


for developing recommendations for long-range planning and submitting them to the Board for action.

○ Advisory Committee: The Board may appoint an ad hoc facilities advisory committee consisting of teachers, principals, parents, students, and other interested parties to serve the Board in an advisory capacity. It will present any recommendations regarding needed changes or additional facilities for school plants. The Superintendent and the executive manager for administrative services will serve the committee in an advisory and fact-finding capacity and will seek experts to assist the committee when deemed desirable.

• Board Policy FEA: Educational Specifications for Construction states that “in construction of new or remodeled educational facilities, include…Information concerning the plan of school organization and estimated enrollment in the proposed building” and “A description of the proposed curriculum and the teaching methods and techniques to be employed.”

• The job description for the superintendent is silent on the area of facilities, however, Board Policy CBA, as noted above, requires the superintendent to assume responsibility for the use of buildings and grounds, and make all recommendation regarding future facility decisions.

• The job description for the Assistant Superintendent for Business Services requires the position to “Coordinate with the Superintendent the development and implementation of a five-year financial and facilities plan which supports the District’s Strategic Plan, Vision, Mission and Goals.”

• The job description for the Director of Maintenance and Operations includes responsibility for all maintenance and construction activities completed in the district, and to “Assist the Assistant Superintendent for Business Services in the preparation of the budget as it pertains to maintenance, construction, remodeling and planning.”

Auditors reviewed several additional documents related to facilities and maintenance, planning for facility needs, and enrollment. Exhibit 5.3.1 lists the documents presented to auditors.

Exhibit 5.3.1

Facility Documents Reviewed by AuditorsPhoenix Elementary School District #1

May 2017

Document DateVarious Board Policies VariesJob Descriptions VariesPhoenix Elementary Mobility Study 2017SFB Building Status List 2017Special Election Informational Pamphlet 2015Financial Report, Arizona Office of the Auditor General 2016First Sale Bond Program PowerPoint; ADM Group 2017District Strategic Plan 2012


No single comprehensive facilities plan was presented to auditors. In the absence of a single facilities plan, analyses of the above documents and the audit criteria used to evaluate them are included in Exhibit 5.3.2.

Exhibit 5.3.2

Comparison of Facilities Planning Efforts to Audit Components Of a Comprehensive Long-range Facilities Plan


Components of a Comprehensive Long-Range Facilities PlanAuditors’ Rating

Adequate Inadequate1. Philosophical statements that reflect community aspirations and the

educational mission of the district and their relationship to short- and long-range facilities goals

X

2. Enrollment projections that take into account any known circumstances that may change the pupil population X

3. The current organizational patterns of the district and identification of possible organizational changes necessary to support the educational program X

4. Identification of educational programs considered by designers of capital projects for renovation or addition of school facilities X

5. A detailed evaluation of each facility, including assessment of structural integrity, mechanical integrity and efficiency, energy efficiency, operations and maintenance, and health and safety requirements

X

6. Prioritization of needs for renovation of existing facilities and the provision of additional facilities X

7. Cost analysis of potential capital projects to meet the educational needs of the district, including identification of revenues associated with capital construction

X

8. Procedures for the involvement of all stakeholders of the school community in the development and evaluation of the long-range facilities plan X

Total 1 7Percentage of Adequacy 12.5%

©2013 CMSi

As described in Exhibit 5.3.2, the documents presented to auditors for review met just one of eight criteria for planning (12.5%), less than the 70% required for adequacy and, therefore, were deemed inadequate in meeting audit criteria for a comprehensive, long-range facilities master plan. The following provide a brief discussion of each criterion and auditors’ assessment of associated district plan components.

Criterion 1: Philosophical statement (Adequate)

Board Policies EA, ECB, and FA discuss the need for adequate facilities in reference to safety, security, cleanliness, energy conservation, maintenance, and support of district operation. The job description for the Assistant Superintendent for Business Services includes the responsibility to interpret education priorities to match facility design. The district strategic plan includes a priority goal, 2.3, to “Ensure all schools and departments have conditions and operation capacity to meet academic and business needs.” And “Including adapting facilities to support Signature Schools,” and “Implement standards for learning environments.”

Criterion 2: Enrollment projections (Inadequate)

While Board Policy FE requires student enrollment projections as part of a long-range facility planning process, no enrollment projections were presented to auditors for review. The Phoenix Elementary Mobility Study includes prior year’s enrollment, but no projection for future years.


Criterion 3: Educational program support (Inadequate)

Auditors were not presented with any documentation of the impact of enrollment trends on school construction, expansion, or remodeling. The aforementioned Phoenix Elementary Mobility Study identified casual factors related to demographic shifts within the district, primarily the impact of open enrollment on the student population. No data related to demographic housing patterns within the district were presented to auditors, however, auditor heard several comments during interviews related to housing issues for families within the PESD #1 district boundaries, “Neighborhoods are changing and families are being pushed out of the community.” (Central Office Administrator).

Criterion 4: Educational program needs (Inadequate)

While a philosophical statement connecting the facility expectations in relation to the educational programming mission of the district does exist (See Criterion 1 above), no documentation was presented to auditors to define such goals. As a result, while board policies expect a thorough and comprehensive survey of the structural integrity and capacity expectations of the district, there is no connectivity between the facilities and how they will be utilized to support the educational programming in the district. During interviews with district administrators, auditors learned of several facility remodels that have been incorporated into existing structures construction. However, such remodels are limited to replacing or repairs existing buildings (e.g., new carpeting, HVAC systems, roof repair/replacement). There is no systemic plan or institutionalized process to link educational program needs to facility design.

Criterion 5: Facility needs assessment (Inadequate)

Board Policy ECB requires a “continuous program of inspection and maintenance of school buildings and equipment.” Auditors were not presented with such documentation. Auditors reviewed the SFB Building List that details information about each district building including date of construction, primary use, gross area, and net area.

Criterion 6: Renovation and new construction prioritization (Inadequate)

Board Policy FE describes several components of required planning activities related to prioritization, including planning, goals of the facility program, assessment of current facilities, and committee membership. None of these components address a prioritization or rank ordering of projects and its impact on learning in the district.

Auditors examined the document, Information Pamphlet for Special Election of November 3, 2015. The document was the public notice of the special election to raise up to $59,500,000 for facility maintenance and repair. While various renovation and repair projects are described in general terms in this document, there is no list of projects or building or a priority ranking of planned activities. Auditors examined a PowerPoint presentation from May 2017 entitled “First Sale Bond Program 2016.” The PowerPoint lists the projects by building and activities resulting from the first round of bond sales as a result of the successful 2015 election. While inclusion on the list of planned projects may imply a type of prioritization, there is no indication of qualifying factors for inclusion or future planning for additional remodeling/repairs resulting from the bond.

Criterion 7: Capital cost analysis (Inadequate)

Information on capital cost analysis was not presented to the auditors for analysis.

Criterion 8: Stakeholder involvement (Inadequate)

Board Policy FE states that “the board may appoint an ad hoc facilities advisory committee consisting of teachers, principals, parents, students, and other interested parties to serve the Board in an advisory capacity. It will present any recommendations regarding needed changes or additional facilities for school plants. The Superintendent and the executive manager for administrative services will serve the committee in an advisory and fact-finding capacity and will seek experts to assist the committee when deemed desirable.” While optional stakeholder involvement by the district is laudable, the policy does not require stakeholder involvement, only that the board may appoint an ad hoc facilities advisory committee.

Additionally, the district website does not provide updates on the progress of facility projects.


Summary

Auditors were presented with no formal facilities master plan. The district has procured bond levies to build and maintain facilities but its planning function was inadequate when compared to audit criteria. Without data on enrollment projections, the district is unable to projects its facility needs for the future, although the governing board is committed to providing adequate facilities for its students’ educational growth.

Facility Condition and Renovations

Audit team members conducted a visual inspection of all buildings. This tour included an examination of many classrooms, media centers, cafeterias, all-purpose rooms, offices, work areas, restroom facilities, and a general inspection of the grounds. Depending on the age of the school, auditors found the buildings to be well maintained and generally clean. Limitations such as an inability to provide technology access, air conditioning and ventilation issues, dim lighting in the hallway areas, some crowded classrooms, and leaking roofs diminish the quality of the educational environment.

Although stakeholders responded that they agree facilities are in acceptable conditions, principals reported a need to improve response times to maintenance issues. Their responses to a survey question on this issue is presented in Exhibit 5.3.3:

Exhibit 5.3.3

Principals’ Responses Regarding Response Time to Maintenance Issues (N=10)Phoenix Elementary School District #1

May 2017

Strongly Agree10%

Agree40%Disagree

40%

Strongly Disagree

10%

I am satisfied with the response time to maintenance issues in my building.

As can be seen in Exhibit 5.3.3, only half of the building leaders who responded agreed that they are satisfied with the response time to maintenance issues in their building. Half disagreed or strongly disagreed.

The Phoenix Elementary School District #1 Governing Board has aggressively pursued a plan to address issues of facility inadequacies, building age, and deterioration through the $59.1 M bond levy passed in 2015.


Age of Facilities

Exhibit 5.3.4 displays each school in the order of initial construction, the years of any additions added to the building, and how it has been impacted by the bond issue with general improvements, cost estimates, and anticipated completion dates. Auditors noted that the age of a facility does not singly determine its suitability for school use.

Exhibit 5.3.4

Facility Construction DatesPhoenix Elementary School District #1

May 2017

School Initial Construction AdditionsKenilworth 1920 2000Whittier 1929 1949, 1959, 1995, 1998, 2004, 2012Dunbar 1949 2004, 2010Heard 1970 1981, 1996, 2004, 2009Garfield 1981 1988, 2004Lowell 1981 1999, 2000, 2004Bethune 1981 2004Emerson 1981 1997, 2000, 2004Edison 1981 2000, 2004, 2010Herrera 1986 2004, 2011Shaw Montessori 1995 1998, 2000, 2004Capitol 1995 1999, 2004, 2009Magnet 1997 NoneFaith North 2000 NoneSource: District provided data

As indicated in Exhibit 5.3.4:

• Two buildings, Kenilworth and Whittier, were built in the 1920s.

• Dunbar was built over 50 years ago.

• All buildings except Magnet and Faith North have had additional space added.

• The latest addition projects were completed in 2012, 2011, and two in 2010.


Schools in the district take advantage of outdoor learning environment for student activities.

Imaginative furnishing in the classroom offers students a special reading corner.

Auditors analyzed non-construction projects that have impacted the district capital outlay in recent years. Exhibit 5.3.5 cites such projects along with the building location of the project and the estimated costs.

Exhibit 5.3.5

Recent and On-going Facility ProjectsPhoenix Elementary School District #1

May 2017

Description Building Estimated Cost

New Buses District wide $1,391,000Instructional and Administrative Technology District wide 4,719,000

Roofing Bethune, Emerson, Edison, Garfield, Kenilworth, Whittier, Faith North, Herrera, Heard, Dunbar 3,029,000

HVAC Bethune, Emerson, Edison, Garfield, Capitol, Faith North, ASU Prep, Ann Ott, Related Fees 4,050,000

Security Upgrades Faith North, Capitol, Dunbar, Edison, Emerson, Heard, Magnet, Shaw 1,042,000

Miscellaneous Remodeling

Emerson: Windows, Transit window panels Faith North: Sidewalks, cabinets, windows Kenilworth: Windows Shaw: Door project Shaw, Faith North, Bethune: Roll up doors ASU Prep: Storm drains Edison: Playground Bethune, Edison, Emerson: Walk-in renovation

1,243,000

Painting Faith North, Heard, Kenilworth 698,000Carpet/Flooring Bethune, Faith North, Shaw, Whittier 320,000

Total $16,492,000Source: First Sale Bond Program PowerPoint, 2017



• The district is projected to spend almost $16.5 million on projects to improve, modernize, and renovate building facilities.

• The largest expenditure—$4.7 million—will be for instructional and administrative technology.

• Every campus except Lowell is receiving one or more upgrades.

During interviews with district personnel, auditors heard many comments concerning the district facilities, their ages, crowded classrooms, overall maintenance and operation costs, and the wisdom of renovations. Auditors examined a few variables for closer review.

Student to Teacher Ratio

To examine the adequacy of school facilities, auditors examined the student to teacher ratio at each school. Exhibit 5.3.6 displays the student to teacher ratio information.

Exhibit 5.3.6

Student to Teacher Ratio by School Phoenix Elementary School District #1

May 2017

School Student to Teacher RatioBethune 14:1Dunbar 14:1Edison 14:1Emerson 14:1Faith North 14:1Herrera 14:1Shaw Montessori 14:1Garfield 17:1Kenilworth 17:1Lowell 17:1Magnet 17:1Whittier 17:1Capitol 20:1Heard 20:1District 17:1Source: District provided data

As noted by auditors from Exhibit 5.3.6:

• The overall average student to teacher ratio for Phoenix Elementary School District #1 was 17:1.

• Seven schools were below the average with a ratio of 14:1.

• Capitol and Heard Elementary Schools had the highest ratio at 20:1.

Despite the ratio displayed above, auditors heard comments from district personnel regarding over crowded classroom sizes in many schools. Auditors noted similar situations during their classroom visits where students numbered thirty or more in some classrooms. Representative teacher comments are below:

• “Too many students in grade K-2” (Teacher Survey Response)

• “Student to teacher ratio should not be over 20:1.” (Teacher Survey Response)

• “We need smaller class sizes.” (Teacher Survey Response)


Auditors would normally compare campus pupil capacity limits to actual student enrollment, but such building capacity information was not presented to auditors for review.

Kenilworth Elementary School portrays the district’s pride in its historical facilities

Cost to Maintain Facilities

Auditors next examined the efficiency with which PESD #1 rates in regard to maintaining district facilities. The Arizona Auditor General provides each district an efficiency comparison to other peer districts as well as the state. Exhibit 5.3.7 displays this information.

Exhibit 5.3.7

District Facility Efficiency Measures Compared to State and Peer DistrictsPhoenix Elementary School District #1

May 2017

Operational Area Measures District Peer Average State Average

Plant OperationsCost per square foot $7.41 $6.27 $6.10

Square feet per student 179 141 154Source: Financial Report, Arizona Office of the Auditor General, 2016


• In plant operations at the cost per square foot, PESD #1 exceeded peer districts by $1.14 and the state average by $1.31 per square foot.

• In the allocation of square feet per pupil, PESD #1 exceeded peer districts by 38 square feet and the state average by 25 square feet per pupil.

During interviews with district staff, auditors heard comments about the adequacy of space in district buildings, as follows:

• “We really have three or four buildings more than is needed, based on enrollment.” (Central Office Administrator)

• “We would be wise to consider new construction, rather than to repair the three old smaller schools.” (Central Office Administrator)


Summary

In contemplating facilities, school districts are continually examining the ever-changing variables that influence their decisions. Among such variables are some that directly impact Phoenix Elementary School District #1 including:

• A changing demographic population that is trending to lower student enrollment.

• Aging facilities that require continual upgrades and repair and exceed both state and peer group averages.

• Aging facilities that may not meet the educational learning needs of the students.

• Class sizes that vary from building to building and some that exceed what is expected by district personnel.

• Square footage per pupil that exceeds both peer district and state averages.

• The requirements associated with an infusion of instructional and operational technology throughout the district.

While auditors make no recommendations regarding the decision to invest in continued maintenance costs for existing facilities, they do suggest that the governing board consider all variables that heavily impact the productivity of system operations. Since cost savings from facilities are directly transferable to new monies to support teaching and learning, all funds that can be recuperated by calculated facility decisions should be examined.

The auditors concluded that Phoenix #1 is presently undergoing many building renovations as a result of the 2015 Bond levy passage to help address the problem of aging facilities. The governing board will need to make decisions to address the adequacy of facilities in relation to the student population size and learning needs. Regardless, the district’s master facilities plan is inadequate by audit standards to guide the process of building utilization (see Recommendation 7).

Finding 5.4: The district’s instructional technology efforts are not guided by a district technology plan. Efforts to increase technology usage among teaching staff are unfocused and lack direction and oversight. Recent efforts are underway to increase instructional technology use in classrooms.

In the 21st century, technology is a critical component of daily life. As technology usage has grown at an increasingly fast pace, so has the expectation in schools for the use of technology. No longer is technology usage simply a teacher using a Smart Board to display slides or students employing the internet for research. On the contrary, it is now imperative that technology assist teachers with instructional delivery while students must use technology to facilitate their exploration and learning.

District leadership aspiring to deliver a world-class education integrates technology into all aspects of the day-to-day operation of the district, including student learning. Technology planning in effective school districts leads to technology implementation that supports deeper and more meaningful learning in district. The systematic, efficient integration of technology into curriculum and instruction serves as a tool to increase student achievement. Technology serves important roles in business and management functions.

Appropriately, funding and directing the use of technology throughout the school district is an essential part of effective management and control. A written plan that outlines expectation, goals, guidelines, and evaluation protocols for the use and integration of technology is an effective means of ensuring consistent implementation across the district.

Auditors reviewed district policies, district and campus plans, job descriptions, and other district documents related to technology. Auditors conducted interviews with board members, district staff, students, and parents, and conducted an online survey of teachers and principals to determine the use of instructional technology within the district. Auditors conducted site visits to classrooms to observe the use of technology in the classroom.

Auditors found that Phoenix Elementary School District #1 lacks a technology plan that meets audit criteria. Board policies and district job descriptions are limited in relation to guidance and responsibility related to


technology for the improvement of student learning. Auditors also found that while technology is available in most classrooms, the use of the technology is either limited or rudimentary.

Auditors reviewed board policies to determine the direction they provide to technology planning, financing, and implementation. Auditors determined that the following board policies had some relationship to technology:

• Board Policy IA: Instructional Goals and Objectives describes the various goals and objectives of instructional programming for students, including “skills in technology.”

• Board Policy IHA: Basic Instructional Program details instructional programs in the district, including a planned sequence in “technology skills.”

• Board Policy IJND: Technology Resources affirms that there is educational value in utilizing movies and videos if they have been planned for in advance and they reinforce concepts taught in the classroom.

• Board Policy IJNDB: Use Of Technology Resources In Instruction is basically an acceptable use policy describing rules and safeguards of technology use in the district.

Board policy does not adequately direct the development, implementation, integration, and evaluation of a district technology plan for the improvement of student achievement or increased efficiency of business and management functions (see Finding 1.3).

Auditors reviewed available job descriptions to determine the roles and responsibilities related to technology in Phoenix #1. The following job descriptions reference technology to varied degrees:

• Only the Director of Educational Technology job description is up to date and relevant to the current Director of Educational Technology:

○ Maintains current knowledge of technology, instructional practices, research and application that relate to the implementation of educational technology…

○ Provides leadership and collaborates with other Departments to develop, implement and update a district educational technology plan aligned to district goals…

○ Provides input in the development of system-wide and building level programs that enables students to use technology as a learning tool based on best practices for technology integration…

○ Collaborates with instructional staff, including teachers, to develop curriculum materials and specific lesson plans that integrate technology into content…

○ Models the integration of technology in all curriculum areas…

○ Works with the principal and school leadership team to provide access to technology resources and develop a plan of action for the integration of educational technology…

○ Conducts on-going needs analysis and evaluation of educational technology initiatives…

○ Coordinates professional development with teachers, administrators, and support staff…

○ Plans and implements educational technology workshops and professional learning communities.”

• Senior Software Support Technician:

○ Works closely with the Curriculum and Instruction departments to determine what assessment software programs are required and helps determine what programs should comprise the base computer image…

○ Aids the MIS department in testing all new assessment software applications to determine if they work within the existing network standards…

○ Aids those responsible for the user and student information contained in assessment web-based applications.


• Classroom Teacher: “Effectively utilizes district technologies appropriate to the position and assumes responsibility for attending district training needed to successfully perform designated responsibilities as directed by supervisor.”

Other job descriptions are vague, outdated, and provide no clear linkage to current or available technology (see Finding 1.1). Auditors were told by district staff that the Director of Educational Technology is a new position this school year developed in an attempt to advance instructional technology use among staff. Job descriptions notably absent of any references to technology include elementary principal, superintendent, Assistant Superintendent of Curriculum and Instruction, Assistant Superintendent of Business Services, and Instructional Coach.

The Phoenix Elementary School District #1 does not currently have a technology plan. In place of a comprehensive technology plan, auditors were presented the district’s E-Rate application, which contains a self-assessment of the current and future technology goals for the district. This document was included for review by the auditors to determine the quality of the technology program in PESD #1. Additional documents included in the review are as follows:

• Board Policies;

• Related job descriptions;

• Computer purchases, 2008-17;

• Computers by location, 2017;

• Software adoption process, 2017;

• District Mission and Vision Statement;

• District Strategic Action Plan, 2012;

• Educational Technology Professional Development, 2017; and

• Sampling of building improvement plans, 2017.

None of the documents reviewed, either individually or taken in total, contain the elements of a comprehensive technology plan. At best are generalized statements of technology usage, such as “use appropriate technology, increase technology use, and implement standards.” None of these statements provide the clarity and specificity to direct the implementation of technology usage for staff and students. Utilizing the information from the above documents, as well as interviews with staff members and classroom visits by the auditors, auditors compared the district technology planning against audit quality criteria for technology planning. To be considered adequate overall, 70% of the quality criteria must be met.


Exhibit 5.4.1 presents audit technology plan criteria and the auditors’ assessment of PESD #1 technology as to adequacy.

Exhibit 5.4.1

Auditors’ Assessment of Quality Criteria for Instructional Technology Related ProgramsPhoenix Elementary School District #1

May 2017

CriteriaAuditors’ Rating

Adequate Inadequate1. Board policy or administrative regulation for instructional technology

exists. X

2. There is a clear statement of program philosophy/vision. X3. A comprehensive view of technology exists. X4. A needs assessment has been completed and evaluated. X5. Measurable student goals and objectives exist. X6. An ongoing student assessment component exists. Partial*7. An ongoing program assessment component exists. X8. There are comprehensive staff trainings with measurable standards for

equipment, application, and technology. X

9. School site equipment standards exist. X10. Internet access standards exist. X11. The role of the school library is stated. X12. An implementation budget has been identified. X13. A maintenance budget has been identified. X14. Technology site plans are aligned with district plans. X

Total 4 10Percentage Adequate 28.6%


As can be noted in Exhibit 5.4.1, the PESD #1’s technology planning efforts do not meet the audit criteria of 70% adequacy. Auditors rated the planning adequate in four (28.6%) of the criteria, partially adequate in one criteria, and inadequate in nine criteria. The following discussion provides more information on what the auditors found with respect to each of the characteristics above.

Criterion 1: Board policy or administrative regulation for instructional technology (Inadequate)

Board policies do not establish a clear expectation for the development of a technology plan or the content for such a plan that would include instructional technology development, implementation, and evaluation.

Criterion 2: Clear statement of program philosophy/vision (Inadequate)

The district’s mission statement makes no mention of instructional technology or technology use in the district to help students achieve learning goals. While the district’s E-Rate application includes several objectives for technology, there are no links to other district documents or student learning outcomes that can be traced back to the E-Rate proposal.

Criterion 3: Comprehensive view of technology (Inadequate)

No mention is made in any documents reviewed concerning equitable access to technology, ongoing professional development, evaluative and ethical use of information and technology, or a high level of proficiency for use in a global society.


Criterion 4: Needs assessment (Adequate)

A needs assessment, as part of the E-Rate application, was utilized to update the application from prior funding.

Criterion 5: Measurable student goals and objectives (Inadequate)

Student technology goals and objectives included in district and building improvements plans are generic and include statements such as, “will use technology” and “utilize laptops computers.” General statements such as these are not measurable.

Criterion 6: Ongoing student assessment (Partially Adequate)

No evidence was presented to auditors to indicate that students are assessed on technology skills or that standards of technology usage are incorporated into the technology program. While some assessment may take place in isolated classrooms/buildings, the evaluation of student skills is not uniform across the district.

Criterion 7: Ongoing program assessment (Inadequate)

No documents were provided to auditors to indicate that the technology program is assessed for effectiveness on a regular and ongoing basis.

Criterion 8: Comprehensive staff trainings with measurable standards for equipment, application, and technology (Inadequate)

Professional development for staff members is mentioned in many school improvement plans, but the plans are vague as to the where, when, and what of the trainings. District plan Goals 1.1.8 and 2.3.4 state that professional development will “Build content area expertise,” “Build the capacity of instructional staff,” and “Use ‘cutting-edge’ technology.”

The district has recently added the position of Director of Educational Technology to the district staff to address instructional technology. This position, however, is a unitary position with no support staff to serve the entire school district. Planned training sessions for the summer of 2017 focus on software use (SMART Notebook and Windows 365) and not on implementing technology use into classroom curriculum and instruction.

Criterion 9: School site equipment standards exist (Adequate)

Expectations for instructional technology for teacher use (SMART Board, document cameras) are clearly delineated in the E-Rate plan, and student computer access is included in this same document.

Criterion 10: Internet access standards exist (Adequate)

Internet access standards are addressed in board policy.

Criterion 11: Role of the school library (Inadequate)

The library as a link to technology and information access is not mentioned in documents presented to the auditors.

Criterion 12: Implementation budget (Inadequate)

Implementation of technology in the district has existed without specified goals or objectives since no technology plan exists. Auditors were not presented with documents that described program expenditure categories for library media, instructional technology, telecommunications/infrastructure, professional development, and curriculum development.

Criterion 13: Maintenance budget (Adequate)

Auditors were provided documentation indicating the disposal plans for outdated and non-functioning technology. As part of the recent 2015 bond levy passage, $9M is earmarked for technology equipment and upgrades in the future.


Criterion 14: Technology site plans are aligned with district plans (Inadequate)

Since building and district plans are only loosely worded regarding technology goals and objectives, auditors could not determine linkages and connections between plans within the intent of district technology planning.

During interviews with district staff, auditors heard several comments about the state of technology planning in the district:

• “The district does not really know if there is a 1:1 plan for computers.” (Campus Administrator)

• “We currently have no good way to evaluate our tech program.” (Central Office Administrator)

• “We have one-to-one learning [technology], but it is not consistently used.” (Central Office Administrator)

• “The current technology plan is from 2012, so it is quite dated.” (Central Office Administrator)

• “The one-to-one plan for computers is for teachers, not for students. We have about two computers for every three students.” (Central Office Administrator)

Recent district technology initiative has provided computer labs with headsets for all schools.

Teacher utilizes her SmartBoard to facilitate teaching her lesson. All district classrooms have been furnished with

SmartBoards.


Availability of Instructional Technology in the Classroom

As school district’s work to meet the learning needs of students, technology can be an effective means of meeting those needs if there is adequate availability in the classrooms throughout the district. Auditors reviewed computer inventories and observation data from classroom visits to determine the availability of instructional technology from campus to campus. That analysis is displayed in Exhibit 5.4.2. Computer to student ratio by campus is also noted.

Exhibit 5.4.2

Computer Inventory by School and Computer/Student RatioPhoenix Elementary School District #1

May 2017

SchoolDesktops Laptops* Total

Comp.Total Student

Comp.Computer/

Student Ratio†Staff Student Staff StudentBethune 3 73 9 199 284 272 1 to 1.7Capitol 5 65 43 325 438 390 1.6Dunbar 2 87 24 140 253 227 1.5Edison 5 160 42 309 516 469 1.4Emerson 2 70 32 193 297 263 1.8Garfield 3 151 45 343 542 494 1.3Heard 10 65 42 345 462 410 1.4Herrera 4 63 51 367 485 430 1.4Kenilworth 3 98 34 252 387 350 1.5Lowell 3 75 37 376 491 451 1.2Magnet 2 43 24 261 330 304 1.6Shaw 3 93 20 130 246 223 1.4Whittier 2 67 33 205 307 272 1.6

School Totals 47 1,110 436 3,445 5,038 4,555 1/1.46

††Emerson Court 81 25 131 316 553††Faith North 2 7 10 2 21††Monterey Park** 0 17 4 62 83††Warehouse 32 7 1 8 48

Totals 1,328 4,415 5,743* Includes laptops, tablets, and Chromebooks** Not classroom buildings, but included due to student computer availability†Calculation: K-8 student enrollment/number of computers ††Computer to student ratios were not calculated for Emerson Court, Monterey Park, or the Warehouse since students are not permanently assigned to these locations. Student to Computer ratios, being K-8, excluded Faith North, a preschool facility.Source: District provided data

As noted in Exhibit 5.4.2:

• The lowest computer ratio was at Emerson, with one computer for every 1.8 students; while the highest ratio was at Lowell with one computer for every 1.2 students. The district average was one computer for every 1.46 students.

• Seven schools met or exceeded the district average of one to 1.46 (Bethune, Dunbar, Whittier, Magnet, Emerson, Capitol, and Kenilworth).


• The district has a total of 5,743 computers, with some type of laptops accounting for 4,415 (77%) of the total. Designated student computers on school campuses are 4,555 (90.4%) of the on-site 5,038 computers.

• While some disparity can be noted across the district in terms of student computer availability, all campuses fall within a +/- 0.3 range of the district average computer/student ratio.

Implementation of Instructional Technology in the Classroom

The auditors observed a total of 138 classrooms in 14 schools. During these brief visits, auditors made note of students utilizing technology in the classrooms. In 120 classrooms visited by auditors, students were using technology in 24 classrooms (20%). Technology was noted in classrooms where Galileo testing was taking place. It was also noted that some technology was used in settings as an intervention strategy without direct guidance from the teacher. In some classrooms, technology was used as a lab, with all students using technology completing Imagine Learning, an intervention strategy program selected for use in the district.

Auditors observed passive use of technology by students rather than students engaging in critical thinking and problem solving, thus indicating that instructional technology in the PESD #1 is not being utilized to increase cognitive rigor. From observation data and conversations with district personnel, there was no evidence of a common understanding of best practices for infusing technology in classroom instruction.

A summary of the auditors’ observations related to the use of classroom instructional technology by teachers is noted in Exhibit 5.4.3.

Exhibit 5.4.3

Observed Teacher Use of Instructional TechnologyPhoenix Elementary School District #1

May 2017

4%

76%

26%

0% 0% 0%5%

0%10%20%30%40%50%60%70%80%

Elm

o/do

cum

ent p

roje

ctor

Whi

tebo

ard/

Smar

t Boa

rd

Com

pute

r/iPa

d/C

hrom

eB

ook Cel

l pho

nes

Scie

ntifi

c ca

lcul

ator

Elec

troni

c re

spon

se sy

stem

Oth

er (p

leas

e sp

ecify

)

Auditors’ noted the following from Exhibit 5.4.3:

• Of 138 classrooms observed, instructional technology was being utilized by teachers in 86 classrooms (62%).

• Use of the Smart Board was the predominate technology used, at 76%. Auditors noted that in almost all classes the Smart Board was used as a screen for projecting an image similar to an overhead projector (substitution).


• Computer use in 26% of the classrooms was primarily for teachers to refer to notes or linked to the Smart Board.

• In 5% of classrooms, videos were being shown to the class and are included in the “Other” category.

During classroom visits auditors examined technology use from the perspective of its impact on teaching and learning. Instructional technology use was categorized using Puentadura’s SAMR model that is described in Exhibit 5.4.4 along with the percentage of classrooms in which the auditors observed particular teacher use of technology. Auditors followed the SAMR model for classification since district personnel had indicated that the model was being utilized in the district for technology integration into classroom teaching.

Exhibit 5.4.4

Teacher Use of Technology Based on SAMR ModelPhoenix Elementary School District #1

May 2017

Category Description# of

Classes (n=49)

% of Obs.

Classes

SubstitutionComputer technology is used to perform the same task as was done before the use of computers. No functional change. (E.g., Teachers use a Smart Board as an overhead projector.)

41 84%

AugmentationComputer technology offers an effective tool to perform common tasks. Some functional benefits. (E.g., Students take a quiz using a Google Form instead of using pencil and paper.)

7 14

Modification

First step over the line between traditional and transformed classroom work. Common classroom tasks are accomplished through use of technology. There is significant functional change in the classroom. (E.g., Students are asked to write an essay around a theme. An audio recording of the essay is made along with an original musical sound-track. The recording is played in front of an authentic audience such as parents.)

0 0

Redefinition

Computer technology allows for new tasks that were previously inconceivable. Functionally, common classroom tasks and computer technology exist not as ends but as support for student-centered learning. Collaboration becomes necessary and technology allows such communications to occur. (E.g., A class is asked to create a documentary video answering an essential question related to important concepts. Teams of students take on different subtopics and collaborate to create on final product. Teams are expected to contact outside sources for information.)

1 2

Source: Classroom observations

Of the 49 classrooms where SAMR classifications were conducted, the following can be noted from Exhibit 5.4.4:

• Substitution, or no functional change, was observed in 84% of classrooms.

• One classroom used technology as redefinition to make significant functional changes in teaching.

Teachers were requested to complete an online survey in which one question inquired about the level of support provided in written curriculum documents to guide technology use. Fifty-eight percent of teachers indicated that technology is not referenced in curriculum documents provided by the district.


During interviews, auditors heard many comments about the state of technology in the district, included comments about unclear expectations for technology use, a lack of professional development for teachers, and computers being used primarily for testing purposes at the expense of classroom instruction. A sampling of comments about technology use expectations can be found below:

• “There is no requirement for teachers to use the software for completing report cards.” (Central Office Administrator)

• “There is no technology use required in the teacher evaluation.” (Central Office Administrator)

• “There is no expectation for technology use in my classroom.” (Teacher)

• “[We] share computers across three middle school classes. Would like to use computers every day, every class. Sharing prevents fluidity and completion of ongoing projects. (i.e., you can’t use Google classroom if you don’t have a computer.) Sometimes it’s easier to just plan a unit without, knowing that more than likely when you need the computer for a solid week you won’t have access.” (Teacher Survey Response)

A lack of professional development for technology use was also cited:

• “There are no opportunities to train teachers unless the principal invites you to the school. So instead we print what to do and send it upon request [for technology use].” (Central Office Administrator)

• “Our office [MIS] is flooded with questions about how to use the grading system, but we know many reports cards go out with missing grades.” (Central Office Administrator)

• “We need trainings for staff on how to use the systems [software].” (Central Office Administrator)

• “We need technology teachers on campus.” (Teacher Survey Response)

• “We have plans for a virtual tech camp this summer for teachers.” (Central Office Administrator)

• “Technology should be integrated into classes, not in labs that won’t be used.” (Campus Administrator)

• “The board decided to have classrooms of the future: new furniture, plugs, whiteboards, etc. I visited a school in October and returned a month later. Nothing was going on with the technology because there was no training for teachers. It still hasn’t happened.” (Board Member)

• “When a school gets technology, professional development needs to be provided by the district.” (Teacher)

• “We have a diverse staff at different ability levels [with technology] and need PD for them. We need the Education Technology person to help us. We need more of them.” (Campus Administrator)

Comments related to using technology for testing are below:

• “We are invested in devices, but mostly for remediation and testing.” (Central Office Administrator)

• “Testing takes weeks in our school because we do not have enough computers for all to test at the same time.” (Campus Administrator)

• “During testing weeks, non-testing classes cannot have use of the computers.” (Campus Administrator)

Summary

Although the board has committed extensive funds for the purchase of technology, Phoenix Elementary School District #1 does not have a technology plan that meets audit criteria. No clear direction is provided in either board policy or planning documents to express intent for technology use in the district, especially with instructional technology. Where technology is available for instruction, its use is limited to “substitution” for antiquated ways of teaching. Professional development has not been provided to train teachers to use instructional technology. Its use has been left to the discretion of the individual teacher (see Recommendation 8).


V. RECOMMENDATIONS OF THE CMSI CURRICULUM AUDIT™ TEAM FOR THE IMPROVEMENT OF THE PHOENIX ELEMENTARY SCHOOL DISTRICT #1Based on the four streams of data derived from interviews, documents, online surveys, and site visits, the CMSi Curriculum Audit™ Team has developed a set of recommendations to address its findings shown under each of the standards of the audit.

In the case of the findings, they have been triangulated, i.e., corroborated with one another. In the case of the recommendations, those put forth in this section are representative of the auditors’ best professional judgments regarding how to address the problems that surfaced in the audit.

The recommendations are presented in the order of their criticality for initiating system-wide improvements. The recommendations also recognize and differentiate between the policy and monitoring responsibilities of the board of education, and the operational and administrative duties of the superintendent of schools.

Where the CMSi audit team views a problem as wholly or partly a policy and monitoring matter, the recommendations are formulated for the board of education. Where the problem is distinctly an operational or administrative matter, the recommendations are directed to the superintendent of schools as the chief executive officer of the school system. In many cases, the CMSi audit team directs recommendations to both the board and the superintendent, because it is clear that policy and operations are related, and both entities are involved in a proposed change. In some cases, there are no recommendations to the superintendent when only policy is involved or none to the board when the recommendations deal only with administration.

Audit recommendations are presented as follows: The overarching goals for the board and/or the superintendent, followed by the specific objectives to carry out the overarching goals. The latter are designated “Governance Functions” and “Administrative Functions.”

Recommendation 1: Establish system-wide expectations, direction, and system guidance through updated and new board policies and administrative regulations to support systemic quality control of the educational program and system operations.

A comprehensive set of board policies and administrative regulations is necessary to establish clear direction for the system, to guide the management of a system, and to express the expectations and intentions of the board legally charged with governance of the school system. Current policies provide updated frameworks for program operations and help create educational focus for ongoing decision making at district and school levels. Policies are relied upon to be a source of reference for campus management in the recurring issues and operational decisions to promote consistency of administrative practices and cohesion of organizational functions.

Auditors found that the current board policies and administrative regulations for the Phoenix Elementary School District #1 do not provide adequate direction or communicate clear expectations for either educational program management or related administrative and operational functions (see All Report Findings). Some policies represent generic educational expectations without giving clear direction. The policies require few elements of a data-driven planning function: an effective curriculum management system, a comprehensive student assessment and program evaluation system, professional development planning, or data-based decision making (see Findings 1.2, 1.3, 2.1, 3.1, 3.2, 3.3, 4.3, 5.1, 5.2, 5.3, and 5.4).

The auditors’ recommended actions address the primary needs in the area of policies as identified through audit analysis. Additional recommendations in this report identify specific areas of policy weaknesses as well. These actions need to be addressed during the next six to 12 months to establish clear parameters for operations and job performance, and to communicate expectations regarding the follow-up actions based on this report.

Governance Functions: The following actions are recommended to the Governing Board of Phoenix Elementary School District #1:

G.1.1: Prepare and present, for review and adoption by the board, drafts of new policies or updated and revised policies that will meet the criteria outlined in Exhibits 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6 (see Finding 1.2) and


address policy deficiencies pointed out in the findings and accompanying recommendations within this report. Address these revisions as a priority in order to establish clear direction and control for educational program management and sound operation of the campus and its schools.

G.1.2: Prepare, for adoption by the board, drafts of policy updates and revisions that clearly delineate the line of authority from the board through the superintendent that directs an aligned, rigorous, and system wide curriculum (PreK-8) be delivered to all students in the district. The policy will expect schools to adhere to the adopted curriculum and to determine and utilize the best instructional practices to ensure the learnings of each student in the district.

G.1.3: Establish an ongoing policy review and update schedule to avoid policies being outdated and ignored. Incorporate and include language needed to specify clearly the board’s intent.

G.1.4: Establish a mechanism to ensure all administrators’ understanding of policies and the expectation that policies be followed throughout the district.

G.1.5: Develop draft policy on the organizational chart and job descriptions that states clearly that only high-quality instruction is acceptable and that the primary role of professional employees is to facilitate and improve the instructional program to ensure student learnings.

G.1.6: Direct the superintendent to prepare administrative regulations and guidelines for consistent implementation of policies. Ensure administrative regulations are developed, maintained, current, and disseminated to all school sites with clear expectations for consistent adherence.

Administrative Functions: The following actions are recommended to the Superintendent of Phoenix Elementary School District #1:

A.1.1: Assist in implementing G.1.1 through G.1.5 above. Provide draft policy language that offers specificity and clarity of expectations where needed to meet the audit criteria in Exhibits 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.2.6, and other findings within the audit report. At a minimum, these revised, updated, or new policies should include:

• A philosophical statement of curriculum approach that the curriculum should be centrally designed and delivered.

• A policy requiring an aligned written, taught, and tested curriculum for all subjects at all grade levels and a multi-grade scope and sequence document covering all grade levels of the taught curriculum (see Recommendations 3 and 4).

• A policy on instructional expectations that includes the types of methods and strategies expected in classrooms and is based on teacher appraisals, and district and school priorities and goals (see Recommendation 5).

• An expanded policy on planning that outlines areas of expected planning across the system (e.g., curriculum management, professional development, student assessment and program evaluation, technology, and budget development) and incorporates the criteria from the respective findings in this report.

• A policy that requires planning, monitoring, and evaluation of all programs and intervention efforts on a systematic schedule/calendar and the use of resulting evaluation data in program and budget decisions (see Recommendations 6 and 10).

• A policy requiring the development, presence, and annual updating of job descriptions for all positions in campus employ (see Recommendation 9).

• A policy requiring annual updating of the organizational chart, with job descriptions for all positions represented on the chart (see Recommendation 9).

• A policy to differentiate clearly between decisions that are made at the school, those made at the central office, and those that are shared, thus centralizing decision making for better oversight of district functions.


A.1.2: Provide copies of updated policies and administrative regulations to all administrators, with additional copies available for staff at the work sites. Include updated policies and administrative regulations on the district website within six months to enable easy and ready internal and external access to the most current policies and regulations.

A.1.3: Include discussion of updated policies and regulations in administrative meetings as revisions are completed, highlighting particular areas of policy at the regular meetings; monitor for consistent implementation at all sites.

A.1.4: Cite references to policies and administrative regulations when making administrative reports and recommendations.

A.1.5: Monitor adherence and compliance with newly developed job descriptions to ensure consistent fulfillment of roles and responsibilities for all district employees.

A.1.6: Develop administrative regulations outlining the process and procedures to be followed in the development, revision, deployment, and use of board policies.

Recommendation 2: Develop a comprehensive system to support the effective delivery of curriculum in Phoenix Elementary School District #1. Clearly define expectations for instructional planning, classroom-based approaches, monitoring, and professional development to provide a clear picture of engaging, challenging, and effective instruction.

Supporting instructional delivery is a critical part of promoting high expectations for students, achieving deep alignment between the written and taught curriculum, and providing teachers, particularly inexperienced teachers, with support in selecting the best ways to teach the assigned objective(s). Flexibility should be allowed in how teachers approach a particular objective, but instruction should be adjusted for each child according to the data regarding that child’s academic needs. A well-developed, district-adopted instructional model provides teachers with the steps and processes to follow in how to use data at the individual level to plan differentiated instruction. Differentiation includes strategies for re-teaching, sheltering content to ensure access for English Language Learners, and enrichment and acceleration for gifted students, in addition to adjusting the contexts of instruction to allow students variation in how they engage in learning activities. The instructional model represents the district’s philosophy regarding how instruction should be adjusted to meet individual needs, as demonstrated by assessment data.

In addition to the instructional model, effective districts provide a framework for instructional strategies for teachers and campus leaders that outlines the type of approaches the district considers effective and most beneficial to its student population. These would typically include strategies that are effective with economically, linguistically, and ethnically diverse students. Such a framework, in conjunction with the instructional model, not only supports teachers in planning effective, student-centered instruction, but also supports principals in monitoring the delivery of curriculum and creates a focus for professional development. Professional development is the critical support piece in equipping teachers to deliver the curriculum effectively in accordance with district expectations.

The auditors found no direction in policy, job descriptions, or plans of the district’s expectations for an instructional model although there was some direction in the teacher evaluation instrument regarding the types of strategies teachers should employ in delivering instruction (see Finding 3.2). There were no expectations for monitoring found in district documents and classroom visits, although frequent in some schools, were not consistent across all buildings and the purposes of monitoring were dependent on individual principals (see Finding 3.2). In visits to classrooms, the auditors found instructional practices were mostly whole-group in nature, differentiation of content was infrequently observed, and the cognitive challenge of student activities was low (see Finding 3.2).

Auditors found inadequate coordination and fragmented professional development at the district level. Most professional development offered is intended to support the implementation of a new program. The auditors found no evidence of long-range planning to direct professional development and connect it to curriculum delivery (see Finding 3.3).


In this recommendation, specific action steps related to instructional delivery, monitoring, and professional development are addressed in separate sections. These actions should be completed in one to three years.

Instructional Delivery


G.2.1: Direct the superintendent to review research-supported instructional strategies that are effective with all student populations (such information is available from CMSi). Require this review of research to focus especially on those characteristics that have been shown to improve student achievement, such as vocabulary development and cognitively engaging instruction (see Appendices D and E).

G.2.2: Direct the superintendent to develop policy that requires an instructional model(s) to be adopted in classrooms throughout the district.

G.2.3: Direct the superintendent to regularly evaluate the effectiveness of the delivery of curriculum across the district and present a report to the board. Such an evaluation should use data from multiple sources: formative assessments, summative assessments, all monitoring data from the principals, and from samples of student work collected from classrooms on a regular basis.

G.2.4: Adopt the policies described above when drafted; direct the superintendent to ensure their implementation.


A.2.1: Direct the development of the policies described above.

A.2.2: Assure effectiveness in curriculum implementation. Although teachers must have discretion regarding specific strategies and approaches they use to delivery instruction, teacher discretion must be balanced within an established district expectation for both data-driven instructional planning and best practices for ethnically, economically, and linguistically diverse students. Train teachers in the new curriculum documents and support them in using those documents to plan and deliver their instruction. Assure that the curriculum is used in a context that prioritizes student needs above all else—the most effective instruction is responsive to the students at an individual level and is a crucial part of effective reading instruction at the elementary level.

A.2.3: Define the framework for strategies and the instructional model expected to be used in classrooms across the district. These are not intended to be a prescriptive, tightly-held requirement. Rather, the framework for strategies and the instructional model are intended to provide a clear picture of the steps district leaders expect teachers to take in planning their instruction in accordance to data. These steps should encompass the following:

1. Framework for Strategies/Approaches: Describe the ways in which district-adopted curriculum is expected to be delivered. In other words, the types of teaching practices district leadership expects to see and that are proven effective with economically, ethnically, and linguistically diverse students should be specifically described in writing and adopted in policy to ensure implementation. Strategies are loosely held, but this is intended to outline those strategies and approaches the district considers congruent with the district-adopted philosophy of teaching and learning. Suggested practices should be research-based, developmentally appropriate as well as relevant, and might include:

• Ensuring that the learning objective and language objective are evident to students and that the students understand what they should be able to know and do.

• Implementing higher-order questioning that helps students see the “big picture” of the concepts, knowledge, and skills being taught, as well as facilitating a deeper understanding on the part of students.

• Differentiating instruction to meet the individual needs of all students.

• Using small group activities, paired tasks, and cooperative learning strategies.


• Using sheltered strategies, such as SIOP1, to provide English language learners and students with low vocabulary ranges access to core curriculum and to support their academic English language development across all content areas.

• Comparing/contrasting new concepts, knowledge, skills, with concepts, skills, and experiences already familiar to students.

• Engaging students in experimental inquiry, problem-solving, and investigation—all hands-on methods of applying or discovering new knowledge and concepts.

• Having students set their own learning goals, develop strategies for attaining them, and monitor their own progress toward meeting those goals.

• Engaging students in meta-cognitive activities, whereby they analyze their own thought processes in approaching test questions, assignments, new information, etc..

• Using non-linguistic ways to support comprehension of, identification with, and the retention of new concepts or knowledge, such as pictures, graphic organizers, outlines, etc.2

• Tailoring instruction to the cultural, economic, and linguistic diversity present in every classroom, recognizing and valuing differences and similarities, and emphasizing the benefits of cultural and linguistic pluralism.

(See also Appendices D and E for additional suggestions for effective instructional strategies.)

2. Instructional Planning and Monitoring of Learning: Clearly describe expectations for how teachers are to use student performance/achievement data to plan instruction based on their specific academic needs and delineate the steps in this process.

3. Consider using the Mastery Learning Model: as a possible model for planning and executing instruction using a variety of strategies and approaches with which the teacher is comfortable. The Mastery Learning Model presents a model of close monitoring of student learning that is data based and relies on flexible, small student grouping to deliver the exact teaching that those students need, rather than relying on whole group, one-size-fits all approaches. This model is presented in Exhibit R.2.1.

1 See www.CAL.org for more information on the SIOP model.2 For more information, see Downey, C., English, F., Steffy, B., Frase, L., & Poston, W. (2003). Fifty Ways to Close the Achievement Gap. See also Marzano, R., Gaddy, B. & Dean, C. (2001). What Works in Classroom Instruction. May be downloaded from http://www.mcrel.org/topics/products/110/


Exhibit R.2.1

Mastery Learning Model

Learning Objectives

Assessment and

Analysis

Instructional Strategy Selection

Information and

ExamplesPractice Evaluation

Feedback Reteach Enrichment

Monitoring and Recordkeeping

Instructional Planning Instructional Practice Evaluation

Require the monitoring of curriculum delivery to include monitoring for these teaching strategies and practices expected to be used in the classroom. The aim is to provide teachers with specific feedback regarding what type of strategies they were using, their effectiveness, and how that strategy could have been more effective or how perhaps another could have been used to improve student achievement.

A.2.4: As part of the instructional model, incorporate the expectation for differentiating instruction in the classroom to meet individual student needs. Differentiation occurs in two important ways: differentiating the content or objective an individual student needs to learn based on where they are in the overall sequence of learning, and differentiating the type of activity or performance product the student is expected to accomplish or create. Both types of differentiation are important, but teachers must learn the difference and apply one or the other or both as needed with each individual child, based on the individual child’s need. A critical part of differentiating effectively is by having a battery of skill-specific diagnostic assessments that give teachers key information on whether a student has mastered a targeted concept or skill.

A.2.5: Communicate widely the expectations for adherence to the framework and instructional model. Integrate throughout all discussions and meetings concerning curriculum delivery the need to not only verbally espouse high expectations for all students and respect and appreciation for cultural, ethnic, linguistic, and economic diversity, but also to model it faithfully in every classroom every day.

The definition and adoption of a framework for research-based, student-centered strategies and approaches with an instructional model that assures differentiation to meet students’ academic needs will not only assist the district in moving forward with improving instruction and student achievement, but also provide a consistent district-wide focus for instructional delivery.

Monitoring

Monitoring is the primary means by which district leaders evaluate the degree to which the district curriculum is delivered with fidelity, and that the instructional model is likewise reflected in classroom activities and instruction. Monitoring is an absolutely critical facet of effective implementation. It is about supporting and facilitating quality and effective curriculum delivery, not just looking for it. No matter who is involved in monitoring (it can be carried out by multiple positions within a building and even by teachers amongst themselves); the principal should still remain the instructional leader on the campus.



G.2.5: Revise board policy to include specific expectations for monitoring instruction. These expectations must:

• Define all purposes of monitoring.

• Specify who is monitoring for what and how those responsibilities are interconnected. (For example, if site coaches share in monitoring responsibilities, how/when are their findings or observation data shared with the principal? What kind of feedback should they share with district-level curriculum staff? How is this to occur and how frequently? Ensure that the building principal remains the key instructional leader in the building and require him/her to oversee all monitoring that occurs by other staff members.)

• Specify what type of data are to be collected for each purpose and with what methods.

• Indicate which data are intended to be collected district-wide for district-level feedback (such as for determining the effectiveness of a professional development initiative), and which data are to be used for teacher evaluation, coaching, and instructional improvement within the building. All monitoring data may be reported to a single grade, rather than split across all grades. Instructional walk-through data is about collecting information regarding the effectiveness and alignment of the delivered curriculum, not an evaluation of teachers, so this should be seen primarily as a curriculum-related function.


A.2.6: Revise the principal and associate superintendent’s job descriptions to include more specific expectations for monitoring, in accordance with the policy changes directed by G.2.5. Require monitoring to be the primary responsibility of the principal in keeping with his/her role as the instructional leader. In monitoring, district leaders should not only keep the learner objectives and effective strategies in mind, but the instructional model as well, focusing reflective questions on those aspects of the model the administrators deem appropriate or desirable.

A.2.7: Revise walk-through observation tools and evaluation procedures to be consistent with the newly adopted framework for strategies and the instructional model.

A.2.8: Once the new instructional model has been incorporated into regular classroom practice, consider two other purposes and types of monitoring that supplement the non-supervisory classroom walk-throughs: gathering trend data with a classroom observation tool to see if dominant teacher and student activities, the objectives taught, and the student work displayed all reflect the district’s instructional model, differentiation, best practices, and expectations for rigor; and a protocol such as Examining Student work, which involves periodically collecting samples of student work and analyzing them for alignment and rigor. These methods for collecting data are for different purposes and all comprise one facet of monitoring that contributes to valuable feedback for decision making.

A.2.9: Revise the job description of principals to include a requirement for regular, informal classroom visits (walk-throughs). Revise job descriptions of central office supervisors to model and support site-level classroom monitoring. Ensure that the goal of monitoring is to mentor, nurture, support, and coach.

A.2.10: Provide training, coaching, and support for principals and central office supervisors in the district adopted instructional model, the alignment between instruction and the written curriculum, techniques for assessing the alignment between the taught and written curriculum, reflective monitoring of instruction, and effective methods for and coaching.

A.2.11: Conduct annual training in informal classroom visits (walk-throughs) for new and current administrators, supervisors, teacher leaders, department heads, coaches, and interested teachers.


A.2.12: Determine with the leadership team the district expectation for the amount of time in the instructional day that principals and co-administrators should be engaged in monitoring the delivery of the curriculum on a daily and weekly basis.

A.2.13: Designate expectations and procedures for central office administrators’ engagement in monitoring classrooms with building administrators. Central office administrators should schedule their site visits and mentor principals and site co-administrators in the classroom monitoring process.

A.2.14: Establish curriculum monitoring as the primary responsibility of building administrators and supervisors in support of their roles as instructional leaders. Administrators should be trained to provide follow-up reflective conversations with teachers and site administrators.

A.2.15: Provide initial and periodic training for district administrators with current instructional best practices relative to various content areas and grade levels. Ensure training is aligned with the district professional development plan.

A.2.16: Develop and implement an evaluation strategy to determine the effectiveness and impact of administrator classroom visits on student learning and teacher effectiveness. Such an evaluation should use data from multiple sources: formative assessments, summative assessments, monitoring data from principals’ and district supervisors’ walk-throughs, and formal teacher appraisals.

A.2.17: Commit adequate resources to support professional development in curriculum development and monitoring to assist staff in designing and delivering a high quality curriculum for every student.

A.2.18: Annually evaluate the effectiveness of curriculum delivery in terms of achievement for every individual student and student subgroups. Clearly identify personnel responsible to implement this step.

Professional Development

The purpose of a quality professional development program is to increase staff effectiveness and student achievement. This is accomplished by developing the skills of teachers, administrators, and support personnel in effective design and delivery of the curriculum. Professional development is a key factor in ensuring the alignment of the written, taught, and tested curriculum. A comprehensive professional development program is based on district goals prioritized and implemented over a stated period of time. Special emphasis must be placed on training teachers and principals to employ instructional strategies that meet the needs of every student and to implement the adopted instructional model to support differentiation and student-responsive teaching.

A comprehensive professional development program requires regular evaluation of the professional development approaches and content to determine if student achievement has improved based on the trainings provided to teachers. An effective professional development program is also linked to the teacher appraisal program designed to provide teachers with constructive feedback to improve classroom performance.

An effective professional development program is guided by a comprehensive plan that provides all staff with the knowledge and skills to be productive and move toward the long-range goals of the system. Professional development that is centrally coordinated with other district and school plans is powerful when it is data-driven and focuses on organizational change.

Effective professional development programs offer a variety of delivery models coupled with intensive follow-up and support and are inclusive of all staff. Follow-up activities include meaningful practice of what was presented. High quality programs are ongoing and utilize research-based strategies to guide improvement. Monitoring professional development strategies helps to effectively measure the success of the training in improving student achievement. Successful professional development is measured against student learning growth, not merely on the tallying of attendance or completion of training activities.

The auditors found that design and delivery of professional development is inconsistent, fragmented, and inadequately planned. Professional development lacks systemic monitoring, coordination, and evaluation to support campus and program achievement goals (see Finding 3.3). The recommendations of the auditors should be implemented within one to three years.



G.2.6: Direct the superintendent to develop policies that establish clear expectations for the purposes and goals of professional development in the district: to support the effective delivery of the district curriculum. In policy, direct the development of a long-rage professional development plan that includes the components outlined in A.2.10.

G.2.7: Adopt a policy that defines roles and responsibilities to centralize and coordinate professional development efforts in relation to:

• Selecting, planning, coordinating, implementing, and evaluating professional development activities in relation to student achievement.

• Requiring that professional development activities be directly linked to the teacher appraisal system. Require that training be evaluated in terms of student learning and growth of professional educators in the delivery of the curriculum.

• Assessing professional development needs based on data-driven analysis of student performance.

G.2.8: Direct the creation of a comprehensive long-range professional development plan. The plan must extend at least three years with annual updates and revisions to ensure connectivity to system priorities. Include the following in the plan:

• Policy that directs professional development activities that are aligned to the district’s strategic plan.

• A process to provide for organizational, district, school, and individual professional development in a systematic and systemic manner.

• Inclusive of all employees.

• An expectation that each supervisor is a staff developer of personnel.

• An expectation that assessment of professional development activities is data-driven.

• A focus on research-based approaches and activities that are documented to show increases in productivity.

• A process to provide for the three phases of the change process: initiation, implementation, and institutionalization.

• A human learning, development, and adult learning approach.

• A variety of professional development approaches.

• A provision to require follow-up, coaching, and on-the-job application to ensure improvement.

• An evaluation process that is on-going, includes multiple sources of information, focuses on all levels of the organization, and is based on actual behavior.

• A process for campus-wide coordination and a process to perform clearinghouse functions.


A.2.19: Assist the board in the development of the recommended policy in G.2.1 that establishes district professional development standards and expectations aligned to the identified training needs. Ensure that professional development is planned and mandatory for all professional staff, including all administrators.

A.2.20: Develop administrative regulations that describe the procedures and guidelines to implement all adopted professional development policies and activities district-wide.


A.2.21: Assign the responsibility to develop a comprehensive, long-range professional development plan for review and approval to the appropriate district administrator. A feasible number of priorities should be established within timelines.

A.2.22: Approve, monitor, and evaluate all district and school-level professional development activities.

A.2.23: Align all staff training with the district curriculum, district and campus improvement plans, student achievement results, as well as performance evaluation data of instructional staff members.

A.2.24: Incorporate goals for professional development that work in concert with the curriculum management plan and serve to support the strategic plan. Professional development goals should be updated (minimally) annually to ensure and maintain alignment with any changing priorities or conditions. This plan should include the following:

• A framework for integrating professional development activities with the mission and plans of the district.

• Expectations for professional growth for all employees.

• A process to provide for organizational, school, and individual professional development in a systematic and systemic manner.

• A process to provide for the three phases of the change process: initiation, implementation, and institutionalization.

• Incorporate in all professional development plans, whether campus-based or district-wide, sufficient provisions for in-depth training, follow-up or on-the-job assistance over time to ensure that professional development is being applied correctly and results in changes in practice.

• Define expectations for administrative monitoring and feedback to staff regarding implementation of new training in the classroom. Provide sufficient, targeted staff development for administrators to carry out this monitoring function, including training not only on the new skills teachers are learning, but also on how to observe classroom implementation of the new strategies and how to provide growth-producing feedback to improve performance.

• Hold teachers, building administrators, and support staff accountable for attending, utilizing, and monitoring the effective implementation of the training through their annual performance evaluations.

• Implement an evaluation process that is ongoing, focuses on all levels of the school district, includes multiple sources of information, and is based on actual behavior documented in the classroom.

• Overall, the focus of professional development should be on the improved delivery of curriculum and its monitoring to assure student learning is maximized.

A.2.25: Develop and present an annual report to the board on the status and outcomes of trainings based on student performance results including:

• An overview of the major learning activities offered at both the district and site levels.

• A review of the identified professional development needs, including what student learning needs were identified.

• A review of the major learning outcomes or specifically what the district and school sites were striving to accomplish from the training activities.

• A categorization showing the alignment of learning activities in relation to performance goals.

• A review of the evaluation procedures used to measure the effectiveness of professional development activities in relation to the planned teacher and student outcomes.

A.2.26: Identify the source(s) of funding for professional development activities and ensure that funding for professional development is incorporated in the annual performance-based budgeting process.


A.2.27: Include personal, individual professional development activities designed as part of employees’ professional improvement plans in the professional development plan. Enhance personal improvement plans to include support and encouragement for teachers to pursue National Board Certification.

A.2.28: Align the professional development plan with the district strategic plan and campus improvement efforts.

A.2.29: Develop supervisory job descriptions to include responsibilities for the implementation and evaluation of the impact of professional development efforts on improving teacher instructional skills and behaviors as measured by improvement in student achievement.

A.2.30: Develop administrative regulations outlining the processes and procedures to follow to coordinate the training efforts of district and site coaches and mentors outlined in the district professional development plan to ensure alignment, consistency, and continuity. Ensure supervisory responsibilities are assigned in the job description of solely one district level administrator for planning, coordination, and evaluation.

A.2.31: Engage in training-of-trainers opportunities to advance school-wide sustainability and reinforcement of priority instructional strategies on a regular basis. Develop in-house mentors and coaches to support teacher implementation efforts.

A.2.32: Include in the professional development plan an induction program for teachers new to the district including:

• Orientation to the district’s educational philosophy and the status of the district’s strategic plan.

• Orientation to Phoenix Elementary School District #1 (including a tour of district facilities and campuses and the district’s local communities).

• Detailed explanation and training on using the district’s curriculum guides for each teacher’s grade level and/or subject.

• Training on the district’s assessment plan/program to include both local and state formative and summative assessments.

• Training on the use of district adopted textbooks and resource materials as tools in the instructional process.

• Presentation and training on specific instructional strategies recommended in district-adopted curriculum guides.

• Discussion regarding the district’s instructional philosophy and planning.

• Presentation and discussion regarding the district’s culture of student and parent services.

• Training of participation in the schools’ Professional Learning Communities (PLC).

• Review of district operational processes and procedures, services provided for students and teachers, educational and intervention programs, and access to district and site coaches and mentors.

• Orientation and training on access to and use of the district’s technology (instructional, operational, software, etc.).

• Assignment of a mentor/coach for support throughout the first year of employment.

Implementing the steps and expectations outlined in this recommendation will require several years to implement fully, in part due to the time involved in development a high-quality written curriculum for all content areas. However, work on instructional expectations should begin immediately and reflect a district philosophy based on respect for every child, appreciation for Phoenix families and their diversity, and a belief in student-centered approaches. These values and expectations should be congruent with the new strategic plan and should establish and reflect Phoenix Elementary School District #1’s commitment to excellence in student learning.


Recommendation 3: Develop and implement a comprehensive curriculum management system that coordinates and focuses all curriculum management functions and tasks across and within departments and schools. Deeply align current district curriculum to the new Arizona Standards; redesign and direct curriculum revision to ensure curriculum documents are of the highest quality and deeply aligned in content, context, and cognitive rigor with district and high stakes assessments.

Sound curriculum management has many components, including:

• A board-adopted curriculum aligned with student needs and state performance requirements, covering all courses and subjects taught.

• A means for conveying the curriculum and performance requirements (usually curriculum and/or pacing guides) to teachers, along with performance expectations for teachers.

• Professional development that equips teachers to implement the curriculum in accordance with the board’s performance expectations. This includes support in the classroom to ensure that training and curriculum materials are properly used.

• Monitoring of classroom activities by principals and other supervisors to identify and propagate productive practices that support learning, correct or eliminate practices that do not, and identify professional development needs.

• Appropriate and timely formative and summative assessments of students and evaluation of programs to identify the extent to which board goals are being achieved and areas where teaching and learning need to be modified to improve achievement and provide feedback for decision making and planning.

Every student deserves quality schooling. This is achieved through a teaching and learning process that is systematic and organized for systemic implementation. The curriculum is designed using a step-by-step process and the design is delivered throughout the district. A district with an effective curriculum management plan focuses its time, energy, and appropriate resources to guarantee equitable access for every student in the district. The goal is always increasing students’ achievement and thereby their opportunities for success.

The systematic process includes quality curriculum documents based on a written, taught, and tested curriculum. The curriculum document is aligned in content, context, and cognitive type. A document that meets the audit standards of quality are aligned with the written, taught, and tested curriculum in a way that is consistent with external assessments. It informs teachers what skills and tasks to teach and how they should be delivered in the classroom so that students will master the objectives and can be successful on those assessments.

A quality curriculum ensures that learning is intentional, rigorous, focused, and that all students have access to the same learning. It has a format that is consistent across grade levels and includes the key components of objectives that have clarity and specificity, assessments that match district and state performance evaluations, prerequisite skills and knowledge needed for new learning, instructional resources and tests that match the objective, and specific classroom strategies for each objective taught. The process also includes an established curriculum review cycle and evaluation of the curriculum on a consistent basis.

The audit provides 15 components for an effective curriculum management plan (see Exhibit 2.1.3). The auditors used these components to assess the district’s approach to curriculum management. There were no components rated as adequate during the auditors’ review.

There is some evidence that curriculum planning has occurred in Phoenix Elementary School District #1; however, there is no comprehensive curriculum management plan currently in place. The associated documentation provided to the auditors included some scope and sequence materials for some courses. However, the scope and sequences offered were inadequate to serve as direction for teachers to follow in delivering rigorous and focused instruction.

The auditors found the scope of the curriculum inadequate (see Finding 2.2). Absent curriculum guides, teachers are left to develop curriculum individually, which results in inconsistent curriculum being delivered to students. Teacher survey responses, when asked “What do you use to guide instruction,” showed that 86.5%


design instruction from their own ideas and resources and 83.3% went online to develop their own curriculum. Curriculum monitoring is inconsistent throughout the district. Teacher survey responses, when asked “How often do the following people visit your classroom to observe instruction,” the response category indicated that principals visited daily or weekly 36% of the time.

Phoenix Elementary School District #1 needs to design and implement a comprehensive system for curriculum management that coordinates curriculum management functions to include the design of curriculum, effective delivery of a rigorous curriculum, process for revision of the curriculum, and consistent evaluation of the curriculum. The focus must be to develop quality curriculum documents for core subject areas that ensure alignment and depth of content, have a format that is used throughout the district, and is consistent. Curriculum documents for all non-core subject areas that use the same format as the core areas will also need to be developed.

Auditors found no one comprehensive board policy requiring planning for curriculum management (see Findings 1.3 and 2.1). District administrators need to develop a comprehensive plan, directed by board policy, to guide the design and delivery of the curriculum, systematic assessment of student achievement, and timely data-driven program evaluation (see Findings 2.1 and 4.1).

The scope of the K-12 written curricula is inadequate in that not all subjects are supported by board-adopted curricula (see Finding 2.2).

Phoenix Elementary School District #1 curriculum guides do not give adequate information to teachers for effective delivery of the curriculum (see Finding 2.3). Internal consistency, which measures the congruence among design elements in a written curriculum and degree of cognitive complexity of the district objectives and assessments, was inadequate to direct teaching of the written curriculum and support attainment of district expectations expressed in the Phoenix Elementary School District #1 Strategic Plan: Our Blueprint for Student Success.

There are many opportunities for teachers to be involved in professional learning through a variety of different venues and in various content areas. However, there is no consistent district-wide focus or coordination.

Elements of professional development planning were found, but a district-wide professional development plan that includes the criteria for a comprehensive plan (as listed in Exhibit 3.3.1) does not exist in Phoenix Elementary School District #1 (see Finding 3.3 and Recommendation 2).

Auditors’ observational data gathered while visiting classrooms and submitted student work examples indicate that instruction and student activities across schools do not meet district expectations expressed in the Phoenix Elementary School District #1 Continuous Improvement Plan or the district strategic plan (see Finding 3.2 and Recommendation 5).

Auditors found that the Phoenix Elementary School District #1 lacks adequate planning for student assessment. In the context of the limited scope of assessment, Phoenix #1 lacks a consistent approach and focus on utilizing student achievement data at all levels of the organization. The district lacks adequate formative and summative assessment tools for teachers to use during instruction, and formal program evaluations that can identify the effectiveness of academic programs in the district (see Findings 4.1, 4.2, 4.3, 4.4, Recommendations 4 and 11).

Based on their findings, the auditors recommend the development and implementation of a comprehensive curriculum management system that is focused on a planned approach to every aspect of curriculum design and delivery—its development, implementation, monitoring, evaluation, and revision—so that learning for all children is maximized. To put such a system in place and support these functions, the auditors recommend the following actions to the Phoenix #1 Governing Board and Superintendent of Schools. This implementation is best accomplished within a one to five years period and must be an essential focus for the district.

These steps will help district leaders prioritize the work that needs to be done and focus all involved personnel on common goals, thereby rendering the attainment of those goals more likely. The recommended steps are organized into the following sections:

I. Curriculum Management Planning

II. Curriculum Design and Development


I. Curriculum Management Planning

The district needs a cohesive and comprehensive plan that directs the management of a quality, deeply aligned curriculum and its effective implementation in every classroom. Such management includes monitoring its delivery to maintain equity and the district’s philosophical and instructional priorities, and evaluating its effectiveness, using the deeply aligned formative, progress monitoring, and diagnostic assessment tools. This plan should be developed in congruence with the Strategic Plan and should assure that the complex interworking of departments within the district is both efficient and effective in achieving district goals.

This plan must integrate and coordinate professional development across the schools, specify and support identified methods (and purposes) for monitoring curriculum delivery, and reinforce the model for instructional delivery. These processes and procedures must be formalized and institutionalized in policy to ensure smooth transitions in the event of staff turnover and to facilitate orientation of new staff during future years of growth and expansion in the communities served.


G.3.1: Direct the superintendent to draft a curriculum management planning policy for board review, revision, and adoption that provides direction for the development, implementation, monitoring, and evaluation of curriculum. Require regular reporting to the board on curriculum effectiveness, and include the following:

• A requirement for the alignment of the written, taught, and tested curriculum;

• The expectation of PreK-12 articulation of learning goals and objectives;

• The expectation that all courses offered be supported by written curriculum guides;

• Formal board adoption of all curricula prior to implementation; and

• A requirement that planning, particularly timelines, within and among departments and schools be aligned to the curriculum management plan.

G.3.2: Direct that the superintendent enforce the requirement of a written, board-adopted curriculum for all subjects taught, starting with core subjects and expanding to non-core subjects.

G.3.3: Direct the superintendent to establish systematic procedures requiring central office staff members to monitor curriculum implementation in schools.


A.3.1: Assist the governing board in developing policies that define the roles of the board, district administrators, and teachers regarding curriculum. For example, the board is primarily responsible for adopting curriculum; administrators are responsible for attending to its development, evaluation, and revision, as well as for overseeing and supporting its implementation; teachers are responsible for delivering the adopted curriculum and sometimes assisting in the writing or reviewing of the curriculum, with support from school personnel, outside consultants, or district administrators.

A.3.2: Develop a curriculum management plan for directing the design, delivery, monitoring, evaluation, and revision of curriculum. The plan should address the following areas (see Exhibit 2.1.2):

1. A philosophical framework for the design of the curriculum:

a. Identify what the underlying beliefs of district leadership are regarding how children learn: What constitutes effective teaching, what is the teacher’s role, what is the student’s role, and what is a district’s role in assuring a student’s learning? Is education a process, a goal, or both?

b. Use this process to specifically identify what the district’s and schools’ respective roles are in providing each child with an education, and establish a picture of what an effective, engaging classroom should look like.


c. Use the philosophical framework as the guiding force behind making decisions regarding curriculum and assessment design and instructional delivery.

2. Timing, scope, and procedures for a periodic cycle of curriculum and resource review/development:

a. Ensure that every content area is addressed and has a written curriculum guide that facilitates effective, rigorous instruction; and that curriculum is kept up-to-date, particularly with changes in state standards or requirements, as well as testing modifications or changes.

b. Such a cycle should establish the timeline for reviewing the alignment, quality, and rigor of adopted resources and materials, and direct their revision or replacement where and when they are inadequate.

c. ALL resources that are referenced by the curriculum should be screened for rigor, availability to all schools, appropriateness, and alignment to district expectations for instruction and student engagement.

3. Stages of curriculum development: Specify the different stages for developing and revising the curriculum. These might include:

a. Backloading and released item analysis; review for alignment with external/target assessments in all three dimensions (content, context, cognition).

b. Assessing the complexity, rigor, and measurability of objectives.

c. Placing objectives in an articulated, PreK-8 sequence that expects mastery of content six to nine months before it is encountered on the state test or other high stakes tests.

d. Developing mastery-level projects and activities (such as any existing formative assessment tools) with accompanying rubrics, materials, and resources for rigor, cultural proficiency/inclusivity, technology integration, and student-centered, active learning.

e. Creating a bank of high quality assessment items and formative/progress monitoring assessment instruments to support differentiated, individualized instruction. See 50 Ways to Close the Achievement Gap3 for more specific suggestions and information.

f. The stages defined in the Phoenix #1 plan must particularly address the way student achievement data, teacher input, and monitoring data are used to evaluate the quality of the written curriculum. Revise the curriculum, accordingly.

4. Staff roles and responsibilities for curriculum management: Who is responsible for what task? How do departments with overlapping responsibilities work in concert to effect improvements in the written curriculum and to better support classroom instruction?

a. This aspect of the plan delineates which tasks are housed where and at what level: which are classroom-based, which are school-based, which are department-based, and which are board-based. (For example, it is the board’s responsibility to determine the content of the educational program, in congruence with state law, and to approve and adopt the written curriculum. It is the teacher’s role to deliver the curriculum, the principal’s to monitor, instructional coach’s and principal’s roles to support teachers in delivering the curriculum, etc.)

b. Curriculum development and implementation should be the sole responsibility of the department of Curriculum and Instruction (staff) (see Recommendation 9).

c. Monitoring of classroom activities should be the primary responsibility of the principals, with support from other designated positions (such as instructional coaches and central office supervisors) to identify and promote productive practices that support learning, correct or eliminate practices

3 For more information, see Downey, C., English, F., Steffy, B., Frase, L., & Poston, W. (2003). Fifty Ways to Close the Achievement Gap.


that do not, identify weaknesses or gaps in the written curriculum, and determine professional development needs.

d. Clarify how monitoring and curriculum support responsibilities of any school-based personnel complement one another to prevent duplication of effort or possible conflicts in carrying out these supportive responsibilities.

5. A format and included components for curriculum guides: For consistency in every content area, specify the components to be included in the curriculum that are nonnegotiable and the other aspects that are “fluid.” The curriculum should include the criteria presented in Exhibit 2.3.2 in addition to supporting assessments and tools to enable differentiation and implementation of the district’s instructional model in the classroom.

6. Direction for how state standards will be included in the curriculum: Decide whether or not to use a backloaded approach in which the curriculum is derived from high-stakes tested learnings (topological and/or deep alignment) and/or a frontloaded approach, which derives the curriculum from the Arizona Standards (Blueprint), but in a refined, more specific format (but in a refined, more specific format). Of critical importance is the emphasis on condensing and streamlining the standards for feasibility and clarity of focus.

7. Require for every content area a focused set of precise student objectives/student expectations and standards:

a. These should be derived from the new Arizona Standards, be reasonable in number so the student has adequate time to master the content and practice it in authentic, rigorous contexts, be very specific so teachers clearly understand what mastery of these objectives look like and what the standard of performance is, and should be measurable (written in measurable terms).

b. The curriculum should not only specify the content of the student expectations, but also include multiple contexts and suggestions for activities and approaches that engage students in critical thinking and analytical cognitive types.

8. Assessment beliefs and procedures to determine curriculum effectiveness and use of data:

a. What are all the instruments that will be used to measure progress toward meeting goals, including the goal of students mastering curriculum objectives? How will the assessments be scheduled to avoid too much testing in a given testing window, and maximize use of the data that will be derived from administration?

b. How will the data be used, who will use it, how will it be collected, analyzed, and disseminated to teachers, administrators, and concerned stakeholders?

c. There must be an expectation for formative assessments included in the frameworks that teachers can use whenever needed to evaluate student progress in mastering learning targets (or to determine whether they already know content about to be taught).

9. Design of curriculum to support differentiation and other expectations for delivery:

a. Directs the curriculum guides to be designed so that they support teachers’ differentiation of instructional approaches (to match student preferences and learning styles), and to support teachers’ selection of student objectives at the right level of difficulty. This ensures that those students who need prerequisite concepts, knowledge, and skills are moved.

b. Whole group, one-size-fits-all approaches, and reliance on test-like contexts cannot meet the majority of students’ academic needs. District curriculum leaders must define what true academic differentiation and rigor look like and how teachers can manage so many different skill levels and varying content knowledge in the classroom without holding certain students back or leaving other students behind. This is critical to meeting the needs of academically at-risk populations and must


be addressed by the design of the curriculum, in addition to all district documents that describe expectations for delivery.

10. Approaches for using diagnostic, formative, and summative test results to plan instruction, evaluate programs, and design interventions at all levels (see Recommendation 11).

11. A staff development program linked to curriculum design and delivery: Professional development that trains teachers in the curriculum, its design, and how to deliver the curriculum in accordance with the board’s performance expectations is absolutely critical. (This includes support in the classroom to ensure that training and curriculum materials are properly used. See Recommendation 2 for more detailed information.)

12. Monitoring (and supporting) the delivery of curriculum:

a. This presents the procedures, philosophy, and intent for monitoring the delivery of curriculum. Multiple means of monitoring (as well as multiple purposes) are suggested, including the Three-Minute Walk-through (Downey, et al.).

b. The key person to monitor curriculum content, context, and cognitive type; instructional strategies, student engagement and mastery, environments, and appropriate selection of learner objectives for teaching is the principal.

c. Monitoring is a line officer responsibility that needs to be assigned, and with accountability for results, to principals.

13. Communication plan: Establish a plan for communicating among and across departments and levels of the district regarding the process, goals, and products/measurable associated with curriculum design and delivery (including professional development and assessment) to maintain constancy of effort, focus, and continuity.

14. A process for integrating instructional technology into the curriculum: Setting the expectation that technology will be incorporated into classroom settings to enhance student learning.

A.3.3: Make periodic reports to the governing board regarding the progress in managing curriculum district-wide, using data from formative and summative assessments, as well as data from monitoring practices. The importance of quality, deeply-aligned written curriculum that raises expectations for student performance and supports those expectations with critical resources for teachers cannot be overstated; curriculum is a key component in ensuring better teaching and higher achievement. Planning for its development, implementation, and revision is essential to impacting student learning in every classroom.

II. Curriculum Design and Development


G.3.4: Require that efforts to develop the written curriculum begin immediately; require that decisions regarding which content areas receive priority be data-based (for example, if math is an area of concern for so many individuals and there is little consistency in its delivery, focusing on that content area first might be prudent).

G.3.5: Review and adopt the written curriculum that is currently in use, and future curriculum prior to its implementation, based on a thorough consideration of documentation and staff advice.

G.3.6: Direct the superintendent (or designee) to review the concepts of deep curriculum alignment and require that those concepts form the basis for curriculum design efforts across the district.

G.3.7: Develop policies that define the roles and responsibilities of the governing board, district administrators, and teachers regarding curriculum. Incorporate into these policies the responsibilities outlined under the administrative functions section of this recommendation.


G.3.8: Direct the superintendent to assist the board in developing a clear, concise, comprehensive policy framework that stipulates that the written curriculum documents be revised to include at least the minimal basic components described in Exhibit 2.3.1 and:

• A clear framework for the development of curriculum;

• A requirement for deep alignment of the written, taught, and tested curriculum from high-stakes tests;

• An expectation that all courses (including electives) offered within the district be supported by written curriculum documents;

• A process for the integration of technology with instructional strategies and resources. The goal is to enhance student learning; and

• A common format for all curriculum documents across subject areas and grade levels, core and non-core.

G.3.9: Adopt board policy that requires the following minimum components needed in every guide (see Finding 2.3):

• Aligned, specific learner objectives (based on the AZCCRS);

• A scope and sequence defining prerequisites (what specific knowledge and skills students must have prior to entering a particular course or level);

• Assessment instruments and sample test items;

• Instructional resources; and

• Suggested strategies and approaches for teaching the objectives.

These are only minimum components; internal alignment, evidence of rigor, clear descriptions of mastery, and clear connections to an aligned formative assessment system are all additional elements to be considered for quality guides.

G.3.10: Direct the superintendent to align professional development to support teachers’ quality delivery of the curriculum.

G.3.11: Direct the superintendent to establish budget development procedures to ensure that district learning and planning priorities are reflected in budgeting and spending and are communicated to staff and the public-at-large.


A.3.4: Require that efforts to revise and refine the written curriculum begin immediately.

A.3.5: Assist the governing board in developing policy that requires the curriculum to reflect the principles and concepts of deep curriculum alignment (see Finding 2.3).

A.3.6: Define what the curriculum still needs to be considered a “model” curriculum. Examine the weaknesses in the format and components identified in Finding 2.3. The following components are minimum requirements:

1. Objectives: The objectives used in the curriculum were from the Arizona College and Career Readiness Standards. They were insufficiently specific to give teachers adequate information regarding what mastery looks like. Objectives should be “refinements” of the state standards: a specific restatement of the intended skill or knowledge to be learned, the contexts in which it is to be learned and practiced, and the standard of performance by which a teacher knows mastery of that skill or knowledge has been achieved. These should all align closely with the state/national standards, but these specific learner objectives give the teacher more precise information of what mastery looks like and clearly define which objectives are assigned to which grade or instructional level (because the first grade objective is clearly different from the second, and so on).


The number of objectives included in the guide must also be manageable. It is better to focus on fewer objectives and address them more deeply than to include an entire of battery of objectives that teachers “might” touch on. Review all objectives for evidence of rigor (see Hess’ Cognitive Rigor Matrix and Curriculum Example in Appendix C), and integrate into the objectives across all content areas expectations for culturally responsive content.

Giving teachers a clear continuum of student learning from preschool through eighth grade allows them to move students ahead at a more appropriate pace, if the students are ready, or to accelerate them if they are behind. This is easier when the teacher knows exactly where a student is on the continuum of learning, knows what content is next in the sequence, and can easily determine what students have mastered when they come into their classroom This is particularly important in districts with high mobility.

2. Assessment: Specific examples of how each objective will be assessed, with what tools, and when must be included in the written curriculum documents. District formative assessments must be cross-referenced throughout, specifying when, how, and with which instrument each objective will be evaluated. Relying on released test items or commercially produced assessments or unit/chapter tests is insufficient; the sample items to be included should be items based on deconstructed, released test items that have been altered and “deepened” to provide students with a challenge level ensuring their success on a multitude of test items related to the same content (English and Steffy, 2001). Additional diagnostic assessments are needed to supplement the ATI Galileo® assessments, which serve as a minimal benchmark but lack cognitive rigor (see Finding 2.3). Teachers must have tools with which to continuously evaluate student progress and move them at the appropriate, individualized pace in all content areas. Consider more authentic approaches that integrate into daily instruction and are more project-based in nature, particularly those assessment tools that require writing.

3. Prerequisites/Scope and Sequence: Place the essential standards and learning targets (PreK-12) within a scope and sequence document to allow teachers to easily discern what content and skills students come in with and what content and skills they are responsible for seeing students leave with. Such a document helps distribute accountability and eliminates gaps and overlaps in student learning—an important factor in an educational environment that must make the most of the time allowed with students.

This will facilitate greater articulation of the curriculum from one level to the next and assure greater coordination across a single level or course, as the mapping out of objectives is already completed, and any “misinterpretation” of the nonspecific state standards/student expectations is avoided.

4. Resources and Materials: Every book, recommended professional resource, audiovisual aid, technological enhancement or program, and other resource should be listed (after ensuring teachers have all that are necessary) in the written curriculum and referenced by objective/strategy, after it has been screened for rigor, quality, developmental appropriateness, and alignment with the content, contexts, and cognitive types of the objectives. All suggested materials and resources should have been analyzed for deep alignment to the curriculum and the tests in use; modifications are also included in the guide to improve alignment.

Materials and resources are suggested—as with strategies and approaches, not required—to allow teachers and buildings flexibility in selecting those materials most effective and appropriate for their students. However, in cases with extremely high mobility, adherence to the sequence of units or objectives in the curriculum guide by teachers across schools becomes more important. This consistency in WHAT is taught is critical to ensure better transitions for students moving from school to school (while allowing for flexibility in how the content is taught).

5. Suggested Strategies and Approaches: This item is a critical part of ensuring high expectations for students and achieving deep alignment to provide teachers, particularly inexperienced teachers, with support in deciding ways to teach the assigned objectives.


Flexibility is always allowed in how teachers approach a given objective, but this component provides teachers with invaluable, research-proven suggestions if they want or need them. The suggested strategies should be developed to ensure they incorporate those contexts and cognitive types known to be part of the tests in use, and these strategies and suggested student activities and projects allow students to become familiar with the context and cognitive type before encountering them on the high stakes tests. This is the main tenet of the “doctrine of no surprises.”

However, such strategies should not ONLY align with test contexts. A wide variety of authentic, student-centered contexts is recommended to ensure a more broad-based, real-life application of the concepts, skills, and knowledge so that students can connect personally with the learning, be more actively and cognitively engaged, and see the overall value of their learning.

Currently, the strategies observed in classrooms are of varying quality and rigor, and the rigor of the sample student work was below that required on the state assessments. Classroom-based activities and strategies should always meet and exceed the rigor found on assessments—students should be challenged in the classroom, not by a high stakes assessment.

Beyond these components, the format for the guides should be determined. These do not necessarily need to be identical for all content areas, but within content areas it is recommended that a common format is selected and adhered to for consistency across the district. The degree of variation in curriculum guides is up to district leaders. Use the data from Finding 2.3 in making decisions concerning curriculum design and development.

A.3.7: Direct the staff to revise/prepare written curricula in alignment with the new Arizona Standards in content, cognition, and context for all taught subjects and courses, starting with core subjects/courses and expanding to non-core subjects/courses. Phoenix Elementary School District #1 has evidenced the beginnings of an effective approach to designing a system-wide curriculum. Some of the existing district-developed curriculum documents have a common format across subjects and grade levels, and key structures have the potential for refinement to attain deeper alignment.

Content, Cognition, Context: Content is typically derived from state standards with local augmentation; cognition refers to the type of cognitive processes students engage in when learning content; and context refers to the way learning is experienced, e.g., write, model x based on y, represent, etc.

Example: Students will order and compare (cognition) whole numbers to 1,000 by using the symbols <, >, = (content) in written form given sets of numbers in mixed sequence from 1 to 1,000 (context).

A.3.8: Take steps to assure that all courses have a corresponding curriculum. Ascertain that every board-approved course in the district is included in the official course list. In the official course list, indicate which courses are offered at each building during the current school year (see Finding 2.2).

A.3.9: Engage in a deep alignment analysis (considering the dimensions of content, cognition, and context) to ensure the objectives, resources, and strategies included in curriculum guides are deeply aligned to the tests in use. Research the methods and ideas presented in the book Deep Curriculum Alignment by English and Steffy (2001), or consider contracting for a deep curriculum alignment training (contact CMSi for more information) to gain the skills necessary to analyze and deconstruct released test items, for information on how to successfully prepare for current and future tests in use, and to more successfully anticipate the direction in which the test is moving. This will assist the district in predicting where the state assessments and other external assessments are going and increase student success on current and future forms of the tests in use by ensuring that the content, context, and cognitive types encountered on any tests are an integral part of daily instruction without compromising rigor, active student engagement, and hands-on problem solving.

A.3.10: Develop or refine formative assessments to more deeply align with the AzMERIT and new Arizona Standards, and to more closely reflect the levels of cognition and type of student performance desired by district leaders. Link them to the written curriculum and identify those for which the data will be entered electronically and monitored at the system and building levels. Some formative assessments should be open for teacher selection, but ALL should be rigorous and incorporate a wide variety of contexts—not just multiple choice.


These assessments will provide teachers with diagnostic data on what skills, concepts, and knowledge students have mastered or are still lacking, so that instructional decisions may be made that target those deficiencies and that teaching is never redundant. Include diagnostic assessments that target specific skills to round out the battery of assessments teachers can use to constantly monitor student progress toward mastery of a discrete concept, skill, or objective. All assessments used in the district, whether classroom-based or district-level, should integrate a variety of student modes of response and performance-based items, as well as incorporate multiple types of cognition.

The assessments should be concise and yield the needed information in a very brief span of time—a few days, at the most. Ideally, the assessments could be quickly scored at each campus, so teachers receive the data immediately and can adjust instruction accordingly. In addition, continue to return benchmark assessment data to teachers in a timely manner. These formative assessments allow teachers to monitor every individual student’s progress toward mastering the intended curriculum, and each student’s performance on the state tests will no longer be a surprise or guessing game.

A.3.11: Reflect in the design of the curriculum the expectation that instruction will be differentiated to accommodate individual student needs (academic) and learning styles. This requires suggestions for remediation as well as enrichment within the guides themselves.

A.3.12: In written curriculum guides, also include the following:

• Integrate instructional technology.

• Include strategies for meeting the needs of English language learners, special education students, and gifted students.

During the curriculum development process:

• Obtain feedback from the curriculum review team.

• Use external consultants as “critical friends” to critique the process and products during the design stage.

• Direct personnel to date curriculum document and show the department/external agency and person responsible for the creation or modification of the document.

A.3.13: Establish a process to ensure that all texts, instructional materials, and ancillary resources for all courses that are suggested through the curriculum guides and provided to teachers by district personnel, including interventions and adopted programs, are screened for quality, rigor, and alignment (in all three dimensions) to the curriculum and with district expectations, prior to presenting to the board for adoption. Include the adoption and/or revision date as a recordkeeping measure to note the history of development.

A.3.14: Wherever possible, integrate strategies and approaches that are most effective with English language learners. When these are integrated within all core and non-core courses, it is more likely to become an institutionalized expectation and practice.

Require principals to monitor whether these approaches are implemented in the classroom. Continue to train teachers in these approaches and monitor for their impact on curriculum delivery.

A.3.15: Prepare for curriculum implementation. At least six months to one year prior to rolling out any new or comprehensively revised curriculum do the following:

• Field-test the curriculum. Pilot the resource materials, assessments, and any other supporting materials.

• Collect preliminary data concerning the pilot curriculum’s effectiveness in terms of student achievement.

• Revise field-tested curriculum guides based on feedback.

• Submit the revised curriculum guides for adoption by the board.

A.3.16: Prepare trainings for teachers in using and effectively implementing the curriculum. Issue a directive that planning within and across departments and schools will be in concert with the curriculum management


plan, especially in the area of providing timely professional development necessary for effective curriculum delivery (see Recommendation 9).

A.3.17: Develop and implement evaluation procedures for curriculum guides. Base guide evaluation on measurable and documented levels of student learning. Align the guide evaluation process with the district process for program evaluation (see Recommendation 10).

A.3.18: Develop administrative regulations to govern the development, deployment, implementation, and evaluation of the curriculum planning process.

A.3.19: Develop processes for posting any newly adopted curriculum guide(s) on the district website for immediate access by instructional staff. Include functions that permit and encourage teaching staff to post their assessments and suggestions regarding lessons they have taught.

A.3.20: Make periodic reports to the board regarding the progress in managing curriculum district-wide, using data from formative and summative assessments, as well as from monitoring practices. The importance of quality, deeply-aligned written curriculum for the district cannot be overstated—curriculum is a key element in ensuring better teaching and higher achievement.

In summary, following the steps outlined above will move the district’s written, taught, and tested curriculum in closer alignment and increase the overall expectations for student cognitive engagement, thereby reducing the likelihood that student performance on tests is predicted by demographic factors rather than by classroom instruction. A key element in curriculum quality is maintaining unwavering focus on how design supports and facilitates delivery; written curriculum must not only integrate content and contexts and rigor that are more challenging and deeper than the tests, it must provide teachers with tools they need to teach most effectively in a manageable format. The current Phoenix Elementary School District #1 curriculum requires refinement in deeply aligning to the taught and tested curriculum and to the new Arizona Standards.

Recommendation 4: Develop and implement a comprehensive written formal student assessment plan to encompass proper scope and quality alignment of the written, taught, and tested curriculum to guide instructional planning across the district.

Leaders of school systems plan for student assessments. The data generated by these activities help to determine the level of student achievement. Effective school systems utilize a comprehensive assessment plan that encompasses formative and summative assessment results to effectively plan for the design and delivery of the district’s curriculum and to achieve desired outcomes of programs and interventions.

System-wide student assessment provides district and school staff information that will enable them to make instructional decisions that support improved student achievement. These instructional decisions include lesson planning and delivery in each classroom, the design and delivery of the curriculum, and how effective programs and interventions are implemented in the district. A comprehensive plan for student assessment builds capacity throughout the district for using assessment data.

The auditors found that several formative and summative assessments have been used to determine student progress and to measure other aspects of district performance (see Finding 4.2 and Finding 5.2). Auditors did not find a comprehensive student assessment program to adequately provide data for curriculum and instructional changes to improve student achievement. Auditors found the scope of assessment to be inadequate for core and non-core courses and, furthermore, the designs of assessments were not aligned to support the delivery of the curriculum (see Findings 2.3 and 4.2).

The leadership of Phoenix Elementary School District #1 needs to make the design and implementation of a comprehensive student assessment plan a priority. An assessment plan is a means to guide instructional planning, ensure teachers are informed about student progress, determine staff and program effectiveness, and make vital decisions about district instructional practices and key educational programs.

In order to strengthen student assessment and program evaluation in Phoenix Elementary School District #1, the following steps are recommended to the governing board and superintendent. The superintendent should develop a timeline to ensure completion of these recommendations within one to three years.



G.4.1: Direct the superintendent to draft a policy that sets forth the expectation of formative and summative assessment of the curriculum by course and grade. The policy should provide specific direction to staff regarding the purpose, scope, and approaches for assessing curriculum at all grade levels and subject areas.

G.4.2: Direct the superintendent to ensure that routine benchmark tests and other formative assessment measures are aligned with the State Academic Standards, district curriculum, implemented across the system, and used to inform instructional decisions and monitor student progress at the student, class, grade, school, program, and district levels.

G.4.3: Expand board policy to direct the superintendent to develop an assessment plan to ensure that all components of an effective assessment plan are embedded and that the plan also includes program evaluation:

• Explicit procedures for carrying out formative and summative assessments;

• Frequent diagnostic (formative) instructional student assessments aligned to district curriculum;

• Student assessment and program evaluation disaggregated for effective use;

• Use diverse assessment strategies in all district programs, across all grade levels;

• Clarify the roles and responsibilities of district and school-based staff;

• Specify the connection(s) among district, state, and national achievement;

• Specify the overall assessment and analysis procedures to determine curriculum effectiveness;

• Ensure student assessment examples and tools are included in curriculum guides;

• Specify how equity issues will be identified and addressed;

• Identify the components that are included in program evaluation;

• Ensure appropriate trainings for instructional use of assessment results;

• Establish procedures for monitoring formative and summative assessment and program evaluation from design to implementation to results;

• Establish process for communicating and training staff; and

• Specify the creation of an assessment data system that supports cost-benefit analysis of programs.

G.4.4: Direct the superintendent to draft a revision of board policy that requires qualitative and quantitative data from a variety of assessments and survey sources to be used in all levels of plan development and implementation.

G.4.5: Commit adequate resources to support the development and maintenance of a comprehensive, system-wide student assessment plan.


A.4.1: Assist the board in developing the policies described in recommended actions above.

A.4.2: Upon board approval of the above policies, prepare and implement a student assessment plan containing the following elements. Supporting administrative regulations should include the audit assessment plan characteristics.

• Include a formative and summative assessment system to carry out board policy.

• Specify explicit procedures to carry out regular formative and summative assessment at all levels of the system.


• Provide for frequent diagnostic instructional assessments aligned to the district curriculum, which teachers will use to make ongoing decisions including assigning student objectives at the appropriate level of difficulty.

• Disaggregate student assessment results for effective use.

• Provide a list of assessment tools, purpose, subjects, and type of students tested, timelines, and related information.

• Identify and provide direction on the use of diverse assessment strategies for multi-purposes at all levels of the system.

• Specify roles and responsibilities of district and campus level staff for assessing all functions and operations of the system.

• Specify overall assessment procedures and analysis procedures to determine curriculum effectiveness.

• Specifies how equity issues will be identified and addressed.

• Require aligned assessment examples and tools to be placed in curriculum, instruction, and assessment guides.

• Provide appropriate training for various audiences on assessment.

• Delineate the responsibilities and procedures for monitoring formative and summative assessment design, implementation, and results.

• Establish a process for communicating procedures, results, and trends of student achievement data.

• Specify the creation of an assessment data system that supports cost-benefit analysis of educational programs.

A.4.3: Directs the leadership team to examine benchmark tests and formative curriculum embedded assessments to verify their regular use and to ensure that they incorporate the district expectations for what students are to know and be able to do within the content area and grade level being assessed.

A.4.4: Direct instructional support staff to provide training and support to principals so that they can assist teachers in the use of assessment data to measure student progress and to make classroom instructional decisions that enable them to differentiate instruction while still moving forward in expected curriculum implementation.

A.4.5: Direct the collaboration of instructional support staff, administrators, and assessment and evaluation personnel to ensure that data from benchmark assessments, AzMERIT, and other measures are available to all teachers and building administrators to support instructional planning decisions and other efforts to improve student achievement.

A.4.6: Assign administrative staff, instructional support staff, and assessment departments to work cooperatively in determining the types and formats of data and data reports needed for decision making and planning and for professional development on accessing and interpreting data.

A.4.7: Develop a district master testing calendar to include all formal formative and summative assessments to be administered to students.

A.4.8: Direct the development or acquisition of testing software that will enable the district to provide local benchmark results to teachers and site principals within 24 hours of testing. Results should be disaggregated for easier use and provide item analysis for teacher use.

A.4.9: Provide training and support to principals so that they can in turn assist teachers and other staff in the use of assessment data to measure student progress and make instructional decisions to ensure the academic success of each student, including differentiating instruction when necessary and appropriate.

A.4.10: Direct the collaboration of central office administrators, instructional support staff, assessment and accountability, and instructional technology staff to ensure that assessment data from district, state, and national


assessments is available to all teachers and administrators for instructional planning decisions in a format that aids their decision-making regarding improving student learning.

A.4.11: Develop a process for consistent monitoring of data used by administrators and teachers and attach to administrative evaluation criteria. Require the inclusion of test data analysis and data-based recommendations in all school department and district level reports and budget requests.

A.4.12: Perform ongoing analyses of written curriculum documents and district implemented assessments to control for bias (see Recommendations 3 and 4).

A.4.13: Revise existing job descriptions for administrators and instructional staff to include more specificity and clarity of responsibilities for assessment design, implementation, and evaluation, including data analysis and use in decision making at all levels.

A.4.14: Prepare and present reports to the board on progress toward completing the tasks on the timeline and annual reports on the overall status of the comprehensive student assessment plan.

A.4.15: Provide professional development training for all teachers on the process of deconstructing test items to better understand the content assessed in high stake tests. Teachers can use identified content (and context) in developing mini-assessments for use in their shared PLC collaboration that monitors weekly student growth.

A.4.16: Provide professional development training for all campus administrators in the design and use of assessment tools and data, the analysis of data, and the implementation of instructional change to impact student learning.

These recommendations provide the district with a means of generating the data necessary to assess student progress, analyze testing results, disseminate those results to the appropriate staff members, and ensure that the results are used to make sound decisions related to curriculum, instruction, and supporting programs.

Recommendation 5: Establish a system for monitoring and supporting equity across the district. Ensure that areas of greatest need receive commensurate support with additional financial and human resources. Establish clear direction for ELL and Accelerated Learning Procedures and the MTSS that outlines program expectations and guidelines to ensure equity of access and effectiveness of services. Commit to effectively communicate with parents and proactively build relationships in the community.

Realizing equity in effective schools districts and assuring equal access to programs and services requires a strong philosophical foundation rooted in the belief that schools exist to serve students, that all students are capable of academic success, that no child’s potential can be measured, and expectations should be relentlessly high for all children, regardless of their ethnic, linguistic, cultural, or economic background. Such a philosophy must be expressed in belief statements, summarized in the district vision, and targeted by the district mission. For equity and equal access to prevail, the message from the top must be that the alternative to equity and equal access is not an option.

Equity is about ensuring that students have equal access to not only quality programs and services but also to academic success. Ensuring academic success means providing instruction and resources to students based on their individual needs not on what works for the majority of students or on a formula or standardized procedure. Equity in the world of public education shifts the district focus from what teachers and administrators want to do for students to what the students need the teachers and administrators to do. This student-centered approach requires involvement from parents and is proactive in seeking it: by venturing into the community, into students’ homes, and building relationships with students and their parents and families. District personnel who attain this level of involvement see improved student attendance, improved school behaviors, and higher student achievement.

At the system level, areas of inequity must be monitored and addressed through system-wide efforts such as new policy directives, professional development initiatives, or even staffing changes. Identifying areas of inequity in a district is achieved through data analysis, as well as anecdotal evidence collected from district stakeholders. Areas of inequity must be identified, monitored, and addressed at individual schools through data analysis, classroom monitoring (walk-throughs), teacher evaluations, and building-level planning.


Programs that provide services to large subgroups of the student population must be clearly and specifically defined to assure their effective implementation and monitoring. These programs are essential at providing students with the supports they need for success. This system-level direction then translates into support for the teachers in delivering the services for the ultimate benefit of the child.

In the classroom, teachers monitor equity and assure equal access in similar ways but with a much smaller population. They look at test data by student subgroups, monitor their own instructional strategies and behaviors, ultimately evaluating whether students are making appropriate gains in achievement despite any demographic factors that might predict failure. The focus is ensuring that every child has the support and services he or she requires to be successful, in terms of resources, materials, instructional time, or targeted interventions.

Students from diverse backgrounds have varying needs and come to school with different background knowledge, experiences, and skills. These differences present challenges to educators who are inexperienced or unfamiliar with the differences, particularly differences in cultural understandings, priorities, and perspectives. Educators must learn to recognize and appreciate differences and teach students accordingly, in a student and culturally-responsive way, for students to be actively and personally engaged in instruction. The driving philosophy behind the concept of equity is that each students can attain academic success if provided with this support. There are no exceptions; expectations must remain high for every child and failure is never allowed to be an option. A child who fails to succeed academically is a failure on the part of the system.

The Phoenix Elementary School District #1 is a small, urban district with a diverse student population. Its students are culturally, linguistically, and economically diverse. Certain subgroups of students are not attaining academic success (see Finding 4.1). Some student subgroups are more likely to be suspended or retained (see Finding 3.1), while different subgroups are over- or under-represented in key district programs, such as the Accelerated Learner Program or in special education (see Finding 3.1).

Auditors examined Phoenix Elementary School District #1 documents that describe the programming for English Language Learners. The district’s program is based on state requirements and includes some suggestions for instructional delivery, but goals and expectations for the program are unclear and training inadequate. Guidelines for implementation are not specified. Delivery and monitoring the program has been inconsistent. Auditors found district-wide disagreement over the clarity of the program and its effectiveness.

Direction for the ALPS is also inadequate. The program currently identifies students at a rate less than half the national average. Auditors found no consistent guidelines for its implementation, nor any direction for monitoring (see Finding 3.1).

Multi-Tiered System of Support (MTSS) is the main process to provide interventions for students who are underperforming but the system is based on a flawed assessment instrument not aligned to the standards (see Finding 4.2) and relies heavily on computer-based interventions with limited scope for meeting student needs. Teachers and principals had low ratings for the quality of interventions (see Finding 3.1).

While parents expressed concerns over safety and food issues, they were most concerned about the lack of communication from teachers and schools. Many stakeholders expressed their concern over the district’s lack of success in engaging parents and about the cultural disconnect they perceive across the system. Others commented on the inequitable distribution of resources that are not allocated according to need (see Finding 3.1).

To address the issues and concerns related to equity, improve parental involvement and increase access to programs and services and to increase the effectiveness of support programs in the system, the auditors recommend that the district take the following actions. These actions are grouped in two main categories: those actions that address general equity concerns; and those that address the ALPS and ELL programs, specifically, as well as the Multi-tiered System of Supports (MTSS). These recommendations should be implemented within one to three years.


District-level Equity


G.5.1: Direct the superintendent to revise board policy to address the issue of equity and prioritize equity district-wide. Policy needs to accomplish the following:

• Clearly establish the philosophy that serves as the foundation for assuring equity and equal access in all aspects of district decision making, processes, and communications.

• Define equity specifically in terms that clearly contrast it with equality. Specify when actions are to be equal (access to resources, materials, courses) and when they are to be equitable (fair, just, and different in order to level the playing field).

• Direct the methods to be used in collecting data on equity across the district; specify the instruments, measures, and procedures to be used to identify equity problems and to determine probable causes.

• Require the disaggregation of ALL centrally-collected assessment data by student subgroups and monitor their performance. Direct district leaders to pay close attention to achievement gaps that fail to narrow over a reasonable amount of time, such as two years.

• When problems with equity are evident, require multiple measures to evaluate reasons for achievement gaps; identify the key factors that contribute to maintaining the gap. Determine the suitability of current effort to ameliorate the gaps based on new data.

• When they are within the scope of the district’s control, require that the factors contributing to inequities be targeted and eradicated using whatever means necessary to make changes that will result in ameliorating exiting inequities (e.g., ALPS enrollment or special education identification rates.).

• Establish a district priority for high quality, student-centered instruction that is culturally responsive and supportive of English language development at all times, in every classroom. Refer to the framework for strategies that specifically describe such instruction in the classroom; require teachers to adhere to the model for instruction (see Recommendation 2).

• Institutionalize the importance of equity in all curriculum management functions throughout the district (all planning, monitoring, curricular revisions, curriculum delivery, etc).

• Direct professional development to be deployed on a regular basis to address equity and equal access issues. Evaluate the effectiveness of the professional development initiatives through ongoing data collection and analysis. Consider every subgroup of equal importance.

• Require application-only programs in schools to monitor the SES and ethnicity of their student population and new applicants to maintain proportionality in their enrollment.

• Establish high expectations for every student, regardless of race, income level, language proficiency, gender, etc. Specifically describe how those expectations are to be actualized in classrooms.

G.5.2: Request periodic updates from the superintendent regarding equity across the district using measures congruent with methods for equity data collection defined in policy (as directed in G.5.1). The data collected and reviewed for the reports are intended to evaluate the district’s success in maintaining an equitable educational program.


A.5.1: Assist the board in revising board policies to prioritize equity across the district and level the playing field for all students (i.e., giving every student the same advantages and opportunities in accordance with his/her needs).


A.5.2: Monitor achievement by student subgroup at all levels, using progress-monitoring tools that align to the standards and that provide meaningful data.

A.5.3: Work through the desegregation office and the curriculum department to ensure that students have equal access to all programs throughout the district. Implement intake procedures at schools with special programs that assure proportional representation of all district students in those programs. The students in any program in the district should look like all students in the district.

A.5.4: Require an instructional model and framework for strategies (both curricular and activities) that are centered on individual student needs. The model should reflect the latest research concerning effective approaches and activities for culturally, linguistically, and economically diverse students. Such approaches are typically characterized by individualized instruction at the appropriate level for each child, in congruence with the teacher appraisal instrument and district belief statements.

A.5.5: Train teachers and administrators over a two-year period on the district model of effective instruction (see Recommendation 2).

A.5.6: Direct principals to monitor instruction for evidence of the instructional model and the framework for effective strategies, and require them to monitor test scores for student gains in achievement. Gains means monitoring a single cohort of students over time—from year to year—to ensure their performance is improving. Academic improvement should not be consistent for every child—students who are below grade level must have accelerated instruction and learning opportunities so they make faster gains than other students, to ensure that they do not fall farther behind. (See suggestions under MTSS for guidance in selecting and implementing interventions more effectively.)

A.5.7: Train principals aggressively in effectively monitoring the delivery of curriculum content as well as providing English language development support in every classroom. Mentor principals how to monitor and coach teachers more effectively to improve teachers’ instruction and their students’ achievement.

A.5.8: Supervise principals’ monitoring of classrooms and implementation of the teacher appraisal process, particularly with teachers who are struggling with instructional differentiation and the use of diagnostic assessment data to drive their teaching and planning.

A.5.9: Coordinate all human resource, curriculum delivery, and building administrator functions to prioritize effective delivery of curriculum and instructional quality in PESD #1. Ensure that those buildings with the greatest needs (such as the highest percentage of at-risk students) have the most experienced and effective teachers and principals. Model and maintain an emphasis on meeting student needs and demonstrating high expectations.

A.5.10: Integrate principal supervisory functions with teacher evaluation and monitoring. In addition, teachers with unsatisfactory evaluations should remain in their current building (not be allowed to transfer) until documented problems are resolved. Provide principals with support in coaching and evaluating teachers and, when necessary, removing ineffective personnel.

A.5.11: Focus professional development on identified areas of individual teacher and principal needs. Require staff development for issues that are identified in teacher evaluations and that impede student success.

A.5.12: Beyond offering or requiring staff development for teachers and administrators, require the implementation and monitoring of teachers’ new learnings in the classroom. Collect classroom observational trend data to determine whether professional development is having the desired impact on teaching and learning (see Recommendation 2). Observational data are collected and analyzed in the following areas:

• Dominant student activities observed;

• Dominant student learning arrangements;

• Dominant teacher activities observed;

• Evidence of student work that gives testimony to adherence to the adopted instructional model;


• Evidence of powerful instructional strategies (SIOP);

• Evidence of cultural and linguistic responsiveness; and

• Evidence of cognitive rigor, in both the materials/resources being used as well as in the students’ activities.

Student Support: MTSS

The current system for MTSS is functional in district schools across the system with existing committees and processes in place. The system has strengths and can be strengthened if observed weaknesses are addressed. Weaknesses in the system include:

• Insufficient direction concerning the implementation of MTSS, expectations concerning the nature/scope of assessments to be used to determine student weaknesses, and steps in the process of identifying deeper issues for those students whose learning challenges exceed the capabilities of regular interventions alone.

• Unknown quality/effectiveness of intervention personnel.

• Effectiveness of teachers in delivering differentiated, effective Tier I instruction.

• Relying on the ATI Galileo for identifying students’ gaps/weaknesses.

• Relying on Imagine Learning as a broad-based intervention for the majority of Tier II students.


G.5.3: Develop board policies that require a plan directing the design and implementation of the PBIS and MTSS systems across all district schools.

G.5.4: Require annual reports concerning the effectiveness of PBIS; require the review of disciplinary data as well as survey data from students, teachers, and parents.

G.5.5: Require annual reports concerning the effectiveness of the MTSS program. Require results from at least three different assessment instruments to show growth, gains, or decline.


A.5.13: Develop a plan to direct the design and delivery of the PBIS/MTSS programs district-wide. In this plan, include the following components:

• Clear philosophy for student behavior management and for academic support aligned with the district’s philosophy of education.

• Goals for both aspects of the program: clear, measurable, with assigned person(s) responsible for each, at every level of implementation.

• Direction concerning the tools used to measure effectiveness of both PBIS and MTSS.

○ With MTSS, use multiple measures to determine weaknesses in student learning. Do not rely solely on an electronic, unaligned assessment such as the ATI Galileo.

○ An assessment to determine reading levels and comprehension must be used at primary and reliance of fluency should be only a small percentage of the data used to determine students’ weaknesses.

○ In primary, the use of a tool that requires miscue analyses and the use of teacher observation is a must. Consider Marie Clay’s inventories for pre- and early-literacy skills, and the Fountas & Pinnell Benchmark Assessment System and Running Records. Consider the DRA II if it is available.

○ These tools can be used up through sixth grade with some students and they provide needed data on students’ levels of comprehension, something fluency does not provide.


• For PBIS, set clear expectations and boundaries for behaviors required in schools.

○ Include in these expectations the proactive approach of soliciting parent involvement and engagement through home visits at the beginning of each year and with any students not experiencing success in managing behaviors.

○ The focus of home visits is to establish positive relationships with parents prior to any negative behaviors so families see the school as a resource and support for their child and their family.

○ Home visits after negative behavior are intended to involve parents in strategizing ways to minimize negative or destructive behaviors and partner with the school in assuring their child’s academic and social success.

• For MTSS, develop a list of multiple, effective interventions for students in mathematics and reading that are focused on specific skills and provide those interventions for all schools.

○ Include tools already available at schools, such as the Leveled Literacy Interventions.

○ Ensure that at least two thirds of interventions are not computer-based to provide students with real-time feedback and teachers with targets of the exact skills, concepts, and knowledge these students need.

○ Interventions are more effective when selected based on the exact diagnosed need of the students and when they are delivered by teachers who can monitor their effectiveness immediately and adjust instruction accordingly.

• A schedule for collecting all data concerning program effectiveness and the data sources to be collected. The schedule should provide for the annual report to the board. Data sources should include disciplinary actions, attendance records, parent volunteers and visits to schools, and additional student and parent surveys. Climate surveys to teachers and students are recommended as part of these evaluations.

Overall, in developing the plans for PBIS and MTSS, the focus is on student success—academic, emotional, and behavior. Additional factors should be considered in providing a comprehensive system of supports for students that include integrating social supports and services available outside the school district.

Support programs: ELL

A well-designed program for English Language Learner (ELL) students advances English language acquisition in a timely manner, provides access to core curriculum content, allows for participation in courses of high level, and targets specific outcomes needed to enable students to graduate from high school and pursue goals to be college, work, and career ready. The success of such a program depends on the organizational capacity of the district in terms of a vision for the education of English language learners, the appropriate resources allocated to it, the professional preparation of staff, and the effective management of its design and delivery.

ELL programming in the district was almost solely reliant on state legislation. Greater specificity in program delivery is necessary to ensure that all teachers and building personnel understand what each component of the Structured English Immersion (SEI) model looks like, how to implement it, and that they have the resources and materials needed to present the program. Auditors found inadequate curriculum and resources to support teachers in delivering SEI and to meet the requirements of the Individualized Language Learning Plan (ILLP). Evidence of sheltered instruction to support access to content and of language objectives and instruction were inadequate (see Findings 3.1 and 3.2).


G.5.6: Require the superintendent to present new or revised policies for board adoption to provide a framework for a comprehensive program for the education of ELL students. The framework should require specifications outlined below in A.5.15.


G.5.7: Require the superintendent to submit an annual report to the governing board that includes progress of ELL students from one language proficiency level to another, the number of ELL students meeting and exceeding the state required proficiency level, progress towards meeting state academic standards, access to accelerated courses, disciplinary statistics, and graduation rates.

G.5.8: Commit adequate resources to support effective program implementation and required professional development to meet the needs of English language learners.


A.5.14: Prepare a board policy for board approval as recommended in G.5.6.

A.5.15: Develop a plan for the design and delivery of ELL programming district-wide. The plan should include the following components, some of which may already exist in district documents:

• The district’s philosophy for ELL programming. Review the statement in current documents that articulates the philosophical approach to English language acquisition. Revise, as needed, to reflect the district’s philosophy of education and instructional approach and ensure the communication of the district’s commitment to respecting and valuing the ethnic, cultural, linguistic, and economic diversity in the system.

• A statement of the district’s commitment to language enrichment for ELL and non-ELL students in the form of developing students’ proficiency in more than one language. Proficiency in multiple languages is seen as a foundational skill, globally, and is a preferred focus area for new and future parents. The linguistic diversity in the district is a decided strength in developing a culture of multilingualism and in fostering proficiency in Spanish, English, as well as any other languages.

• Specific goals for the program in terms of students’ language progress, content mastery, as well as their sense of self-efficacy and positive self-image as a language learner. Keep the goals measurable, observable, and specific assigned to specific personnel for execution and monitoring with a timeline for completion. Establish a number of goals that are feasible and aligned to the district strategic plan.

• Provision for the integration of specific objectives related to English language development. These should address minimum vocabulary needs, language structures and syntax, and essential phrases and academic constructs. These ELD objectives should be integrated into EVERY content area to support the sheltered instruction model adopted by the district. ELL personnel should participate and be included in all curriculum development stages.

• Provision for the design of curriculum that explicitly embeds sheltered strategies in the core content areas in addition to language development strategies and approaches.

• Provision for the alignment of district and site programs with the specific objectives of the ELL/dual language enrichment programs, such as interventions used by MTSS or any other site-based adopted programs.

• Provisions that ensure access by ELL students to all core curriculum content, accelerated or Accelerated Learner courses, the Montessori program, and academic English language development.

• A schedule for the professional development of instructional, non-instructional, and administrative staff in terms of the effective delivery of services for ELL programming and dual language development. Connect this training to the focus on the framework for strategies and instructional model presented in Recommendation 2.

• Expectations that all instructional resources are available to all teachers in the district at the required proficiency levels, be research-based, and analyzed and evaluated for their appropriateness, rigor, quality, and alignment with the district-adopted sheltered instruction model and curriculum, such as SIOP.


• Provisions that delineate roles and responsibilities for all professionals who have a responsibility to administer the design/delivery of ELL services.

A.5.16: Define approaches to formal and informal assessment and provide guidance on the use of data for diagnostic purposes, to monitor progress, and to inform intervention for all ELL students. Establish transition plans for students moving from elementary grades to middle school grades who have not met expected proficiency levels in English.

Parent Outreach and Engagement

The auditors heard multiple concerns from a variety of stakeholders about the lack of parental engagement and the district’s need to establish firmer relationships with the community and parents in certain locations.


G.5.9: Direct that all job descriptions for office-based personnel, including school liaisons, be reviewed for accuracy and appropriate chain of command. Ensure that all job descriptions include the requirement that all parents and students are to be welcomed in full accordance with the district’s philosophy and beliefs (good customer service). Require the appraisal process for these personnel to include performance evaluation on this criterion.

G.5.10: Include in board policy an expectation for a proactive approach to developing relationships with parents at every school.

G.5.11: Require an annual report from the superintendent regarding trends in parental engagement and involvement district-wide.


A.5.17: Establish expectations at the district level regarding how every student, parent, and community member is expected to be greeted and welcomed at every school site. Communicate the commitment to being a community institution of friendliness and respect in every neighborhood. Establish a high quality expectation of customer service.

A.5.18: Assign responsibility for the professional development of office personnel to the appropriate district level administrator. Train all office personnel (and supervisory administrative personnel) in appropriate interactions with parents and students. Direct principals to evaluate the effective implementation of these expectations and conduct a survey of parents and students regarding school climate. Those schools with extremely positive climate ratings should be highlighted and allowed to present to their colleagues the “secrets of their success.”

A.5.19: Define the role of the school liaisons. Include in their job descriptions the role of social service liaison, as well. Ensure that they are kept up-to-date regarding internal and external social services and supports that students and their families may request that exceed what the school itself can provide.

A.5.20: With the beginning of each school year, establish the requirement that all teachers, liaisons, principals, and other identified personnel (such as school resource officers) make visits to the families or homes of students in pairs or trios.

• If parents are uncomfortable having school personnel in their home, arrange to meet them at their place of employment or a neutral location.

• Use translators as needed, with appropriate translating protocols.

• As stated in A.5.13, the purpose of home visits is to pro-actively and positively engage parents in a discussion about their child and how the school can best support their child’s learning and development.

• Use this time to communicate to parents the district’s (and the school personnel’s) commitment to serving their child successfully in accordance with district philosophy and beliefs.


The time and effort needed to accomplish this daunting task is offset by the undoubted improvements in student attendance, academic success, and behaviors. Parents are more likely to come to school for subsequent conferences, presentations, and events when they already have had a positive interaction with their child’s school. Such approaches are already in use by charter schools; this is the district’s opportunity to communicate to the community its own programs and resources.

A.5.21: Establish a parent welcome and student intake center for students new to the district.

• A welcome center can provide not only information on registering for classes but also work with families to lower mobility within the district and connect them with needed social services at the first point of contact.

• A welcome center can identify any language issues with incoming students who do not speak English or Spanish.

• Alleviates the burden of processing registrations from school sites who can then focus effectively on their current students and families.

The steps outlined above will distinguish Phoenix Elementary School District #1 as the foremost provider of a positive, student-centered and family friendly education in the valley. Given the anticipated shifts in population and the competition always offered by charter schools, these actions promise to increase enrollment and improve students’ and parents’ satisfaction with their education while positively impacting student achievement.

Recommendation 6: Develop and implement an expectation among all district staff that data are to be used to improve the district’s curriculum design, curriculum delivery measured by improvements in student achievement results, and to evaluate based on a quality designed formal assessment system.

The quality of any educational organization is influenced by the presence of an assessment process that facilitates informed decision making about levels of productivity and the quality of the curriculum. As a critical element of sound curriculum management, assessment is comprehensive in nature and guides the district as it considers instructional needs. To establish a comprehensive picture of the progress of students, formal assessment should occur in all subjects (core and non-core) and at all grade levels. Effective districts use readily available formative and summative assessment data to diagnose problems in its curriculum design and delivery system. Assessment measures include formative and summative assessment, student performance data. Effective districts use assessment data on an ongoing basis to evaluate services, programs, and instructional strategies.

Auditors found district wide student assessment data results are below state average in English language arts and mathematics, and show significant gaps in various subgroups. The design of assessment to support effective delivery of the district curriculum is inadequate to support student learning and success on state tests (see Finding 4.1 and Finding 2.4). Data analyzed by district personnel is based on summative state testing results even though other mechanisms and tools are available for assessing learning both formatively and summatively. Assessments found in the district such as the Galileo assessment are not aligned with the state standards and are prevalent in use for staff evaluation purposes rather than student achievement to measure the effectiveness of the district’s curriculum and instruction.


G.6.1: Direct the superintendent to include in a local policy that includes the development of formative, diagnostic assessment instruments are aligned to district curriculum and are administered to students regularly to provide teachers with information for instructional decision making.

G.6.2: Direct the superintendent to revise job descriptions to include the responsibility for developing the scope and monitoring of the development and implementation of formative district assessments (benchmark testing for all core subjects and grades).



A.6.1: Assist the board in developing the policy described in action G.6.1 and G.6.2 above.

A.6.2: Revise existing job descriptions to include responsibility for development and monitoring of formal assessments for the district.

A.6.3: Design and implement a comprehensive, system-side formative assessment framework that includes:

• Formal formative assessments for all curriculum objectives that are available for teacher use in determining initial acquisition of learning;

• Informal formative assessments for all objectives that are in place for teachers to use prior to teaching an objective to determine prior student mastery;

• Informal formative assessments for all appropriate course/grade level objectives that are available for teachers to use prior to teaching an objective to determine pre-requisite knowledge; and

• Pools of informal assessment items for all objectives that are available for teachers to use during instruction to diagnose the current status of learning.

A.6.4: Provide for the monitoring of the formative assessment framework. Components to be included in the monitoring would include:

• Presenting teachers with formative student achievement data for incoming students at the beginning of the school year. Data from the prior year(s) assessment should be organized so that each teacher has historical achievement data by objective on each of his/her students.

• Identifying for the teacher individual formative student data on each objective for level of achievement and where each student is within those levels each time formative district assessments takes place. Data should include group and/or subgroup levels of achievement for a given objective.

• Presenting the individual formative student achievement level by objective connected to a district’s schedule of objectives or pacing chart.

• Presenting longitudinal data for each student by class roster and specifies the required gain needed to close the achievement gap within three years.

• Identifying pre-teaching formative assessments to use for individual student diagnosis for one or more years. Subsequently, this allows teachers to determine when students are making desired progress over time.

A.6.5: Assess the extent to which the available formative assessments align with the AzMERIT assessments in the grade levels and content areas that are assessed by the state. If the available formative assessments provide a match to AzMERIT items in content, context, and cognitive type, require use of these formative assessments by teachers and hold principals accountable for monitoring student progress using these formative assessments.

Recommendation 7: Design and implement a comprehensive, curriculum-driven budget development process that emphasizes cost-benefit analyses that links resources to instructional priorities. Update long-range facility planning, providing evidence of the tangible connections between the district’s educational goals for students and the facility needs within the plan.

Linkages between the budgets and programs that lead to predetermined priorities, goals, and strategies for improving student achievement are critical to the district’s overall success. Intended results are lost or delayed when there is not a thorough, systemic process to ensure that the financial plan represents the district’s learning priorities. To allocate resources without comprehensive evaluation of results ignores the annual opportunity to strategically re-establish priorities and aggressively pursue intended results with new direction. In the absence of such comprehensive budgeting practices, system-wide effectiveness is often a matter of chance and special political interests than of intentional design.

Auditors found the district’s budget remains stable, however, decreasing enrollment expectations and the undertaking of facility renovations are two variables the district has identified as possibly negatively impacting


district finances over the coming years. The auditors found that the system’s budgeting process historically has been lacking in critical steps and elements that provide connections from data to decisions and from allocations to results (see Finding 5.1). The lack of effective cost-benefit processes has resulted in an inability of the district to determine the effectiveness of programming weighed against the program cost. In short, productivity of programs cannot be determined.

Essentially, the Phoenix Elementary School District #1 needs to refine and revise budgeting processes to ensure that the district accomplishes the district’s mission to deliver quality learning experiences to its student clientele. The recommendations that follow are aimed to help the system cope with the phenomenon of decreasing enrollment, facility renovation, and rising academic expectations.


G.7.1: Direct the superintendent to prepare for board consideration a new policy and revised related job descriptions to provide overall direction for budgeting using criteria noted in Exhibit 5.1.4 and A.7.1.

G.7.2: Direct the superintendent to establish procedures and prepare and/or revise documents that communicate the budgeting process and goals throughout the system and requires budget and staffing proposals to reflect a direct connection to established data-driven priorities.

G.7.3: Direct the superintendent to set a budget development timeline to incorporate procedures identified in A.7.3.

G.7.4: Require, as part of the budget development process, a presentation from the administration to communicate how the proposed budget addresses the goals and priorities and responds to student and program evaluation data. The presentation should include an evaluation based upon measurable criteria of the effectiveness of the previous year’s budget in achieving district priorities and those programs and interventions that are being revised or terminated on the basis of lack of effectiveness or lack of specificity in their objectives or intentions. This presentation should specifically include student achievement results connected to the budget proposals.

G.7.5: Direct the superintendent to establish a three-year plan that, when implemented beginning with fiscal year 2018-19, will lead to successful implementation of actions G.7.1 through G.7.4.

G.7.6: Require that long-range facility planning include clear linkage of the facility needs and planned actions with the educational program priorities and student needs reflected in school and district improvement plans. Communicate expectations that the documents prepared for board, staff, and public information include narrative demonstrating these linkages and clear explanation of parameters for decisions on capital projects.


A.7.1: Draft and propose the policy identified in G.7.1 and revise related job descriptions to include specific budget and related financial responsibilities in time for implementation with the 2018-19 fiscal year.

A.7.2: Establish procedures and prepare and/or revise budget documents that communicate the budgeting process and the goals the system is attempting to address, and require budget and staffing proposals to reflect a direct connection to the established data-driven priorities and assessment results (G.7.2).

A.7.3: Set the budget development process and timeline (G.7.3) to ensure that the budget planning processes are focused on specific, time-bound, and measurable goals. Clear connections must be established between the student performance information and the basic instructional and support areas of the budget. Undertake steps similar to the following to increase the connection of programs and priorities with budgeting decisions4:

1. Using the current construction of the budget, identify various educational activities or programs and group them into areas of need or purpose served. This would include a more expansive inclusion of sub-components found in the current budget.

4 For more information, see Poston, W. (2011). School Budgeting for Hard Times—Confronting Cutbacks and Critics.


2. Assign a budget/program manager to each component. Direct them to prepare a concise and meaningful budget proposal for their respective areas. Insure that campus level budgets are fully accountable to the policy identified in G.7.1 and the requirements described in A.7.3. Otherwise, the district is further subject to fragmentation, which may contribute to unequal results in achievement.

3. Goal statements need to be attached to each program area or budget request to state the program’s linkage to established goals and priorities, its purpose, the criteria for identifying success, and specifically how results will be evaluated and reported (the actual evaluation of the program’s effectiveness should be physically attached to each budget package request).

4. Each request should be described so as to permit evaluation of the consequences of funding or non-funding in terms of performance results. It is essential that this element be added to the current budget planning process. Teachers and principals must participate in the actual budget decision-making process to assure that valid data are used, knowledge of actual practice is available, and instructional efficacy is served.

5. Compile the goal/linkage statements and budget packages and give them to appropriate staff to gather data that best describe needed service levels, program outcomes, and cost-benefits.

6. Define program performance expectations and accountability for each program area with the involvement of various stakeholder staff (including principals, teachers, and support staff). Current results should be compared to desired expectations and related service level requirements. For example, to be successful, a specific program may need to be established at 110% of previous spending levels or at 75% of previous spending levels. Changes in funding may necessitate a comparable reduction from some other program or allow an increased allocation for another program judged to be of greater consequence.

7. Each program manager must create three to five program alternatives that deliver an adequate and workable program at different levels of allocation.

8. Prepare guidelines and recommendations and give them to the Director of Budget and Finance, who will then combine all recommendations into a single budget proposal listing funding by program increments and corresponding line items for each incremental package.

9. Compile past cost information, especially expenditure percentages of budget, with performance data and recommendations to guide preliminary budget estimates. Assessment and documentation of previous program results are essential.

10. Appoint a budget planning team, representing the various stakeholders, that will eventually bring the draft budget documents to the superintendent to study the goals, priorities, and parameters inherent in the decisions being made for program funding. Discussions of cost-benefit information are critical at this stage. Where needed, budget plans should be extended over multiple years to assure consistency of effort and focus (G.7.5).

11. The budget planning team evaluates and ranks the budget packages. Budget requests need to compete with each other for funding based upon data derived from evaluation of the priorities of need and level of program effectiveness. For example, specific academic curriculum standards should be set regarding students’ optimum access to and mastery of technology. Present inventories should be evaluated to determine the gap between current availability and access and what is judged as being optimum. Once completed, a budget package would compete with all other expenditures. To ensure equity, an individual school’s ability to generate its own grants or gifts should be considered in this equation. The result will be that budget decisions are made deliberately on the basis of highest priority rather than by default.

12. Compile results of the evaluation and ranking and publish them in a tentative budget with programs listed in priority order. Use this draft with administrators for input before a draft is prepared for use as the presentation document.


13. Finalize budget allocations based on available revenues, the appropriation levels to be authorized, and program funding priorities and rankings. Prepare the recommended budget to be taken to the governing board for final evaluation and ranking.

14. Use the public hearing process to communicate broadly the financial planning link with student needs, program priorities, and the results sought through the actions taken. Allow time for individual comment and questions. Prepare the final document after considering public information and board decisions.

15. Establish final program and services to be funded at the level approved by the governing board and set the budget in place.

A.7.4: Design the budget management process to allow for an acceptable variation (such as a plus or minus 3 to 5%), permitting program managers sufficient stability to achieve the desire results. Budget revisions should only occur when acceptable variations have been approved; failure to do so would violate board bylaws, sound accounting practices, and/or place the district financial jeopardy.

A.7.5: Provide training and consultation as needed to all affected staff members during the transition to a curriculum-driven budgeting process and format. In addition, all district and campus level personnel who have accounting or program evaluation responsibility should be required to demonstrate competency in their respective duties in order to avoid frustration and inefficiencies that occur when such competencies are not present.

A.7.6: Direct all leaders with responsibilities in the long-range facility planning process to respond to the direction in G.7.6 above and prepare documents congruent with that expectation. Components of the plan should include the following:

• Philosophy statements that review the community aspirations and the educational mission of the district and their relationship to short- and long-range facilities goals.

• Enrollment projections that take into account any known circumstances that may cause significant change in pupil population.

• The current organizational patterns of schools and an identification of possible organizational changes necessary to support the educational program as the district grows and as aging facilities become too expensive to maintain.

• Identification of educational program needs (see Findings 4.3, 5.2, and 5.4) to be considered by designers of capital projects for renovation or addition of school facilities (e.g., space for specialty grouping of students, technological infrastructure, lab requirements, etc.).

• A detailed evaluation of each existing facility, including assessment of structural integrity, mechanical integrity and efficiency, technology capacity, energy efficiency, operations and maintenance, and health and safety requirements.

• Prioritization of needs for renovation of existing facilities and the provisions for additional facilities.

• Cost analysis of potential capital projects to meet the educational needs of the division, including identification of revenues associated with capital construction.

• Procedures for the involvement of all stakeholders of the county-wide community in the development and evaluation of the long-range facilities plan.

A.7.7: Assist the facility planners in preparing public information related to facility needs and their alignment with educational needs and priorities, as well as with the district goals.

A.7.8: Include the expanded facility planning information in all presentations to the governing board and the public related to plans for expansion, remodeling, or replacing current buildings.

A.7.9: Continue emphasis among all staff of quality maintenance and custodial care for all buildings.


With an approach to budgeting based on individual program costs, results, and performance, the governing board and superintendent will be better equipped to monitor both finances and program effectiveness simultaneously. It is important to note that such a system cannot be implemented hastily. Needed policies and related job descriptions should be completed in the next six months. Budget package descriptions, if prepared, can remain in place for the FY 2017-2018. Evaluation components are added to each package as the district collects and interprets meaningful student achievement data, which should improve each year and be fully implemented in three years. Given this approach to budgeting, changes in funding or allocation levels are truly based on, “How well are students doing?” instead of, “How much did we spend last year, or how much do we think we may need?”

The Phoenix Elementary School District #1 needs a credible rationale and an effective system for appropriating and/or reallocating finances, especially from unproductive programs and activities to new, emerging programs or activities of high priority based on organizational effectiveness, changing needs of clientele, or produced results. Moreover, valid linkages need to be identified among organizational objectives, results, and costs in the process of improving quality control and system prudence with its financial resources. Such linkages will provide evidence to support program increases and/or decreases each year.

Recommendation 8: Develop, adopt, and implement policy to support the use of technology, both instructional and management, across the district that enhances student learning and staff productivity. Clearly identify expectations for technology usage in classrooms and provide adequate professional development and oversight to ensure its implementation and links to student learning.

Technology planning in effective school districts leads to technology implementation that supports deeper and more meaningful learning in district. The systematic, efficient integration of technology into curriculum and instruction serves as a tool to increase student achievement. Technology also serves important roles in business and management functions. Appropriately funding and directing the use of technology throughout the school district is an essential part of effective management and control. A written plan that outlines expectation, goals, guidelines, and evaluation protocols for the use and integration of technology is an effective means of ensuring consistent implementation across the district.

Auditors found that the Phoenix Elementary School District #1 lacks a technology plan that meets audit criteria. Board policies and district job descriptions are limited in relation to guidance and responsibility related to technology for the improvement of student learning. Auditors also found that while technology is available in most classrooms, use of the technology is either limited or rudimentary.


G.8.1: Direct the superintendent to prepare for review and adoption a policy that outlines the criteria for use and implementation of technology at the district and campus levels.

G.8.2: Require through the policy described above in G.10.1 that a schedule be established for reporting evidence of the effectiveness of technology use to the board and other decision makers. Require that effectiveness be calculated in terms of impact on student achievement.

G.8.3: Establish an annual reporting cycle for technology for administrators to present results that include student performance data linked to the goals and objectives of the technology as well as recommendations to continue, modify, or terminate the practices.

G.8.4: Approve funding for instructional technology based on completed needs assessment, information regarding alignment with the curriculum, student performance data, and the criteria in G.8.1.


A.8.1: Draft the policy recommended in G.8.1 above and present to the board for adoption.


A.8.2: Establish criteria and create a plan for developing, adopting, implementing, and monitoring technology use that is aligned to district priorities and student learning goals and is responsive to student assessment data. The following components should be included:

• Insure board policy or administrative regulation for instructional technology exists which is congruent with G.8.1.

• Develop a clear statement of program philosophy/vision which aligns with other philosophy/vision expectations across the district, including the district improvement plan, campus improvement plans, professional development plan, and curriculum management plan.

• A comprehensive view of technology exists for both instructional technology and staff level management.

• A needs assessment has been completed and evaluated.

• Measurable student goals and objectives exist which align with student learning expectations.

• An ongoing student assessment component exists which identifies student technology usage and links to learning objectives.

• An ongoing program assessment component exists and is used make informed decisions about the effectiveness of the instructional technology program.

• There are comprehensive staff trainings with measurable standards for equipment, application, and technology.

• School site equipment standards exist.

• Internet access standards exist.

• The role of the school library is stated.

• An implementation budget has been identified.

• A maintenance budget has been identified.

• Technology site plans are aligned with district plans.

A.8.3: Require that district curriculum documents include recommendations for the use of instructional technology (see Recommendation 8).

A.8.4: Establish a plan which allows technology to be utilized for student assessment purposes, yet least interferes with technology availability for classroom instruction across the district/building.

A.8.5: Align technology use in classroom for instruction purposes with teacher instructional expectations. Consider incorporating classroom technology use as part of the teacher evaluation protocol.

A.8.6: Provide professional development for program administrators and principals on identifying effectiveness technology use in the classroom.

A.8.7: Require the teachers utilized management technologies (e.g., Synergy) and provide necessary professional development as needed to insure its completion.

The implementation of the above recommendations will advance the instruction and management technology program within the district. Such progress will increase productivity of district personnel, and also enhance instructional models to more closely align with the expectations expressed in district planning documents.


Recommendation 9: Revise, adopt, and implement job descriptions and the organizational chart to meet audit criteria, establish clear lines of authority, and improve the clarity of functional relationships.

A district’s chain of command is reflected in its organizational chart. It defines the role relationships between supervisor and subordinates, outlining a scalar relationship between district officials. Adherence to the adopted chain of command ensures that the authority of the board is channeled through the superintendent to all employees in the district.

Job descriptions are clearly written explanations of duties and qualifications of persons employed by the school district. They provide employees with information regarding what background is necessary to successfully prepare for the job and how positions are to function within the organization, including assignment of supervisory relationships and the critical components of the job. A clear set of job descriptions supports the district’s internal and external communications by explaining who performs what duties within the organization and how various positions interact to accomplish the board’s expectations. The board must adopt all job descriptions to further ensure adherence to board policy and directions. To communicate graphically the responsibilities and functional relationships within a school system, a table of organization (organizational chart) and job descriptions must be present, aligned, current, and accurate.

In Phoenix Elementary School District #1, the organizational chart does not meet all audit criteria for sound organizational management (see Finding 1.1). Problems in meeting the characteristics outlined in Exhibit 1.1.1 result in some confusion about responsibilities for the reader, and some vagueness occurs in indications of the curriculum design and delivery, assessment, program evaluation, and campus improvement functions. Scalar relationships are not accurately represented on the organizational chart, with positions at different levels of responsibility and compensation depicted on the same horizontal plane.

The district’s job descriptions are numerous but do not adequately meet audit criteria. Job descriptions must meet audit criteria for accurate and clear specifications of responsibilities and relationships in the district (see Finding 1.1). All job descriptions must include appropriate accountability for the relationship of the position to the design and delivery of the district’s aligned curriculum.

Auditors recommend that the district modify the position of Associate/Deputy Superintendent (the second highest administrative position in the district) and adopt a job description for a Chief Academic Officer (CAO) to bring curriculum development, revision, professional development, assessment, curriculum-based budgeting, and the sequencing and pacing of change under system macro-level control. Phoenix Elementary School District #1 is in need of bold, competent curriculum leadership so that classroom teachers and campus administrators are provided clarity regarding expected student learning.

It is important that the CAO maintain a balance between staff functions, such as curriculum development and professional development, and implementation, which is a line function. The CAO can only be held accountable if she can control the pace of learning as it impacts schools and classrooms. The CAO’s office must also function as a clearinghouse for programmatic changes that involve textbook adoption, system program change, curriculum development and evaluation, instructional strategies, and the purpose of testing and assessment as it pertains to student achievement in the district.

Auditors recommend that the district establish a position of Coordinator of Professional Development that can consolidate the scattered training and staff development responsibilities. All job descriptions must be revised to reflect such consolidation, coordinated under the direction of the Director of Curriculum.

The auditors provide suggested steps to be taken in order to remedy the areas of deficiency noted in the audit analysis and recommend that these be accomplished within the next year, and sooner for any positions modified or added to the current administrative and staff team. These actions should be completed within two years to meet audit criteria.

Governance Function: The following actions are recommended to the Governing Board of Phoenix Elementary School District #1:


G.9.1: Direct the superintendent to draft, for board review and adoption, a policy requiring annual review and adoption of the district’s organizational chart and provide assurances to the board that all positions and job descriptions are aligned, current, accurate, and adopted by the board. Job descriptions must meet audit criteria for clear specifications of responsibilities and relationships in the district (see Finding 1.1).

G.9.2: Direct the superintendent to develop, for board review and annual adoption, an organizational chart that meets Curriculum Audit™ criteria for sound organizational management. The organizational chart must reflect the central design and delivery of curriculum and congruence among all district functions related to student learning.

G.9.3: Adopt a job description for a Chief Academic Officer as proposed by the superintendent (see Exhibit R.9.2). Revise appropriate board policies to reflect the scope and responsibilities of the CAO to plan, coordinate, implement, and evaluate the direction of the district-wide curriculum.

G.9.4: Adopt a job description for a Coordinator of Professional Development as proposed by the superintendent.


A.9.1: Assist the board in development of the policy described in G.9.1 and present the draft policy to the board for adoption. Develop administrative regulations and procedures to keep the organizational chart and job descriptions current and consistent with board policy.

A.9.2: Disseminate new board policies and administrative regulations to all district employees.

• Include copies of newly adopted organizational chart for employee reference.

• Direct staff to review and revise all district job description to incorporate all essential job description components outlined in Finding 1.1.

A.9.3: Revise current job descriptions for all district positions consistent with audit criteria for job descriptions outlined in Finding 1.1.

• Develop or update job descriptions for all positions depicted on the organizational chart.

• Begin review process with all other employees to verify accuracy of the responsibilities and duties.

• Include the date of drafting and board approval for the most recent revision on all job description drafts and approved final documents.

• Ensure all job descriptions have been signed upon adoption by the board.

A.9.4: Develop a cycle of review of all district job descriptions to ensure they are current. Notations of any review and/or revisions should be posted on each job description as a record of the review process.

A.9.5: Finalize the job description for the Chief Academic Officer (CAO) (see Exhibit R.9.2), make recommended adjustments to the organizational chart (see Exhibit R.9.1), and submit to the board for adoption.

• Revise appropriate board policies to reflect the authority, scope, and responsibilities of the CAO.

• Present policies to the board for adoption.

• Create new administrative regulations outlining the duties of the revised CAO position.

A.9.6: Develop and finalize a job description for the Coordinator of Professional Development make adjustments to the organizational charts. Consolidate professional development responsibilities from other positions under one person to coordinate professional development planning, needs assessments, implementation, and evaluation.

A.9.7: Update the district’s organizational chart to meet the audit design requirements included in Exhibit 1.1.1 and address the deficiencies noted in Finding 1.1, especially focusing on the logical grouping of functions and scalar relationships. Include the following characteristics in the design of organizational chart:


• The span of control that requires direct responsibility for no more than 12 employees.

• Not more than one supervisor to avoid being placed in a compromised decision-making situation.

• Logical grouping of functions to keep tasks of a similar nature grouped together.

• A separation of line and staff positions.

• A scalar relationship that shows positions at the same level with similar responsibilities, authority, and compensation.

• Full inclusions of all central functions that facilitate quality control are included in the organizational structure.

A.9.8: Ensure that all organizational chart drafts and adopted documents bear the date of drafting and/or adoption and that the most recent revision replaces earlier versions in document collections and any other communication media.

A.9.9: Annually provide the board with a review of the organizational chart for adoption and provide assurances that all job descriptions are available, listed, and current.

A.9.10: Disseminate and review the job descriptions and the organizational chart with all administrative staff to ensure consistent adherence to the chain of command.

A.9.11: Create a representative task force to study and consider the creation of a district middle school program (grades 6-8) including implementation of a middle school philosophy, curriculum, staffing, scheduling, funding, programs, etc. A middle school program would include redesignating identified schools buildings as middle school campuses and consolidating PreK-5 students within the remaining schools.

A.9.12: Direct the director of human resources to examine and forward recommendations to the superintendent regarding the titles of administrative position in order to create a common hierarchy of each designation (e.g., Principal, Administrator, Director, Coordinator, Supervisor, Manager, etc.). Consideration should be given to the different responsibilities, authority, and remuneration of certificated and classified positions to avoid confusion.

A.9.13: Clarify the role and responsibilities of the preschool “Administrator” position, whether it is a “Principal” position with commensurate authority, responsibilities, and remuneration for consistency within the district.


Exhibit R.9.1a

Recommended District Organizational ChartPhoenix Elementary School District #1

May 2017G

over

ning

B

oard

Supe

rinte

nden

t

Ass

oc. S

up./

Chi

ef

Aca

dem

ic O

ffice

r (C

AO

)

Ass

ist.

Sup.

for B

usin

ess

Serv

ices

All

Prin

cipa

ls (1

4)

Dire

ctor

s (7)

Dire

ctor

of

Hum

an

Res

ourc

es

Dir.

of M

an. I

nfo.

Sy

stem

s

Dir.

of B

udge

t &

Fina

nce,

Chi

ld

Nut

ritio

n Se

rvic

e

Dir.

of P

lant

Se

rvic

es &

Tr

ansp

orta

tion

Coo

rdin

ator

s (8)

Coo

r. of

Tec

h.,

Info

. Sys

. Man

., So

ftwar

e M

an.

Ener

gy

Educ

atio

n C

oord

inat

or

Ass

ist.

Prin

cipa

l

Hum

an

Res

ourc

e Su

perv

isor

Tran

spor

tatio

n Su

perv

isor

Mai

nten

ance

&

Gro

unds

Su

perv

isor

Con

stru

ctio

n Su

perv

isor

Haz

Mat

/ C

usto

dial

Su

perv

isor

Mat

eria

l D

istri

butio

n Su

perv

isor

Res

pect

ive

Teac

hers


Exhibit R.9.1b

Curriculum and Instruction Organizational ChartPhoenix Elementary School District #1

May 2017G

over

ning

B

oard

Supe

rinte

nden

t

Ass

oc. S

up./

Chi

ef A

cade

mic

O

ffice

r (C

AO

)

Dire

ctor

of

Ass

essm

ent &

A

ccou

ntab

ility

Dire

ctor

of

Spec

ial

Educ

atio

n

Dire

ctor

of

Educ

atio

nal

Tech

nolo

gy

Dire

ctor

of

Cur

ricul

um

Dire

ctor

of

Prin

cipa

l Ef

fect

ivem

enss

Dire

ctor

of

Coa

chin

g Ef

fect

iven

ess

Dire

ctor

of

Fede

ral

Prog

ram

s

IEP

Com

plia

nce

Coo

rdin

ator

Coo

rdin

ator

of

Psyc

holo

gica

l Se

rvic

es

Hea

lth &

W

elln

ess

Coo

rdin

ator

Coo

r. of

Soc

ial

Wor

k &

Stu

dent

Su

ppor

t Sys

tem

s

Coo

rdin

ator

of

Prof

essi

onal

D

evel

opm

ent

Bef

ore

and

Afte

r Sch

ool

Prog

ram

C

oord

inat

or

Pare

nt

Enga

gem

ent

& E

d. C

oor.

Spec

ial

Popu

latio

ns

Coo

rdin

ator

Hig

h Sc

hool

Im

plem

enta

tion

Spec

ialis

t


Exhibit R.9.2

Job Description for Chief Academic OfficerPhoenix Elementary School District #1

May 2017

JOB TITLE: Associate/Deputy Superintendent—Chief Academic Officer (CAO)

REPORTS TO: Superintendent

DEFINITION: The Chief Academic Officer (CAO) is responsible for the design and delivery of all curriculum, programs, and related educational program support functions for the school system, including the piloting, implementation, alignment, and timing and sequence of program innovation and change, K-12. The CAO is expected to integrate all curricular departments and functions so that there is a continuous and manageable upgrading and implementation of curriculum across the school system. The CAO is responsible for ensuring coherent, focused professional development, and improvement of curriculum utilizing internal and external student assessment results, student services, special education, gifted education, and instructional/information technology. The CAO is expected to develop, with input from professional staff (teachers, principals, and central office personnel), a district-wide curriculum management plan that is updated annually.

SUPERVISES: Directly supervises five elementary principals (including Preschool Principal), the Directors of Assessment and Accountability, Curriculum, Educational Technology, Special Education, Federal Programs, Principal Effectiveness, Coaching Effectiveness.

ESSENTIAL DUTIES:

1. Responsible for the design, delivery, and monitoring of all district curricula aligned to the appropriate state standards. Curriculum is predicated on the results of formative and summative student assessment results. Curriculum is prepared for every course (including electives) offered in the district.

2. Monitors and analyzes the results of student assessments to ensure individual student progress toward established district academic achievement goals and targets.

3. Develops annual curriculum management plan in which central curricular functions are integrated and focused with professional development and assessments.

4. Ensure equitable access and initiation of equity measures to address the needs of every district student.

5. Ensures that curriculum change follows a timetable of logical and timely development, sequenced so as not to overload teachers and principals with implementation responsibilities;

6. Coordinates all district-wide professional development, ensuring that it is congruent with district goals and priorities;

7. Coordinates curriculum and services for all students, including special education, ALPS, English language learners, economically disadvantaged students, historically underperforming students, at-risk students, and instructional technology;

8. Ensures alignment of technology services with needed instructional use, curricular support, and identified district curriculum needs;

9. Guide the implementation and enhancement of Professional Learning Community (PLC) principles and strategies;

10. Participates in the district performance-based budgeting process to ensure that the district budget is aligned with and driven by the desired and targeted results of student achievement (performance-based).

11. Develops related educational policies for board adoption and administrative regulations in all assigned areas (e.g., curriculum, assessment, professional development, monitoring instruction, instruction, students, etc.).


12. Ensures that curriculum documents are consistent in format to promote user friendliness for use by classroom teachers; and

13. Arranges pilot programs for all curricular, textbook/software, and instructional materials to be used in the district; arranges systematic evaluation of all curriculum, programs, instructional materials and textbook adoptions; and, identifies, researches, and implements any intervention programs deemed essential to increasing the level of student learning and success.

MINIMUM QUALIFICATIONS:

The Chief Academic Officer must possess experience in successfully designing and implementing rigorous and aligned curriculum in an educational system that is functioning within and responsive to high-stakes testing and accountability systems. The CAO should possess working knowledge of testing and assessments, and experience in designing highly successful professional development programs, curriculum development, successful instructional and intervention strategies, and the management of change.

REQUIRED SKILL SETS:

1. Knows how to design and implement deep curriculum alignment based on test item deconstruction and other forms of feedback.

2. Knows how to direct successful curriculum design and delivery programs that have resulted in improved student learning gains on high-stakes accountability measures while maintaining challenging and rigorous standards for all students.

3. Knows the requirements, parameters, and successful strategies for all special programs offered in the district.

4. Knows how to analyze, interpret, disseminate, and utilize student assessment data for critical decision making for the district’s instructional program.

5. Knows various instructional strategies and approaches to successfully implement differentiated instruction within a high-stakes accountability system for varied student groups, including the use of instructional technology.

6. Demonstrates the ability to work collaboratively.

7. Demonstrates the ability to perform effective professional development as a leader and model differentiated instruction.

8. Knows how to initiate, implement, and monitor Professional Learning Communities (PLC).

9. Knows one or more forms of classroom walk-through models.

10. Knows how to engage in internal curriculum consistency analysis.

11. Knows how to deconstruct test information and connect various forms of classroom instruction in order to improve teacher effectiveness.

12. Knows how to identify, research, analyze, and evaluate potential intervention programs.

13. Knows various types of curricular formats, the strengths and weaknesses of each, and what works best in different contexts.

14. Knows the major trends (local, state, and national) of curriculum in each of the core curriculum content areas: math, science, language arts, and social studies.

15. Understands approaches to curriculum design and delivery and instructional responsiveness to the diverse needs of students, particularly exceptional learners (special education, ALPS, English language learners, at-risk learners, and economically disadvantaged learners).


Exhibit R.9.3

Recommended Modifications to the Organizational ChartPhoenix Elementary School District #1

May 2017

1. Change Assistant Superintendent for Curriculum and Instruction to Associate Superintendent/Chief Academic Officer (CAO);

2. Change Director of Human Resources to report to Superintendent;

3. Superintendent to supervise nine elementary principals; remaining five principals, including Preschool Principal, to report to CAO;

4. Associate Superintendent/CAO to supervise five principals (TBD), including Preschool Principal;

5. Add new Coordinator of Professional Development (reporting to Director of Curriculum);

6. Eliminate Coordinator of Gifted Education—reassign responsibilities to Director of Curriculum;

7. Change Administrator of Preschool to Principal of Preschool;

8. Change High School Implementation Specialist to report to Director of Curriculum;

9. Director of Educational Technology to report to CAO;

10. High School Implementation Specialist to report to Director of Curriculum; and

11. Exclude any non-administrative positions from organizational chart(s) (e.g., IEIS, specialists, coaches, mentors, etc.).

Recommendation 10: Develop and implement a policy and procedure that standardizes program and intervention selection based on diagnosed needs and design and implement evaluation of program objectives with feedback linked to student achievement. Develop and implement program decision-making in the initiation, modification, continuation, or termination of programs and interventions that are assessed on valid and impartial knowledge of potential value and measured results.

Leaders of effective school systems intentionally plan to systematically evaluate the efficacy of their curriculum and instructional programs against pre-determined criteria which are based upon student achievement outcomes. These data are also used to determine selection, value, and funding priorities. Such a system creates and maintains a culture of accountability and related transparency that enables the district to direct its resources towards achieving its established goals and objectives.

As with all curriculum program development, program interventions should follow a rational selection and evaluation process to ensure that they meet desired outcomes, sustain district productivity, and lead to the improvement of student academic achievement performance with documented assurance. In designing procedures and processes for implementation, it is necessary to control the number of interventions implemented at any point in time to minimize fragmentation and loss of quality. When the system is overburdened with an excessive number of programs, principals and teachers are unable to distinguish which, if any, program actually helped to improve student achievement. Likewise, in absence of clearly written procedures, it is difficult to sustain the fidelity of effective programs through changes in leadership and staff.

PESD #1 does not have a process for the establishment, implementation or evaluation of program interventions. Many of the required components were missing from planning documents. Further, there was no documentation that any district-wide review process is being followed due to the heavy reliance on campus implementation of intervention programming. Job descriptions do not identify responsibility for oversight of the program. Imagine Learning, originally selected as an intervention pilot, is now a required program for all children and no longer fits the definition of an intervention. There is lack of processes to determine intervention effectiveness, consistency in the adoption of programs, implementation, and evaluation of program interventions in the district (see Finding 5.2).



G.10.1: Direct the superintendent to prepare for review and adoption a policy that outlines the criteria for selection and evaluation of interventions at the district and campus levels. Include criteria for selecting, piloting, modifying, continuing, evaluating, and eliminating interventions.

G.10.2: Require through the policy described above in G.10.1 that a schedule be established for reporting evidence of the effectiveness of interventions to the board and other decision makers. Require that effectiveness be calculated in terms of impact on student achievement.

G.10.3: Establish an annual reporting cycle for programs and interventions for administrators to present program/intervention results that include student performance data linked to the goals and objectives of the program/intervention as well as recommendations to continue, modify, or terminate the program/intervention.

G.10.4: Approve funding for programs/interventions based on completed needs assessment, information regarding alignment with the curriculum, student performance data, and the criteria in G.10.1.


A.10.1: Draft the policy recommended in G.10.1 above and present to the board for adoption.

A.10.2: Establish administrative regulations for developing, adopting, implementing, and monitoring interventions that are aligned to district priorities and student learning goals and are responsive to student assessment data. The following components should be included:

• A description of the intervention that states the purpose, number of students to be served, and the cost per student;

• A description of the documented need supported by diagnostic and analysis data collected and considered in the selection;

• A statement ascertaining alignment with district goals and priorities and clear alignment with the district curriculum;

• The relationship to other programs operating in the district;

• A listing of required resources and funding sources that includes needs for space, staffing, material costs, and the potential for long-term sustainability;

• A budget for the proposed intervention that includes implementation costs as well as sustained costs over time;

• A plan for professional development for all staff responsible for implementing, monitoring, and evaluating the program. This plan should include initial professional development costs as well as costs to maintain delivery effectiveness over time;

• A method of evaluation for both formative and summative purposes; and

• Evaluation results reported on a regular basis to staff and the board.

A.10.3: Design an evaluation and reporting format and schedule for programs/interventions that include student performance data sources, alignment to curricular goals, criteria used to measure effective implementation, and the statistical analysis used to measure program effectiveness.

A.10.4: Establish an annual schedule for the review of program evaluations.

A.10.5: Develop and implement procedures for eliminating programs that do not demonstrate effectiveness in improving student performance.

A.10.6: Provide professional development for program administrators on selecting or designing, monitoring, and evaluating program and intervention effectiveness.


A.10.7: Evaluate and confirm the current list of interventions to determine all programs/interventions that are currently available, currently being utilized, and which should be evaluated for need and effectiveness.

A.10.8: Allocate funding to effectively design, implement, and assess programmatic interventions using student performance data to evaluate effectiveness. Provide future funding in the budget for effective programs/interventions from existing internal funding sources or from long-term external funding sources.

A.10.9: Research, develop, and implement a district-wide model for Response to Intervention/MTSS. That model should include the following:

• A framework that includes a multi-level prevention system (high quality core instruction—Tier 1; evidence-based intervention—Tier 2; and individualized intervention—Tier 3) that provides increasingly more intense levels of support. The framework should also include appropriate screening measures and progress monitoring tools to make quality decisions based on data that support student learning and growth.

• Select appropriate interventions based on direction found in A.10.2 and A.10.3.

• A plan for professional development for all staff to define the multi-level prevention system and create a common understanding and vocabulary across all classrooms, campuses, and the district.

• Define expectations for implementation of the model district-wide and establish procedures for monitoring implementation of the RTI/MTSS Model and evaluating its impact on student achievement.

A.10.10: Designate personnel who will provide oversight for intervention selection and implementation within the district. Revise job descriptions as necessary to indicate levels of responsibility across the district.

These recommendations should give the district a means of developing and implementing policies and procedures that standardize program and intervention selection and Response to Intervention/MTSS based on diagnosed needs and design and implement the evaluation of program objectives with feedback linked to student achievement. Implementing these recommendations will improve decision making on the initiation, modifications, continuation, or termination of programs and interventions based on valid and impartial knowledge of potential value and measured results.

Recommendation 11: Develop comprehensive assessment procedures to provide reliable data for decision making at all levels of the system. Establish a formalized process for selection, implementation, and evaluation of programs and interventions utilizing data that will be used as feedback for initiation, modification, continuation, or termination of programs and practices.

Effective system wide student assessment and program evaluation processes provide district and school staff with quality feedback helping to make informed instructional decisions ranging from individual student lessons to overall design and delivery of curriculum and the effectiveness of programs and interventions. Evaluation strategies are determined in advance, and implementation of programs and interventions are monitored on a regular basis. Reports of progress or problems identified through student achievement provided on a periodic basis help to guide modification or elimination of programs and interventions.

Auditors found a lack of coordinated district wide procedures to provide direction for the proper selection, implementation and evaluation of instructional programs and interventions (see Finding 4.4 and 5.2). Auditors did not find a singular and focused comprehensive program and intervention evaluation plan that defines the critical characteristics essential for directing and providing direction for the design of a program evaluation system. Auditors found the district lacks a systemic process for the use of data for proper implementation, continuation, and termination of programs (see Finding 4.4). Auditors also did not find any job descriptions assigning roles and responsibilities for how data are to be used for the formal implementation, continuation, and termination of programs and interventions (see Finding 4.3, 4.4, and 5.2).

Recommendations should be implemented in six to 12 months to establish a process for the selection, adoption, implementation, and evaluation or programs and interventions.



G.11.1: Direct the superintendent to draft a policy outlining criteria for the selection, implementation, and evaluation of programs and interventions at the district and school levels. Include in the policy the requirement that all programs and interventions must be adopted by the board prior to implementation.

G.11.2: Adopt a policy requiring that all programs and interventions undergo a system review process linked to student achievement data at least every three years with linkages to district goals.


A.11.1: Draft a policy responding to G.11.1 and G.11.2 above, and develop administrative procedures to implement the policies.

A.11.2: Develop administrative procedures with detailed selection criteria for programs and interventions recommended to be adopted by the board. The criteria should address the following questions:

• Does the program or intervention align with the district’s mission and goals?

• Does the program or intervention align with district curriculum?

• Is there an evaluation design that will produce objectives and reliable data on programmatic effectiveness?

• Do the program budgets include a cost of accounting for materials, staffing, equipment, and facilities requirements long term?

• Is the program likely to strengthen the overall programmatic efforts of the school system?

• What indicators suggest that the program or intervention will improve student achievement?

• How will improvement in student achievement be measured on a frequent and regular basis?

A.11.3: Develop a prioritized list of programs and interventions to be evaluated with timelines for review.

A.11.4: Design an evaluation and reporting format for programs/interventions that includes the following:

• The source of student performance data used for program evaluation.

• The alignment of curriculum goals to program assessments.

• The criteria used to measure effective implementation.

• Frequent formative evaluations and annual summative evaluations.

• Statistical analysis applied to measure program effectiveness based on student performance data.

A.11.5: Develop specific district criteria and procedures to determine whether to initiate, continue, or terminate programs/interventions.

A.11.6: Establish an annual schedule for the review of program/intervention evaluations with appropriate administrative staff.

A.11.7: Provide professional development for administrators and teachers assigned to lead the process of data analysis for program and intervention reviews.

A.11.8: Identify fiscal resources available to effectively support the design, implementation, and assessment of programs and interventions.


Recommendation 12: Develop and implement coordinated planning processes that produce congruent district and campus plans and link the planning for curriculum management, professional development, student assessment, program evaluation, and budget development with those plans.

Effective planning is essential for focusing and organizing district resources and priorities to meet changing student needs. Long-range planning provides a systemic means to sustain constancy of purpose as a district works towards achieving its goals. Comprehensive planning benefits students by increasing the probability that effective programs, practices, and facilities will be available to them regardless of district growth, economic changes, and other effects of community changes. School improvement planning that is aligned with district planning and considers all student data and unique school needs enhances the chance of increasing student achievement with improved focus and coordination of site and district efforts.

The auditors found a current district strategic plan and were informed of present efforts to create a new plan. There is little board direction for planning in areas of curriculum management (see Finding 1.2), professional development (see Finding 3.12), program interventions (see Finding 5.2), student assessment (see Finding 4.1), budget development, and technology planning (see Findings 5.1 and 5.4). Current documents do not fully meet audit criteria to provide quality direction for ongoing improvement efforts and to cohesively link those efforts across units and functions within the school district.

Common focus is emerging under the leadership of the superintendent in the area of student learning, curriculum and instruction, and technology. Board policies are inadequate to direct planning consistently across the district and among all stakeholders.

The following recommendations are designed to improve system planning for the effective integration of all planning functions and the successful accomplishment of district goals. Their implementation is best accomplished over a one- to two-year period. Some steps represent refinement rather than full-scale development and can be accomplished more quickly.


G.12.1: Direct the superintendent to assist the board in developing a clear, concise, comprehensive policy framework that stipulates the following:

• The requirement that all system sub-functions engage in planning and create plans that are connected to the overall system learning priorities, reviewed annually by the superintendent, interrelated to budget development, become part of the evaluation of the system administrators charged with these sub-functions, and reported publicly to the board at regular intervals.

• Requirement that all system initiatives be subjected to evaluation after a specified time period, ensuring that unsuccessful or even moderately successful initiatives may not be continued.

G.12.2: Adopt board policies to provide overall direction for a full scope of long- and short-range planning aligned with the current efforts in strategic planning. Policies should address development, implementation, monitoring, and evaluation of plans and their ongoing use in relation to the budget planning process. Policies should include expectations for public reporting to communicate plan implementation progress.

G.12.3: Direct the superintendent to design a planning process to comply with the policy described in G.12.1. Require that any district-wide strategic plan incorporate all components listed in Exhibit 1.3.2, 1.3.3, and 1.3.4. Require linkage of campus improvement, curriculum management, student and program assessment, professional development, and budget planning to the district’s long-range plan.

G.12.4: Direct the superintendent to establish budget development procedures to ensure that district learning and planning priorities are reflected in budgeting and spending and are communicated to staff and the public at-large.

G.12.5: Direct the superintendent, with the administrative team, to prepare regular reports to the governing board on all district plans and their results as they pertain to the attainment of district goals and objectives.



A.12.1: Assist the board in the development and/or revision of board policies as noted in G.12.1 to meet Curriculum Audit™ criteria.

A.12.2: Implement a comprehensive planning process that:

• Provides a vision of the future in each of the functions and divisions in the district.

• Uses data information to frame district wide student performance measures over several years with increasing challenges.

• Requires campus improvement plans to set student performance measures aligned to the district measures and at a higher achieving pace.

• Develops SMART goals, objectives, and action steps, ensuring they are succinctly measurable, focus on student learning results, and are not merely tallies of completion or attendance.

• Require clear and explicit connections in campus improvement plans to the strategic directives of the school system.

• Ensures all professional development activities at each school site are clearly connected to the strategic directives.

• All school plans need to be indexed to relevant assessment data. Initiatives that are not so connected need to be eliminated. These actions will ensure system focus and accountability.

A.12.3: Direct that all initiatives previously implemented are to be systematically evaluated with measurable outcomes as to their effectiveness in reaching the goals for which they were adopted, and their anticipated effectiveness in supporting the strategic directives. Initiatives that have not been successful or are even only moderately successful may not be continued. Any program that is not evaluated will not be continued after a specific time period established by the superintendent.

A.12.4: Design a consistent format for district departmental and campus plans that includes the following components:

• Connectivity to the district mission, beliefs, and goals;

• Clearly established and measurable goals to meet student needs;

• Goals based on the analysis of student achievement data and other data;

• Strategies that address goals to be accomplished;

• Resources and funding required for each strategy/objectives;

• Methods of monitoring and evaluating plan progress included with plan design;

• Actions and intervention strategies that are built around effective change strategies;

• Evaluation based on measurable student results data;

• Deployment strategies to disseminate the steps and tasks to be accomplished in the plan;

• Identification of specific persons responsible for implementing strategies; and

• Professional development needed to reach the goals.

A.12.5: Provide training for all administrators and key instructional staff pertaining to the following:

• Understanding and adhering to the critical components of an effective planning process that is data driven;


• Building capacity to address components of the planning process as they assess campus and department needs; and

• Setting realistic goals and performance-based activities.

A.12.6: Monitor new or revised plans to ensure that these plans are aligned with the curricular priorities in the district and that implementation is resulting in progress toward those goals.

A.12.7: Direct administrative staff to directly utilize plan elements in their daily roles and responsibilities. References to specific plan goals and objectives are incorporated in all reports, presentations, and analyses to ensure routine usage of plans to focus on and improve student learning.

A.12.8: Develop administrative regulations to govern the development, deployment, implementation, and evaluation of the planning process.

A.12.9: Establish an ongoing communication plan to orient all staff and keep current the knowledge of the planning process and results.

A.12.10: Create a district task force to examine and analyze all existing plans in the district and to develop a process to consolidate the planning processes and eliminate duplication and contradictions between those plans. The charge of the task force is to:

• Assist campuses and district personnel to singularly focus on one concerted improvement effort.

• Reduce the amount of staff time devoted to writing and completing plans that simply meet compliance deadlines.

• Provide professional development for school and district staff in writing improvement plans and training staff on effective implementation of plans.

Attention, adherence, and fidelity to the twelve recommendations will provide Phoenix Elementary School District #1 with significant improvement in the management of its curriculum. Some recommendations can be implemented fairly quickly while others will require steady, continual efforts and attention. These recommendations are intended to guide the district in its effort to improve student achievement with a central focus on each student’s learning.


VI. APPENDICES


Appendix A

Auditors’ Biographical Data

Roger C. Antón, Jr., M.A., Lead Auditor

Roger Antón retired as Superintendent of the Salinas Union High School District in Salinas, California. In his 40 years in education, he served as Superintendent, Associate Superintendent—Instructional Services, a junior high/middle school principal, and assistant principal for 33 years. He was a junior high school classroom teacher and activities director for seven years. He has extensive experiences in all aspects of district operations: curriculum and instruction, budgeting, personnel, staff development, strategic planning, categorical programs, leadership training, and pupil services. He is a certified

national trainer in the “Downey Walk-Through” process, “Deep Alignment” of curriculum, “50 Ways to Close the Achievement Gap” (a process for school improvement), “Examining Student Work,” “Performance-Based Budgeting,” and the Curriculum Management Audit process.

Roger received his B.A. in History from the University of California, Riverside and his M.A. in Education/History from California State University, San Bernardino. He completed his Curriculum Management Audit (CMA) training in California in 1997. He has served as an auditor of school districts in Texas, Arkansas, Mississippi, New Jersey, Pennsylvania, California, and North Carolina, and an External Evaluator for the “School Review for Higher School Performance” process in California. He has personally experienced an audit when his own district underwent a CMA audit in 1997 and a Post-Audit in 2005.

Penny Gray, Ph.D.

Penny Gray has been an educator for over 40 years, as a teacher and an administrator, in Indiana and California. She taught elementary school for 20 years and was Director of Curriculum Services in the San Marcos Unified School District in California. She has taught graduate courses in educational leadership and supervised students in the Administrative Credential Program for San Diego State University. Dr. Gray co-authored articles on state testing programs and labor relations and three books, From Good Schools to Great Schools: What Their Principals Do Well, Leading Good Schools to Greatness: Mastering

What Great Principals Do Well, and The New School Management by Wandering Around. She received her Ph.D. from Claremont Graduate School and completed her audit training in Burlingame, California in 1998. Dr. Gray has served on thirty-one curriculum management audits in thirteen states and Bermuda.

Holly J. Kaptain, Ph.D.

Holly J. Kaptain is currently the Executive Director of Curriculum Management Solutions, Inc. She has worked in public education for over 20 years and most recently in higher education at Iowa State University, where she was a research assistant in bilingual and two-way immersion programming for culturally and linguistically diverse students. She is a CMSi (Curriculum Management Solutions, Inc.) licensed trainer in deep curriculum alignment and has participated in over two dozen audits in 11 different states since 1996. Dr. Kaptain graduated with a B.A. from St. Olaf College in Minnesota and completed

curriculum management audit training in St. Paul, Minnesota in July of 1996. She completed her M.S. in Curriculum and Instruction and her Ph.D. in Educational Administration at Iowa State University. She has presented at regional and national conferences on bilingual education research, instructional efficacy, and curriculum design. Dr. Kaptain is a member of Phi Delta Kappa, the National Association for Bilingual Education, the American Council of Teachers of Foreign Languages, as well as other honor and professional organizations.


Appendix A (continued)Auditors’ Biographical Data

Ronnie Thompson, M.Ed.

Ronnie Thompson is currently Superintendent for the Liberty-Eyalu Independent School District in Texarkana, Texas. Formally, he served as Superintendent for Hooks Independent School District and Assistant Superintendent for the Texarkana Independent School District in Texarkana. He has also served as Executive Director of School Improvement, Associate Principal, Assistant Principal, and a classroom teacher. Thompson is an experienced teacher in the Career and Technology field and has served as an Adjunct Professor of Education at Texas A&M University-Texarkana.

Thompson has a total of 26 years of experience in public education. He has extensive experience in all aspects of district operations: curriculum and instruction, financing, personnel, staff development, strategic planning, special programs, leadership and student services.

Mr. Thompson is a graduate of Texas A&M University in Business Management and received his Master’s Degree in Educational Administration from Texas A&M University-Texarkana. He completed his Curriculum Management Audit Training in Arizona in 2008.

Jeffrey Tuneberg, Ph.D.

Jeffrey Tuneberg has over 30 years experience in education, including 25 years as Director of Curriculum with the Mercer County Educational Service Center, Celina, Ohio. His teaching background includes experience in urban (Cleveland, OH Public Schools) and suburban settings, as well as overseas (Guam). He was selected as a Fulbright Memorial Fund Teacher Program representative to Japan in 1997. Dr. Tuneberg is also an adjunct professor at Wright State University Lake Campus, Celina, Ohio, and Ashland University, Ashland, Ohio, where he has served as a supervisor of student teachers. Additional

consulting includes serving as a credentialed faculty member with Battelle for Kids, Columbus, Ohio, on the topic of value-added growth measures in schools.

Dr. Tuneberg received his B.S. in Education, M.Ed., and Ph.D. from Bowling Green State University, Ohio. He has served as a consultant to school districts in Ohio, Tennessee, and Oklahoma on issues of teacher licensure, school improvement, and value-added student growth measures. He received his Curriculum Management Audit training in Lima, Ohio in 1999 and has conducted or served as a lead auditor on curriculum audits in Ohio, Oregon, Washington, Michigan, Pennsylvania, Iowa, Wisconsin, Kentucky, Arizona, Maryland, Texas, and New Jersey. Dr. Tuneberg has also presented throughout the U.S. and Canada on the Classroom Walk-Through Program, SchoolView, the Baker’s Dozen Program, and Deep Curriculum Alignment.

Olivia Elizondo Zepeda, M.Ed.

Olivia Elizondo Zepeda graduated from Northern Arizona University with a B.A. in Elementary Education. She began her teaching career upon graduation from NAU and later earned a Master’s degree in Bilingual and Multicultural Education.

Olivia has been the Associate Superintendent for the Gadsden Elementary School District since 2000. She was Curriculum and Staff Development Director for two years and principal for two and a half years. Olivia has taught graduate and undergraduate classes at the university level as well as middle school and elementary grades. Olivia has 39 years of

experience in education. She in is fully bilingual in English and Spanish. Zepeda underwent her Curriculum Audit training in Arizona.

Olivia currently serves on the Board of Trustees for Arizona Western College, interested in education for children and adults. She has a passion for service and enjoys serving in agencies that provide assistance to children and adults for educational purposes.


Appendix B

List of Documents Reviewedby the

Phoenix Elementary School District #1 Audit Team

Accelerated Learning ProceduresAnnual Financial Reports—Arizona Department of Education, 2013-2016Arizona School Tax Credit VideoAssessing the Quality and Alignment of Literacy Instructional Materials to the Common CoreAssessment Letter to staff, parents, and/or guardians, 2016Board Agendas and Minutes, VariousBoard Policies, Regulations, and Exhibits, VariousBook Room Materials, 2015-2016Budgeting Process DocumentsBuilding Improvement Plans, Various, 2017Building Title I Budgets, 2016Class Size Data, April 2017Coach Professional Development, April 2017Coach Boy Schedule, 2016-17Comprehensive Annual Financial Report (CAFR), 2016Computer Purchases, 2008-2017Computers by Location, 2017Criteria for the Adoption of Instructional MaterialCurriculum & Instruction PowerPoint Presentation, 2017Curriculum Documents, VariousDemographic and Enrollment Data, VariousDistrict & School Title I Continuous Improvement Plans, Various, FY 2014, 2015, 2016, 2017District Assessments, VariousDistrict Continuous Improvement Plan, 2016-2017 District Disciplinary Data, 2013-14 to 2015-16District Retention Data, 2013-14 to 2015-16District Vision and Mission StatementsEducational Technology Professional Development, 2017ELA Leadership Team Governing Board Information Session, 2013ELA Leadership Team Session 2, 2013English Language Arts (ELA) Adoption CommitteeEnglish Language Learner Program, 2016-17English Language Learner (ELL) Program PhilosophyFinancial Report, Arizona Office of the Auditor General, 2016Financial Reports, Arizona Office of the Auditor General, 2013-2016


Appendix B (continued)List of Documents Reviewed

First Sale Bond Program PowerPoint, ADM Group, 2017Free/Reduced Lunch—100th Day, 2015-16 and 2016-17Galileo/AIMS web Test Security AgreementGeneral and Gifted Population Description, 2016-17Job Descriptions, VariousLiteracy Instruction in the Phoenix Elementary School District—Goals and Beliefs, 2013LOI—Teacher Appraisal Instrument, July 2012“Looking Through the Lens of Coaching Effectiveness, March 2017Master Schedule—All Schools, 2016-17Move On When Reading School K-3 Literacy Plan, 2016-2017New Hire Survey Results, April 2017Online Survey ResultsOrganizational Charts, VariousParent and Family Engagement Survey, 2017Phoenix Elementary Mobility Study, 2017Phoenix Elementary School District #1 Organizational ChartPhoenix Elementary School District #1 Single Audit Reporting Package, 2016Phoenix Elementary School District #1 Technology Plan, January 2013“Principals Weekly Update”—Emails to School PrincipalsProfessional Development Convocation, 2015 and 2016Professional Development for the Instruction of ELLsProgram Enrollment Data, 2014-15 to 2016-17REIL—TNG Year 4 Survey Results, July 15, 2016Request for English Language Arts Instructional MaterialsSchool Mission and Vision StatementsSession Outcomes, 2013SFB Building Status List, 2017Software Adoption Process, 2017Special Education: Guided Implementation Plan, 2017Special Election Informational Pamphlet, 2015Strategic Plan: “Our Blueprint for Student Success,” 2012Student Handbook, 2016-17Support Staff Building Assignments by Number and Role, 2017Title I District Budgets, 2014-2016Working copy of Strategic Plan: Our Blueprint for Student Success Action Plan, 2016-2017Writing Position Statement


Appendix C

Blooms vs. DOK MatrixPhoenix Elementary School District #1

May 2017


Appendix D

Characteristics of Culturally Responsive Teaching

1. Consistently compares and contrasts different cultures, languages, experiences, and values with the dominant community cultures in the classroom, regardless of the content area.

The teacher consistently allows students the opportunity to discuss their own and their families’ experiences, values, and cultural experiences during the course of lessons and activities, within a context of acknowledging differences and similarities with the predominant community culture. The teacher shows an appreciation of presenting them in a positive light.

2. Actively researches different cultural perspectives and examples connected to instructional content and incorporates these into classroom lessons and discussions.

The teacher actively seeks examples from his/her students’ own representative cultures as well as from other cultures that tie into classroom lessons and discussions. For example, in a lesson on basic mathematical algorithms (division/multiplication), the teacher researches common global approaches to the same and introduces them in the classroom.

3. Involves students, parents, and the community in contributing to cultural awareness and appreciation.

Whenever possible, the teacher invites contributions from students, parents, and the community at large in learning activities that focus on curriculum content being taught with diverse cultural perspectives.

4. Facilitates and encourages students to discuss concepts and new learnings in their native language in earlier stages of language development (not translating).

When possible or desirable, the teacher allows small groups or pairs of students to discuss new learnings in their native language, to assure understanding of key curriculum concepts and vocabulary. For example, when reading a novel in class, students are occasionally grouped by native language to allow discussion of the plot and themes in the book, so students’ comprehension is supported.

This approach is not to be confused with translating for students, although occasional translation (among students only) is acceptable. The teacher also allows students to contribute to classroom discussions in their native language if their English is not yet strong enough, with another student translating. This enables all students to contribute to discussions and activities.

5. Incorporates cross-language, as well as cross-cultural comparison and development.

The teacher facilitates comparing languages and cultures in a deliberate way. For example, word walls, graphic organizers, and concept maps may be used with bilingual terms and expressions.

6. Respects and values student input and frequently (daily) elicits student involvement and supports their personal connection to the learning.

Students are always encouraged to contribute to classroom activities and discussions, sharing personal experiences that relate to new content. Such approaches also support scaffolding of curriculum content and makes learning more personally relevant.

7. Respects students’ affective needs with regard to participation and involvement in classroom activities and discussions, particularly during the early stages of English development.

The teacher allows students periods of silence or non-involvement if a student feels uncomfortable participating or is struggling with communication issues. Such scenarios can be extremely stressful for children and emotionally challenging, and the teacher responds accordingly with sensitivity and tolerance. Every student is unique and should be encouraged but never forced to participate in every activity. Consider alternative forms of involvement if the activity is a type of assessment.

©2016 CMSi


Appendix E

Characteristics of Cognitively Engaging Instruction

Note: The term, “Cognitively engaging instruction” is intended to describe classrooms where the emphasis is on meaningful, challenging student learning that makes kids think, involves them in their own academic progress, and creates a climate that encourages risk-taking, thinking outside the box, and real-life scenarios.

Cognitively engaging instruction is focused on the most important role schools play: promoting student learning. It is built on the foundation of rigor. Rigor is not determined by the quantity of work a student completes; rather, rigor refers to the nature of the work a student performs in completing an assignment or project; i.e., the amount of thinking that is involved, the nature of that thinking, and how it is manifested in their work.

The following characteristics are extrapolated from research and have been shown to be effective in improving achievement among all student groups: at-risk students, gifted students, learning disabled students, and ELL students. These characteristics, when coupled with challenging academic content, describe courses that would be considered “advanced” or “enrichment” courses.

Teaching approaches and student learning activities reflect a constructivist philosophy regarding student learning. Such approaches are typified by the following characteristics:

• The focus of all learning activities is to keep them meaningful for the student. The student understands why he/she is doing the activity, the goal or purpose behind it, and how he/she will ultimately benefit from completing it. Activities are student-centered, not teacher-centered.

• Learning focuses more on larger, connected, or related concepts rather than on discrete, specific facts.

• Students can relate their learning to real-life scenarios; the learning is seen as relevant to themselves, personally, or to their social context.

• Every student is an active participant in his/her learning. Students are involved in setting learning goals and in monitoring their own progress in mastering objectives and meeting their goals.

• Learning activities are intrinsically interesting. They are modified to suit student preferences, learning styles, and academic needs. Students have a certain degree of autonomy, or choice, in their learning activities and the product they are responsible for.

Students are divided into smaller groups (or pairs) for various instructional purposes. These groupings are accomplished in the following ways:

• Students are grouped or paired heterogeneously to foster collaboration with others and to encourage communication and positive, productive social interaction. Working in heterogeneous, collaborative groupings involves accountability and respects prevailing rules governing group members’ conduct (to ensure accountability for all group members).

• Students are grouped homogeneously, typically by need, to allow for instruction at the students’ level and in response to diagnosed gaps in learning. These groupings are never static; they change constantly—usually weekly or even daily—to reflect varying rates of student progress in mastering objectives.

• Groupings may be cooperative, where students work with each other to accomplish assigned tasks; pairs, where students review and learn from one another; or varied-size groups, pulled together to allow for small group, targeted instruction.

Activities are personally relevant and culturally responsive. Such activities are characterized by the following:

• Students are led to connect their learning to real-life scenarios or personal experiences, such as things they’ve seen or done themselves.


Appendix E (continued)Characteristics of Cognitively Engaging Instruction

• Learning scenarios are culturally responsive—learning activities always take into account and build on students’ linguistic, ethnic, and socioeconomic diversity.

• Students are encouraged to view new learning through a lens of their personal cultural perspective: what about that learning has significance in their own ethnic/cultural context? What is similar? What is different? What learning is culturally neutral?

Students are encouraged to think independently and critically:

• The overall focus of learning activities is on thinking, not acquiring facts or knowledge. Knowledge acquisition is accomplished through projects and assignments.

• Students engage in learning scenarios and activities that require them to think independently—in contrast to mainstream thinking or against majority opinion or stance. In such scenarios, students are encouraged to adopt a specific position or formulate an argument, whether it reflects their personal opinion or not, and research and defend that position to those possessing opposing viewpoints.

• Students are involved in analytical thinking—breaking down concepts or processes into its various parts and demonstrating an understanding of how the parts relate to one another, or evaluating the advantages and disadvantages of all parts or perspectives.

• Students are given tasks that require reviewing large quantities of information and data and summarizing it into brief, meaningful synopses.

• Student activities reflect active cognitive processing, as first conceptualized by Bloom’s Taxonomy of Learning.

The teacher engages students in metacognitive strategies:

• Students are asked to think and reflect on their own thinking. They can explain how they arrived at an answer, describe their thought processes in completing a task or solving a problem, and describe their progress in mastering a specific concept or skill.

Language structures and vocabulary are deliberately, consciously taught and integrated into all learning activities across all content areas:

• Classroom activities explicitly integrate and teach vocabulary using authentic text and context-embedded approaches.

• Learning activities across content areas simultaneously focus on content mastery as well as language skills: language structure, punctuation, vocabulary.

• Students are engaged in multiple modes of communication—speaking, reading, writing, listening.

Instruction is differentiated to meet specific student academic needs and preferences:

• Teachers utilize a variety of student groupings and multiple diagnostic tools and instructional resources to determine and teach required content (concepts, skills, knowledge, and vocabulary).

• Teachers plan instruction based on data from formative, diagnostic tools, which reveal gaps in student learning and specific weaknesses in student mastery of intended objectives.

©2016 CMSi