crafting a performance culture: anxiety, climate and school district reform katharine neem destler...
TRANSCRIPT
Crafting a Performance Culture:
Anxiety, Climate and School District Reform
Crafting a Performance Culture:
Anxiety, Climate and School District Reform
Katharine Neem Destler
University of Washington
“Improving Education Through Accountability And Evaluation: Lessons From Around the World”
APPAM and INVALSI Rome, Italy
October 3, 2012
Sponsored by the US Department of Education, Institute of Education Sciences (#R305B090012)
Katharine Neem Destler
University of Washington
“Improving Education Through Accountability And Evaluation: Lessons From Around the World”
APPAM and INVALSI Rome, Italy
October 3, 2012
Sponsored by the US Department of Education, Institute of Education Sciences (#R305B090012)
Shifting Tides in Education Reform
Shifting Tides in Education Reform
Standardized Practices--> Site-based control Instructional Practice --> Managerial Behavior Increased Funding --> Accountability for Results
Standardized Practices--> Site-based control Instructional Practice --> Managerial Behavior Increased Funding --> Accountability for Results
England
New Zealand
Victoria, AustraliaU.S. Charter Schools
A turn towards “Portfolio” School District Reform
A turn towards “Portfolio” School District Reform
Parental School Choice Site-Based Decision Making Performance Monitoring Closure of Chronically Underperforming Schools Aggressive Development/Recruitment of New
Schools, including those run by private providers (Bulkey and Henig 2010, Hill et al. 2012)
Parental School Choice Site-Based Decision Making Performance Monitoring Closure of Chronically Underperforming Schools Aggressive Development/Recruitment of New
Schools, including those run by private providers (Bulkey and Henig 2010, Hill et al. 2012)
Moving too quickly?Moving too quickly?
Despite the attention to high stakes accountability reforms, research suggests that the results have been mixed: District-wide improvement Teaching to the test Narrowing of the Curriculum Focus on “Bubble” Kids
(Corcoran and Beverage 2010; Jacob 2005; Kemple 2010)
Despite the attention to high stakes accountability reforms, research suggests that the results have been mixed: District-wide improvement Teaching to the test Narrowing of the Curriculum Focus on “Bubble” Kids
(Corcoran and Beverage 2010; Jacob 2005; Kemple 2010)
Reforms in Education aren’t “new” everywhere
Reforms in Education aren’t “new” everywhere
Performance Management Reform:An ongoing, systematic approach to improving results through evidence-
baseddecision making, continuous organizational learning, and a focus onaccountability for performance”(National Performance Management
Advisory Commission, 2010).
Performance Management Reform:An ongoing, systematic approach to improving results through evidence-
baseddecision making, continuous organizational learning, and a focus onaccountability for performance”(National Performance Management
Advisory Commission, 2010).
Managerial Discretion Publication of Performance Data Accountability for Outcomes
Education reflects findings in other sectors
Education reflects findings in other sectors
Performance management is well-established in other sectors
The provision of data does not ensure that public leaders or policymakers will act on that data
Incentives have perverse, as well as positive, results
(Heinrich and Marschke 2010; Jennings and Haist 2004; Moynihan
2008; Radin 2006)
Performance management is well-established in other sectors
The provision of data does not ensure that public leaders or policymakers will act on that data
Incentives have perverse, as well as positive, results
(Heinrich and Marschke 2010; Jennings and Haist 2004; Moynihan
2008; Radin 2006)
Formal Systems Change is Not Enough…
Formal Systems Change is Not Enough…
Need to understand the conditions under which schools and other organizations experience a change in front line culture.
Under what conditions do front-line workers and managers espouse performance management values and enact performance management values in their daily behavior?
Need to understand the conditions under which schools and other organizations experience a change in front line culture.
Under what conditions do front-line workers and managers espouse performance management values and enact performance management values in their daily behavior?
Understanding Performance Management Behavior
Understanding Performance Management Behavior
Formal Performance Management Reform:
Relevant and Reliable Data
Performance Accountability
Managerial Discretion
Learning Forums
Performance- Oriented
Front-Line Behavior
Understanding Performance Management Behavior
Understanding Performance Management Behavior
Formal Performance Management Reform:
Relevant and Reliable Data
Performance Accountability
Managerial Discretion
Learning Forums
Performance- Oriented
Front-Line Behavior
Performance Management Values
Cultural Characteristics
Organizational Climate
Survival Anxiety
Performance- Oriented
Front-Line Behavior
Two Alternate Paths to Organizational ChangeTwo Alternate Paths to Organizational Change
The “Incentive” Approach Increased performance leads to change in culture and
behavior. “Survival Anxiety” will provoke greatest need for change
(Khademian and Harttman 2010; Kotter 1990; Schein 2006)
The “Internal Dynamics” Approach Not every organization is equally “ready” to change A positive Organizational Climate (trust, support) is an
important prerequisite for cultural shift. (Bryk and Schneider 2005; Childress et al., 2011; Hou, Moynihan, & Ingraham, 2003; Schein 2006)
The “Incentive” Approach Increased performance leads to change in culture and
behavior. “Survival Anxiety” will provoke greatest need for change
(Khademian and Harttman 2010; Kotter 1990; Schein 2006)
The “Internal Dynamics” Approach Not every organization is equally “ready” to change A positive Organizational Climate (trust, support) is an
important prerequisite for cultural shift. (Bryk and Schneider 2005; Childress et al., 2011; Hou, Moynihan, & Ingraham, 2003; Schein 2006)
Research QuestionResearch Question
What is the role of external incentives (in the form of organizational survival anxiety) and organizational climate in the espousal of performance management values and the adoption of performance management values-in-use in schools?
What is the role of external incentives (in the form of organizational survival anxiety) and organizational climate in the espousal of performance management values and the adoption of performance management values-in-use in schools?
Data and MethodsData and Methods
Sequential, Mixed-Method Study Cross-Sectional School System Analysis
NYC Learning Environment Survey, Progress Report, Quality Review 2007-2011
n : 115-1,153 depending on model Analysis conducted using OLS with schools as the unit of
analysis
Comparative Case Study of Four Schools
Sequential, Mixed-Method Study Cross-Sectional School System Analysis
NYC Learning Environment Survey, Progress Report, Quality Review 2007-2011
n : 115-1,153 depending on model Analysis conducted using OLS with schools as the unit of
analysis
Comparative Case Study of Four Schools
New York City: A “Critical Case” for Performance Management Reform in Education
New York City: A “Critical Case” for Performance Management Reform in Education
Mayoral Control, dating from 2005 Abolishment of the internal school district
structure “Empowerment” strategy:
Devolution of decision-making power down to school principals
School-level accountability for performance
Mayoral Control, dating from 2005 Abolishment of the internal school district
structure “Empowerment” strategy:
Devolution of decision-making power down to school principals
School-level accountability for performance
Key VariablesKey VariablesDEPENDENT VARIABLESEspoused Values: explicitly embraced by organizational
members (Argyris and Schon 1974,1977; Schein 2006)
Values-in-Use: tacitly revealed through organizational behavior. (Argyris and Schon 1974,1977; Senge 1990; Schein 2006.
INDEPENDENT VARIABLES OF INTERESTOrganizational Climate: level of organizational trust;
perceptions of support and norms of open/honest dialogue (Childress et al 2011; Moynihan & Pandey, 2010; Schein, 2006; Senge, 1990)
Survival Anxiety: perceived risk of inaction to organizational wellbeing (Kotter, 1996,Schein, 2006)
DEPENDENT VARIABLESEspoused Values: explicitly embraced by organizational
members (Argyris and Schon 1974,1977; Schein 2006)
Values-in-Use: tacitly revealed through organizational behavior. (Argyris and Schon 1974,1977; Senge 1990; Schein 2006.
INDEPENDENT VARIABLES OF INTERESTOrganizational Climate: level of organizational trust;
perceptions of support and norms of open/honest dialogue (Childress et al 2011; Moynihan & Pandey, 2010; Schein, 2006; Senge, 1990)
Survival Anxiety: perceived risk of inaction to organizational wellbeing (Kotter, 1996,Schein, 2006)
Measurement ChallengesMeasurement Challenges
High Correlation between Organizational Climate and Performance Values Construct Validity Common Source Bias
Endogeneity Risk
High Correlation between Organizational Climate and Performance Values Construct Validity Common Source Bias
Endogeneity Risk
High standards for student learning
Satisfaction with governing regime
Focus on innovation/autonomy
Survival Anxiety
Values-in-Use
OrganizationalClimate
Espoused Values
PerformanceCulture
FormalPerformanceManagement
ReformManagerial Discretion
Accountability for Results
Valid/Reliable Data
Learning Forums
Psychological Safety
Norms of Collaborationand Open Dialogue
Perceived School Quality
Parental Engagement
Parental Support for Reform
Organizational strategizingand goal setting
Performance data use
Continuous improvementprocesses
Perceptions of Support
Leadership Behavior
Conceptual Model
Socio-Political Context
HypothesesHypothesesH1: Survival anxiety will have a positive and
statistically-significant effect on the espousal of performance values and the adoption of performance values-in-use.
H2: Organizational Climate will have a positive
and statistically-significant effect on the espousal of performance values and the adoption of performance values-in-use
H1: Survival anxiety will have a positive and statistically-significant effect on the espousal of performance values and the adoption of performance values-in-use.
H2: Organizational Climate will have a positive
and statistically-significant effect on the espousal of performance values and the adoption of performance values-in-use
Findings--Espoused ValuesFindings--Espoused Values
• Organizational climate = strong predictor
• Survival anxiety= weak but significant predictor
• Limited interaction effect
• Few statistically-significant formal controls
n -0.13* (0.08)
-0.33* (0.18)
R2 1,240 300
All
Schools Low 2007
Value Schools
(A) (B)
Organizational Climate 0.80*** (0.02)
0.86*** (0.03)
Survival Anxiety 0.04*** (0.01)
0.07*** (0.03)
Survival Anx*Org Climate 0.05*** (0.01)
0.01 (0.02)
Peer Index 0.01*** (0.00)
<0.001 (0.00)
Part of Empowerment Zone
-0.16*** (0.03)
-0.08 (0.07)
Affiliated with Partner Support Org
0.04 (0.05)
0.22** (0.09)
Proportion of <4 yr teachers
-0.004*** (0.002)
0.002 (0.004)
Proportion of 15+ yr teachers
0.01** (0.00)
0.01** (0.01)
Stable leadership (dummy) 0.003** (0.001)
0.01** (0.002)
Number of full-time teachers
-0.25*** (0.06)
-0.11 (0.13)
Values in 2006-2007 -0.05** (0.02)
-0.02 (0.08)
Intercept 0.00*** (0.001)
0.00** (0.001)
Findings: Values-in-UseFindings: Values-in-Use
• Organizational climate = strong predictor
• Survival anxiety= negative and significant predictor
• No evidence of interaction effect
• Even fewer statistically-significant formal controls
All Schools Schools
with low
values 2007
(A) (B)
Organizational Climate 0.25*** (0.05)
0.35*** (0.11)
Survival Anxiety -0.25*** (0.04)
-0.23** (0.09)
Survival Anx*Org Climate
0.05 (0.04)
-0.03 (0.08)
Stable leadership -0.19* (0.11)
-0.04 (0.23)
High School (dummy) -0.81*** (0.18)
-0.91** (0.38)
Number of full-time teachers
0.18*** (0.05)
0.02 (0.13)
Values in 2006-2007 0.001** (0.002)
0.004 (0.01)
Intercept 0.47** (0.22)
0.41 (0.50)
n 426 108 R2 0.388 0.359
Summary of FindingsSummary of Findings
Internal Dynamics Perspective Strong support for both espoused values and values-in-use
External Dynamics Perspective Weak support for espoused values Contrary findings for values-in-use
Lack of support in formal controls
No evidence that incentives and climate shape behavior together
Internal Dynamics Perspective Strong support for both espoused values and values-in-use
External Dynamics Perspective Weak support for espoused values Contrary findings for values-in-use
Lack of support in formal controls
No evidence that incentives and climate shape behavior together
Why the Limited Impact of Survival Anxiety?
Why the Limited Impact of Survival Anxiety?
Inadequate Time Frame for change High-anxiety lack capacity and control:
Policy Complexity Political Uncertainty
Low-Anxiety schools go beyond incentives Stewards of organizational mission Efforts to define district mandates in line with
organizational mission
Inadequate Time Frame for change High-anxiety lack capacity and control:
Policy Complexity Political Uncertainty
Low-Anxiety schools go beyond incentives Stewards of organizational mission Efforts to define district mandates in line with
organizational mission
ConclusionConclusion Performance incentives are, on their own,
insufficient to change organizational behavior. Organizational change--even the development of
performance behaviors-- takes time. Policymakers and school leaders need to focus
less on perfecting accountability instruments and more on how to create the internal organizational conditions for change.
Performance incentives are, on their own, insufficient to change organizational behavior.
Organizational change--even the development of performance behaviors-- takes time.
Policymakers and school leaders need to focus less on perfecting accountability instruments and more on how to create the internal organizational conditions for change.