efficacy and impact of key performance indicators as perceived efficacy and impact of key...

Download Efficacy and Impact of Key Performance Indicators as Perceived Efficacy and Impact of Key Performance

Post on 02-Jan-2020

0 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  •  

    Efficacy and Impact of Key Performance Indicators as Perceived by Key Informants in

    Ontario Universities

    by

    Vivian Chan, MBA, BA, CGA

    A thesis submitted in conformity with the requirements

    for the degree of Doctor of Philosophy

    Department of Leadership, Higher and Adult Education

    Ontario Institute for Studies in Education of the

    University of Toronto

    © Copyright by Vivian Chan 2014

  • ii  

    Efficacy and Impact of Key Performance Indicators as Perceived by Key Informants in

    Ontario Universities

    Doctor of Philosophy, 2014

    Vivian Chan

    Leadership, Higher and Adult Education, Ontario Institute for Studies in Education of the University of Toronto

    Abstract

    The issue of performance indicators for colleges and universities in Ontario was first

    raised in the early 1990s by the Ministry of Education and Training’s Task Force on

    University Accountability. The decision to develop and use Key Performance Indicators

    (KPIs) was made in the 1990s (Ministry of Education and Training, 1998). The three KPIs

    for Ontario universities are Graduation Rates, Employments Rates, and OSAP Loan Default

    Rates.

    The declared purposes of the publication of the KPIs evolved over time. Initially, they

    were to enable parents and students with data to inform post-secondary education choice. The

    purposes then became benchmarks without any clear indication of what constituted

    satisfactory or unsatisfactory performance. Performance funding based on KPIs introduced a

    third phase. Finally, they were to influence the universities' programming behaviour without

    government’s direct intervention. The overall intent was to help universities improve their

    performance (MTCU, 2012). My study focused on the impact of KPIs on Ontario

    universities 10 years after they were instituted.

    This exploratory and descriptive study examined the history of why and how KPIs

    were introduced in Ontario and the perception of 12 key informants of 11 participating

  • iii  

    universities regarding the efficacy and effectiveness of the existing KPIs. The study sought

    to identify areas of concern of the participants and their suggestions on how the KPIs can be

    altered to improve their effectiveness. The findings provide an understanding of the impact

    of the KPIs and suggested alternatives, and provide valuable information and a potential

    source for evidence-based MTCU policy decisions that impact Ontario’s universities and

    their stakeholders.

    The findings suggest that a review and redefinition of the theory of accountability as

    applied in Ontario universities are required as the initial interpretation of accountability

    defined by the three KPIs is too narrow to be pragmatically useful. Most university

    participants perceived that the current KPIs are not having the intended impact. I recommend

    that both parties work together and be clear on each other’s goals and expectations to develop

    effective measures on institutional performance and accountability, and to satisfy the needs of

    the government, the universities and the public.

  • iv  

    Acknowledgements

    I would never have been able to reach the finish line of my dissertation journey

    without the generous guidance of my committee members, as well as help and support from

    my colleagues, friends, and family.

    I would first like to express the deepest appreciation to my thesis supervisor, Dr.

    Katharine Janzen, who shepherded me throughout the whole journey starting from my

    preparation of the proposal to completing the dissertation. Her recommendations and

    instructions inspired me to have the confidence to work hard and finish the thesis to the best

    of my ability. This dissertation would not have been possible without her continuing support

    and guidance.

    I would like to thank my committee members, Dr. Dan Lang and Dr. Tim McTiernan.

    Dr. Lang is a walking encyclopedia of the postsecondary education systems – not only of

    Ontario and Canada, but also of international jurisdictions. Dr. McTiernan’s advice and

    unsurpassed knowledge of the government systems have been invaluable. Both of my

    committee members have directed me through various challenging situations and truly

    enabled me to achieve this accomplishment, for which I am extremely grateful.

    I will always be grateful to Dr. Paul Stenton and Janice Winton for their generous

    advice, support, and mentorship. Their insight and thorough knowledge of the post-

    secondary education system in Ontario have enabled me to learn, understand, and navigate

    this system effectively.

  • v  

    I also wish to mention the contributions of the human subjects of my study.

    Regrettably, I cannot acknowledge them by name, but without their participation this

    dissertation would not have been possible.

    I cannot thank Dr. Wendy Cukier enough for her invaluable advice, encouragement

    and generosity in providing me with the flexibility to have the time away from my daily work

    to prepare for examinations and complete the thesis. I would also like to thank my colleagues

    at Ryerson University, especially Jennifer MacInnis and Marsha McEachrane, for always

    being there for me.

    Finally, my friends – especially Dr. Heather Munroe-Blum, Patrick Gutteridge,

    Pauline Lee, Linda Palanica, Margaret Streadwick and Jennifer Yang – have supported and

    helped me along the course of this dissertation by providing me enormous strength and the

    encouragement that I needed to complete this journey. Above all, I owe my gratitude to my

    parents, brother, sister, and their families, and special thanks to my uncle, aunts, and cousins

    who are thousands of miles away yet offered their personal support and patience at all times.

    To my friends and family, I am extremely grateful; words cannot express how much!

  • vi  

    Table of Contents

    Abstract ..................................................................................................................................... ii 

    Acknowledgements .................................................................................................................. iv 

    Table of Contents ..................................................................................................................... vi 

    List of Tables ............................................................................................................................ x 

    List of Figures .......................................................................................................................... xi 

    CHAPTER ONE: INTRODUCTION ....................................................................................... 1 

    Background ........................................................................................................................... 3 

    Statement of the Problem Situation ....................................................................................... 7 

    Purpose of the Research ........................................................................................................ 8 

    Rationale and the Researcher’s Perspective .......................................................................... 9 

    Research Questions ............................................................................................................. 11 

    Theoretical Framework ....................................................................................................... 12 

    University governance. ..................................................................................................... 12 

    Accountability. ................................................................................................................. 14 

    Conceptual Framework ....................................................................................................... 17 

    Scope and Limitations of the Research ............................................................................... 18 

    Outline of the Rest of the Chapters ..................................................................................... 19 

    Terms and Definitions ......................................................................................................... 21 

    Performance Funding ....................................................................................................... 21 

    Performance Budgeting .................................................................................................... 21 

    Performance Reporting ..................................................................................................... 21 

    Pragmatic Worldview ....................................................................................................... 21 

    Effectiveness ..................................................................................................................... 22 

    Efficacy ............................................................................................................

Recommended

View more >