evaluation system of government –funded research institutes 2011. 11. 3 woo chul chai research...
TRANSCRIPT
Evaluation System of Evaluation System of Government –Funded Research Government –Funded Research InstitutesInstitutesEvaluation System of Evaluation System of Government –Funded Research Government –Funded Research InstitutesInstitutes
2011. 11. 3
Woo Chul ChaiResearch Fellow
R&D Program Evaluation Division
ConclusionConclusionⅣ.Ⅳ.
Contents
Outline Outline Ⅰ.Ⅰ.
Utilizations of Evaluation ResultsUtilizations of Evaluation ResultsⅢ.Ⅲ.
Directions and Evaluation SystemDirections and Evaluation SystemⅡ.Ⅱ.
ⅠⅠ. Outline. Outline
11
1. Purpose and Object
To Set up Roles of GRIs and Promote R&D Productivity through
Reviewing Research- and Management-Performances
36 Government-funded Research Institutes under the MEST, DAPA, etc.
Management Performances
Research Performances
Comprehensive Performance
Purpose
Object
Section
22
1991• Performed by the Prime Minister’s Office• 22 GRIs evaluated together• According to the Evaluation Results, Merge and Abolition of some GRIs made
1998• Benchmark of Foreign GRIs
1999-2005 • 4 Research Councils were Established• Each Research Council performed Evaluation of Affiliated Institutes
The First Evaluation
Management Diagnosis by Foreign Consultant
Research Council System
2006-
• Achievement of Performance Goals in R&D and Management Evaluated
2. History of Evaluation System
Performance-based Evaluation System
3. Structure of GRIs’ Evaluation
Meta-Evaluation
Self-Evaluation
33
Ministry of Education & Science Technology
Ministry of Education & Science Technology
Ministry of Knowledge Economy
Ministry of Knowledge Economy
National S&T Council
MEST(KISTEP)
MEST(KISTEP) KRCFKRCF ISTKISTK DAPADAPA
Manag
-ment
KAIST, etc
Research project
Manag-ement
KIST, etc
Research project
Manag-ement
KRISS, etc
Research project
ADDADD
Ⅱ. Directions and Evaluation System
1. Directions (’11)
To Alleviate Burden of Researchers, Evaluation Interval in Management Section is
Expanded
※ 1 Year Every 3 years
Alleviation of Evaluation Burden
Reinforcement of Utilization of Evaluation Results
For Promoting Effective Performance of the Mission and Function of GRIs,
Consulting is Provided to the GRIs concerning the Appropriate Direction of R&D,
etc.
Provision of Consulting
55
Depending on Evaluation Results, Budget is Allocated Differently to Research
Programs (10% Decrease of Budget to Unsatisfactory Programs)
66
2. Evaluation Sections (Self- Eval.)
SectionSection ObjectObject IntervalInterval ItemsItems
Research Performance
Projects funded by Government
3 years - Achievement of Performance Objectives
Management Performance
Management Activities
1 year- Improvements of Management Activities and Achievement of Management Goals
Comprehensive Performance
Comprehensive R&D and Management
Performance3 years
- Qualitative Evaluation concerning the Development Strategy of Institutes in R&D and Management
88
3. Criteria in the Evaluation of Management Performance (Self- Eval.)
Evaluation Items Indicators
Establishment of Vision and Strategy of Institute
• Excellence of President’s Leadership and Level of Achievement of Management Goals
• Efforts of Promoting S&T Culture
Achievement of Monitoring• Improvements and Utilization of Internal- and External -Evaluation Results
• Efforts for Customer’s Satisfaction
Development ofHuman Resources
• Employment of Excellent Manpower and Establishment of Global Research Networks
• Efficiency of Institute Management and Utilization of Manpower
Research Management System
• Establishment of Cooperative System among Academia, Institutes and Industry
• Utilization of Performance Management System
• Utilization of Knowledge Management System
Budget and Finance Management
• Clarity of Budget Management and Execution
4. Criteria of Meta-Evaluation
To Review the Appropriateness of Self-Evaluation System
99
ItemsItems IndicatorsIndicators
1. Appropriateness of
Evaluation Plan (30)
• Appropriateness of Evaluation Objectives
• Appropriateness of Evaluation Planning
• Appropriateness of Evaluation System (Indicator,
Weight, etc)
2. Appropriateness of
Evaluation Process (35)
• Fairness of Evaluation Committee’s Composition
and Operation
• Appropriateness of Evaluation Indicators and Methods
• Fairness of Evaluation Execution
3. Effectiveness of
Evaluation Results (35)
• Composition of Evaluation Report
• Objectiveness of Evaluation Results
• Appropriate Utilization of Evaluation Results
5. 5. Grade of Meta-Evaluation
Grade is Divided into 5 Level
Final Grade : “Appropriateness (above 70)” or
“Inappropriateness (under 71)”
- Adding up the Score of 3 Evaluation Items
Grade Score Remarks
Excellent 100 - 91
“Appropriate”Good 90 – 81
Moderate 80 – 71
Bad 70 – 61“Inappropriate”
Worst 60 -
1010
Ⅲ. Utilization of Eval. Results
1212
1. Utilization of Evaluation Results
Evaluation Results of Management Performance
- Increase or Decrease of Operating Cost of GRIs (±10%)
- Increase or Decrease of President’s Annual Salary of GRIs
Evaluation Results of R&D Performance
- Decrease of Budget in Unsatisfactory Projects ( - 10% )
- Increase of Budget in Excellent Projects ( + 10%)
Evaluation Results of Comprehensive Performance
- Guideline for Planning of Research Projects and R&D Directions
Ⅳ. Conclusion
1. Improvements
To Introduce the Absolute Evaluation System that Reflects the
Characteristics of R&D Institutes
- GRIs are Classified into 3-4 Groups by the Research Area
To Alleviate Burdens by Simplifying the Indicators in the Evaluation of
Management Performance
- Deleting the Indicators related with R&D such as ‘No. of Patents’, ‘No.
of Technology Transfer’, etc.
To Reinforce the Specialization of Evaluation
- Expanding the Tenure of Evaluation Committee Members
1414
Thank You !Thank You !