REVIEWFRAMEWORK

FOR

CENTRES OF EXCELLENCE (CoE) PROGRAMME

April 2013

Contents

ACRONYMS AND ABBREVIATIONS........................................................................4

TABLES AND FIGURES............................................................................................4

1. BACKROUND.........................................................................................................5

1.1 STRATEGIC CONTEXT.................................................................................5

1.2 AIMS AND OBJECTIVES OF THE CoEs.......................................................6

1.4 CoE REVIEW PHASES................................................................................12

2. CoE REVIEW OBJECTIVES..............................................................................12

3. REVIEW PRINCIPLES.......................................................................................13

3.1 Confidentiality...............................................................................................13

3.2 Access to Information...................................................................................14

3.3 Conflict of Interest.........................................................................................14

3.4 Ethical Consideration....................................................................................14

3.5 Commitment to Excellence...........................................................................15

3.6 Alignment......................................................................................................15

3.7 Transparency................................................................................................15

4. OVERVIEW OF REVIEW PROCESS.................................................................16

4.1 Phase 1 Review............................................................................................16

4.2 Phase 2 Review............................................................................................17

5. REVIEW CRITERIA AND SCORING.................................................................22

5.1 Phase 1 Panel Review..................................................................................22

5.2 Phase 2 Postal Peer Review........................................................................23

2 | P a g e

5.3 Phase 2 Panel Review..................................................................................31

6. APPEALS...........................................................................................................33

7. MEETING PROCEDURES.................................................................................33

7.1 Phase 1 Panel Meeting.................................................................................33

3 | P a g e

ACRONYMS AND ABBREVIATIONS

CoE Centre of Excellence

CV Curriculum Vitae

DVC(s) Deputy Vice-Chancellor(s)

DST Department of Science and Technology

MoA Memorandum of Agreement

NRF National Research Foundation

NR&DS National Research and Development Strategy

NSI National System of Innovation

PAIA Promotion of Access to Information Act

RE Reviews and Evaluation [Directorate]

TABLES AND FIGURES

Table 1 Scorecard for Evaluation of CoE Phase 1 proposals

Table 2 Scorecard for postal Review of Phase 2 proposals

Table 3 Review Panel Recommendations on CoE Phase 2 proposals

Figure 1 Flow Diagram presenting Phase 1 Review Process

Figure 2 Flow Diagram presenting Phase 2 Review Process

4 | P a g e

1. BACKROUND

1.1 STRATEGIC CONTEXT

Centres of Excellence (CoEs) are physical or virtual entities of research which

concentrate existing capacity and resources to enable researchers to collaborate

across disciplines and institutions on long-term projects that are locally relevant and

internationally competitive in order to enhance the pursuit of research excellence and

capacity development. CoEs have become a common research funding instrument,

having already been established in several countries including Australia, Canada,

and the USA.

In 2004, the Department of Science and Technology (DST) and the National

Research Foundation (NRF) developed a CoE Programme Framework document

and signed a memorandum of agreement (MoA) for the establishment and

management of CoEs. The five key performance areas of CoEs are:

Research/knowledge production;

Education and training;

Information brokerage;

Networking; and

Service rendering.

According to the MoA, the operational management of the CoE programme would be

performed by the NRF in line with and to address the DST policy, and strategic

initiatives and priorities, respectively. In the same year (2004), the DST-NRF CoE

Programme was launched with the establishment of seven CoEs. In subsequent

years, two additional CoEs were established.

In 2009, the five-year review of both the CoE programme (as a funding and research

support instrument) and the first cohort of (seven (7))CoEs that were established in

2004/2005, was conducted. A number of positive findings and challenges were

identified and recommendations were made. The programme and individual centres’

Review Reports, and the respective management responses were made publicly

5 | P a g e

available, and both the NRF and DST implemented most of the recommendations.

For instance, the significant contributions (and potential) made by the CoEs, were

highlighted as positive impacts in knowledge production and human capital

development, the NRF employing a Programme Director responsible for CoEs and

the DST doubling the funding of two centres and committing to further funding the

first CoEs for an additional five years, beyond the initial ten year funding period.

1.2 AIMS AND OBJECTIVES OF THE CoEs

The aims of centres of excellence as research promotion and support instruments

are multi-pronged. They are to:

Promote knowledge and human capital development in areas of strategic

importance to South Africa;

Promote collaborative research;

Promote and develop interdisciplinary research;

Systematically develop a creative research training environment that is

internationally competitive;

Strive for the highest standards of quality, international competitiveness and

esteem of their science; and

Diffuse knowledge to where it is needed.

The objectives of the DST-NRF CoEs are to:

Ensure the integration of several smaller and related research initiatives into

large science programmes;

Reward, retain, sustain and improve scientific excellence;

Exploit the competitive advantage vested in outstanding researchers;

Promote knowledge and human capital in areas of national strategic

importance (including indigenous knowledge);

Promote collaborative research;

Promote and develop interdisciplinary research;

Systematically develop a creative research training environments that are

internationally competitive;

Raise the quality, international competitiveness, visibility and esteem of South

6 | P a g e

African science e.g. by an increase in global share of research outputs;

Promote better diffusion and exploitation of the knowledge produced by

tertiary institutions;

Achieve economies of scale through the optimisation of resources and effort

through sharing personnel, equipment, data, ideas etc.;

Ensure secure and stable funding for research and dissemination;

Allow for planned, strategic, long-term research; and

Reduce micro-management of resources by the funding agency.

1.3 New CoEs

In 2012, the Department of Science and Technology (DST) and the NRF decided to

establish a further 5 (five) new CoEs. These new centres will add to an already highly

successful cohort of CoEs that have been established since 2004. Of the five new

centres, three (3) will be in pre-determined themes or disciplines, namely, (i) Food

Security; (ii) Science, Technology and Innovation (STI) Policy and Measurement and

(iii) Mathematical and Statistical Sciences. The two remaining CoEs will be open to

any discipline or field. However of the two open CoEs, at least one centre MUST be

in the social sciences and/or humanities area. This is in consideration of the

important role that both the social sciences and the humanities play in development

and the need to strengthen both human and research capacity in those fields.

1.3.1 Pre-determined themes/disciplines CoEs

(i). Food Security

Food security is critical in the well-being of our society. The right to access sufficient

food is embedded in Section 26 and 27 of the Constitution of the Republic of South

Africa, 1996, and has been identified as one of the top priorities by the government

(State of the Nation Address, 2010). It is also aligned to the aims of the Millennium

Development Goals (MDGs) that of halving the proportion of poverty and

unemployment by 2014.

The CoE in Food Security will focus on focus on all or some of the following

objectives:

7 | P a g e

Sustainable production: To build the evidence base on agricultural

production systems that are economically, socially, and environmentally

sustainable. Research will include testing alternative land and water use

patterns, developing new models and approaches to improving soil health and

increasing production sustainability, etc; studies on socio-economic

determinants of the choices of land use options among competing claims, e.g.

cropping for food or for fuel, land use for agriculture or for industrial

development; technological innovations to optimise land productivity –

intensification vs. extensification, and the implications for land degradation

and on management of water resources.

Post-harvest technology and processing: Research on post-harvest

technology as a means of reducing food waste, increasing value and

generating employment. Specific attention will be given to research on under-

utilized and high-value crops for food and nutrition security and employment

creation; fortification of local staples for improved nutritional value; and

development of technologies to extend shelf life of perishable foods.

Access to food: Research on emerging alternative retailing systems to

increase access to nutritious food in remote areas and informal urban

settlements; food distribution channels and infrastructure; food trade policies

(national, regional and inter-regional trade regulations and their impacts);

impact of alternative social support mechanisms (e.g. conditional transfers,

grants, vouchers) on access to nutritious food and the development of

alternative markets and rural agribusiness initiatives;

Consumer demand, food utilization and nutritional outcomes: Research

on the impact of labeling, advertising, and nutrition and health education, on

food choices (at different socio-economic levels). Nutrition and health impact

of agricultural innovation; and

Knowledge transfer and leadership in the food system: Research on

institutional dimensions of food security, including leadership development,

mentorship programmes, public-private partnerships, policy dialogue

processes including scenario-planning; food security monitoring systems and

programme evaluation approaches; perceptions and understanding of food

security at household, community, and national levels.

8 | P a g e

(ii). Science, Technology and Innovation Policy and Measurement

Lack of a coherent and authoritative single source of information on STI indicators

was identified as one of the key challenges of the National System of Innovation

(NSI). The appropriate use of these indicators in monitoring and measuring the

performance of the NSI, review and effective implementation of STI policies are

critical to NSI development and international competitiveness Therefore, the

objectives of this CoE will be:

Building capacity: Drive the development and sharpening of in-country skills

and capacity in developing STI indicators, measurement and analysis of

national and international STI environments, thereby enhancing NSI policy

analysis, development and review to improve the general understanding of the

interaction between scientific development and the economy;

Consolidation: Draw on the existing pockets of excellence in the STI

measurement arena to provide a single source expertise and information that

will support the policy advisory structure likely to emerge from the Department

of Science and Technology’s response to the Ministerial Review Committee

Report (2012) and future strategic documents. ;

Research: Provide research leadership and support for STI policy

measurement and related research evaluation studies and science and

innovation policy studies.

(iii). Mathematical and Statistical Sciences

The importance of a sound research base in mathematical sciences is absolutely

critical for national development in science, engineering, commerce, and technology.

The two independent reviews of mathematics pointed to the need for an urgent

system intervention. The review of Mathematical Sciences Research at South African

Higher Education Institutions (international Review Report, 2008) identified among

other issues that research in mathematical sciences in South Africa is characterised

by isolation at many levels, especially identifying academic isolation of individuals

(postgraduates and researchers). The review report argued that “…the inward,

institution-oriented approach to research in the mathematical sciences needs to be

transformed into a national approach. Notwithstanding a number of strong examples

9 | P a g e

of connectivity, overall there is geographical isolation from international centres of

research; there is geographical isolation of institutions within the country; there is

academic isolation of individuals (postgraduates and researchers) in their research

fields; and there is institutional isolation of the mathematical sciences from its

applications and related disciplines”. In reference to the field of statistics, the review

saw it as a special case, in that it has experienced all the blockages to research and

human capital development, but has reached now a stage that is so critical that the

field is in danger of disappearing through lack of academic capacity. The closure of

academic departments in statistics is a real possibility.

The Academy of Science of South Africa (ASSAf)’s State of Science (2009)

publication reveals a worrying statistic that researchers in mathematical sciences

lowers than 40 years were responsible for only 10% of journal outputs, and over 80%

of article output is produced by white researchers with black researchers producing

only about 10%.Both the 2008 Review of Mathematics and the ASSAf study

acknowledged that there are pockets of good-to-excellent research (by international

standards) in all the mathematical sciences (pure mathematics, applied mathematics,

statistics, and mathematics education) in Higher Education Institutions) but there is

need for consolidation. The means and the environment to produce effective

research needs to be addressed.

The importance of a sound foundation and research base in the mathematical

sciences is absolutely critical for national development in science, engineering,

commerce, and technology. Without this foundation, attempts at establishing a

knowledge-based economy and generating innovation are doomed to failure. The

development and transformation of human resources in science, engineering, and

technology depend in a fundamental way on mathematical understanding, and a

significant presence in each field of very highly skilled mathematicians. The need for

highly skilled human resources to effectively use and benefit from the massive data

to be generated through the square kilometre array (SKA) for example cannot be

over-emphasized.

10 | P a g e

The CoE in mathematical and statistical sciences will focus on some or all of the

following objectives:

Knowledge and partnerships: build knowledge and research partnerships in

South Africa between the fundamental disciplines of Mathematics, Theoretical

Physics, experimental Biosciences and Public Statistics;

Enable collaboration: Address the isolation in mathematical sciences

challenges prevalent in the field;

Human capital generator: Generate the necessary human capital in

mathematical sciences, especially among the young generation; and

1.2.1. Open non pre-determined Themes/fields CoEs

Unlike the CoEs in pre-determined fields or themes, proposals for CoEs with non-

predetermined fields MUST primarily show existing capacity and resources to enable

researchers to collaborate across disciplines and institutions in addressing locally

relevant, government/national priorities and internationally competitive areas in order

to enhance the pursuit of research excellence and capacity development (see

http://www.dst.gov.za/index.php/resource-center/strategies-and-reports).

Although proposals for hosting the CoEs in the non-predetermined disciplines and or

themes will be accepted from any discipline, at least one of the CoEs will be

awarded in the Social Sciences and or the Humanities. In this regard, such the CoE

should aim at addressing some of or the majority of the issues emanating from the

(Department of Higher Education and Training’s (DHET’s) Report Commissioned by

the Minister of Higher Education and Training for the Charter for Humanities and

Social Sciences; and the Academy of Science of South Africa’s (ASSAf’s)

Consensus Study on the state of the Humanities in South Africa: status, prospects

and strategies, and ASSAf’sThe State of Science in South Africa among others.

All CoE proposals MUST take cognisance of the philosophy and principals of

Centres of Excellence as reflected in the Framework for the establishment of DST-

NRF Centres of Excellence ver.2.0, 2012

11 | P a g e

1.4CoE REVIEW PHASES

The new DST-NRF Centre of Excellences will be awarded in an open and

competitive process comprising two phases, namely, Phase 1 and Phase 2.

Submission of proposals in both phases will be assessed for the extent to which they

address the requirements of each phase and the scientific merit of the submissions.

Soon after posting the call, Deputy Vice Chancellors, Directors of Research and /or

Executives responsible for research at South African publicly funded universities,

science councils and museums will be invited to the DST-NRF briefing session to

explain the purpose of the CoE, expectations of hosting and collaborating institutions

and the process that will be followed for selection.

2. CoE REVIEW OBJECTIVES

The objective of Phase 1 review will be to evaluate:

Alignment of the proposed CoEs with the mission and research strategy of the

host and collaborating institutions;

Alignment of the CoEs with national research and development strategies;

Readiness and commitment of the institution/s to host the proposed CoEs

(management potential of the host institution/s, existing capacity, the track

record of the host and collaborating institutions in the relevant fields);

Potential for the proposed CoE to concentrate existing capacity and resources

to enable researchers to collaborate across disciplines and institutions on long

term projects;

Brief scientific focus of the proposed CoE.

Note: An institution is only allowed to submit a maximum of two (2) proposals as potential host for CoEs. However, institutions are allowed to participate as (potential) collaborators in as many CoEs as they see fit.

Phase 2 will primarily focus on the assessment of the scientific quality, human capital

development capacity of the full proposal and the CV of the proposed candidate

(Centre Director). The full proposals must show how they plan to address the

12 | P a g e

objectives of the DST-NRF Centre of Excellence initiative as reflected in the CoE

Framework document and highlighted above.

Through a process of postal peer review, national and international reviewers will

primarily focus their assessments of the Phase 2 proposals on the following:

How the five key performance areas of CoEs, namely research/knowledge

production, capacity development, networking, information brokerage and

service rendering will be addressed;

Research track record of the nominated Director of the Centre to be based at

the host institution;

Relevance and scientific merit of the proposed plan;

Strategies and processes for promoting and developing collaborative and

interdisciplinary research; and

Plans for leveraging other sources of finding in the future.

Site visits to shortlisted institutions will form the last leg of the Phase 2 of the review

process. The Review Panel including the NRF and DST staff (as appropriate) will

visit the institutions to assess the facilities and commitment of the members of the

proposed CoE.

3. REVIEW PRINCIPLES

3.1 Confidentiality

In line with the NRF’s principles on confidentiality, openness and fairness in the peer

review process, CoE Programme treats information contained in all submitted

documents as confidential. All individuals involved with the review and awarding

process will be bound to confidentiality agreements. Where a reviewer identifies a

need to consult with another expert on a specific aspect of a proposal, the NRF must

first be consulted and the confidentiality of the proposal and anonymity of the

applicant and the proposal must both be maintained.

13 | P a g e

3.2 Access to Information

The NRF complies with the Promotion of Access to Information Act (PAIA) and in this

respect an NRF PAIA Manual provides specific guidelines for effecting the Act when

and where appropriate. The NRF will keep a signed written record of panel

consensus and a written record and voice recordings of Review Panel meetings.

Reviewers’ comments submitted to the NRF may be provided to an applicant, on a

confidential and anonymous basis to allow applicants to respond to issues raised as

part of the peer review process in an attempt to benefit the research programme and

to improve future applications for unsuccessful applicants.

3.3 Conflict of Interest

To ensure a fair and unbiased review process, in line with NRF procedures, CoE

Programme requires all individuals involved in the review processes to declare any

personal and/or professional interests in applications under review. This will enable

the NRF and Review Panel Chairs to identify and manage any conflicts of interest.

Conflicts of interest may arise from close links with, or interest in, an institution from

which a funding proposal is being considered. Such conflict may arise due to

employment or academic collaborations; commercial or financial interest; non-

financial interest where a member has other interests that may be thought to

influence them, either intentionally or unintentionally; and personal or family relations.

A potential conflict of interest may also arise where the reviewer is located at the

same department or institution as the applicant(s). Where such a conflict arises, the

concerned individual shall recuse his/her from the assessment and or review

processes.

3.4 Ethical Consideration

The review process relies on the integrity and accountability of reviewers. Reviewers

will be selected based on their expertise relating to one or more aspects of the

proposal(s) under review. However, reviewers must also be aware of subtle biases

14 | P a g e

that can influence their judgment and recommendations and ensure impartiality at all

times.

During the review of a proposal, a reviewer may discover ethical issues that must be

considered and addressed by the CoE. It remains the responsibility of the host

university to ensure that all research undertaken by CoEs have the necessary ethical

approvals.

3.5 Commitment to Excellence

The CoE Programme drives and maintains quality and excellence in research and

innovation through its rigorous review process and careful selection of suitable

reviewers and panellists based on their academic and research expertise and,

understanding of the South African Higher Education System and National System of

Innovation (NSI).

Quality reviewer reports add tremendous value to the peer review process by

providing balanced reports with clear comments and recommendations, a justification

for overall rankings and constructive criticism and feedback to applicants.

3.6 Alignment

CoE Programme is a national strategic intervention of government; therefore

applications shall clearly indicate alignment between the proposed research area and

national research and social development priorities. To ensure sustained university

commitment and prioritisation of research areas in which CoEs are applied for, the

proposed research area shall be aligned with the host university research strategy.

The programme is aimed at increasing research and scientific innovation at South

African universities through the development of human capacity and stimulating the

generation of new knowledge.

3.7 Transparency

The peer review process and the criteria for assessing proposals are made public to

the research community through this framework document. The names of the review

15 | P a g e

panel members and of the adjudication panel members will also be made available

on the NRF website at the end of the awarding and approval process.

4. OVERVIEW OF REVIEW PROCESS

The application for Centres of Excellence consists of two phases namely, Phase1

and Phase 2.Proposals may be returned to the institution without review following an

NRF screening process, if the submitted proposal:

does not meet the announced proposal submission deadline date;

does not meet the eligibility criteria;

does not include the requested supporting documentation;

does not have the required institutional approvals;

has sections of the application form that have not been completed in full; or

substantially exceeds the recommended page limitations.

4.1 Phase 1 Review

4.1.1 Purpose of Phase 1 Review

The purpose of Phase 1 of the CoE application process is to shortlist institutions that

demonstrate readiness, commitment and suitability to host and support the CoEs as

assessed against Phase 1 review objectives. The shortlisted applications will

proceed to the final stage of review (Phase 2) in which the NRF will make an award

to the successful institution.

4.1.2 Review and Awarding Process

Phase 1 proposals will be reviewed by review panels comprising of representatives

from the Department of Science and Technology, National Research Foundation as

well as experts in the field. The Phase 1 review panels will make recommendations

on whether the applications will progress to the 2nd Phase of review or not.

16 | P a g e

4.2 Phase 2 Review

4.2.1 Purpose of Phase 2 Review

Phase 2 will award the CoE to the successful institution. The review process will

involve the evaluation of both the nominated candidate as the Centre Director and

the proposed research programme against Phase 2 review objectives.

4.2.2 Review Panels

Review Panel meetings will take place at the NRF or a venue indicated by the NRF.

The NRF will host briefing meetings before commencement of reviews to orient panel

members on the strategic intent of CoE Programme and on review processes and

procedures. All reviewers will be expected to attend at least one of these briefing

meetings.

Composition of Review Panels

The credibility and quality of the review process depends on the integrity, expertise

and experience of the reviewers. Review panels will be constituted based on areas

identified for this call for CoE applications. Reviewers will be appropriately allocated

to panels based on their area of expertise.

The composition of the review panels will take cognisance of the principle of equity in

terms of the race and gender diversity of the South African research community and

the institutional diversity of the universities and research institutions.

Each panel will comprise the following participants:

An external Chairperson;

An independent Assessor;

Discipline based Reviewers;

An NRF resource person (Executive Director or Director);

Secretariat; and

A Scribe.

Role of Review Panel Members

17 | P a g e

Panel Chairperson

The panel Chairperson will be an individual of acknowledged credibility and authority,

who commands respect within South Africa’s NSI. The Chairperson must have

research and management experience and be well versed with facilitation methods,

such as dealing with points of dispute that may arise during the panel meeting. The

chairperson will be responsible for overseeing and providing leadership during the

review process. He/she plays a role in developing a sense of common purpose

amongst the panel members. During the entire course of the panel deliberations, the

chairperson ensures that panel members are mindful of and observe the following:

The context and focus of the review against the background of CoE

Programme objectives;

The code of conduct and procedures that apply to the review process; and

The roles and responsibilities of the panel members.

In addition, the chairperson will facilitate discussions on each application and guide

panel members towards a consensus decision in an impartial manner.

The Assessor

The Assessor will be a person who has sufficient experience in granting and

proposal review processes, in particular those of the NRF. The role of the assessor is

to conduct quality assurance of the panel review process to ensure compliance with

the following:

that proposals presented for review meet eligibility criteria;

that the scoring is fair and independent and, that the same criteria are applied

consistently by the review panels in accordance with prescribed scoring rules;

and

that proceedings at the panel meeting are in accordance with the CoE Review

Framework.

At the conclusion of the panel meetings, the assessor will provide the NRF with a

consolidated report that will assist CoE management to make any necessary

amendments that will improve the review process.

18 | P a g e

Reviewers

Reviewers will be individuals from broad disciplines covering the area/s identified for

this call for CoE applications. Individuals with substantial research and/or research

management experience and have an understanding of the South African Higher

Education System and NSI. Furthermore, they will be individuals with no direct

association with any of the proposals for which he/she is serving as a reviewer.

The role of reviewers will be as follows:

Review proposals submitted by universities against Phase 1 review objectives

outlined in this document and the criteria described in the scorecard presented

in this document; and

Provide a quantitative (scoring) as well as detailed qualitative written

evaluation of the proposal. This will be used by the NRF to provide feedback

to the applicant for improvement of the research programme or of future

applications. Reviewers are required to use the evaluation form provided by

the NRF for evaluating proposals and the scorecard provided for scoring

proposals.

All proposals will be presented by one or two assigned reviewers, referred to as

discussion leaders, before being discussed by the review panel and unanimously

agreed upon scores being allocated to each application. The review panel will give a

consensus recommendation on whether or not the proposed CoE should be

awarded.

NRF Executive Directors or Directors

Each panel will be assigned an Executive Director or Director from the NRF. These

individuals will function as resource persons to the Chairperson and the panel in

providing context with regard to the strategic intent and purpose of CoE Programme

and to address any matters requiring clarity.

Panel Secretariat

19 | P a g e

The secretariat will be drawn from Reviews and Evaluation (RE) staff and with the

assistance of the CoE Programme staff members. The role of the panel secretariat is

to provide the panel Chairperson, assessors and reviewers with the following

support:

Administrative and logistical support services;

Prepare all required review documentation; and

Prepare a report of the meeting proceedings in consultation with the scribe.

Scribe

The scribe from RE directorate will keep verbatim notes of the discussions during

panel meetings, record the proceedings and provide a report for the NRF. If

necessary the voice recording of the review panel deliberations will be used to

ensure accuracy and completeness of the information captured. This report together

with reviewers’ written qualitative evaluation will be used by the NRF to provide

feedback to the applicant.

4.2.2 Review and Approval Process

Phase 2 proposals will be evaluated firstly through a postal peer review process by

persons with relevant discipline/field expertise in the respective proposals’ research

areas. Review panels comprising of individuals from broad disciplines will draw on

the expert reviewers’ written feedback and make recommendations to the NRF for

the approval of the proposed research programme and on the suitability of the

candidate for appointment as a Centre Director.

4.2.3 Postal Peer Reviews

Postal peer reviews will be conducted by subject experts that are individuals with

high academic and professional credibility and have achieved international

recognition for their research contribution in their field. Individuals will be selected

from active researchers from both the national and international research

communities.

20 | P a g e

Role of Postal Reviewers

The role of postal reviewers who are subject experts is to:

Review proposals against Phase 2 review objectives;

Provide expert opinion on the scientific merit of the proposal and feasibility of

the proposed research programme;

Assess how the five key performance areas will be addressed;

Review the nominated candidate against set criteria for the profile of CoE ;

and

Provide a quantitative (scoring) as well as detailed qualitative written

evaluation of the proposal. This will be used by the NRF to provide feedback

to the applicant for improvement of future applications. Reviewers are required

to use the evaluation form provided by the NRF for evaluating proposals and

the scorecard provided for scoring proposals.

All proposals will be reviewed by at least three subject experts and three useable

review reports for each proposal will be sourced and presented in the Phase 2 panel

reviews.

4.2.4 Review Panels

Role of Review Panel Members

All panel members, except for reviewers, will have the same roles as for the Phase 1

panels. Phase 2 panel review members will have access to written reviewer reports,

submitted by subject expert reviewer’s, on proposals evaluated against Phase 2

review objectives. The review panel will consider the following in making

recommendations for the Phase 2 proposals:

Profile of the nominated candidate;

The proposed plan to fulfil CoE Programme objectives, including the strategic

considerations indicated in the university proposal;

The proposed research programme of the CoE;

Specific objectives, outputs and outcomes for the five-year period.

21 | P a g e

The proposed budget allocation of the research grant and projected funding to

be leveraged.

The Phase 2 review panel will provide a quantitative (score) as well as detailed

qualitative written evaluation report of the Phase 2 proposal. This will be used by the

NRF to provide feedback to the applicant(s) for improvement of the research

programme or of future applications. Reviewers are required to use the evaluation

form provided by the NRF for evaluating proposals and the scorecard provided for

scoring proposals.

All applications will be presented by one or two assigned reviewers, referred to as

discussion leaders, before being openly discussed. Unanimously agreed upon

scores will be allocated to each proposal.

5. REVIEW CRITERIA AND SCORING

5.1 Phase 1 Panel Review

Assessment of the Phase 1 proposal will not focus on the detailed scientific

programme, but rather on the technical issues which include:

Alignment of the proposed CoE with the university research strategy and the

identified research area;

Research strengths, capabilities and competencies of the university in the

area of a proposed CoE;

Commitment of the institution to provide an enabling environment to ensure

the success of the CoE; and

Brief scientific focus of the proposal.

The scoring below will be used by reviewers to assess individual sections of the

Phase 1 proposal submitted by universities.

22 | P a g e

Table 1.Scorecard for evaluating CoE Phase 1 Proposals

Numeric Score Meaning of score Descriptor

5 Excellent

The proposal is well thought through and

motivated and, has fully addressed all the

issues/requirements in this section.

4 Good

The proposal has addressed all the

issues/requirements in this section although

there are some issues that the institution is

advised to bear in mind.

3 Satisfactory

The proposal has adequately addressed the key

issues/requirements in this section. However,

there is one or more minor issues that should be

addressed before an award may be made.

2 Unsatisfactory

The proposal partially addresses the

issues/requirements in this section. However,

some key issues/requirements have not been

addressed.

1Insufficient

information

The proposal has not provided sufficient

information in this section to enable a fair

evaluation.

5.2 Phase 2 Postal Peer Review

Phase 2 proposals will be evaluated by subject experts. The subject expert reviewers

will evaluate proposals and make recommendations to the review panel on the

following aspects of the full proposal:

Profile of the nominated candidate;

The proposed plan to fulfil CoE Programme objectives, including the strategic

considerations indicated in the university proposal;

The proposed research programme of the CoE;

Specific objectives, outputs and outcomes for the five-year period; and

The proposed budget allocation of the research grant and projected funding to

be leveraged.

23 | P a g e

The reviewers will evaluate and score the Phase 2 proposals using the criteria detailed in Table 2. The qualitative comments and

scoring will guide and inform Phase 2 panel review members in making recommendations to the NRF for approval of the nominated

candidate and proposed research programme:

Table 2.Scorecard for postal peer review of CoE Phase 2 proposals

SCORE 1 2 3 4 5DESCRIPTOR Internationally

leadingInternationally competitive, and leading nationally

Leading nationally, not yet internationally competitive

Nationally competitive

Uncompetitive

REVIEW CATEGORYTrack record of nominated candidate:

Research outputs

Impact of research

National and international standing

The candidate is

outstanding and

has a track record

that is world

leading in terms of

the quality and

impact of research

outputs and,

The candidate is

excellent and has a

track record that is

internationally

competitive in

terms of the quality

and impact of

research outputs

The candidate has

a track record that

is at the forefront

nationally and is

becoming

internationally

competitive in

terms of the quality

The candidate has

a track record that

is satisfactory but

is not at the

leading edge in

terms of the quality

and impact of

research outputs

The candidate has

a track record that

is unsatisfactory in

terms of the quality

and impact of

research outputs

and, training and

mentoring of

Masters and doctoral postgraduate training

Training of emerging researchers, e.g., postdoctoral fellows and academic staff

training and

mentoring of

postgraduate

students and

researchers.

The candidate

meets the

requirements for

appointment as a

Centre Director.

and, training and

mentoring of

postgraduate

students and

researchers.

The candidate

meets the

requirements for

appointment as a

Centre Director.

and impact of

research outputs

and, training and

mentoring of

postgraduate

students and

researchers.

The candidate

does not meet the

requirements for

appointment as a

Centre Director.

and, training and

mentoring of

postgraduate

students and

researchers.

The candidate

does not meet the

requirements for

appointment as a

Centre Director

postgraduate

students and

researchers.

The candidate

does not meet the

requirements for

appointment as a

Centre Director

Scientific merit of proposed Centre programme

The proposed

research

programme is

highly innovative

with novel design

and

methodologies.

The proposed

research

programme is

innovative with

novel design and

robust

methodologies.

The proposed

research

programme is

original with robust

design and

methodologies.

The proposed

research

programme is

original with sound

methodologies but

has a limited

number of flaws or

omissions that

require significant

The proposed

research

programme has

major omissions or

conceptual flaws

and requires

comprehensive

revision.

25 | P a g e

revision.

Potential Impact The research

programme

addresses a crucial

knowledge gap and

will be an

internationally

unique resource in

the discipline. The

research outputs

have a high

potential for

scientific

innovations or for

social or economic

benefits.

The research

programme

addresses a crucial

knowledge gap and

will be a unique

resource in South

Africa. The

research outputs

have a high

potential for

scientific

innovations or for

social or economic

benefits.

The research

programme

addresses a

knowledge gap in

South Africa and

will be a useful

resource. The

research outputs

have a reasonable

potential for

scientific

innovations or for

social or economic

benefits.

The research

programme

addresses

worthwhile

scientific

questions. The

research outputs

have a moderate

potential for

scientific

innovations or for

social or economic

benefits.

The research

programme

addresses

potentially

worthwhile

scientific questions

but requires

comprehensive

revision. There is a

low probability for

producing scientific

innovations or for

social or economic

benefits.

Human resource development

The proposed

human resource

development plan

for training masters

The proposed

human resource

development plan

for training masters

The proposed

human resource

development plan

for training masters

The proposed

human resource

development plan

for training masters

The proposed

human resource

development plan

for training masters

26 | P a g e

and doctoral

students and

postdoctoral

fellows is realistic

and achievable

within the

timeframe. There

are low probability

risks that can be

managed and that

present negligible

consequences for

the CoE

programme.

and doctoral

students and

postdoctoral

fellows is realistic

and achievable

within the

timeframe. There

are low probability

risks that can be

managed and that

present moderate

consequences for

the CoE

programme.

and doctoral

students and

postdoctoral

fellows is realistic

although there are

some potential

risks that may

present significant

consequences for

the CoE

programme.

and doctoral

students and

postdoctoral

fellows is

satisfactory

although there are

some potential

risks that may

present high

consequences, for

the CoE

programme.

and doctoral

students and

postdoctoral

fellows is

unsatisfactory and

requires

comprehensive

revision.

Collaboration The proposed

national and/or

international

collaborations,

partnerships and

networks to be

formed are

outstanding and

The proposed

national and

international

collaborations,

partnerships and

networks to be

formed are

comprehensive.

The proposed

national and

international

collaborations,

partnerships and

networks to be

formed are good

although there are

The proposed

collaborations,

partnerships and

networks to be

formed are

satisfactory

although there are

some omissions

The proposed

collaborations,

partnerships and

networks to be

formed are

unsatisfactory and

require major

27 | P a g e

represent world

leading standards.

Collaborators and

partners have been

identified to

develop and

maintain

excellence in this

research area.

Some collaborators

and partners have

already been

identified to

develop and

maintain

excellence in this

research area.

some omissions

that may present

significant

consequences for

the programme,

that the Director is

advised to bear in

mind.

that may present

high

consequences, for

the programme,

that the Director

must address.

revision.

Succession plan The proposed

succession plan for

developing and

mentoring

emerging

researchers and

new leadership is

realistic and

achievable within

the timeframe.

There are low

probability risks

that can be

The proposed

succession plan for

developing and

mentoring

emerging

researchers and

new leadership is

realistic and

achievable within

the timeframe.

There are low

probability risks

that can be

The proposed

succession plan for

developing and

mentoring

emerging

researchers and

new leadership is

realistic although

there are some

potential risks that

may present

significant

consequences for

The proposed

succession plan for

developing and

mentoring

emerging

researchers and

new leadership is

satisfactory

although there are

some potential

risks that may

present high

consequences, for

The proposed

succession plan for

developing and

mentoring

emerging

researchers and

new leadership is

unsatisfactory and

requires

comprehensive

revision.

28 | P a g e

managed and that

present negligible

consequences for

the continuity of the

CoE programme.

managed and that

present moderate

consequences for

the continuity of the

CoE programme.

the continuity of the

CoE that the

Director is advised

to bear in mind.

the continuity of

the CoE

programme that

the Director must

address.

Research activity plan

The proposed

activity plan is

outstanding and

represents world

leading standards.

It is realistic and

achievable within

the five-year

timeframe.

The proposed

activity plan is

comprehensive. It

is realistic and

achievable within

the five-year

timeframe.

The proposed

activity plan is

good although

there are some

potential risks, that

may present

significant

consequences for

the programme,

that the Director is

advised to bear in

mind.

The proposed

activity plan is

satisfactory

although there are

some potential

risks that may

present high

consequences, for

the programme,

that the Director

must address.

The proposed

activity plan is

unsatisfactory and

requires

comprehensive

revision.

29 | P a g e

Budget The resource

allocation of the

grant is

reasonable. There

are other sources

of research funding

available. Overall

there is the

potential for a high

return on

investment.

The resource

allocation of the

grant is

reasonable. There

are other sources

of research funding

available. Overall

there is the

potential for

significant return

on investment.

The resource

allocation of the

grant is

reasonable. There

are other potential

sources of

research funding.

Overall there is the

potential for return

on investment.

The budget section

has a limited

number of

omissions or flaws

that require

significant revision.

The budget section

has major

omissions or flaws

and requires

comprehensive

revision.

30 | P a g e

5.3 Phase 2 Panel Review

The panel will consider postal reviewer reports and allocated scores submitted by

subject experts. In summarising the decisions, the panel member will focus on the

following areas:

Profile of the nominated candidate;

The proposed plan to fulfil CoE Programme objectives, including the strategic

considerations indicated in the university proposal;

The proposed research programme of the CoE;

Specific objectives, outputs and outcomes for the five-year period; and

The proposed budget allocation of the research grant and projected funding to

be leveraged.

For each Phase 2 proposal the review panel will make recommendations to the NRF.

Table 3. Review panel recommendations on CoE Phase 2 proposals

Nominated candidate Recommended for approval

Does not meet the criteria for appointment as a

CoE Director and is not recommended for

approval.

Not recommended for approval as the

Curriculum Vitae of the nominated candidate

provides insufficient information and/or is

presented in a format that does not enable a

fair evaluation of the candidate.

Research programme and research activity plan

Recommended for approval without any

revisions;

Recommended for approval with minor

revisions that may be approved by the NRF

without further peer review; or

Not recommended for approval and requires

major revisions and further peer review.

Proposed Budget Recommended for approval without any

revisions;

Recommended for approval with minor

revisions that may be approved by the NRF

without further peer review; or

Not recommended for approval and requires

major revisions and further peer review.

Note: Following the successful review of the Phase 2 proposal, the NRF may, in

consultation with the approved candidate and executives of the research

office, make adjustments to the budget prior to submitting the Conditions of

Grant for approval by the CoE and the host university.

32 | P a g e

6. APPEALS

Applicants will only be allowed to lodge an appeal in the event of unfair or suspected

faulty processes. No appeals will be entertained with respect to scientific merit and

or quality of proposal as determined by peers. Appeals MUST be lodged within three

(3) months of announcement of the outcomes. Appeals MUST be lodged by

Designated Authorities within the research or higher education institution

7. MEETING PROCEDURES

7.1 Phase 1 Panel Meeting

7.1.1 Briefing Meeting

A briefing meeting will be convened by the NRF Secretariat before the panel

meetings to orientate the panel Chairpersons, assessors, reviewers and other panel

members on the proceedings, procedures and documents to be used during the

review process. This meeting will:

Provide an opportunity to clarify and discuss issues that the panel members

regard as particularly important for the review process; and

Ensure that the panel members fully understand their responsibilities during

the review.

7.1.2 Pre-Evaluation Processes

Panel members will have received all documentation pertaining to proposals

assigned to them at least two weeks before the panel meeting. They will therefore

have adequate time to acquaint themselves with the proposals in preparation to

undertake the review.

33 | P a g e

Briefing Sessions for DVCs and Research EDs(April 2013)

Call for Phase 1 proposals from universities(April – May 2013)

Panel Reviews to shortlist applications(June 2013)

Feedback to institutions (June 2013)

Targeted institutional visits(April 2013)

Shortlisted institutions prepare to submit full proposals& CV of the nominated candidate

(June/July 2013)

Figure 1: Flow Diagram presenting Phase 1 Review Process

34 | P a g e

Call for Full Proposals (July – September 2013)

Approval(November 2013)

Postal reviews for subject expert assessment of research proposals and nominated candidates

(October 2013)

Panel reviews for approval of proposals and nominated candidates

(October 2013)

Feedback to institutions (December 2013)

Award(December 2013)

Site Visits

Figure 2: Flow Diagram presenting Phase 2 Review Process

35 | P a g e

NRF Contact Persons

Dr Andrew Kaniki, Executive Director: Knowledge Fields Development and Acting Executive

Director: Reviews and Evaluation (012 481 4260; email-andrew@nrf.ac.za

Dr Ndanduleni B. Nthambeleni, Executive Director: Grants Management and Systems

Administration (phone – 012-481 4182; email - bernard@nrf.ac.za)

Dr Romilla Maharaj, Acting Executive Director: Research Chairs and Centre of Excellence

(phone – 012-481 4087; email - romilla@nrf.ac.za)

Dr Nthabiseng Taole, Programme Director: Research Chairs and Centre of Excellence

(phone – 012-481 4245; email – Nthabiseng.taole@nrf.ac.za)

Mr Raven Jimmy, Grant Director: International Research Grants and Centre of Excellence

(phone – 012-481 4069; email – raven@nrf.ac.za)

36 | P a g e