The non-trivial choice among innovation indicators

by Alfred Kleinknecht, Professor, Economics of Innovation, TU Delft &

Visiting Professor, Université Panthéon Sorbonne, Paris I

Key note speech to the Ester/ESF Workshop: "The Economic History of Patents and Innovation",

Eindhoven University of Technology, 23-26 June 2009

Related literature can be downloaded from: www.eci.tbm.tudelft.nl

Indicators of innovation:• R&D (Research & Development): Man hours of R&D

as a percentage of total personnel; or R&D budget as a percentage of total sales; or, a country's R&D spending as a percentage of its National Product

• Total innovations costs (Including non-R&D innovation costs)

• Patents granted or patent applications (at the European, US or national patent offices)

• Sales of new products: The share in total sales taken by technologically new or substantially improved products, introduced during the past 3 years. Distinction between "New to the firm" (→Imitation) and "New to the market" (→Innovation)

• New product announcements in trade journals

Note:

• These five indicators have a very weak correlation with each other: it does make a difference which one you use!

R&D (personnel or budgets):

Strengths:+ Regularly collected since the 1950s

+ Can be subdivided by product and process effort

+ Easily accessible (in aggregated form);

+ Time series available

+ Analysis of inter-sectoral technology flows is possible

R&D (personnel or budgets):

Weaknesses:- It is an input indicator- Usually it cannot be split by technical fields- It is only one of several input indicators (besides design, trial

production, tooling-up, market analysis, training, investment in fixed assets etc.)

- Undercounting of small firm R&D (calls into question usefulness for comparison across firm size classes, sectors, regions and even countries): serious measurement problems due to subsidy schemes.

- Problems with the interpretation of the Frascati definition- Use of micro data by researchers is often hampered by secrecy- “Singapore effect”- Hard to disaggregate by regions (multi-plant firms)

Total innovation expenditures:

Strength:+ Measures many more aspects than just R&D, e.g.

design, trial production, tooling-up, market analysis, training, investment in fixed assets.

Weaknesses:- Input measure

- Difficult to report in surveys (high item non-response)

- Inclusion in a questionnaire may have negative impact on overall response

Patents granted or patent applications:

Strengths:+ Long historical time series available

+ Publicly available (no secrecy problems)

+ Relatively consistent over time

+ Classification by technical field is possible

+ Citation analysis is possible

Patents granted or patent applications:

Weaknesses:- Misses non-patented innovations; note that the

probability to patent an innovation differs (substantially!):- Across firm size class;- Between firms that do or do not collaborate on R&D;- Between sectors

- Patent figures may be obscured by strategic behaviour

- Hard to classify by economic sectors

Sales of innovative products:

Strengths:+ Direct measure of innovation+ Only successful innovations (positive cash flow)+ Interesting distinction between imitation and

innovation+ By relating R&D input to innovative output, we can

say something about factors that influence efficiency of the innovation process

Sales of innovative products:

Weaknesses:- Sometimes suffering from low response rates

- Will the firms adequately interpret our definitions?

- Accuracy of answers?

- Probably quite sensitive to the business cycle

- Inter-sectoral comparisons can be problematic (length of life cycles differs)

New product announcements in trade journals:

Strengths:+ Direct measure of innovation+ Not expensive to collect+ No secrecy problems+ Can be classified by type of innovation+ Data can be extended to the past+ Covers innovations in very small firms+ Tracing of inter-sectoral flows is easily done+ Easy to disaggregate by regions+ Allows analysis of success and failure of innovation

New product announcements in trade journals:

Weaknesses:- Less suitable for comparative studies as statistical

properties of database appear dubious- Reliability depends on adequate selection of journals- Process innovations are neglected- Innovations from small market niches may be missed- Influenced by publication policy of journals and firms- Larger firms appear to be clearly under-represented

Final remarks:

• Note that the various innovation indicators have a weak correlation with each other. It therefore matters which one you use; the choice between indicators depends on your research purpose.

Further reading:• Kleinknecht, A., K. van Montfort & E. Brouwer: "The

non-trivial choice between innovation indicators", in Economics of Innovation and New Technology, Vol. 11 (2002), p. 109-121.

• OECD Oslo Manual: Guidelines for Collecting and Interpreting Innovation Data, 3rd edition (2007); download: http://www.oecd.org/document/