www.uis.unesco.org innovation data collection: methodological procedures & basic forms regional...
TRANSCRIPT
www.uis.unesco.org
Innovation data collection:Methodological procedures
& basic forms
Regional Workshop on Science, Technology and Innovation (STI) Indicators for Gulf countriesDoha, Qatar
15 to 17 October 2012
www.uis.unesco.org
Ch. 8 OM - Survey procedures
Guidelines - collection and analysis of innovation data;
Comparable results over time and across countries;
Particular circumstances may require other methodology comparability.
www.uis.unesco.org
The survey approach
The “subject” approach:
• Innovative behaviour and activities of the firm as a whole;
The “object” approach:
• Specific innovations (“significant innovation” of some kind, firm’s main innovation).
www.uis.unesco.org
Populations (1)
The target population:The target population:
• Innovation activities in the business enterprise sector (goods-producing and services industries);
• Minimum: all statistical units with at least ten employees:
• Small: 10-49;
• Medium: 50-249;
• Large: 250 and above.
www.uis.unesco.org
Populations (2)
The frame population:The frame population:
• Units from which a survey sample or census is drawn;
• Based on the last year of the observation period for surveys;
• Ideal frame = up-to-date official business register NSOs;
• If the register forms the basis for several surveys (innovation, R&D, general business), the information can be restricted to innovation.
www.uis.unesco.org
Survey methods (1)
Mandatory Mandatory surveys increase response rates;
Census or sample surveys?Census or sample surveys?
• Sample surveys - representative of target population (industry, size, region) stratified sample;
• Census - costly but unavoidable in some cases.
www.uis.unesco.org
Survey methods (2)
Domains (sub-populations):Domains (sub-populations):
• Subsets of the sampling strata;
• Potential sub-populations: industry groupings, size classes, regions, units that engage in R&D and innovation-active;
• Guidelines:» Same statistical units and classifications;
» Consistence of the methods for results calculation;
» Documentation of deviations in data treatment or differences in the quality of the results from the domains.
www.uis.unesco.org
Survey methods (3)
Sampling techniques:Sampling techniques:
• Stratified sample surveys: size and principal activity;
• Sampling fractions should not be the same for all strata;
Cross-sections: Cross-sections: standard approach - new random sample for each innovation survey;
Panel data:Panel data: alternative/supplementary approach.
www.uis.unesco.org
Survey methods (4)
Suitable respondents:Suitable respondents:
• Methods: e.g., postal surveys, web-based questionnaires, personal interviews;
• Unit’s most suitable respondent - very specialised questions that can be answered by only a few people;
• Try to identify respondents by name before data Try to identify respondents by name before data collection starts.collection starts.
www.uis.unesco.org
Survey methods (5)
The questionnaire:The questionnaire:• Pre-test;
• Simple and short;
• Order of the questions;
• Questions on qualitative indicators - binary or ordinal scale;
• International innovation surveys - attention to translation and design;
• Short-form questionnaires - units with little/no innovation activity previously reported.
www.uis.unesco.org
Survey methods (6)
Combination of Innovation and R&D surveys:Combination of Innovation and R&D surveys: Reduction in the overall response burden;
Scope for analysing the relations between R&D and innovation activities;
Increase in the frequency of innovation surveys;
Country experiences - it is possible to obtain reliable results for R&D expenditures;
Longer questionnaire;
Units not familiar with the concepts of R&D and innovation may confuse them;
Different frames for the two surveys.
www.uis.unesco.org
Survey methods (7)
Guidelines for conducting combined surveys:Guidelines for conducting combined surveys:
• Questionnaire: two distinct sections;
• Smaller individual sections;
• Comparison of results from combined and stand-alone surveys should be done with care - surveying methods should be reported;
• Samples extraction from a common business register.
www.uis.unesco.org
Estimation of results (1)
Weighting methods:Weighting methods:
• Weighting by the inverse of the sampling fractions of the sampling units, corrected by the unit non-response;
• If a stratified sampling technique with different sampling fractions is used, weights should be calculated individually for each;
• Based on the number of enterprises in a stratum;
• International and other comparisons: same weighting method.
www.uis.unesco.org
Estimation of results (2)
Non-response:Non-response:
• Unit non-response: reporting unit does not reply at all;
• Item non-response: response rate to a specific question - % of blank or missing answers;
» Disregarding missing values + applying simple weighting procedures based on the responses received assumes that respondents and non-respondents are distributed in the same way biased resultsbiased results;
» Possibility: imputation methodsimputation methods.
www.uis.unesco.org
Presentation of results
Descriptive analysis: no generalisation of results;
Inferential analysis: conclusions about target population;
• Variance for the results: (average) values for innovation indicators and their coefficients of variation and/or confidence intervals;
Results presentation: metadata (data collection procedure, sampling methods, procedures for dealing with non-response, quality indicators).
www.uis.unesco.org
Frequency of data collection
Every 2 years;
If not economically feasible frequency of 3 or 4 years;
Specify an observation period;
• The length of the observation period for innovation surveys should not exceed 3 years nor be less than 1 year.
www.uis.unesco.org
Annex A - 5. Methodological issues for developing country contexts (1)
Information system specificities:
• Relative weakness of statistical systems:
» Absence of linkages between surveys and data sets;
» Lack of official business registers;
• Involvement of NSOs;
• When lacking, basic variables about firms’ performance can be included in the innovation survey.
www.uis.unesco.org
Annex A - 5. Methodological issues for developing country contexts (2)
General methodological considerations:
• Survey application:» In-person;
» Trained personnel;
• Questionnaire design: » Sections can be separated to allow different persons in the
firm to reply them;
» Guidance/definitions;
» Language and translation of technical terms.
www.uis.unesco.org
Annex A - 5. Methodological issues for developing country contexts (3)
General methodological considerations:
• Frequency:» Every 3 to 4 years (e.g., timed to CIS rounds);
» Update a minimum set of variables every year;
• Purpose of surveys;
• Clear questions;
• Adequate legislative base;
• The results should be published and distributed The results should be published and distributed widely.widely.
www.uis.unesco.org
Example - product innovation/degree of novelty
(Product innovations) Yes No
New or significantly improved goods? ( ) ( )
New or significantly improved services? ( ) ( )
(Degree of novelty) Yes No
New to your market? ( ) ( )
Only new to your firm? ( ) ( )
During the three years 200X to 200Y, did yourenterprise introduce:
Were any of your product innovations during the threeyears 200X to 200Y:
www.uis.unesco.org
Example - innovation activities and expenditures for product and process innovations
(Innovation activities and expenditures forprocess and product innovations)
Yes NoAmount of
expenditureIn-house R&D ( ) ( ) $
External R&D ( ) ( ) $
Acquisition of machinery, equipment and software ( ) ( ) $
Acquisition of external knowledge ( ) ( ) $
Training for innovative activities ( ) ( ) $
Market introduction of innovations ( ) ( ) $
Other (preparations) ( ) ( ) $
During the three years 200X to 200Y, did your enterprise engage in thefollowing innovation activities:
www.uis.unesco.org
Example - co-operation
Within country
Abroad
A. Other enterprises within your enterprise group ( ) ( )
B. Suppliers of equipment, materials, components, or software ( ) ( )
C. Clients or customers ( ) ( )
D. Competitors or other enterprises in your sector ( ) ( )
E. Consultants, commercial labs, or private R&D institutes ( ) ( )
F. Universities or other higher education institutions ( ) ( )
G. Government or public research institutes ( ) ( )
LocationType of innovation co-operation partner
During the three years 200X to 200Y, did your enterprise co-operate on anyof your innovation activities with other enterprises or institutions?( ) Yes ( ) No
www.uis.unesco.org
Example - hampering factors
High Medium LowNot
relevantExcessive perceived economic risks ( ) ( ) ( ) ( )
Innovation costs too high ( ) ( ) ( ) ( )
Lack of appropriate sources of finance ( ) ( ) ( ) ( )
Organisational rigidities within the enterprise ( ) ( ) ( ) ( )
Lack of qualified personnel ( ) ( ) ( ) ( )
Lack of information on technology ( ) ( ) ( ) ( )
Lack of information on markets ( ) ( ) ( ) ( )
Insufficient flexibility of regulations or standards ( ) ( ) ( ) ( )
Lack of customer responsiveness to new goods or services ( ) ( ) ( ) ( )
Economic factors
Internal factors
Other factors
Degree of importanceHampering Factors
If your enterprise experienced any hampering factors during the period 200X-200Y, please gradethe importance of the relevant factors.
www.uis.unesco.org
Example - organisational innovation
(Organisational innovations) Yes NoNew business practices for organisingprocedures
( ) ( )
New methods of organising workresponsibilities and decision making
( ) ( )
New methods of organising external relationswith other firms or public institutions
( ) ( )
During the three years 200X to 200Y, did your enterpriseintroduce:
www.uis.unesco.org
Basic forms - CIS
The Community Innovation Survey (2010) (CIS 2010):The Community Innovation Survey (2010) (CIS 2010):
1. General information about the enterprise
2. Product innovation
3. Process innovation
4. Ongoing or abandoned innovation activities for pdt/pcs innovation
5. Innovation activities and expenditures for pdt/pcs innovation
6. Sources of information and co-operation for pdt/pcs innovation
7. Objectives for pdt/pcs innovation
8. Factors hampering pdt/pcs innovation activities
9. Organisational innovation
10. Marketing innovation
11. Creativity and skills
12. Basic economic information on your enterprise
www.uis.unesco.org
Basic forms - AU/NEPAD
AU/NEPAD Standard Innovation Questionnaire:AU/NEPAD Standard Innovation Questionnaire:
1. General information about the enterprise
2. Product innovation
3. Process innovation
4. Ongoing or abandoned innovation activities for pdt/pcs innovation
5. Innovation activities and expenditures for pdt/pcs innovation
6. Sources of information and co-operation for pdt/pcs innovation
7. Effects/Objectives of pdt/pcs innovation
8. Factors hampering (pdt/pcs) innovation activities
9. IPRs
10. Organisational and marketing innovation
11. Specific innovations by your enterprise