regi.tankonyvtar.hu€¦ · web viewevaluation and impact assessment of business climate...

Evaluation and Impact Assessment of Business Climate DevelopmentWith case studies from Small Business

Development Policy and Regulatory PolicyPéter Futó

Created by XMLmind XSL-FO Converter.

Evaluation and Impact Assessment of Business Climate Development: With case studies from Small Business Development Policy and Regulatory PolicyPéter Futó

Publication date 2011

This research was supported by the Hungarian 'Social Renewal Operational Programme' (Társadalmi megújulás operatív program) in the framework of the project „TÁMOP-4.2.1.B-09/1/KMR)


Table of ContentsIntroduction .......................................................................................................................................... 71. 1. Evaluation and impact assessment activities: industry or science? ............................................. 12. 2. Research design of impact assessment and evaluation studies ................................................... 3

1. Conceptual framework ........................................................................................................... 32. Components of research design .............................................................................................. 5

2.1. The research question ................................................................................................ 52.1.1. Research questions of impact assessments ................................................... 52.1.2. Research questions of evaluations ................................................................ 6

2.2. The underlying theory: impact mechanism of SME development measures ............ 82.2.1. The difficulties of a theoretical approach ...................................................... 82.2.2. Understanding how SME development measures function .......................... 9

2.3. The type of data to be collected ............................................................................... 102.3.1. Qualitative studies ....................................................................................... 102.3.2. Sampling strategies for quantitative impact evaluation .............................. 112.3.3. Compromises and pitfalls in sampling ........................................................ 12

2.4. The use of data in order to make inferences ............................................................ 132.4.1. Qualitative approach: comparing causal and evaluative inference ............. 132.4.2. Inference based on highly aggregated quantitative data ............................. 142.4.3. Inference in quantitative impact studies using micro data .......................... 152.4.4. Pitfalls of impact assessments ..................................................................... 18

3. 3. SME Development Policy ......................................................................................................... 191. SME development policy and its impact mechanisms ........................................................ 19

1.1. The rise of small business development .................................................................. 191.2. Special features of SME development in Central and Eastern Europe an transition countries ......................................................................................................................... 201.3. The relationship between SME policy and other policy fields ................................ 22

2. Evaluation of SME development projects, programmes and policies ................................ 242.1. Project and programme level evaluations ............................................................... 242.2. Case study “SME Employment": Survey for evaluating subsidies given in order to enhance the employment capabilities of SMEs ............................................................. 29

2.2.1. Policy context: harmonising employment and competitiveness priorities in small enterprises ............................................................................................................ 292.2.2. Purpose and method of the evaluation ........................................................ 31

2.3. Case Study “SME Innovation". Assessment of a subsidy scheme facilitating SME innovations ...................................................................................................................... 43

2.3.1. Policy context: the relationship between innovation policy and SME development ................................................................................................................................ 432.3.2. Purpose and method of the questionnaire based evaluation ....................... 452.3.3. Results of the questionnaire-based survey [bib_18] .................................. 49

2.4. Policy level evaluations .......................................................................................... 523. Revealing the effects of factors that are exogenous for SME policy ................................... 55

3.1. Explaining the business climate .............................................................................. 553.2. Comprehensive business surveys ............................................................................ 563.3. Business demography analyses ............................................................................... 573.4. Case Study “Balaton": The use of small enterprise surveys in regional enterprise development .................................................................................................................... 58

3.4.1. Purpose, genre and method of the research ................................................. 58


Evaluation and Impact Assessment of Business Climate

Development3.4.2. Policy context: local embeddedness of SMEs and the role of tourism in enterprise development ......................................................................................................... 603.4.3. Tourism in the region of Lake Balaton ...................................................... 623.4.4. Dependence of local SMEs on tourism ....................................................... 633.4.5. Company demographic indicators .............................................................. 633.4.6. Results of the questionnaire-based survey .................................................. 65

4. 4. Regulatory Policy ...................................................................................................................... 711. The quality of the legal environment of enterprises ............................................................. 712. Assessing the aggregate impact of the administrative–regulatory environment .................. 72

2.1. Regulatory reform initiatives in OECD member states .......................................... 722.2. The OECD Economic Regulation Index ................................................................. 752.3. The “Doing Business" project of the World Bank .................................................. 76

2.3.1. Purpose and method .................................................................................... 762.3.2. Selected results of the research ................................................................... 78

3. Impact assessment of individual regulations ........................................................................ 793.1. Use, application fields and institutionalisation of RIA ............................................ 793.2. Conceptual framework of RIA ................................................................................ 813.3. Data collection for regulatory impact assessment ................................................... 823.4. Analytical methods of impact assessment ............................................................... 833.5. Simplified regulatory impact assessments ............................................................... 86

4. Projects and case studies of Regulatory Impact Assessment ............................................... 884.1. Regulatory Impact Assessment in the central organisations of the EU ................... 884.2. Regulatory Impact Assessment in Western European member states of the EU ..... 91

4.2.1. United Kingdom ......................................................................................... 914.2.2. Netherlands ................................................................................................ 934.2.3. Belgium ...................................................................................................... 934.2.4. Germany ..................................................................................................... 944.2.5. Sweden ....................................................................................................... 94

4.3. Regulatory Impact Assessment in Central and Eastern Europe ............................. 944.3.1. Impact assessment culture in Hungary ....................................................... 97

4.4. Assessing the administrative environment of enterprises in the USA ..................... 994.4.1. Regulations reducing bureaucracy and their implementation ..................... 994.4.2. Impact of the federal regulations on small enterprises in the USA .......... 100

4.5. Case Study “Croatian RIA": Introducing EU technical regulation in Croatia ..... 1024.5.1. Policy context: legal harmonisation as part of Europeanisation process . 1024.5.2. Research design ....................................................................................... 1034.5.3. Results ....................................................................................................... 1044.5.4. Follow-up .................................................................................................. 106

5. 5. Conclusions and recommendations ......................................................................................... 107A. 6. Appendices .............................................................................................................................. 110

1. Outlines of the questionnaires used in the case studies ..................................................... 1101.1. Outline of company questionnaire used for SME labour subsidy evaluation (Case Study “SME Employment") ................................................................................................... 110

1.1.1. Introductory text ....................................................................................... 1101.1.2. Questions .................................................................................................. 110

1.2. Outline of company questionnaire used for SME innovation subsidy evaluation (Case Study “SME Innovation") ............................................................................................ 115

1.2.1. Introductory text ....................................................................................... 1151.2.2. Questionnaire ............................................................................................ 115

1.3. Outline of company questionnaire used for SMEs around Lake Balaton in Hungary (Case Study “Balaton") .......................................................................................................... 119

1.3.1. Introductory text ....................................................................................... 119


Evaluation and Impact Assessment of Business Climate

Development1.3.2. Questions .................................................................................................. 119

1.4. Outline of company questionnaire used for RIA on technical regulation (Case Study “Croatian RIA") ............................................................................................................ 123

1.4.1. Introductory text ....................................................................................... 1231.4.2. Questions .................................................................................................. 124

2. Definitions - Glossary ....................................................................................................... 1253. References .......................................................................................................................... 1394. Research, consultancy and educational projects mentioned in the text ............................. 145

BIBLIOGRÁFIA ............................................................................................................................. 150


List of Tables2.1. Box 1. .......................................................................................................................................... 133.1. Box 2. .......................................................................................................................................... 253.2. Box 3 ........................................................................................................................................... 273.3. Box 4. .......................................................................................................................................... 281. Distribution of supported enterprises by firm size and sectorNumber of enterprises ................... 322. Distribution of refused enterprises by firm size and sector Number of enterprises ...................... 323. Purpose of the planned project as stated in the application document Respondents could select more than one option. .................................................................................................................................. 334. Why was the project needed? Content analysis of the responses of applicant enterprises ............ 345. What has changed in your enterprise as a consequence of the support? ....................................... 346. In what ways has the supported project improved your abilities to employ persons? ................... 357. Changes following the implementation supported project ............................................................ 368. In what ways has the support improved the competitiveness? ...................................................... 369. What would you do differently today in implementing the project? ............................................. 3710. What would have happened if you had not received the support? .............................................. 3811. What would have happened if you had received the support? A content analysis of the answers of 132 refused enterprises in words .............................................................................................................. 3912. In which ways could the employment capability of Hungarian enterprises be improved? ......... 3913. In what ways could the competitiveness of Hungarian enterprises be improved? ...................... 413.17. Box 5. ........................................................................................................................................ 4414. The composition of responding enterprises by size classes ......................................................... 4715. The composition of responding enterprises by legal form ........................................................... 4816. The composition of responding enterprises by sector of their main activity ............................... 4817. Does the company have a foreign owner or co-owner? ............................................................... 493.22. Box 6. ........................................................................................................................................ 523.23. Box 7. ........................................................................................................................................ 5318. The distribution of small- and medium enterprises around Lake Balaton according to their size Balaton Region, 2000 ..................................................................................................................................... 6319. The distribution of small- and medium enterprises around Lake Balaton according to their sector of activity Balaton Region, 2000 .......................................................................................................... 6420. The distribution of small- and medium enterprises around Lake Balaton according to small regions Balaton Region, 2000 ....................................................................................................................... 6421. The distribution of small- and medium enterprises around Lake Balaton according to the type of settlement where the premises of the enterprises are located Balaton Region, 2000 ....................... 644.1. Box 8. .......................................................................................................................................... 874.2. Box 9. .......................................................................................................................................... 954.3. Box 10. ........................................................................................................................................ 954.4. Box 11. ........................................................................................................................................ 9622. Estimated value of compliance costs caused by federal regulations United States of America, 2004 .......................................................................................................................................................... 1014.6. Figure 9.: Simplified scheme of impact mechanism of European legislation for ensuring the free movement of goods ........................................................................................................................ 10223. Significance of various impacts of the Low Voltage Directive in Croatia ............................... 105


IntroductionThe climate of business, also called the environment of business activity is an invisible, complex and elusive concept. However, if the business climate of a country becomes beneficial, it has very spectacular and measurable consequences in terms of lively market places, the number new firms entering the market each year, the share of new high-growth firms and an increasing attractiveness of the country for foreign investors.

Business climate is a term which indicates how business development is supported by state, regional and local policies, by local communities, how business-to-business networks and labour relationships facilitate business activities. A good business climate allows businesses to conduct their affairs with minimal interference from authorities, while enabling the access to high quality inputs and to customers at low costs, offering investment possibilities with few risks and higher returns when compared to other places. 1

New firms are created by a combination various factors, such as the availability of skilled people, access to capital and the existence of promising business opportunities where risks are outweighed by expected benefits. Analogous factors affect the growth of firms and the decisions of foreign investors. These factors can be affected by several different policy areas that are responsible for business climate development.

• The availability of skilled people can be affected by regulations that improve the functioning of the labour markets, by supporting the offer of business and technology services, moreover by the availability of training, with special respect to the spreading of entrepreneurial and management knowledge.

• The amount of capital can be positively influenced by creating a good legal and institutional framework the capital markets, and by offering government supported loans and loan guarantees can affect available for entrepreneurs.

• Business opportunities are also affected by regulations, e.g. regulation of entry into different sectors of the economy, administrative simplifications for start-ups, government control on business activities, access to international markets, bankruptcy legislation, by regulation of transfers of knowledge from universities.

Business climate development - as any other policy area - is regularly evaluated. The decision makers of various government agencies, business organisations and also taxpayers in general need regular feedback on the performance of measures taken in order to improve the business environment. Evaluations can address the whole strategy, or may focus on a particular set of measures such as a subsidized project, a campaign, a programme or a regulation. The resulting evaluation reports should offer insight and expert opinion on whether these interventions were successful in terms of various criteria such as relevance, efficiency, effectiveness, impacts and sustainability. In particular, if the evaluation report focuses on the issue of impacts, i.e. to what extent the events following the policy measure can be attributed to the assessed intervention itself, then the report is called an impact assessment.

Out of the various areas of business climate development, this book focuses on better regulation and small business policies. Further studies are under preparation on the application of evaluation and impact assessment in other policy fields of business climate development.

The book offers various analytical instruments that can be applied in research conducted on business climate development policies. At the same time, the substantive 2 dimension of the book provides analyses on specific strategies and instruments of small business development policy and regulatory policy.

• Methodological dimension. The book aims to systematically portray and to classify those methods of impact analysis and evaluation that have already become influential in policy making. These methods have been applied to assess the success or failure of projects, programmes and regulatory measures, ranging from small business development policy, regional development policy, environmental policy and other fields. These methods will be placed into an institutional context by giving an overview about those international, national and regional donor organisations and public services that have institutionalised and routinely apply these

11The OECD has defined the following areas of business climate development: Investment Policy and Promotion, Privatisation Policy and Public Private Partnerships, Tax Policy, Trade Policy and Facilitation, Policies for Better Business Regulation, SME Policy and Promotion, Anti-Corruption, Corporate Governance, Business and Commercial Law, Conflict Resolution, Infrastructure, Human Capital Development Policy. [OECD 2010]22 Substantive in this case means non-methodological aspects, i.e. related to particular policies, programmes and projects.


Introduction

methodologies in order to obtain feedback about their development activities. Special attention is paid to those conditions in which evaluation and impact assessment have been introduced in the region of Central and Eastern Europe, with some focus on Hungary. The methodological components of this book are not specific to any policy area, since some of these methods may be applied for a wide variety of policy fields.

• Substantive dimension. The book focuses on the application of impact analysis and evaluation in SME development policy and regulatory policy. In the case studies we examine the functioning of these methods in Central and Eastern European transition countries.

• SME development policy. A wide range of supportive and regulative, direct and indirect SME development instruments are presented that facilitate the access of SMEs to finance, business development services, certain factor inputs such as labour and technology and to output markets. The case studies highlight various aspects of the historical transition process of creating a viable and strong SME sector in various countries. The case studies were selected with a view of demonstrating how SME development is interdependent with various other policies ranging from innovation policy through labour policy to regulatory policy.

• Regulatory policy. The case studies devoted to regulatory impact assessment discuss the efforts of Central and Eastern European countries to harmonise their legislation with that of the European Union in order to create well functioning product markets and to institutionalise the free movement of goods.

The structure of the book is as follows.

• Chapter 1. gives an overview about the status of evaluation and impact assessment activities within the current practice of development policy and discusses evaluation reports in the contexts of consultancy and applied social science.

• Chapter 2. offers a methodological framework for doing research or consultancy work by using evaluation and impact assessment methods. The differences are shown between the genres of (a) impact assessment, which is a causal explanation of policy interventions and their perceived or expected consequences, and (b) evaluation, which is an act of assessing the value of these policy interventions against certain criteria. A coherent family of research designs has been constructed that includes both causal explanations and value judgements. The components of research design are identical for both quantitative and qualitative studies. These models are applicable to the analysis of regulations, projects, programmes or policies as well.

• Chapter 3. first explains the emergence of SME policies internationally and in particular in the post-communist countries of Central and Eastern Europe. SME development policies are positioned in the context of governance and public administration. The chapter proceeds with the evaluation of subsidised projects, support programmes and SME development policies. It is explained how to transform the abstract criteria of evaluation (e.g. relevance, efficiency, effectiveness, sustainability and others) into concrete questions of structured interviews or questionnaires. Finally methods for assessing the impact of exogenous factors (e.g. historical events, regional endowments, business cycles and infrastructure developments) on SME development are presented.

• Chapter 4. presents various analytical tools of regulatory policy. The major instrument is regulatory impact assessment, a method for assessing the expected effects of individual regulations - or those of families of interconnected regulations – on enterprises and other stakeholders. The chapter presents the types of impact assessment systems that have been established and institutionalised in the developed countries and promoted by international organisations, such as the European Union or the OECD. Special attention is given to regulatory impact assessment (RIA) projects implemented in the new member states of the EU and those with an ambition to become members of the EU. The chapter also analyses the overall business environment of countries or regions, i.e. the system of regulations, administrative measures and institutions. Systems of indicators are presented that measure the influence of business environment on the economy, based on the practice of the OECD and that of the World Bank.

Case studies and examples. Throughout the book, the methods, genres and approaches are demonstrated by short examples in numbered boxes and also by detailed case studies. The case studies and examples analyse real-life measures of the following policy areas:

• Legal approximation of technology-policy of the European Union in the candidate countries.


Introduction

• Investment promotion by business-friendly regulatory reforms.

• Financial policies as exemplified by facilitating loans for capital to small- and medium size enterprises.

• Labour policy as exemplified by specific support schemes, encouraging the willingness of small- and medium size enterprises to increase and improve their activities as employers.

• Innovation policies as exemplified by the subsidising of innovative activities of small- and medium enterprises.

• Improving regional development policy by lessons learnt from a survey of small- and medium enterprises located in a given tourist region.

The Appendix offers an outline of the questionnaires used in the case studies. On the basis of the simplified questionnaires it is easy to reconstruct the data collecting activity of the researchers and consultants which, after all, constitutes the empirical basis of every impact assessment and evaluation.

Definitions. The glossary explains the terms used in the book.


Chapter 1. 1. Evaluation and impact assessment activities: industry or science?Both evaluation and impact assessment projects have become regularly implemented routine tasks to be performed on behalf of governments and donor organisations. For this reason, some methodologists refer to them as “industries" 1 Impact assessments and evaluations are products of administrative cultures. Experts are routinely producing these types of analyses in order to provide feedback about the expected or experienced success or failure of interventions. 2 Decision makers of every policy area rely on such analyses, including social policies, environment protection, fiscal policies and others.

In the particular case of small business development policies most impact assessment and evaluation studies refer to the impacts of regulations, programmes and projects, in particular to measures involving credit and guarantee provision, equity financing, developing enterprise culture, offering advice, assistance and training to SMEs in the domains of technology and management, moreover institution development and infrastructure development.

Institutionalised methodologies. During the past two decades governments and international donor organisations have issued a wide selection of evaluation and impact assessment methodologies and guidelines. These documents describe recommended or mandatory methods to be followed by evaluators and impact assessors of development measures. Sectoral methodologies apply to specific policy areas (e.g. environment protection, enterprise development, etc.) On the other hand, methodologies specifying a particular research design define certain evaluation and impact assessment genres, which have been applied by thousands of individual studies to a wide range of interventions and policy areas.

Consultancy or scientific research. In practice, evaluation and impact assessment are regarded as consultancy activities than scientific research. Consultants routinely produce large quantities of such reports in order to meet the needs of their donor clients. Scientific research institutions and academic bodies also issue a wide range of evaluations. Many of these reports are classified documents that can be read only by the donors and the experts and project managers of intermediary organisations and beneficiaries. 3 On the other hand, the final reports of most impact assessments will be placed on the websites of the beneficiary government or that of the donor organisation. Only a handful of evaluations comply with the high standards of scientific journals and will be published in such periodicals.

Applied social research. Evaluation and impact assessment, if performed properly, are activities that can be classified as applied social research or applied policy research. Evaluation relies on the same methodological principles as political science. The difference between evaluation and political science is analogous to the difference between sociological research on the one hand and public opinion research and market research on the other hand.

Evaluators must use the techniques of applied social science in order to arrive to objectively verifiable inferences. The paradox and the challenge of evaluation is that evaluators can obtain the most valuable information from stakeholders lacking objectivity due to the fact that they are interested in continuing and expanding the evaluated programme at the same time: in most cases these are the initiators, managers and beneficiaries of the programme.

The techniques of evaluating policy interventions and measuring their effects are in a constant change, thus they cannot be regarded as fully developed yet. The views concerning the utility, necessity and best ways of obtaining feedback about measures are mixed with ideological elements. In most countries the relevant stakeholders of the public sector insist to use the traditional solutions and procedures of evaluation and impact assessment already tested and institutionalised. It is the big international organisations, think tanks and donors of aid programs which do their most for the unification, standardisation, development and spreading of the culture

11[Conlin-Stirrat 2008]

22 [OECD 2007]

33 E.g. the so-called „interim evaluations" of projects co-financed by the Phare Fund of the EU.


1. Evaluation and impact assessment activities: industry or

science?of impact assessment and evaluation.


Chapter 2. 2. Research design of impact assessment and evaluation studies1. Conceptual frameworkFrom projects to policies. The research design of evaluation and impact assessment projects depend strongly on the nature and the aggregation level of the examined intervention. The methods used for policy evaluation are substantially different from those methods that are used for project evaluation or those applied in regulatory impact assessment exercises. The concept of programme is wider than that of a project, and the concept of policy is wider than that of a programme. [bib_1]

The following classification of small business development policy measures helps to clarify these differences.

• Individual regulations may directly and indirectly influence the performance of small businesses. An example of an individual regulation supporting SMEs is a law on penalising late payment in supplier-buyer relations. Such a measure, if implemented properly, creates equal opportunity for subcontractors, which in most of the cases are smaller companies than their clients.

• Wider families of regulative measures may also serve the needs of small businesses. Examples of such interventions are offered in deregulation campaigns, regulatory simplification reforms or legal harmonisation measures which may reduce the transaction costs of small enterprises when doing business.

• Individual projects have become the most widespread building blocks of aid delivery. Non-governmental agencies, governments and international donor organisations finance or co-finance tens of thousands of projects yearly involving training, consultancy, credit provision, direct non-returnable financing of SMEs, institution development or infrastructure development.

• Programmes are smaller or larger aggregates of more or less homogenous projects serving similar purposes. The aggregate of projects affecting a specific sector, or having identical objectives can be called a programme. In SME development a typical programme is the provision of micro-credits, consisting of several thousands of smaller individual projects, each of them facilitating the start-up or investment activities of a particular beneficiary SME.

• Policies are the joint aggregates of the above mentioned types of measures on the national, regional or local level. In particular, SME development policies are strongly interrelated with other policy fields. Policies are built up from subsidised projects and programmes and frequently the delivery of policies is supported by legal measures and the respective institutional development as well. Policies are more than a simple sum of programmes, because they are characterised by higher level strategies, involving aims and priorities of a more abstract nature than those of the individual projects or programmes.

Target group. Policies, programmes and projects are implemented on behalf of a certain target population. In the particular case of SME development the target population can be selected as a well defined subgroup of small businesses, such as start-ups, potential subcontractors, innovative firms, companies in lagging behind settlements, etc. In case of support programmes these firms are also called eligible or final beneficiary companies. Alternatively, in case of indirect support the members of the target group are intermediary organisations such as government agencies, financial intermediaries or business development service providers.


2. Research design of impact assessment and evaluation studies

Figure 1. The interrelationship between policy, programme, project and their target groups

Typology of consequences of policy interventions. Impact assessments may focus on specific types of impacts.

• Economic impact assessments highlight the impact of the examined intervention on companies in general, or on winners and losers among bigger and smaller companies, moreover consumers and government budgets.

• Social impact assessments focus on those impacts of decisions that affect certain social groups, such as women, minorities, unemployed people or immigrants, entrepreneurial stakeholders or the social stratification in general.

• Environmental impact assessments are revealing the impacts of a decision on the environment-friendly behaviour of various stakeholders on the pollution of air, water or soil, on the level of noise or vibration, or on waste management strategies of companies, households or institutions.

Impact assessments and evaluations can be applied before the intervention (ex-ante) or after the measure (ex post). Ex-ante analyses should deliver estimates about the expected future feasibility features, outcomes and consequences of a yet non-realised intervention.

Opinions, ratings, indicators. The success or failure of policy interventions is judged (a) by their outcomes and (b) by attributing these outcomes to the necessary resource inputs, e.g. subsidies or efforts. The attained outcomes may be interpreted in terms of resource inputs or of previously intended outcomes.

Impact assessors and evaluators make an extensive use of project indicators which can be used as empirical bases for formulating statements about causal relationship and ratings ranging from „highly satisfactory" to „unsatisfactory". Indicators should have the following properties: (a) indicators are relevant if they are expressing the major properties of the program being evaluated (b) indicators are reliable if they are biased with as little measurement error as possible. Moreover, (c) it should be possible to obtain the data necessary for the computation of the indicator.

Quantitative characteristics. In most cases inputs and outcomes are measured by quantitative indicators. The use of quantitative indicators enhances the objectiveness of evaluations. In many cases a double set of measurements is needed „before" and „after" the implementation of the assessed intervention. However, the



differences in the numerical values of the respective indicators cannot be automatically interpreted as impacts. Specific considerations and analyses are needed to attribute the changes to the assessed policies. Such indicators may refer (a) either to the final beneficiary SMEs by showing the improvement in terms of turnover or productivity or (b) to the regions in which the beneficiary SMEs work, by showing the impact in terms of poverty reduction or unemployment, or (c) to the intermediary organisations that deliver the intervention for the final beneficiary SMEs: in case of microcredit organisations such indicators may show the proportion of SMEs repaying their debt in time.

Qualitative characteristics. In many cases the outcomes of interventions can be measured only on a qualitative level. The report of an expert commission on entrepreneurial culture may consist of qualitative statements, since this phenomenon is hardly quantifiable. Qualitative rating is frequently the result of a comparative effort, e.g. the impact of an assessed regulation can be rated as a success if its outcomes are better than the outcomes of a previous attempt to regulate this field. Or, a reform of the SME support system may be rated by the stakeholders as worse than the analogous interventions implemented in a neighbouring country.

The components of research design. A research design is a plan that shows how the researcher expects to use his/her evidence to make inferences. Research designs are composed of the following four major factors:

1. the research question

2. the underlying theory

3. the data and the sampling strategy for collecting the data

4. the use of data in order to make inferences. 1

In order to compile a theoretically well founded systematisation of the underlying methodologies of evaluation and impact assessment studies, it is useful to recognise these activities as applied research projects and to classify them according to their research designs. The rest of this chapter classifies analyses and compares the genres of impact assessment and evaluation by using the above mentioned four parameters of research design.

2. Components of research design2.1. The research question2.1.1. Research questions of impact assessments

It is the research question of the investigation that defines the differences between the genres of evaluation and impact assessment. While in case of evaluation one searches for the value of some policy intervention in relation to criteria defined earlier, in case of impact assessment the task is to reveal for some causal explanation about the expected impacts of a set of measures.

Impact assessments are analyses, explanations about the expected results and consequences of certain social-political interventions, policies and measures, based on empirical evidence. Therefore the research question of an impact assessment is always related to the existence and intensity of causal relations. In other words, impact assessors must demonstrate that the observed changes can be attributed to the measures examined or at least that these measures have contributed to them.

The definition of impact. Social researchers have, in connection with impact assessment, increasingly emphasised the requirement that impact assessments should apply the conceptual framework of causality as defined by social sciences. 2, 3 A researcher is justified to claim that a change that has occurred within the circle of affected enterprises is the consequence of a certain measure, if he or she clarifies first, what is regarded as a change and what are the key variables measuring this change. Theoretically, an inference about the existence of an impact of an intervention can be made only by comparing two scenarios: (a) the one in which the assessed measure was taken and (b) another scenario in which the measure was not taken. 4 One of these scenarios cannot

11 [King-Keohane-Verba 1994]

22 [Moksony 1999]

33 [Bartus and others 2005]

44 The previous sentence applies for the case of ex post impact assessment only. In case of ex ante impact assessment future tense must be



be observed, and therefore it is called the counterfactual scenario. In some cases the impact of an intervention is measurable: this is the difference between the values of a key outcome indicator expressing change under the two scenarios.

Probabilistic impacts. The notion of impact can also be defined in probabilistic settings in the following manner: a cause raises the probability of an event, and the increase of this probability is the impact.

• In the framework of SME development policy this approach can be exemplified in the following interpretation: small enterprise development initiatives have positive impact to the extent that their implementation raise the probability of certain previously defined aims being attained among SMEs of the target group.

Complex structures of causes and impacts. In more complex settings causes and impacts can build up inter-dependent and interwoven structures. An example for such structure is the causal chain (the so-called "domino effect"). Another example for such inter-dependent causal structure is when a cause leads to the desired impact only if some other condition is also met (i.e. another cause, which alone could not trigger the desired impact) also occurs.

• An example for the “causal chain" structure is the chain of non-payment, i.e. when each firm within a group of companies owes money to the previous one, and the bankruptcy of one company may trigger the failure of all other companies that follow this firm in the line of non-payment.

• An example for a more complex causal structure in SME development is the joint effect of a soft loan scheme and a credit guarantee scheme. The establishment of a soft loan scheme for start-up companies can facilitate the financing of these companies only if an affordable credit guarantee scheme is established as well.

The fundamental problem of impact assessment. Every impact assessment must cope with the basic problem of causal inference. Since the outcomes under the counterfactual (hypothetical) scenario are not observable, the comparison of the two sets of outcomes under the two scenarios cannot be based on a solid empirical basis. This leads to the so-called fundamental problem of causal inference, i.e. that causality is not directly observable. However, a series of impact assessment methods have been developed to make indirect inferences to causality.

Examples of impact assessment research questions. The basic research question of impact assessments appears explicitly in the methodological apparatus of many impact assessments conducted about small business development measures.

• Research designs involving control groups may attempt to respond to research questions which tackle the difference between changes that have occurred in the beneficiary group and the control group. Example:

• Did the competitiveness of subsidy receiving beneficiary companies improve, in comparison with other, comparable, similar applicant companies that have not received subsidies?

• In case of research designs where no such control group can be constructed, the members of the target group can be asked about the difference between the observed and the counterfactual scenario. Examples:

• What would have happened to your firm if you had not received the subsidy?

What difference would it have made to your company, if you had received the required support?

You have hired two new employees since having received the subsidy: did you employ them as a consequence of having received the subsidy?

The government plans to introduce this regulation: will this be advantageous to your company or rather disadvantageous? Why?

2.1.2. Research questions of evaluations

Evaluation is a systematic determination of value, merit, and worth of interventions by using previously defined criteria or standards. Therefore the basic research question to be answered by evaluators is, whether the examined policy, programme or project is (or was, or will turn out to be) good or bad according to some criteria

applied: in this case (a) the scenario in which the measure will not be taken is compared with the scenario in which the measure will be taken.



of success. Evaluators are expected to give their opinion about these interventions and in most evaluation cultures this opinion should be summarised in numerical ratings. Evaluators should judge the success or failure of interventions in meeting their intended outcomes or objectives, must distinguish a program's effects from those of other factors, and should find ways to improve the programme through a modification of current operations.

Evaluations are supposed to deliver expert opinions and the corresponding ratings about the success or failure of interventions, about the positive and negative characteristics of laws and enforcing organisations, about the advantages and disadvantages of aid delivery regimes.

Evaluations are prepared not only in order to establish the impact of policies and programmes. Such studies inform decision makers and taxpayers about the allocation of public funds. Evaluations may facilitate the improvements in the design and administration of programmes if these reports are used as feedback into current policy making and stimulate informed debate.

Impact assessments as part of evaluations. Evaluations of policies, programmes or projects are frequently revealing causal relations between interventions and outcomes. However, evaluators are expected not only to name the consequences and assess the extent of the expected impacts but also to evaluate these impacts whether they are satisfactory or not. Evaluation reports should answer not only the question of whether interventions or treatments work, but also a wide range of questions about when, how, under what circumstances the impacts occur, and what lessons can be learned from this particular intervention. And besides naming, assessing and evaluating impacts, evaluations must give an account of the relevance, efficiency, effectiveness and sustainability of the examined intervention.

In order to account for this complexity, the genre of impact evaluation or impact assessment has been complemented by the methods of process evaluation as well. While the methods of impact evaluation are identical with those of impact assessment, process evaluation is a form of monitoring designed to determine whether the measures under the policy have been delivered as intended to the targeted recipients. Process evaluation (also known as implementation assessment) can be based on an analysis of the attitudes and behaviour of beneficiaries as they interact with the donor organisation or with the intermediary organisation during their involvement in the programme. 5

Process evaluations. In many cases impacts are not the priority focus of the evaluation project, rather the major aim of the researchers is to qualify the design and the process of aid delivery, e.g. the administrative and client service work of the intermediary institutions on behalf of the beneficiaries.

Examples of research questions of evaluations. The basic research question of evaluations appears in the methodological apparatus of many evaluations devoted to small business development measures. These questions are directly or indirectly concerned with the evaluation criteria against which the measure has to be assessed.

• Relevance . You have financed from the subsidy a new technology which you have recently introduced in your firm: do you consider this an innovation?

• Efficiency . Why were the results of your project delivered with such a big delay?

• Effectiveness . Have the training materials of the subsidised course been published on the Internet?

• Impact . Who else is going to profit from the subsidised project besides your company?

• Sustainability . The development of the Enterprise Resource Programme IT system of your company has been co-financed by the donor organisation. What kind of contractual guarantees has your company received from the contractor software developer?

The fundamental problem of evaluation is that it inevitably leads to normative statements. Evaluation judgments explicitly or implicitly involve statements about which aims, strategies, project designs or outcomes are good or bad, which operations are right or wrong. 6 Since project and programme evaluation is regarded as an exercise in applied social science, and evaluators arrive to value statements, it raises the issue of value in social science. 7

55 [Purdon – Lessof – Woodfield – Bryson 2001]

66[House 1999]

77 [Szántó 1992]



The main question connected with value statements is to what extent these can be verified of falsified. However, in case of project and programme evaluation, the findings are expressed in form of so-called instrumental statements, which qualify certain lines of actions according to whether they are suitable to reach some previously defined aims or not. Such statements are combinations of norms and scientific statements and can be supported (if not proven) with the help of empirical data and with the application of a valid inference mechanism.

The opinions, conclusions and recommendations of evaluators influence decisions either (a) by affirming and encouraging certain procedures or outcomes for policies, regulations, projects programmes, or (b) by defining other procedures or outcomes as anomalies, discouraging policy makers to take these directions. Although evaluators increasingly use a wide range of data and a large apparatus of descriptive models and explanations to justify their judgments, however, a certain risk of arriving at subjectively or even emotionally influenced opinions still remains. Evaluation scores express normative judgments and it is nearly impossible to create universal, rationally applicable standards for all domains of evaluated activities. In principle there is no way to attain a perfectly rational falsification or proof for evaluation findings.

2.2. The underlying theory: impact mechanism of SME development measures2.2.1. The difficulties of a theoretical approach

Evaluations and impact assessments rely either explicitly or implicitly on certain underlying substantive principles, on social, economic and political theories. These theories may be explained in text or can be stated in form of a quantitative model. Researchers promoting the method of so-called „theory-based evaluation (TBE)" maintain that it is not sufficient to assess the impacts of a programme by proving that certain outcomes can be attributed to the program: rather, evaluators must be able to explore and explain how and why these measures have lead to success or failure. Theory-based evaluations explain the mechanisms of the determining or causal factors judged important for success, and how they might interact. The results of a theory-based evaluation should be converted into “lessons learnt" and recommendations, moreover, the design and implementation of the programme should take into consideration the revealed underlying impact mechanisms. Based on these insights, it can then be decided which steps should be monitored as the programme develops. 8

While the aims of SME development are analogous to the aims of other fields of local, regional or national development policies, the instruments of this policy field, the professional content of policy interventions on behalf of small businesses covers a wide range of heterogeneous activities. Major examples of policy instruments are as follows:

• Creating an enterprise-friendly regulatory framework.

• Setting up, closing down, or reforming public agencies in order to offer up-to-date services for enterprises.

• Setting up loan guarantee schemes

• Programmes to encourage young disadvantaged individuals to start businesses.

• Programmes to encourage graduates to start businesses.

• Creating of Science Parks, small business incubators and promoting links between universities and SMEs.

• Granting tax relief for business angels 9.

• Enhancing investment and innovation readiness of SME owners.

• Encouraging SMEs to export.

• Subsidising new technology based firms.

88 [WB 2004]

99A business angel is an informal investor, usually a successful entrepreneur, who is willing to invest in high-risk, high-growth firms at a very early stage, and adds value by supplying hands-on business advice as well.



• Subsidising management training.

• Developing disadvantaged settlements or regions by offering subsidies to companies for starting business, investing or relocating to these areas.

• Developing regional clusters and local innovation systems.

• Promoting rural entrepreneurship.

• Promoting business networking.

• Promoting Enterprise Zones.

There is a wide body of available theories to explain the role of public policies in SME development. Evaluators apply certain financial models in order to describe the decrease of transaction costs for SMEs as a result of certain policy interventions. 1010 In other cases evaluators of institution development projects may apply general principles of organisational sociology in order to explain the impacts of these interventions (e.g. of decentralisations or regionalisations) on SMEs. Theories about the dissemination pattern of innovations can be applied in order to explain the spreading of e-business or e-government applications among small enterprises. However, it would be nearly impossible to apply uniform, standard theoretic principles for modelling the impact mechanisms of all types of policy interventions on behalf of SMEs.

2.2.2. Understanding how SME development measures function

In spite of the lack of a widely accepted theory of SME development, experts responsible for the planning of small business policies, moreover the evaluators and impact assessors of these measures invariably apply certain general laws and principles of development economics.

The performance of small enterprises depends on a wide range of factors. These factors can be classified according to the following system 1111:

• the availability and cost of financial resources such as credits and venture capital,

• the availability and cost of non-financial business development services,

• the availability and cost of factor inputs such as labour, technology, knowledge and some locally scarce natural resources (e.g. water or energy), and

• the access to the markets of the outputs of companies (including product or service markets).

Impact mechanisms of SME development measures. Policy makers make use of the fact that the availability and the costs associated with the use of the above mentioned factors may be influenced by support policies and by reforming the legal and regulatory environment. In order to influence the development of SMEs through the above four factors, donor organisations have implemented the following types of initiatives or approaches:

• Developing financial services. This approach involves debt and equity financing which is facilitated through offering credit lines to intermediaries such as financial institutions; financial services are developed by subsidised consulting and training, and some subsidy schemes facilitate direct investment into small enterprises. Donors choosing this approach support banks, leasing organisations, credit guarantee services and other financial organisations such as local microcredit delivery organisations in order to support SMEs in an indirect manner. The impact of these interventions is exerted through the reduction of the costs of investments made by SMEs.

• Facilitating the delivery of business development services. This approach involves the direct delivery or the subsidisation of consulting, training, management or marketing services on behalf of SMEs. The access to business development services may be facilitated by grants and vouchers offered for SMEs. Donors choosing this approach support consultancies as intermediaries to develop the respective services in order to support SMEs in an indirect manner. The impact of these interventions is exerted through an increased use of business development services by SMEs.

1010 [Kállay 2005]

1111[Oldsman – Hallberg 2002]



• Improving the business environment. This approach involves the simplification of business regulations, reinforcing property rights and the enforcement of contracts, fighting corruption, improving the policies responsible for labour, trade and taxes. Donors choosing this approach support government agencies, professional associations and non-profit organisations in order to support SMEs in an indirect manner. The impact of these interventions is exerted through the reduction of the transaction costs of doing business, the costs of entering and expanding new markets for SMEs.

Intermediaries . Indirect programmes supporting the SME sector involve also a wide range of intermediary organisations.

• Government agencies. Support to SMEs is often provided indirectly by seeking to improve the regional and local business environment in which SMEs operate, and such programmes may target local and regional governments in order to reach the final beneficiary small businesses. Programmes aiming at simplifying the regulative and administrative environment of companies must first reach the central, the subordinated and the decentralised government agencies by re-organising them, developing their e-government capabilities and training their employees.

• Financial intermediaries. Programmes granting preferential credits for small companies must first motivate banks and train bank employees. Micro-credit programmes are delivered through several types of intermediaries such as banks, professional associations or local small business development foundations. The delivery of preferential credit schemes are frequently accompanied by establishing and subsidising loan guarantee organisations. Loan guarantee schemes are preferred instruments used by governments to intervene in the credit markets.

• Business development service providers. Some programmes aim at developing the capacity of SMEs to become subcontractors of multinationals. The resulting projects involve a wide range of technical consultants, such as engineering companies. Other programmes developing the e-commerce capabilities of SMEs are delivered with the help of software consultancy companies.

Figure 2. Schematic impact mechanism of SME interventions 1212

2.3. The type of data to be collectedEvaluations and impact assessments are empirically based efforts. Research designs are thoroughly influenced by the choice of information sources, by the availability of data and by the strategy of collecting the necessary data.

2.3.1. Qualitative studies1212Own compilation, based on [Oldsman – Hallberg 2002].



In most research situations the access to qualitative data is easier and cheaper than to perform a questionnaire based quantitative survey among beneficiary companies. In case of studies are based on qualitative data, inference is based on the analysis of aid delivery documents, on interviews made with owners and managers of affected companies, with officials of regulatory agencies, or conducted with project managers of the aid delivery process. Causal inference is frequently based on the comparison of “cases", i.e. on the secondary analysis of previous studies made in comparable countries where the same interventions have not been taken, or by analysing the effects of previous comparable interventions in the same country or in other regions. Quite frequently, qualitative studies made by international organisations compare countries: in such cases the implementation and reception of similar or analogous interventions – e.g. subsidy schemes, regulatory reforms, campaigns, etc. – is being compared. In qualitative studies the issue of sampling arises as the proper selection of (a) cases to compare, (b) documents to analyse and (c) interview subjects to visit.

Interviewing some representative members of the target group. Most impact assessment studies rely on a small scale sample of in-depth interviews made at companies affected by the examined policies. Sampling strategy as a rule is restricted to having a quota of at least one or two company of each type in the sample. For example, researchers may collect interviews from companies of various sizes, sectors and legal forms, and firms working in various regions and settlement types should also be represented. The collected empirical material contains important information about company characteristics, strategies and behaviour, about the awareness of companies of the examined policy intervention and their responses to the examined measures, about the opinions of experts and stakeholders about the institutions responsible for SME development. The results of these in-depth interviews are then contrasted with each other, classified by explanatory variables and aggregated. Inference to causal statements is made by asking the interviewed experts, regulators and enterprises about the counterfactual: what is their opinion about “What would happen (or what would have happened) if the intervention did not take place (had not taken place)". While this method is relatively cheap, flexible and feasible, it can be easily biased by subjectivity or by lack of skills on the side of the interviewers.

2.3.2. Sampling strategies for quantitative impact evaluation

Quantitative impact assessments are prepared with the ambition to make statistically reliable statements about the relation of the cause (i.e. the policy intervention) and its consequences: about their impact exerted on companies. These efforts are always based on business surveys.

The sampling strategy of rigorous quantitative impact assessments of SME development measures consists of the following stages: (a) statistically defining the enterprise population that is the target group of the measure (b) selecting a sample of enterprises affected by the measure and (c) selecting a comparable sample of other companies, that have not been exposed to the measures. If possible, data collection from "treatment" and “control" companies is repeated both before and after the intervention – this is called a “before-after design". The literature of this body of policy research often uses the language of experimental design used for medical research: the companies exposed to the intervention are called the “treatment group" and a comparable group of companies that has not been exposed to the intervention is called the “control group". In such studies the statistical inferences result in impact statements that refer to the “treatment group", but these statements are on based on a comparison between the “treatment group" and the “control group".

However, due to lack of data, lack of resources or because of ethical reasons, it is not always possible to involve a control group into these research efforts. In such cases the inference about the causal relationship (i.e. the connection between the measure and its impacts) is weaker.

The following sampling strategies have been routinely applied in quantitative impact assessment designs. These designs have been ordered by the decreasing reliability of the causal explanations that can be based on them. On the upper end of this scale, the random selection of both the treatment group and the control group is considered to yield the most credible impact statements. The other extreme is when there is no control group at all: although such a sampling strategy is perfectly justifiable in case of descriptive studies, but any inference to impacts or causal relations can be made only if it is strongly reinforced by additional qualitative information which has been gathered independently from the quantitative data collected by the survey.

(1) Random assignment. This experimental design is used to measure the observed results of the intervention by comparing (a) a random selection of enterprises having been exposed to the intervention with (b) a random selection of enterprises not having been exposed to it. This sampling strategy is the so-called “randomist" approach. Although this approach leads to impact statements that have very high reliability, there are some ethical, methodological and feasibility considerations against its use. Since regulations and subsidies have well defined target groups, it is not feasible or not ethical to experiment with companies by randomly choosing a



“treatment" and a “control" group. Researchers are not in the position to decide, by way of random choice, which enterprises should receive subsidies or which companies should be exempted from the force of a regulation.

(2) Matched control group. In this case the control group is not randomly selected, but is constructed to be as similar as possible to the group affected by the intervention. Similarity is attained by selecting a control sample of companies of which the composition by size, sector, region and other important explanatory variables are identical with that of the affected group.

• In most cases this is ensured by using company quotas according to some previously selected explanatory variables such as sector, size and region.

• Another approach to matching is called “matched area comparison design". In this case the outcomes in the pilot area – i.e. where the intervention is introduced – are compared with a control group that is chosen by picking enterprises of another, comparable, possibly similar area (e.g. region) where the intervention is not implemented.

• In more sophisticated cases a better match can be achieved by computing the so-called “propensity scores". Propensity score is the conditional probability that a company will be supported (“treated"), given the set of its values on the explanatory variables. Each company in the supported (“treated") group is matched to a subject that has a similar propensity score in the untreated group. The control group consists of the non-beneficiary companies selected by this method.

(3) Treatment group and control group selected by convenience . In the practice of SME development policy evaluation, company samples are very often determined by the willingness or capability of companies to respond to the questions of evaluators.

Examples of this approach are those research designs where the responses of a group of responding beneficiary companies are compared to the responses of those applicants who were applying unsuccessfully to the same subsidy scheme. This research design is somewhat biased for two reasons. (a) Voluntary respondents do not necessarily represent properly the target group. (b) Companies that have applied to the subsidy but were rejected by the selection committee have performed weaker during the application process and are not comparable to the beneficiary group. Consequently, by aggregating their responses the researchers cannot directly respond to the counterfactual question “What would have happened without the intervention?" 1313

(4) Research design without control group . In many situations of SME research interviews are made with - or questionnaires are collected from - only the companies affected by the intervention. In other words, the research design involves no control group. Certain causal statements with weaker reliability can be formulated even in such cases, by comparing internal sub-groups of the target group. This type of inference can be facilitated by the following sampling strategy: a sample with the possibly highest variability should be compiled. A high variability can be attained if the sample includes companies from various sectors, regions, size classes and levels of exposure to the (planned) intervention. 1414

For example, let’s suppose that some previous qualitative information supports the hypothesis that the positive impact of the examined intervention is proportional to the size of the beneficiary company. If the empirical findings show that the expected positive changes do not occur among small companies, but do occur among medium sized enterprises, and even more so among bigger companies, then this might be interpreted as a reinforcement of the hypothesis.

2.3.3. Compromises and pitfalls in sampling

The range of data to be collected depends strongly on the financial resources available for the evaluation or impact assessment project. Research projects can save significant costs by applying weaker principles to sampling strategy. Such compromises might be unavoidable, but in such cases the report must (a) unanimously point out the lower reliability of the causal statements that it has arrived at and (b) should refer to the direction of the possible bias. The following methodological pitfall demonstrates how poor sampling can lead to biased impact statements.

1313 E.g. Case Study C in this document.

1414 E.g. Case Study A in this document.



Selection of the observations on the dependent variable. This is a misleading sampling strategy when selecting the members of the target group of the intervention. This mistake is committed if the researcher examines the change that was brought about by a previously implemented intervention only in that group of companies in which the positive effect of the intervention was able to exert its influence.

Table 2.1. Box 1.

Example of selection on the dependent variable

Bankruptcy regulations in Hungary

An example of bankruptcy research should illustrate the consequences of selecting the sample on the dependent variable. In the early 1990s in Hungary the transformation crisis has lead to an unusually high number of indebted companies. Long chains of non-payments have evolved, whereby each company in the chain was indebted to the next one. One of the causes of this anomaly was that the country lacked a well functioning bankruptcy law. Consequently a series of laws and regulations were issued with the aim of ensuring orderly bankruptcy procedures, giving chances of recovery to the indebted companies, but at the same time defining fair rules of liquidation in order to assure that the creditors – their clients, the state, other companies and banks - will be compensated by the assets of the indebted companies.

Two years later an evaluation was made about the interventions implemented in favour of those companies that got into troubles. The evaluation was based on a survey among surviving companies and among companies being still under liquidation procedure at the time of the survey. Only a handful of those companies were included in the sample that have been already liquidated between the bankruptcy regulations and the survey date. The reason for this omission was that at this time only a few managers could be reached who were able to speak about the liquidated companies.

The evaluation report clarified that (a) the sample of the survey had been selected on several of the dependent variables, i.e. the survival of companies and the length of the liquidation procedure; (b) the impact statements were somewhat biased, because the results of the examination could not be generalised to the full population of enterprises that were under the impact of the law.

The consequences of the lack of quantitative data. Researchers preparing quantitative impact assessment studies always face resource and the data constraints: surveys are expensive and data obtained from official statistics are in most cases outdated or irrelevant. In most research situations the group of enterprises affected by the examined intervention constitutes an aggregate of very specific character which does not correspond to any one of the widely used statistical categories of economic sectors, company size classes or geographic regions. For this reason, many researchers are compelled to adopt ad hoc methods by combining the use of qualitative data with the available quantitative data obtained from statistical offices, by doing a secondary analysis of previous surveys and by doing interviews by using a project-specific questionnaire.

In most cases it is the lack of available data that reduces the applicability of quantitative evaluation research methods 1515 rather limited in their actual working practice. Due to a chronic lack of statistical data, impact assessments and evaluations related to the entrepreneurial sector do, in practice, depend on document analysis, on administrative and company interviews which are complemented by business surveys and by official statistical data in favourable cases only. Most guidelines of existing impact assessment and evaluation cultures do not count on the availability of relevant time series or survey data.

2.4. The use of data in order to make inferences2.4.1. Qualitative approach: comparing causal and evaluative inference

Inference in qualitative impact studies. Qualitative impact studies make causal inferences by comparing detail-rich descriptions of carefully selected cases according to some previously defined sets of criteria.

The cases which offer themselves for comparison may be of the following types. In comparative qualitative research the researchers come to conclusions about impact mechanisms by studying various comparable items:

1515 [Mohr 1995]



• by comparing countries having introduced analogous sets of regulations - in order to show the impact of specific regulatory strategies;

• by comparing different laws reforming the regulatory environment of SMEs in the same country - e.g. in order to show the impact of deregulation campaigns on SMEs

• by comparing public agencies that are instrumental in enforcing SME-relevant laws - in order to show the impact of institutional arrangements on SMEs;

• by comparing various banks that provide loan or equity capital for SMEs - in order to show the impact of various financial arrangements on SME development;

• by comparing local chambers or small business development centres - e.g. in order to show the impact of local entrepreneurial traditions on SME development;

• by comparing regions in order to show how national SME policies are adapted to local endowments and influence the specific local economic structure.

As a rule, the aim of the comparison is to arrive to some conclusions about the impact mechanism of the assessed interventions, and to explain why are some types of measures are successful in some institutional settings and unsuccessful in other circumstances.

Inference in qualitative evaluations. Most evaluators do not have access to survey data or to suitable time series. Frequently, the indicators of the evaluation consist only of statements about the accomplishment of some previously set objectives. The results of project, programme or policy evaluations are empirically based on documents, interviews and personal observations and the evaluators use certain economic, sociological and organisational analyses for interpreting this material. The statements and judgements to which the evaluators arrive are based frequently not only on theoretical considerations but also on expectations and previous personal and institutional experiences. Frequently, inferences are made with the help of sound judgement about what is considered good or bad in order to attain the aims of the declared SME policy or in order to comply by the development principles of the donor agency.

Evaluation is frequently a comparative effort, because it may be based

• on a comparison of the results of the intervention with previously made plans and promises, sometimes based on indicator values, or alternatively,

• on a comparison of the outcome of the intervention with some existing benchmarks.

The findings of evaluations are frequently presented in the form of a SWOT analysis, which is a qualitative assessment of the strengths, weaknesses, opportunities and threats related to the evaluated intervention.

2.4.2. Inference based on highly aggregated quantitative data

In quantitative impact studies the design of making inferences depends strongly on the aggregation level of the data used. Different models have been developed for analysing micro, meso and macro data such as time series, or data aggregated to sectoral or regional level.

Interrupted Time Series Design . If the researchers have access only to aggregated level data, a widely used model that can be used for inference is the so-called Interrupted Time Series Design. This is a design using the time series of a key variable for a time interval which starts well before the date of the intervention and ends well after that date. Researchers first compile the monthly or yearly time series of this aggregated performance variable. Afterwards, based on the time series values before the intervention, a so-called pre-intervention trend is computed, which is projected to the post-intervention period and interpreted as the counterfactual: “this is what would have occurred without the intervention". This hypothetical trend is subsequently compared with the actually observed trend of the variable, as observed after the intervention. If the two trends differ significantly, this indicates that the intervention had some effect.



Figure 3. Schematic impact assessment inference with Interrupted Time Series Design

2.4.3. Inference in quantitative impact studies using micro data

Impact studies relying on micro level company data are frequently calculating certain indicators both for the treatment group and for the control group, both before and after the intervention. Causal inferences are made by comparing these numerical values. Researchers working with such “before-after" models use the values of a key performance variable. This variable may be selected – depending on the aims of the intervention - as the number of trained employees, company creditworthiness, or some other indicator. The ex post (after intervention) values of this variable are compared with the ex ante (before intervention) values of the same variable. This “before-after" model is applied both for the treatment and for the control group.

In these research situations the following statistical models are most frequently used.

(1) Difference-in-difference method . A key variable is chosen which is able to express the success of the intervention, e.g. the profitability, creditworthiness or productivity of the beneficiary and non-beneficiary companies. The average values of this variable are computed for the four groups of observation, i.e. for the beneficiary companies exposed to the intervention (A) before and (B) after the intervention, and for the control group (C) before and (D) after the intervention. (See next Figure.) Impact is demonstrated by highlighting a significant difference between the improvement in the beneficiary group as compared to (i.e. subtracted from) the improvement in the control group.



Figure 4. Difference-in-difference design of SME support programme evaluation. Inference to positive impact if (A-B)-(C-D) is significantly greater than zero.

(2) Before-after regression. This approach is based on a bivariate regression model which is also called pretest-posttest regression. The calculation differs from the previous one insofar as in this case the researchers compare regression lines (as characterised by their slopes and vertical positions, i.e. intercepts) instead of averages. Geometrically, in terms of an (X,Y) system of coordinates, if there were no changes in the key variable, the regression line would be the X=Y line, i.e. the 45 grad line between the X and the Y axis. If the regression line of the treatment group differs significantly from that of the control group, a corresponding impact statement is justified, because this can be attributed to the net effect of the intervention, controlled for other factors.



Figure 5. Before-after regression design of SME support programme evaluation

(3) Regression discontinuity design . This model is a special case of the previous bivariate regression. It is applied following the application of a specific sampling design: (a) those beneficiaries are selected to participate in the programme which before the intervention had performed better than a threshold value of a previously selected performance variable, while (b) companies performing under this threshold value are assigned to the control group. For example, the credit application of creditworthy companies is accepted, while those showing creditworthiness lower than a previously defined creditworthiness threshold, are rejected. After the intervention has taken place, the outcome is measured by the ex post level of the same variable (in this case: creditworthiness). The computation of regression discontinuity design answers the counterfactual question “What would have happened to the treatment companies if they had not participated in the programme" by comparing the before-after regression line of the “treatment" group with that of the “control" group.

An example of the application of the regression discontinuity design follows. Companies participate in the evaluated micro-credit programme if and only if the value of their ex ante (i.e. before-intervention) creditworthiness index is above the threshold value of 50, and all other companies are excluded from the programme. After the intervention has taken place (i.e. the microcredit was disbursed to the beneficiaries and they have used it for their respective projects), every company reports its ex post level of creditworthiness and the impact of the programme is measured by comparing the improvement pattern of recipients with that of the rejected companies.

Figure 6. Regression discontinuity design of microcredit programme evaluation where companies with low creditworthiness are not eligible

(4) Regression model with statistical controls . This is a multivariate regression model explaining the variations in the key outcome variable with the help of a series of explanatory variables. The most important explanatory variable is the so-called “participation variable" which expresses the fact of being selected into the target group of the intervention, or not: it takes the value 1 for companies of the treatment group and 0 for companies of the control group. The aim is to highlight the effect of this variable, by controlling for the impacts of all other variables, i.e. to filter out the impact of other interfering factors.

For example a programme is expected to increase the export turnover among recipient companies. A multivariate regression model explains the change in export turnover by several explanatory variables. One of these variables is the X “participation variable" which expresses participation in the programme (X=1) or no participation in the programme (X=0). Other explanatory factors are changes in the number of employees, and



some useful balance sheet data of the company such as investments, credits, turnover or profits. The aim of the calculation is to show the regression coefficient of the X “participation variable" by controlling for all other explanatory variables, i.e. by filtering out their effects on export turnover.

2.4.4. Pitfalls of impact assessments

Let us analyse various types of pitfalls and false inferences when preparing impact assessments.

(1) Mistaking temporary impacts for sustainable ones. In case of ex post impact assessments, in the first place the robust question should be answered, whether the expected changes have occurred at all and if yes, whether they are sustainable. In certain research situations some impacts may be observed, but these impacts may not be significant or may be of temporary nature.

An example of research on regional clusters should illustrate why such research questions are justified. A subsidy scheme offered by the Hungarian government in the years 2001-2002 has supported a series of business networks in order to enhance the co-operation between profit-oriented actors, non-profit organisations and local governments. The aim was the facilitation of the establishment of regional clusters in Hungary . The impact evaluation 1616 of this support scheme has asked the research question, whether and to what extent the supported networks have subsequently developed into sustainable regional clusters. The results of the interview-based survey have shown that many networks supported by the intervention were either of temporary nature or did not meet some of the criteria of a regional cluster.

(2) Ignoring suppressor effects. In some research situations the measure has had some effects but it is not noticeable because an opposite impact mechanism has suppressed it. This phenomenon is called a suppressor effect.

The example of a subsidy scheme facilitating employment may illustrate suppressor effects. 1717 One of the aims of the subsidy is to increase the number of persons employed by applicant SMEs. An impact evaluation investigates the beneficiary companies but has no access to a control group. Most companies exposed to the intervention have hired additional employees during the programme period. However, the aggregate number of recently hired persons is within the limits of the usual volatility of staff. In order to highlight the impacts of the programme, evaluators must control for the effect of other factors such as the business cycle, changes in labour costs or in labour regulations.

(3) Attribution by mistake. Even if some change in the expected direction has taken place, the question remains open, whether it has really been the intervention that brought about the positive change or, perhaps it is due to another impact mechanism, to another cause competing with the one examined. If the effect of other factors is falsely identified as the effects of the examined intervention, it may lead to invalid conclusions.

An example taken from environmental impact assessment may illustrate how regulatory impact assessment may come to false conclusions. In the candidate countries of the EU many researchers have examined the impact of harmonising national environmental legislation with that of the European Union . 1818 In this example the evaluator works in an ex-ante situation, i.e. a specific legal rule of environment protection is being examined which has not been yet introduced. This planned rule demands that companies must reduce a certain type of pollution by decreasing a pollution limit. The majority of the interviewed stakeholders express the opinion that this rule if passed and enforced will substantially increase the compliance costs and the administrative burdens of most affected companies. However, previous regulations have already set certain – somewhat lower - limits to the same type of pollution. Therefore the evaluator must focus on the additional efforts that companies must take in order to meet the additional requirements. In other words, only those additional compliance costs and administrative burdens should count as impacts, that are attributable to the increase of the pollution limit. Otherwise the impact statements based on these interviews will refer to the impact of regulating the type of pollution at all, instead of qualifying the impact of European legal harmonisation.

1616E.g. see Reference XXX at the end of this document about cluster development in Hungary.

1717See Case Study C in this study.

1818E.g. see Reference VIII at the end of this document about legal harmonisation in environmental protection in Hungary.


Chapter 3. 3. SME Development Policy1. SME development policy and its impact mechanisms1.1. The rise of small business developmentA small business is a business that is privately owned and operated, with a small number of employees and relatively low volume of sales. Small businesses are normally privately owned corporations, partnerships, or sole proprietorships. The legal definition of "small" varies by country and by industry. The abbreviation SME (Small and Medium Enterprise) occurs commonly in the European Union and in international organizations, such as the World Bank, the United Nations and the WTO. EU Member States traditionally have their own definition of what constitutes an SME. SME development policy is a specific area of public policies aimed at the development of the SME sector in a particular country or region.

Small businesses are common in many countries, depending on the economic system in operation. Typical examples include: convenience stores, other small shops (such as a bakery or delicatessen), hairdressers, tradesmen, lawyers, accountants, restaurants, guest houses, photographers, small-scale manufacturing, and online business, such as web design and programming, etc.

Since the 1980s international donor organisations - such as the European Union, the World Bank and the OECD – moreover national, regional and local governments have increasingly recognised that the isolated measures for supporting small and medium sized enterprises need to be integrated into what has become a relatively new policy field: small business development policy.

Governments and international organisations have increasingly taken measures to influence entrepreneurship through 1

• macro-economic policies that encourage potential entrepreneurs to invest,

• regulations which take into consideration their differential impact on firms of different sizes

• direct support measures and programmes offered for small firms

• developing market economy institutions

• and developing entrepreneurial culture and the social acceptance of entrepreneurship.

This dimension of development policy was justified by the increasing positive role of SMEs in increasing the level of employment, in enhancing competition, in spreading innovations, in diversifying the supply of products and services and in maintaining supplier chains. Moreover, on the social level, it was understood that a strong SME sector promotes the stability of democracies through strengthening the middle classes, offering realistic perspectives to young people, unemployed persons and minorities.

The attention of governments to SME development policy was especially raised in the 80s when in developed countries it was discovered that the majority of new working places were created in small firms and more generally that entrepreneurship generates economic growth. In several countries SME research was directed to discover the dynamics of new firm creation. In the first phase of SME policy, decision makers have focused on the aim that new companies should offer more employment, than the number of working places lost by disappearing firms.

Local and regional governments have recognized that the birth of new firms or the re-settlement of existing companies into the localities administered by them can positively affect local economic development in a wider

11[Smallbone-Welter 2009]


3. SME Development Policy

sense than just increasing employment. It was acknowledged that small businesses improve the income position of the population, increase tax revenue, and provide additional services for local consumers and businesses. 2

Moreover, it was acknowledged that entrepreneurship may increase social capital on the local and regional level by providing good life perspectives for young people, minorities, women and other groups, moreover by creating networks of co-operation and trust.

As a consequence, the various levels of government have become a major enabling or facilitating factor of entrepreneurship by removing obstacles to enterprise creation, by establishing a facilitating environment for private sector development. The awareness of decision makers has been raised about the importance and widely ranging effects of rules, procedures and administrative practices affecting the starting and developing of businesses. One of the major tasks of regulatory activity is to reduce informality, to fight the grey economy, to favour legal economies as opposed to second or illegal economies. Therefore, regulations must be developed and implemented in a transparent way and enforced in a consequent manner in order to motivate compliance and to reduce informality.

Governments and international organisations have issued an increasing number of strategy documents on SME policies. These documents have formulated major aims and objectives of this policy area, such as creating equal opportunities for smaller and larger companies, simplifying the legal, administrative and institutional environment, facilitating access to finance, know-how and markets. In the first phase of SME development policy interventions have focused on creating equal opportunity for small businesses, but in the 1990s the priority of those aims which were connected to enhancing competitiveness of SMEs has gradually increased. Consequently those measures and interventions have gained ground that were related to the removal of trade barriers, to the introduction of new business models, to the organisation of new business networks, to the promotion of innovative practices and to the development of electronic business.

The direct subsidies and incentive programs that were offered for companies by states, regions and cities were also highly criticized. It was frequently stated that such programmes waste public money without creating new jobs, subsidize companies for economic actions that they would have taken anyway, create unfair competition by helping some arbitrarily selected firms, and distort the prices in international trade. Another recurring subject of criticism was that many regulations were superfluous and clumsy and that the administrative procedures were bureaucratic and over-centralised. 3

In the subsequent debates about the justification of small business development policies and programmes, the use of evaluation and impact assessment studies has become an important component of good governance.

1.2. Special features of SME development in Central and Eastern Europe an transition countriesMost case studies and examples of this book that have been selected to demonstrate the methodological approaches of impact assessment and evaluation are taken from the practice of SME development in transition economies of Central and Eastern Europe . This does not restrict the applicability of these methods in any way: they are generic, i.e. their relevance is not restricted to a specific level of economic development, political circumstances or regulatory environment.

In those countries which were or still are in transition from a centrally planned economy into a market economy, small businesses are facing challenges that are quite different both (a) from the challenges of SME development as they appear in developed countries and (b) from those in developing countries. Consequently, the portfolios of the applied policy instruments that have been applied in these countries for facing these challenges have been rather special to the post-socialist region.

In Central and Eastern Europe , during the decades of central planning, many forms of private business ownership were restricted, have been criminalised or subsisted in the grey or informal economy. Private business activities were tolerated to a certain extent in each socialist country and in certain periods, policy makers have experimented with encouraging some forms of private initiatives. However, following the geopolitical changes that took place between 1989 and 1991, the former socialist countries and Soviet republics have begun their transition from a centrally planned economy towards a market economy favouring private sector development and entrepreneurship and an emancipation and re-emergence of private enterprises took

22 [Nolan 2004]

33[Schweke 2000]



place. [bib_2] The countries of the region became transition economies that were facing deep economic, social and political crises while struggling to transform the public sector and the production systems just inherited from socialism.

The reform process called “transition to market economy" included the following major components: 4

• Changing the ownership structure of the economy. This was implemented through the privatisation of former state-owned enterprises and land, and also through the creation of new privately owned enterprises. In particular, ’small-scale privatisation’ (e.g. of shops, garages and restaurants) has contributed to the evolution of a new class of entrepreneurs. In the same time, effective protection had to be granted to property rights.

• Economic liberalisation. This included price liberalisation, the liberalisation of factor markets (such as the labour markets, the capital markets, the markets for raw materials, and the markets for management or entrepreneurial resources), as well as the liberalisation of markets for final products and services. The power of monopolies had to be restrained and the principles of fair trade had to be enforced by introducing a brand new policy field, competition policy.

• Building and consolidating market institutions. This included the establishment of company registries, SME support centres and also the transformation of the financial infrastructure.

• The promotion of Foreign Direct Investment (FDI) was important, partly because of the shortage of capital, and partly because FDI is an integral part of an open, international economic system and a major potential catalyst for development through linkages to subcontracting SMEs and through spillover effects such as the spreading of know-how.

The first measures were to reform the banking system, to safeguard property rights, to liberalise prices and to privatise a large proportion of the state property. Among the new owners there were foreigners, surviving entrepreneurs of the socialist epoch, former landowners or any person being in the possession of the necessary capital, to buy the assets in all post-socialist countries. But privatisation procedures have also favoured former and incumbent company directors, leaders of the former public administration and ruling parties, other decision makers moreover their family members. This latter process has contributed to the evolution of the so-called „nomenclatura entrepreneurship"

The above measures were very important interventions, but still not sufficient to create a viable SME sector. The institutions inherited from the socialist system, the under-capitalisation of most SMEs and the lack of entrepreneurial culture were serious obstacles to a sustainable small business development.

Within a few years millions of small enterprises were created in the Central and Eastern Europe a region. Experience has shown that it was not sufficient to introduce policies that facilitate the creation of many new firms. Additionally, measures had to be taken to ensure the survival and growth of these companies by improving the business environment, by facilitating the access to loan and equity capital and by developing the capabilities of these companies to employ persons or to implement innovations of technology or organisation. While the numbers of self-employed individuals and micro companies has grown spectacularly, there was a clear shortage of middle sized firms.

In the post-socialist transition countries the following major deficiencies of SME development and the respective challenges to small business development policy should be highlighted.

• Necessity-based entrepreneurship. A significant proportion of new companies has come into being as a reaction to the deep crisis accompanying the transformation and was serving the immediate consumption needs of the entrepreneur and his/her family. Wide social groups have escaped from unemployment into self-employment. Most frequently, the small firms created by them have not employed anybody except the owner, or only offered informal or part-time employment to a few family members. Most small firms were unable or unwilling to separate the budget of the household from that of the business and lacked any ambition to grow.

• Dual economies. A number of Central European countries were rather successful in attracting foreign investors. By the 1990s in some of these countries the contribution of foreign owned companies to production and export has surpassed the contribution of domestically owned firms. At the same time there were scarcely any linkages between local SMEs and multinationals, due to weaknesses of SMEs in quality management, just-in-time delivery and marketing. Policies were needed to facilitate and encourage subcontracting relations

44[Smallbone-Welter 2009]



between bigger and smaller firms.

• Attitudes to innovations. Most new enterprises were based on some traditional business idea. Only a minority of small enterprises were based on innovative business models. Businesses based on some technical or organisational innovation were rather an exception than the rule.

• Trust and compliance. At the early stage of transition frequently forms of behaviour could be observed such as survivalist muddling through and a tendency to avoid the compliance with rules. In the early years of transition, entrepreneurs have faced a high level of uncertainty and ambiguity. The result was a low level of trust which did not facilitate the development of legal forms of inter-firm co-operation, neither the evolution of entrepreneurial networks. In particular, wide groups of SMEs in the Central and Eastern Europe an transition countries can be characterised by avoidance and non-compliance behaviour, which can be explained by the weaknesses of entrepreneurial culture, by the instability of the institutional environment and by the deficiencies of enforcement.

It was not expected that spontaneous market processes would lead to competitive economic structures in the medium or even in the long term. 5 Therefore, for the last two decades a series of progressive measures were taken in order to develop the business climate in these countries. For example, a wide range of such interventions were made in order to facilitate the access to credit and to credit guarantee services for SMEs [bib_3] and access to microcredit [bib_4]. The development of the business climate was also extended to investment promotion and Central European countries have become competitors both in attracting multinational investors to outsource their business processes to them and in motivating local SMEs to start a business on their territories [bib_5]

The Europeanisation process . Although in the countries of Central Europe the formation of SME development policy was under the authority of the nation states and local governments, still the major blueprints of the necessary institutional reforms and their financial background were determined to a large extent by the European integration process. During the 1990s most of these countries became candidate countries of the EU, subsequently joining to the European Community in the early 2000s. Moreover, a series of other Central and Eastern European countries have intended to join the EU or at least to participate in the advantages of the Common European Market. Consequently, these countries have joined various SME development initiatives of the EU and have signed agreements, charters and other key policy documents about aims and instruments of their national SME development policies. These moves were followed by the harmonisation of a wide range of European legal provisions and by the channelling of a vast amount of EU aid into the region aimed at developing the SME sector. The programmes and projects financed or co-financed by the EU were facilitating SME development by simultaneously spreading European institutional patterns. In those transition countries which became EU member states, the development of the business climate continued well after joining the EU.

1.3. The relationship between SME policy and other policy fieldsIt is not only the targeted SME and entrepreneurship policies that influence the creation and the development of small businesses. Governments also implement other policies without specific focus on SMEs that strongly influence small and medium sized companies, such as monetary and tax policies, education, social and labour policies, environmental, regional and sectoral policies, and trade and innovation policies. As a rule, the government departments responsible for the above policy areas are only indirectly concerned with small business development, in fact, their responsibilities for small businesses are very limited. However, these institutions spend much public money on programmes that benefit small firms. Analogously, in most countries the responsibilities for delivering local and regional support to SMEs is shared by many public and non-profit organisations of various legal forms, such as local SME development centres, regional chambers and others.

The following examples demonstrate how policy areas not directly concerned with small businesses may influence the development of this sector.

• Fiscal and monetary policies . Entrepreneurship can be inhibited by interest rates that are substantially higher than inflation, by macroeconomic stagnation or instability. Entrepreneurial performance can be discouraged by high taxes on profits, sales and inheritance However, high tax regimes also contribute to the rise of informal economy. Tax evasion under certain circumstances is easier for SMEs than for bigger companies 6.

55[Dallago 1999]

66 See Reference XLVIII at the end of this document about the size of informal economy in Hungary.



• Regulatory policy . SME activities are reduced by heavy regulations - causing much administrative burden - on licensing, on starting a business, on hiring and firing workers, on registering property, on getting credit, on protecting investors, on paying taxes, on trading across borders, on enforcing contracts and on closing a business. 7 Policies simplifying administrative procedures and fighting corruption contribute to SME development. In particular, business registration is an important government function, because it prevents fraudulent or unqualified entrepreneurs to appear on the markets with their products and services and cause harm to consumers, to their employees or to the environment. However, cumbersome licensing regulations or those with high compliance costs keep entrepreneurial persons and groups in the informal economy or reduce their willingness to start businesses.

• Competition and consumer protection policies . Policies ensuring fair competition can contribute to equal treatment of bigger and smaller firms by banks, by other service providers and in the public procurement process. [bib_6] However, policies ensuring the conformity of products to certain norms or facilitating the entry of new firms to the markets, are frequently debated by existing companies, in spite of the fact such measures may contribute substantially to better product quality and lower prices.

• Education policies are not only responsible for the training of potential employees of companies. Additionally, in an increasing number of countries, attention is paid to developing entrepreneurship education in secondary schools and in universities. 8

Decision makers of SME and entrepreneurship policy must co-operate with other departments of the government responsible for the above policies. They are expected to lobby at other Ministries and government departments for regulations and for support that takes into consideration the interests and the special needs of small businesses. In particular, the development of inter-departmental interest representation within the government might be necessary:

• For implementing reforms of regulatory policy in order to simplify administrative duties of SMEs;

• For creating equal opportunities for SMEs in public procurement;

• For increasing SME related subsidies within funds devoted to regional development;

• For creating specific support schemes for innovative SMEs under technology and innovation policies.

And in many other issues that were under the responsibility of fiscal, environmental, labour and other authorities.

In particular, it is an important function of evaluations of SME interventions that they should facilitate discussions between the respective departments of the government.

77See Case Study „B" about the World Bank’s „Doing Business" indicators in this document.

88[Szirmai 2005]



Figure 7. The relationship between SME policy and other policy fields

2. Evaluation of SME development projects, programmes and policies2.1. Project and programme level evaluationsDonor organisations, whether public or private, collect a wide body of information in order to decide, which project they are going to support, and likewise, these organisations must receive reliable feedback about the programmes and projects that have been financed by them. Both the World Bank 9 1010 and the European Union 1111

have implemented strict policies about the mandatory preparation of a priori and a posterior evaluation of supported projects and programmes. These organisations have published guidelines on how to collect data for evaluation and define those criteria against which the projects and programmes should be evaluated. The application of some of these guidelines has a mandatory nature, while others are issued as recommendations to be followed by evaluators.

Enterprise support projects aim to enhance the competitiveness and the creditworthiness of companies or to develop a particular function of the companies such as innovation, human resource management or marketing. For this reason, the evaluation of enterprise support projects has much in common with traditional business consultant activities where the competitiveness or the creditworthiness of SMEs [bib_7] is analysed or various functions of companies are assessed [bib_8]. However a major difference between such descriptive studies on the one side and the evaluation of subsidised projects on the other side is that while the genre of evaluation always involves the elements of causation and impact, this is not necessarily the task in case of diagnostical analyses.

99[WB 2007]

1010[WB 2003]

1111[EAR 2005]



What is a project. It is reasonable to take a look at the definition of the notion “project", since in most cases this is the entity which is evaluated by the analysts. Moreover, the process of evaluation is just one of the phases in the life cycle of a project, i.e. the evaluation of projects is embedded into the wider activity called “project cycle management". Within the aid delivery methodology of the European Union 1212 the concept of a project is defined as a system of activities, which has a clear and well defined purpose, a planned duration, a budget, target groups, furthermore it has co-ordination-, management- and finance mechanisms. As a rule, projects are selected as building blocks to accomplish a political programme of a government or an international organisation.

What is a programme. According to the conceptual framework used in EU documents, the notion of the programme has a somewhat wider meaning than that of a project: it is a set of homogenous projects co-ordinated by a particular policy aim. In case of programme evaluation the subject of evaluation goes beyond the individual projects and a wider portfolio of support projects having common objectives or being implemented by the same institution is evaluated in a comprehensive way. For example, in the practice of the EU aid delivery, the so-called operative programs have to be evaluated before, during and after their realisation. Within the policy area of SME development a wider enterprise-supporting programme can consist of several projects, each of which offers subsidised training or consultancy services for eligible small enterprises. In this framework, eligibility can be defined according to the overarching aim of the particular policy which is served by the programme, e.g. in case of innovation policy, only innovative SMEs can be eligible to receive these subsidised services.

In the standard aid delivery methodology of the European Union the criteria of project evaluation are as follows.

• The relevance criterion is appropriate for the analyst to ask, whether the design of the project and its implementation corresponds to the challenges facing the target group, whether it is harmonised with the relevant goals of public policies, and whether the project can be / has been properly adapted to the existing institutional structure.

• The efficiency criterion is needed to assess whether the project adequately transforms the existing resource inputs to outputs. In particular, evaluators ask here about the quality of time management, cost efficiency and the level of inter-organisational and intra-organisational co-operation.

• The effectiveness criterion needs to be applied in order to assess the contribution of project outputs to the direct goals of the policy. In particular, in this phase of the evaluation, researchers have to ask about the actual achievements of the project.

• The impact criterion is appropriate for assessing the contribution of outputs to a wider range of economic and social aims, including local and regional externalities, such as organisational learning or the improvement of entrepreneurial culture.

• The application of the sustainability criteria is needed to check the long term viability of the results of the project, in particular the extent of its donor dependence.

Table 3.1. Box 2.

Evaluating the Phare Microcredit Programme in Hungary

In Hungary , following the political changes of 1989, a wide range of small enterprise development projects were launched by various public bodies and non-profit organisations. These projects and programmes were frequently co-financed by the European Union , directly or indirectly. All of them were several times evaluated with the standard evaluation methodology of the EU. A major result of these subsidised projects is a well functioning network of local enterprise development foundations which were established by the end of the 1980s. a Subventions of the EU have financed the launching of micro-credit and other preferential credit schemes as well. The Phare Microcredit Scheme was called into being by the fact, that one of the major barriers for development of the young Hungarian small enterprise sector was the undercapitalisation of these firms b. In 1992 the Microcredit Fund was established and soon the county-level c enterprise-development foundations started to disburse these credits for start-up and growth-oriented small enterprises. This preferential credit was given to those promising small firms, which were not creditworthy enough to receive credits from the

1212[EC 2004c]



Evaluating the Phare Microcredit Programme in Hungary

commercial banks. During the existence of Hungarian micro-crediting a major question was whether the programme would / will be viable in real market conditions or it is only prospering with governmental support, in a “donor-dependent" way. d During its existence the Micro Credit Scheme was several times evaluated [bib_9].

In 1999 the Microcredit Programme was evaluated by independent consultants [bib_10]. Their report begins by demonstrating the Phare Microcredit Scheme as a financial product: its interest rate, guarantees, maturities, the financial sources of this credit scheme, which beneficiaries are eligible to receive microcredit, what kind of appraisal mechanism is used to select beneficiaries. Statistical information follows: the volume of the disbursed and repaid credits, the composition of the beneficiaries by sector and county.

The core of the evaluation report contains the opinions of the evaluators according to the selected evaluation criteria, expressed in numerical scores and justified in text. The 5 evaluation criteria were given in advance by the donor organisation and it was the task of the evaluators to transpose these criteria into operational evaluation questions. The main evaluation questions were as follows:

• Relevance . What kind of sources other than microcredit were available to satisfy the needs for capital of SMEs? Were the parameters of the microcredit scheme (e.g. the interest rate and the limit of disbursable credit for one beneficiary) relevant with the goals of subvention and with the needs of the SME sector? Was there sufficient demand for the microcredit offered?

• Efficiency . Were there delays in the implementation of the programme? If yes, why? How much time did it take for the beneficiaries from their application to the disbursement of the credit? What was the level of transactional and handling costs of micro credit, and from which sources were these charges covered? Do the regional enterprise-support foundations charge reasonable price for managing the programme? Is there a good co-operation between the various organisations active in the implementation of the programme?

• Effectiveness : Has the programme succeeded in reaching the originally targeted SME sector? What types of investments, developments were accomplished which could not have been financed without the micro credit? Has the microcredit programme offered equal opportunities across the various sectors of the economy, such as services, trade, industry and agriculture? Did SMEs in all of these sectors receive sufficient micro credit? Was microcredit provision decentralised across the regions?

• Impact: Has the microcredit scheme succeeded to attract further capital from other sources to finance the SME sectors? Did the micro-credit scheme stimulate the commercial banking sector to provide credits for SMEs? Did the programme increase the prestige and legitimacy of the county–level enterprise development foundations? Did the availability of the microcredit scheme motivate SMEs to reduce their participation in the informal economy? Did the scheme strengthen the local ties; facilitate the local ties of SMEs?

• Sustainability. What are the characteristics of credit repayment and credit default among beneficiary SMEs? Can the Micro Credit Fund, which is the source of micro credit provision, sustain its operations from the repaid capital and from the interests of micro-credits? Are the selection, control and motivation systems appropriately defined in order to increase the proportion of repaid credits? Is there any appropriate fund involved to cover the respective credit risk?

The answers to the above questions need to be confirmed by available statistics, project management documents as well as by interviews made with project managers and grantees.a[Bateman 2000]b[Szirmai 2003]cThe territory of Hungary is admnistratively divided into 19 counties plus the capital, Budapest.d[Kállay 2003]

Logical Framework . Several donor organisations such as the USAID and the EU have decided to apply a standardised version of the so-called “Logical Framework" method to co-ordinate the process of project planning, implementation, monitoring and evaluation. The method compares the inputs and outputs of the project and summarises them in the so-called Logical Framework Matrix, which is developed in the project planning phase and used a major document of project implementation, continuous monitoring and periodical evaluation. It defines the hierarchy of goals and risks of the project, furthermore it specifies the so-called



objectively verifiable indicators, which are supposed to measure the achievements of the project in quantitative terms or prove its success with the help of empirical facts.

The life cycle of projects financed or co-financed by the EU consists of the following phases:

• Programming. In the phase of project identification and specification the representatives of the donor agency and the supported government make decisions about the main priorities, the type of subvention and sort of financing, analysing the viability of the project and the sustainability of its results. This is a planning activity which precedes the selection of the beneficiaries and the launching of the projects.

• In the implementation phase of the project the previously accepted activities are performed. During the lifetime of the projects the supporting organisation obtains monitoring reports which give feedbacks about how the implementation of the project is proceeding. In case of failures the project plan and the implementation of the project can be modified, or in extreme cases the project can be stopped.

• Monitoring is a continuous activity which goes into more detail in terms of feasibility and efficiency than evaluation. Monitoring experts may help the day-to-day activities of project managers by concentrating on the inputs and results of project activities.

• In the phases of preliminary, interim and ex post evaluation independent experts examine the project against the criteria of relevance, efficiency, effectiveness, impact and sustainability. These criteria have been used by the EU during the last decade, but other donor organisations may recommend the use of other, more or less equivalent criteria. The evaluators analyse the economic uses and administrative rationality of the project, assess whether the results are sustainable and collect the lessons to be learned from the project. Depending on their findings, the donors may modify some project instructions, may decide to take corrective actions.

• In the audit phase auditors are controlling whether the relevant laws, the provisions of the support contract, the financial, accounting, purchasing and reporting rules of the subsidy scheme have been observed. Auditors examine the procurement procedures and decide, whether the expenditures were proportional to the purchased products and services during project implementation. While evaluators do not control the legitimacy of project procedures, decisions and payments, auditors do exactly this type of work. The main information sources of audit work are the contract documents and the bills and invoices issued and received during the project.

The main information sources of evaluation are as follows.

• Documents of project planning and management such as invitations to tender and monitoring reports. Evaluators must always pay attention to account for eventual biases contained in source documents with special respect to withheld information.

• Interviews made by the evaluators with the persons responsible for project design and implementation, such as the employees of co-ordinating and support agencies, representatives of the beneficiary institution and if applicable, the final beneficiaries. Evaluators must always pay attention to account for the hidden agenda of interviewed persons.

• Quantitative surveys. Large or medium sized sample surveys are very informative; in fact the statistical evaluation of the responses can convey the best and most objective picture about the success or failure of a project. However, due to its extensive costs, evaluators can rarely rely on a questionnaire based survey conducted among beneficiaries and with the members of an appropriate control group. Within the questionnaires of evaluation surveys the particular questions correspond to one of the above mentioned five evaluation criteria. The major task of questionnaire design is to operationalise these criteria and to transform them to relevant micro-level questions. Good questionnaires cover all relevant issues of the project, including (a) the evaluated support scheme, (b) the institutionalisation of the support and (c) the characteristics of the target group.

Table 3.2. Box 3

Evaluation of PHARE and Transition Facility subsidies

An example for project evaluation methods is given by the standard template of the PHARE Interim



Evaluation of PHARE and Transition Facility subsidies

Evaluation Reports. In Hungary during the years 2003-2008 several dozens of interim evaluation reports have been prepared about the planning and implementation of programmes and projects co-financed by the PHARE Fund of the European Union and the Hungarian government. Each of these reports was devoted to one single “sector". In the terminology of PHARE support, a sector is a policy are such as agriculture, SME development, justice and home affaires, environment, etc.

For evaluating the aid absorption capacity of such a sector, a sample of projects is taken and evaluated. The main chapters of each report are as follows.

• Sector Background: Major events that happened in the sector during the time period evaluated, including its institutional environment and the major challenges to aid delivery.

• List of projects, beneficiaries, tables of the allocated and disbursed finances.

• Description of aims, activities and corrective actions listed by projects.

• Evaluation results are sorted as follows: the five criteria (e.g. relevance, efficiency, etc.), and under each criterion the individual projects are evaluated according to the selected criterion.

• Conclusions and Recommendations.

An application of the above template can be illustrated by the evaluation of the following specific project.

Table 3.3. Box 4.

Evaluating an SME-support project for developing e-business services

„E-business as a tool of SME-development" was a European Community co-financed project within the framework of the Hungarian PHARE program, on which the donors have jointly spent 4 million Euros during the time span of the project, between 2003 and 2005. The activities financed for the beneficiary SMEs were training courses for groups of entrepreneurs and customised counselling services for individual firms to support the development of company level ICT strategies. These activities were performed on 20 different locations of Hungary , altogether for about 800 SMEs. The beneficiary SMEs were able to publish their offer on an e-commerce website which was prepared within the framework of this project. These services have provided real added value for the beneficiary companies, because at the time of the project most Hungarian SMEs were not ready to invest into sophisticated e-commerce services or into the relevant training courses under the conditions offered by the open market.

The Sector Background chapter of the document has described the state of ICT applications within the SME sector of Hungary and previous efforts of spreading this technology among small businesses. The most important chapter of the document was the application of the five evaluation criteria on the specific project. This was operationalised by asking the following research questions in the course of the interviews and document analyses.

• Relevance : Evaluators wanted to know if the project design has met the real needs of SMEs and whether the major issues of entrepreneurial culture and competitiveness were addressed by the project.

• Efficiency : Was the project carried out in due time, with the possibly lowest drop out rate? Was the project successful in establishing good cooperation among the participating organisations such as project managers, trainers, and ICT consultant companies?

• Effectiveness . Was the project successful in reaching a wide enough stratum of SMEs? Did the project



Evaluating an SME-support project for developing e-business services

create good quality training materials and were these materials published on the Internet? Is it possible to successfully offer, sell, buy and pay products on the e-business website created in the framework of the project?

• Impact. Was the project successful in diversifying the activities of the beneficiary SMEs and their co-operating partners? Did the project enhance the cooperation among the participating firms and those which are vertically or horizontally were connected with them?

• Sustainability : Following the closure of the project, when support for such activities will stop, will beneficiaries be able to develop their ICT strategies and achieve success in e-business? Will the website created in the framework of the project survive, will it be competitive enough besides the already existing e-commerce websites, and eventually will it be privatisable? Will the training material compiled in the frame of the project be good for use, following the end of the project, too?

2.2. Case study “SME Employment": Survey for evaluating subsidies given in order to enhance the employment capabilities of SMEs 1313

2.2.1. Policy context: harmonising employment and competitiveness priorities in small enterprises

The support scheme to be evaluated offered subsidies to Hungarian small firms in order to increase the number of their employees, and to improve the quality of employment at these firms by subsidising company level investment projects. Only those applicants were eligible to benefit from the programme who have submitted project plans which have promised simultaneously (a) to develop their capabilities to act as employers and (b) in the same time to enhance their competitiveness as a company.

In order to understand better the impact mechanism of this programme, let us first examine the relationship between employment and competitiveness in the SME sector.

Job creation can be regarded as the traditional aim of small business development policies both in developed and in developing countries. The way to enhance the role of small businesses as employers is decrease their costs and risks related to employment with the help of regulatory and fiscal measures, and by support programmes.

In Hungary the role of SMEs in employment is very significant: more than two-thirds of employees working in companies or sole proprietorships are employed by SMEs. However, during the decade after 2000 it has been observed that the SME sector has not contributed to the stabilisation of the labour market, in fact, between 2000 and 2007 the number of employees working in companies and sole proprietorships has decreased. 1414 The number of persons employed by micro and small enterprises tends to fluctuate, as a function of labour supply and business cycle, and this volatility is greater then that observed by bigger companies. The life expectancy of small enterprises is significantly lower than that of bigger firms; and always when a small enterprise goes bankrupt, the primary losers are the employees themselves. In case of small firms the willingness to grow as expressed in the number of employees is considerably smaller than that of the large ones.

Theoretically it can be expected from the SME sector to apply more flexible forms of employment (part-time employment, time-sharing, renting of labour force, seasonal work, distance work, etc.). However, Hungarian small enterprises have contributed to the spreading of these forms of labour only to a limited extent. The above forms of flexible employment are most popular among enterprises without a solid financial background and among enterprises of the so-called social economy. [bib_11]

1313The evaluation was performed on behalf of the National Public Foundation for Employment which has approved the publication of the method applied and the results appear in the present text. The questionnaire based survey was carried out by the SEED Small Enterprise Development Foundation.1414Source: [GKM 2004]



SMEs have a lower willingness of to enter into co-operations with other companies than bigger enterprises: smaller firms typically hesitate to outsource some of their production to subcontractors, even in cases when this would be a cost sparing and flexible solution. Typical SMEs wage to outsource only their most peripheral activities.

Compared to bigger enterprises, small enterprises pay, as an average, less wages to their employees with comparable qualification and employ people of lower qualification for comparable jobs. Although during the past decade an increasing number of Hungarian small enterprises have entered into activities based on qualified work, still in general, small enterprises have less willingness to develop the qualification of their employees than the large ones.

Small enterprises operate, in general, more labour-intensively than large ones. This relationship between productivity, employment and the size of the enterprise has been observed throughout the 20th century and can be demonstrated in industrial statistics across several sectors of the economy. In other words – compared to bigger enterprises - for the creation of one unit of output small enterprises SMEs need to appropriate more inputs in terms of labour. An improvement of productivity by means of modernisation has frequently the consequence of dismissals of employees.

While in the 1990s the SME development policy documents of the EU have predominantly stressed the importance of employment capabilities of SMEs, after 2000 competitiveness became the priority aim of this policy area. In March 2000 the European Council held a meeting in Lisbon and has declared the strategic goal of making the European economy, for the next 10 years to come, the most competitive and knowledge based economy of the world. One of the resolutions of the conference has called upon the EC Committee to prepare a so-called Multiannual SME development programme for the years 2001-2006 1515. During the following years the new hierarchy of development goals has made its way into the strategic documents of European and national SME policies and the result was that the instruments of development policies, including aid delivery, were re-oriented accordingly.

In 2003, however, the so-called Wim Kok Report of the EC announced, that the fulfilment of the ambitious competitiveness goals of the Lisbon Meeting were endangered. The report contained important findings about employment policy as well. 1616 According to the report, the European Community can harmonise the dual objectives of increasing both employment and productivity, only if the following requirements will be met:

• The adaptability of employees and enterprises should be improved;

• More people should be attracted to the labour markets as potential employees;

• Investment into human capital should increase and these investments must become more effective;

• Reforms aiming at better governance must be consequently implemented.

Among other countries, the Report also touched upon the human resources management strategy of Hungarian SMEs as well. The Report recommended that the start-up of businesses and their investment into human resources should be facilitated (a) by the support of consultancy services and (b) by the establishment of one-stop consultancy systems.

Strategic analyses published by the Hungarian government have revealed that the competitiveness and export orientation of small enterprises is lagging behind that of the big companies, their productivity, innovative possibilities being lower, their technical development slower. 1717 Although there is an increasing demand for differentiated services and the pattern of outsourcing is spreading among SMEs, the proportion of loss making firms is much higher among micro companies than among small companies, and higher among small companies than among middle sized firms.

The above characteristics of the Hungarian SME sector demonstrate that one of the great challenges of support policy is to enhance the employment capabilities of small firms by simultaneously increasing the productivity and competitiveness of the sector. For this reason, employment oriented subsidy schemes granted for small and medium size enterprises should motivate a sufficient number of small enterprises (a) on the one hand to employ

1515 [EC 2005b]

1616 [EC 2003a]

1717 Source: [GKM 2004]



more people - or at least to retain their previous employees - , and to improve the conditions of employment, and (b) on the other hand to increase the competitiveness of the beneficiary enterprises. Evaluators should keep in mind that the above requirements should be applied as criteria of their evaluation.

The issue of direct support. The specific support scheme to be evaluated is a direct subsidy. Each potential beneficiary firm must apply for a certain amount of non-returnable financial aid by submitting an application with a feasible project. The subsidy will be allocated on the basis of individual decisions of a selection committee. In EU member states the direct support of companies by government organisations is limited to a certain extent: it is constrained to specific objectives such as small and medium size enterprise development, regional development, research, the protection of the environment, and the enhancement employment. Allocations not exceeding a certain value limit do not fall under the effect of these prohibitions. These limitations have been accepted because direct subsidies frequently distort competition by enhancing donor dependence and by artificially placing certain enterprises in advantageous position against other firms, i.e. by “picking the winners".

2.2.2. Purpose and method of the evaluation

The main purpose of the applied research [bib_12] was to prepare an ex post evaluation of a specific support scheme implemented by the Public Foundation for Employment (OFA). This Foundation has operated the „Support programme promoting the development and strengthening of small enterprises" in the years 2001 to 2004 in form of a grant scheme. During this time the Foundation allocated 1,019 billion HUF (approximately EUR 4 million) to grantees supporting altogether 212 SMEs.

The basic research questions of the evaluation were as follows:

• What were the effects of the programme on the labour market position of the supported companies as well as on their competitiveness ?

• What was the level of satisfaction of the applicant SMEs with the support and with the supporting institution?

• What was the opinion of the applicants about Hungarian SME development policy ?

The empirical bases of the evaluation were provided by the following sources.

Analysis of documents. As a first step, the researchers examined the most important documents of the grant scheme: the annual announcements about the tender, the lists of the applicants and those of the winners, the criteria used by experts selecting the beneficiaries among the applicants, the project files of the individual winning companies, and the annual reports about the programme.

Interviews with project managers. As a second step, interviews were made with the managers of the programme about the success rate and success factors of subsidised projects, moreover about the lessons learnt and to be applied in subsequent support schemes.

Questionnaire based survey among supported and refused enterprises. As a third step, a survey was made with the applicant enterprises, both with beneficiaries and with those refused by the donor. In April-May of 2005, interviewers contacted 278 such small and medium size enterprises, out of which 146 enterprises had won the demanded support by the OFA, and 132 SMEs had applied but were rejected subsequently. This was a relatively high sample size, since during the years 2001-2004, altogether 742 SMEs applied for the OFA tenders in question.

Due to high sample size the evaluators were able to apply statistical testing of hypotheses. In particular, findings were significant on a 5 % level of significance in which the researchers compared (a) winner and rejected companies, (b) small and bigger firms, (c) companies working in various sectors, (d) firms having applied for subsidy in various years, or (e) firms being located in various regions. One of the main aims of the survey was to compare the supported and the refused enterprises.

The Questionnaires. The interviewers have used two distinct questionnaires: one for the supported enterprises (containing 146 questions) and a slightly different questionnaire for applicant enterprises that have been refused (87 questions). A specific feature of the questionnaires was, that they contained a relatively high number of open questions which could be answered in freely formulated text. This enabled the researchers to learn more about the attitudes and decision mechanisms of the questioned enterprises (e.g.: their satisfaction, expectations,



optimism, fears, their motives to employ additional people or their willingness apply a growth oriented strategy for the enterprise). The responses to open questions were assessed with the help of content analysis.

The sample of the survey. The sample of the survey has proportionally represented the target group of the subsidy scheme, because the inner proportions of the sample have approximately corresponded to the inner proportions of the population of SMEs in terms of company size, economic sector, and region of Hungary .

• Size. About half of the SMEs in the survey were micro-companies.

• Legal form. More than half of the companies were limited liability companies, one fifth were sole proprietorships, one sixth was a partnership, but co-operatives, joint stock corporations and companies without legal personality were also represented in the sample.

• Age. The sample has contained enterprises of all ages, ranging from “older" firms which were established 45 years ago to those founded 2 years ago. The median age of companies was approximately 8 years, both in case of supported and refused enterprises.

• Ownership. In most cases, the responding firms had two owners. Responding co-operatives had a large number of owners. Nearly all the answering enterprises had a Hungarian private person among the owners, more precisely; the proportion of Hungarian ownership was 100 % in nearly each case.

• Premises. About two-thirds of the applicants have worked on the same single location. In the circle of the supported firms the proportion of those with 2 or 3 premises was 30 per cent, while in the case of refused SMEs this proportion was only 20 per cent. Firms with more than 3 premises occurred only very rarely.

The questionnaire based interview was conducted personally with a high ranking manager of the applicant company, in 80 percent of the cases, the owner (or co-owner) of the enterprise. In all other cases the interviewed manager was selected as a competent person, who knew well about the project supported (or rejected) by OFA and/or personally participated in it.

Table 1. Distribution of supported enterprises by firm size and sectorNumber of enterprises

Sector

Company size group (number of full time employees)

Industry Agri-culture Services Trade Total

Micro – 1 (0-5 employees)

9 10 18 5 42

Micro - 2 (6-10 employees)

10 4 7 6 27

Small (11-50 employees)

30 13 13 15 71

Medium sized (51-250 employees)

0 5 0 1 6

Total 49 32 38 27 146

Table 2. Distribution of refused enterprises by firm size and sector Number of enterprises



Sector

Company size (number of full time employees)

Industry Agri-culture Services Trade Total


20 12 29 17 78


4 2 6 4 16

Small (11-50 employees)

12 1 10 12 35

Medium sized (51-250 employees)

1 1 0 1 3

Total 37 16 45 34 132

Results of the survey

Profitability. In the year of the application about 86 percent of the enterprises were profitable, and less than 7 percent have made losses in that year. There was no difference between the supported enterprises and those which were refused, from the point of view of profitability at the time of the application.

Financial characteristics of the submitted project plans. The companies have applied, as an average, for subsidies in the range of 4 to 6 million Hungarian Forints (16.000 to 24.000 Euro). Supported firms on average had applied for somewhat higher subsidies firms that were subsequently refused. Most firms have offered to co-finance the project from their own sources in order to have the project realised. About 8 % of the applicants have declared that the project was co-financed by a third party, other than the supported firm. Such co-financing was, in most of the cases that of the character of a credit, and the co-financing institutions, in all such cases, were banks. 23 % of the supported enterprises received smaller amounts than they had applied for.

Professional content of submitted project plans. Nearly half of applicant companies wanted to spend the required amount of money on employing new persons. One third of applicants intended to purchase new appliances, another third wanted to embark on building operations. There were many projects aiming at social, educational, innovative purposes as well as projects aiming at the protection of the environment. In many projects, the above aims were attained with the help of consultancy services, which have helped the entrepreneur to implement the project or to establish relationships with other enterprises. Within the group of supported enterprises some two thirds have committed themselves in their applications to at least keeping the existing level of employment in the firm, while within the group of those applications that were refused subsequently, only half of the applicant SMEs have entered such obligations.

Organisational characteristics of supported projects. The time span of the supported projects was 15 months as an average, but as a maximum, projects with a duration of 2 years have also occurred. 46 % of the supported firms have relied on suppliers and subcontractors d, in the course of implementing the project.

Table 3. Purpose of the planned project as stated in the application document Respondents could select more than one option.



Purpose Supported Refused

N= Number of respondents =146 N=Number of respondents=131

Proportion of those who mentioned the given purpose, %

Hiring new employees 45 50

Acquisition of appliances 36 47

Construction operations 32 37

Introduction of new product or procedure

12 12

Purchase of services 11 3

Acquisition of ISO or other qualification

10 9

Training of employees 6 10

Consultancy or marketing 5 3

Self-evaluation of the programme by the beneficiaries. The majority of the supported enterprises have expressed the opinion that the subsidy was useful and reached its goal both with respect to facilitating employment and competitiveness .

Table 4. Why was the project needed? Content analysis of the responses of applicant enterprises

A content analysis of the various verbal responses given to the above question led to the following results.

One fifth of the responses emphasised elements connected with competitiveness : they mentioned the introduction of some new product, technology, a system of organisation of work (e.g.: ISO). About one fifth of the emphasised elements were connected with employment. About one tenth of the respondents said that their survival depended entirely on the support.

Characteristic individual answers:

• Focus on human resources. „The subsidy has helped to employ people, it improved the conditions of employment" - „It helped to establish a nice dressing room for our employees." – „It has helped us to retain the working places."

• Focus on competitiveness. „ISO qualification is hardly dispensable in our sector." - „The catering unit needed the subsidy in order to comply by EU requirements." - „We wanted to develop, and development was indispensable for our further operation." - „The subsidy was used for the acquisition of computers and for the creation of an up-to-date information system." - „We badly needed such a machine which we could not have bought without the subsidy. The work of the employees has become easier."

Table 5. What has changed in your enterprise as a consequence of the support?

A content analysis of the verbal responses of supported enterprises.



The content analysis of the verbal responses has led to the following results.

The subsidy had the following effects:

• Increase of capacities;

• improvement of the market situation (mentioned by 37 );

• Improvement of employees’ general feelings (mentioned by 36);

• Improvement of quality (mentioned by 9); Increase in company employment (mentioned by 18);

• No significant effect (mentioned by 10);

• Improvement of the financial situation (mentioned by 10).

Other types of changes, less frequently mentioned: More satisfied clients; ISO qualification; Increased stability of the firm; retaining the staff; survival of the firm; improvement of the qualification of the employees.

Characteristic individual answers:

• Focus on employment. „Two of our employees have obtained a higher level of professionalism." - “The working conditions of our employees have improved" - “We did not have to dismiss workers." - “The social block of the job-shop has been completed; the conditions for workers to douche and dress have become more convenient." - “We have hired new employees, but later we were compelled to dismiss them."

• Focus on competitiveness. „Our turnover has increased; the general feeling of the employees has improved." - “We introduced an ERP software and created a website." - “Complaints of consumers have decreased drastically." - “Better equipped, enlarged kitchen, a bigger capacity of the catering establishment." -„We are working with modern machinery" - “We have become known with the help of our websites, there is no need to explain things, customers can calculate their needs themselves and come to us with the already compiled order." - „We have gained experience in how to apply for further subsidies."

Effectiveness and efficiency. About 90 percent of the supported companies completed all the activities planned in the project in time. However, within the group of the smallest companies one fifth of the supported enterprises have reported a delay in properly implementing the project.

Impacts on employment. As a result clearly attributed to the subsidy, within the group of the supported enterprises 68 percent of the enterprises has employed new persons, on average five employees. Three quarters of the firms who employed new persons, reported that among the new colleagues there had been persons who were registered unemployed beforehand. Besides increasing or retaining the number of previously existing workplaces, in many cases the subsidy scheme improved the qualitative parameters of the working environment, such as the conditions of work, the safety of labour and the qualification of the employees.

Impacts on competitiveness. 73 % of the supported enterprises have reported that the subsidy has improved the competitiveness of the firm. As a result of the subsidy, a significant number of these enterprises has increased its turnover or its profitability or has stabilised its financial standing. In some cases new spheres of activities were added to the old ones, in others innovations were introduced. It is typical, in most of the above mentioned cases, that improvements were the direct consequences of the support given. The subsidy has helped several beneficiaries to purchase their inputs from an enlarged circle of suppliers. Some of the beneficiaries have reported that the support has increased their capacities or has improved their quality strategy.

Table 6. In what ways has the supported project improved your abilities to employ persons?



Respondents could select more than one option. Impacts are listed in decreasing order of the number of times they have been mentioned. Base: supported enterprises, N=143

Impacts of the subsidy on the quality of employment

The proportion of those whose response was „Yes"

Improved the employment possibilities of the enterprise

81

Improved the conditions of labour 65

Improved the safety of labour 54

Eased the overburdening of workers 44

Improved the qualification of workers 32

Improved the qualification of managers 20

Table 7. Changes following the implementation supported project

Base: supported enterprises, N=146

Since the implementation of the supported project…

Yes, as a result of the subsidy

Yes, but not as a result of the subsidy

There were no such changes

Proportion of those who agreed

Increased turnover? 46 20 31

Improved profitability? 42 16 38

Widened range of activities? Any new sphere of activity introduced?

34 8 55

Introduced any innovation in the operation of the enterprise?

22 8 67

Table 8. In what ways has the support improved the competitiveness?

Content analysis of the verbal responses given by supported enterprises Number of respondents agreeing with the statement. Respondents could select more than one option.

Statement Frequency

It has not improved competitiveness 31

It has improved competitiveness 123

It has improved competitiveness, in the following



ways:

..Increased turnover 21

..More efficient work 20

..Introduced quality assurance (QA) 19

..Improved trust of customers/clients 13

..Survival of the firm 11

..Technological advantage compared with other enterprises

11

..Entry to new market 9

..Marketing and PR advantages (e.g.: the company has become more known)

7

..Cost reduction 7

..Wider range of products 5

Opinions about the supporting institution and about the efficiency of administering the application . The respondents have expressed their level of satisfaction in school marks. The following aspects were rated:

• intelligibility of the tender document,

• provision of the applicants with clear and useful information,

• counselling for applicants,

• helpfulness and professional expertise of the officials of the donor organisation.

Representatives of supported companies were much more satisfied with the work of the donor institution than those of refused companies. Representatives of supported companies have given marks between 4 and 5 for all of the above aspects, while refused companies gave marks between 3 and 4 for the same aspects. The following typical opinions can be highlighted:

• Favourable opinions. Companies which had already applied before for support at other donor organisations have found the evaluated tender more favourable than those of other donors. The following reasons were given: this tender was more fair, more simple, it was available for a wider group of eligible firms, it provided more friendly conditions for enterprises, it was more problem oriented and helped survival instead of forced development.

• Unfavourable opinions. Criticisms concerning the donor institution were related to the slow assessment of the applications, to insufficient justification of refusals, and to the strict requirements of project documentation. Some 48 percent of the refused enterprises found either the decision of refusal or one or other elements of the proceedings unjust.

Table 9. What would you do differently today in implementing the project?

Content analysis of the responses given by supported enterprises. N=131 responses.

Three quarters of supported enterprises were satisfied with the way they had completed the project. The rest



pointed out the following elements which might serve as a lesson in case of applying for an other grant:

• Preparation for application must start in an earlier phase of the realisation of the project,

• Employees and contractors must be chosen with greater care and cautiousness.

• Invoicing to the donor organisation must be prepared with greater care.

• Companies should commit themselves to their funded projects for the long term.

• Companies should employ new persons only in justified cases, after due deliberation.

Characteristic individual responses:

• Controllability. „I would organise the working processes in a more transparent way." - „I would pay more attention to the employment of new employees." – “I would make the process quicker and would assign it more into the scope of authority."

• Accounts. I would break down the expenses and the deliverables into smaller units." – „One must pay more attention to book-keeping aspects, receipts and invoices of the project."

• Planning. „More time should have been devoted to the planning of the system."

• Keeping contact. „I would like to keep closer connections with the donor organisation."

Impacts pointing beyond the benefited enterprises. Most of the supported enterprises believed that the influences of the project went well beyond the enterprises themselves and affected the whole of the settlement in terms of newly established working places. Others have pointed out that the project had a positive impact on the environmental characteristics of the settlement. Larger enterprises have frequently mentioned the project had a positive impact on the cultural development of the settlement as a whole.

Sustainability of the results . The results of those projects which have facilitated the introduction of new products or new production processes, the construction of buildings or the acquisition of new devices were sustainable in most of the cases. On the other hand, „softer" project deliverables such as training courses, certification, consultancy and marketing services were thought to be sustainable only in two-thirds of the cases. More than half of those projects where the results have directly improved competitiveness (such as the extension of production, the development of technology or the introduction of quality systems) were thought to be sustainable for more than 5 years. However, projects with employment related results (such as the hiring of new workers) were thought to be sustainable for a shorter period: in two-third of the cases such outcomes were to last only for less than 5 years. Representatives of beneficiary micro firms have considered that the deliverables of their supported projects were less sustainable than what representatives of larger companies thought about their own projects.

Table 10. What would have happened if you had not received the support?

Content analysis of the verbal responses of 143 supported enterprises.

This question is very important for impact assessment reasons because it asks directly about the opinion of the beneficiaries about the counterfactual scenario.

The responses of the beneficiaries demonstrate a significant impact. Only a few respondents stipulated that their company would, without the help of the support, be the same as it is now. Some typical answers:

• Focus on competitiveness . “ There would have been no development then." - “We would have to work in a wet place and receive the clients here."

• Focus on employment. “ We ought to have dismissed two people immediately." “6 people would be without jobs now."



• Time dimension. “Investment would have been delayed 1-2 years." - - “It would have been completed much more slowly if we had relied on our own sources only, or would not have been completed at all." - “The development of the technology could have been done only later, which would have caused serious financial troubles."

• Financial dimension. “We would have been compelled to finance the investment out of credit which would have caused serious losses to the co-operative." - “We would have made losses and therefore we would have excluded ourselves from further subsidy application possibilities." - “The Public Hygiene Authority (KÖJÁL) would have had the unit closed which would have meant the collapse of our enterprise." - “We would have been compelled to apply again for a subsidy; otherwise the activities of the company could not have been started solely on the basis of our own resources."

• It made no difference.“ I would have done the development all the same and everything would be as it is now."

Table 11. What would have happened if you had received the support? A content analysis of the answers of 132 refused enterprises in words

Refused applicants answer to this question mostly in terms of investment and employment. In this sample there was no respondent to say that the firm would look the same even if they had received the support. Some typical individual answers:

• Investment “ I would have purchased the piece of real estate." - “The fence could have been built so that no stranger would be able to enter." - “I could have realised my planned investment and we could have remained competitive." - “We would have shaped a new premise for glazing and could have employed new people, too." - “A new, modern technical development would have been started in our enterprise."

• Labour. “Then, another 10 people would be working here with more up-to-date machinery." - “I would engage two full time workers." - “We would have more up-to date dressing room and showers."

• Organisation. „It would have been easier to handle the introduction of the ISO quality assurance system."

• Financial dimension.“ We are leasing the vehicles and labour machines now but if we had been supported, we ought not to have increased our debts to such an extent." - „We would have survived, at the moment the company is on the verge of bankruptcy.“

• Emotional difference. “ I would have been happy."

Employment related attitudes and recommendations. Most respondents have reckoned that typical Hungarian SMEs hire new employees on the basis of long term strategic considerations, not for short term tactical reasons. An overwhelming majority of respondents has stipulated that the most important obstacles of hiring new employees are related to the high costs of labour (wages, taxes and social contributions) compared to turnovers. Compared to these obstacles, the legal and psychological obstacles are insignificant. However, especially within the group of micro and small firms the avoidance of hiring new employees can be frequently explained by the risk avoiding attitudes of the owners. Several firms have recommended to launch more and more tendentious employment related subsidy programmes. In particular, there was an explicit need for support schemes facilitating equal opportunity employment by SMEs.

Table 12. In which ways could the employment capability of Hungarian enterprises be improved?

Number of respondents mentioning the measure. Respondents could select more than one option.

Supported enterprises Refused enterprises Total

Number of respondent 122 101 322



Number of respondents mentioning the measure. Respondents could select more than one option.

Supported enterprises Refused enterprises Total

enterprises

Ways of improvement and the number of respondents mentioning the measure

Reduction of fees, duties, contributions, tax burdens

88 68 156

Increase the flexibility of the regulation of the labour market

17 13 30

Support of investments 3 14 17

Measures increasing firm competitiveness

9 8 17

Measures to develop the markets, to increase purchasing power

9 7 16

Support of working capital financing

3 9 12

Prevention of informal economy

4 4 8

Administrative simplification

3 2 5

Supporting the employment of people in disadvantageous position (mothers on child care, elderly employees, ethnic minorities)

2 1 3

Reform of professional training

0 2 2

Enforcement of contractual discipline

1 0 1

Number of references together

139 128 267

Factors influencing competitiveness of SMEs . While smaller enterprises have mentioned direct financial burdens in the first place, such as the high and increasing level of costs, bigger companies have stressed the appearance of new competitors as a major threat to competitiveness. Import competition was somewhat less frequently mentioned; this has worried smaller enterprises more than the larger ones. In particular, more than one third of the respondents have stipulated that with Hungary ’s joining the European Community, the competitive position of their company has become worse. On the other hand, nearly one fifth considered their



situation to be better as a consequence of the accession of the country into the EU. One tenth of respondents have experienced both the advantages and the disadvantages of Hungary’s being a community member because of the increased openness of the market, by the changed regulations and legal rules.

Recommendations to increase SMEs competitiveness . Most respondents have stressed that by reducing public financial burdens (taxes, social contributions) competitiveness could be improved. This item is followed by suggestions aimed at the proper choice of the aims and target groups of direct subsidies and have stressed that SMEs should receive equal opportunity in being subsidised. Innovations should be subsidised to a higher extent. A wide range of respondents would have preferred subsidies with immediate tangible results as compared to subsidies supporting the improvement of know-how.

Table 13. In what ways could the competitiveness of Hungarian enterprises be improved?

Content analysis of the responses. Number of enterprises mentioning the measure. Respondents could select more than one option.

Measures improving competitiveness

Supported company Refused company Total number of respondents

Total number of respondents

134 126 260

Reducing taxes, contributions, public burdens

27 25 52

Support of innovations 23 23 46

Market protection, same support for SMEs and for multinational enterprises

33 31 64

Simplification of legal rules

11 5 16

Support of quality policy 11 5 16

Increasing the purchasing power of the population

8 7 15

Marketing support 9 3 12

Consequent enforcement compliance with regulations

3 5 8

Provision of information to SMEs

3 4 7

Weaken the Hungarian currency (Forint)

5 0 5

Supporting export 3 1 4



Content analysis of the responses. Number of enterprises mentioning the measure. Respondents could select more than one option.

Measures improving competitiveness

Supported company Refused company Total number of respondents

Reduction of energy prices 1 2 3

Improvement of paying morals

3 0 3

Developing infrastructure 1 1 2

Conclusions and recommendations. The evaluation stated that the 5-year programme which has offered non-returnable subsidies to small- and medium size enterprises has contributed significantly to the employment abilities and competitiveness of the enterprises within the target group.

Employment related impacts have appeared at the beneficiary enterprises, which have profited directly by the extension of their capacities, by offering better working conditions or gaining more qualified labour force. Moreover, many beneficiaries have obliged themselves contractually to employ additional persons and to guarantee their working places for the medium term. In many cases the new employees were registered unemployed persons or career starters. All this has contributed to the stabilisation of the position of their families as well as to the quality of life in their settlement.

The competitiveness of the benefited enterprises has been improved in three quarters of the cases, partly by way of optimising human resource management and, in certain cases, with the help of subsidising (a) the extension of the product portfolio, (b) the improvement of technology, (c) the quality policy of the enterprise, (d) the purchase of buildings or (e) that of devices. These advantageous impacts go together, in several cases, with such indirect benefits as (a) the modernisation of supply or (b) an improved entrepreneurial culture in a given profession, or in some cases (c) company start-up or (d) the survival of the company.

Most respondents were satisfied with the design of the support project and with the work of the supporting institution, both compared with their own expectations and compared with the practices of other donor organisations. The evaluators have advised to improve (a) the customer service and (b) the project management abilities of the donor organisation. In particular, better information should be provided for applicants. Furthermore, the donor organisation was advised to take proactive steps towards ensuring that the parties on both sides of the subsidy contract - i.e. the supporting and the supported side - are equal. Many applicant firms have devoted a lot of administrative work to prepare the application for support, and still, they were refused. In fact the number of refused companies has surpassed that of the supported ones. In such cases a high priority should be attached to properly justify the rejection decision. The rejecting document should be not only precisely and informatively worded but emphatically, too.

The issue of market distortion and the criterion of cross-financing. Although the evaluated support scheme has increased the competitiveness of hundreds of directly subsidised enterprises, the survey did not point to a market distorting effect of the programme. Nevertheless, the evaluators have made the following recommendation. In case of future subsidy schemes, the process of selecting beneficiaries from applicant firms should be supplemented with a specific examination of the business model of the applicant firms, whereby the evaluators should assess separately and comparatively the various business lines of the applicant firms. If the suspicion of cross financing among various business lines of the applicant company arises, and if it can be assumed that the applicant intends to finance some loss making production or service with the help of the subsidy, then the application will have to be refused.

Monitoring of the supported projects . During the lifetime of the supported projects, the controllers of the donor organisation have visited the beneficiary enterprises several times. Nevertheless, the evaluators have made the following recommendation. In future subsidy schemes to be implemented by the donor, it would be useful to involve independent advisors who would, at certain intervals, visit at least a smaller sample of the supported firms, and would write monitoring reports on the basis of their experiences on the spot. These reports would not be used in the first place for monitoring individual projects, instead, they would serve the fine tuning of the



programme.

Dissemination of the lessons learnt. Regarding the follow-up of the support programme, the evaluators have made the following recommendation. It would be useful to examine the afterlife of the subsidy scheme in the framework of a conference meeting. The conference could be devoted to good practices illustrating how the competitiveness of small enterprises can be increased parallel to the improvement of their employment capability. Proper attention should be paid to the relevant risks as well. The results should be published.

Further it was recommended that the experiences of analogous SME support schemes implemented in EU Member States should be used as models for improving direct SME supporting activity running in Hungary , both in the work of governmental and non-governmental donors. This aim can also be served by well targeted, designed and widely published pilot projects and by the implementation of research and evaluation activities.

2.3. Case Study “SME Innovation". Assessment of a subsidy scheme facilitating SME innovations2.3.1. Policy context: the relationship between innovation policy and SME development

Economic theory has highlighted innovation as a key aspect of entrepreneurship. 1818 However, the contribution of small firms to innovation is a debated issue. True, there is a minority of highly innovative SMEs. In particular, many SMEs are effective in marketing innovative products, e.g. information technology hardware, others rely heavily on innovative products such as mobile telephone. However, most SMEs operate in market niches that have not seen any technological change fro decades. Some researchers argue that smaller firms have less commitment to existing practices and products than larger companies, and therefore there is a bigger likelihood of small firms introducing innovations than larger firms. On the other hand there is considerable empirical evidence for the thesis that SMEs make fewer innovations - both in terms of technologically new products and processes - than large enterprises. If only radical innovations are considered, the gap between small and big business is even wider. 1919

Strategy documents of regional and national innovation policies in the European Community mention frequently the aim of promoting knowledge transfer, the diffusion of up-to-date technologies, and furthermore, with a growing emphasis, the intensification of the innovation activities of small- and medium size enterprises. Although in most public administrations innovation policies, regional polices and SME development policies have different traditions and are implemented by different organisations, these policy fields have the common aim of establishing institutional networks which can promote the spreading of up-to-date technological and organisational innovations among small firms.

In Hungary the respective policies of enterprise development, regional development and sectoral development belong to the responsibilities of various organisations of the existing governmental and non-profit structures, and the successful coordination of all this is one of the key issues of innovation policy. [bib_13] Immediately following the political changes of 1989, so-called entrepreneurial incubators [bib_14] were established in several regions of the country with the help of national and European financial support. A decade later, following international examples [bib_15], another wave of support schemes has followed which has facilitated the establishment and functioning of regional clusters [bib_16]. The membership of these loose and often project-oriented networks was recruited from large and smaller enterprises, but local governments, educational and counselling institutions, and interest representations also participated as members.

The need for an improved European innovation policy has come to the forefront during the 1990s. It was observed that although European economies produced outstanding results in the fields of research and development, but as regards utilisation and competitiveness they were lagging behind the North American countries and in many respects behind Japan, as well. 2020 In 2000 the EC announced the so-called Lisbon Programme that stressed the ambitious aim of making the EU, by 2010, the most dynamic and most competitive knowledge based economy in the world. The Programme stated that European sustainable development should rely on an increasing number of good quality working places as well as on strong social cohesion. The Lisbon Strategy explicitly referred to the importance of increasing the innovativeness of SMEs by stating that its aim is

1818[Schumpeter 1961]

1919Cited and discussed in [Smallbone-Welter 2009] page 12.

2020[Török –Papanek 2004]



„to create a friendly environment for start-up companies and also for the developing innovative enterprises, especially for small and medium size enterprises among them." The strategy called for establishing a knowledge based economy by applying new methods and introducing new media in order to create, spread and utilise knowledge.

Impact assessments, evaluations with innovation-political content. Following the initiation of the Lisbon Programme , a wide range of innovation programmes were launched for supporting innovative SMEs by way of non-returnable subsidies, preferential credits, export credits, credit guarantee programs, and through the establishment of „business angel" networks. These programs are regularly evaluated. Also, the EU has launched several policy-level impact assessments with the aim of revealing the effects of innovation policy, formulating the challenges that individual countries to face, enhancing the willingness of small- and medium size enterprises to innovate.

Investment efficiency model. An impact analysis of an innovation support measure may be based on the hypothesis that the projects supporting SME innovation can be regarded as investments of which the profitability can be analysed analogously to the profitability of any other physical investment, e.g. a motorway or a piece of machinery. Evaluators using this approach start by making sure that (a) the supported SME activity has been implemented, (b) the support has been used for the purpose it was asked for, that is, for the realisation of the project. In case of positive answers to the above questions, evaluators using the investment efficiency model perform traditional cost-benefit calculations 2121.

Questionnaire based surveys. Business survey s are other preferred methods of assessing the impact of measures encouraging innovation activities. These surveys are implemented in order to collect the responses of thousands of companies about (a) innovation related company attitudes, (b) activities and (c) their results. The underlying causal mechanisms are complex, because an increase of firm competitiveness may be the result of other causes as well, such as the appearance of a new strong competitor or the national or regional business cycle. Therefore, evaluators relying on business surveys often use control groups in order to reveal the net effect of innovation facilitating programmes, in order to control for the effects of other factors. By aggregating the responses, various country-level indicators of innovation activity can be calculated.

Table 3.17. Box 5.

European Innovation Scoreboard

Let us illustrate the questionnaire based approach with an all-European innovation survey of enterprises. The so-called European Innovation Scoreboard (EIS) is a large scale enterprise survey project sponsored by the EU covering 35 countries. The questionnaire focuses on specific innovative activities of the responding companies and addresses the following issues:

• purpose of innovation (e.g.: widening product portfolio, increasing market share, extension of capacity, reduction of costs, applying standards, etc.)

• type of innovation (product- or process innovation)

• status of innovation ( completed, ongoing, failed)

• the extent of the necessary input and financial resources (own resources, cooperation, external partner)

• obstacles of the innovation.

The resulting Community Innovation System (CIS) is a set of indicators about (a) the human resources of innovation (5 indices) (b) knowledge production (4 indices) (c) knowledge transfer (3 indices) and about (d) financing of innovation, results and markets (7 indices). Due to the large sample size, a breakdown of these indices is possible according to the enterprise size classes and countries. Additional data for the Scoreboard is provided by Eurostat, the statistical office of the EU and by national statistical institutions. Data originating

2121 See the sub-chapter „Methodological aspects of RIA" in this study.



European Innovation Scoreboard

from the above survey constitute the basis of the calculation of the Summary Innovation Index (SII). This influential composite index is based on country level characteristics influencing innovation activities such as (a) the protection of intellectual property, (b) entrepreneurial research and development expenditures, (c) human resources of innovation, (d) selected outstanding innovative developments and other characteristics of innovation.

For Hungary , the Summary Innovation Index in 2005 was within the range of the respective indicators of other countries having recently joined the EU. Other business surveys 2222 focusing on innovation activities of Hungarian enterprises have registered a wide range of research and development activities and results in enterprises employing less than 50 persons. However, the proportion of enterprises with outstanding developments is far more significant within the size class of companies employing more than 50 persons than in small and micro enterprises.

2.3.2. Purpose and method of the questionnaire based evaluation

In Hungary the government authorities responsible for innovation policy 2323 have implemented a wide range of innovation related subsidy schemes which have been afterwards evaluated. Small enterprises were found among the beneficiaries of both the successful and the less successful grants. The next case study describes one of the above evaluations which focus entirely on the innovative activities of SMEs.

In 2000 the Hungarian State Technical Development Committee (OMFB) entrusted a group of independent experts with the task of evaluating the impact of various subsidy schemes that were offered by the Committee as a donor. Three subsidy schemes of the Committee, called „Applied R+D tender", „Competitive products tender", and „Regional innovation tender" with different eligibility and management rules, have allocated support to a wide range of SMEs. The aim was to report, how through these projects the donor organisation has supported SME sector and to evaluate the contribution of the Committee to Hungarian SME development and innovation policy.

The evaluation [bib_17] report consisted of two major parts:

• The institutional overview was based (a) on the analysis of official documents (e.g. the government’s SME- and technological-political documents, the tender documents issued by the donor organisation) and (b) on interviews with the experts of the Committee.

• The small sample business survey relied on questionnaires returned by beneficiaries.

The institutional overview started with the identification of the examined subsidy schemes. The evaluation report characterises the subsidy schemes by describing (a) their dates of implementation (b) the declared aims (c) the administrative procedures of decision making, (d) the financial allocations and disbursements related to these schemes, and (e) the number and other characteristics of the benefited enterprises. The document explains the context of these subsidy schemes within the enterprise supporting system and positions the development of these schemes within the donor organisation.

Following these fact finding exercises, the evaluation switches to its main task, i.e. to give the opinion of the evaluators about these schemes. From the point of view of methodology this means, that a general system of evaluation criteria had to be adapted, applied for the subsidy schemes both (a) in document research as well as (b) during the interviews made. The following list of questions explains how the researchers have transformed the five abstract criteria of evaluation into specific questions concerning the examined subsidy schemes

Relevance :

• Are the evaluated subsidy schemes properly adjusted to the relevant governmental tasks, to the system of goals and methods, as well as institutions, to the corresponding policy fields (SME policy, technology policy), to the innovation activities of small- and medium size enterprises?

2222 [Peredy 2007]

2323 During the 1990s: the State Technical Development Committee (OMFB). Afterwards: its legal successors such a sin 2009 the National Office for Research and Technology (NKTH)



• Do the declared goals of the schemes correspond to those principles and internationally tried and tested practices which have been declared by relevant Hungarian and international innovation support organisations?

• Have the target groups of the tenders been chosen in accordance with the declared purposes? (e.g.: innovative small enterprises, enterprises following a modern market niche strategy, new technological enterprises)

• Have these schemes been evaluated earlier? Have the experiences of the earlier evaluations been taken into consideration when shaping, designing the present schemes of support? Would it be possible to use the indicators of previous evaluations with comparative purposes?

Efficiency :

• Is the invitation to these tenders intelligible, transparent and client- oriented enough?

• Are the selection aspects of winning companies announced in advance and are they clear enough?

• Do the documents of individual subsidy schemes consequently and intelligibly communicate the goals to be supported?

• Does the performance of the funded projects, in each of their phases (announcement, admission of the applications, preparation, selection, disbursement, control), follow the procedures of the donor organisation and the rules of the particular subsidy schemes?

Effectiveness :

• Were the supported projects effective, did they produce the expected positive results? Has a system collecting information and feedback been established?

• Was the announcement of the tenders and the selection procedure successful in finding those enterprises that belong to the declared target groups of the subsidy schemes? Was the support distributed among many firms, or rather, concentrated to a few beneficiary enterprises? Which approach has served best the original aims of the relevant policies?

• Did the choice of the financial model of the subsidy well serve the original aims of the relevant policies? Were the options of (a) non-returnable subsidies, (b) preferential credits and (c) public procurements applied in a segmented, target group specific way?

• Did the subsidy schemes motivate the supported enterprises to take full responsibility for the projects and to assume the respective risks?

Impacts benefiting others than the immediate beneficiaries that extend beyond the actual projects:

• Did the positive impacts of the project extend beyond the benefited enterprises and reach a wider range of stakeholders, such as their clients, subcontractors, co-workers, and the settlements in which these enterprises operate as well?

• What were the proportions between supporting (a) “soft deliverables", such as developing company knowledge base, knowledge transfer and organisational development on the one hand, and (b) “hard deliverables" such as physical investments with the purpose of innovations, on the other hand? Were these preferences serving well the underlying aims of the respective policies?

Sustainability :

• Are the supported enterprises able to maintain the results of the projects after the phasing out of financial subsidies?

• Has the donor organisation sufficient administrative capacity to trace the evolution of the supported projects afterwards?

• What legal and institutional guarantees have been built in to make sure that those enterprises which break their contractual obligations in the medium term or in the long run will have to pay back the support?



Small sample business survey. It is a basic principle of international donor organisations such as the World Bank or the EU that for evaluating support schemes, the beneficiaries must be always asked about their opinions and recommendations. The quantitative part of the evaluation of this case study consisted of an analysis of a questionnaire based survey of small sample of supported enterprises which were ready to respond to the disseminated questionnaires.

The purpose of the survey was to obtain responses to the following research questions:

• How do supported companies interpret the concept of innovation?

• What is the attitude of the enterprises towards research and development activities?

• Which are the characteristic features of the innovative activities in small enterprises?

• What results have the supports achieved?

• What are the entrepreneurial attitudes towards innovation oriented subsidies?

• What kind of opinions have been shaped about the supporting organisation in the group of supported enterprises?

Sample and sampling considerations. The enterprises providing the answers had won the support of one of the 3 evaluated OMFB projects during the 5 years preceding the tender research. Surveys based on postal dissemination of questionnaires and voluntary responses of companies are preferred tools of firms conducting market research and business climate research. However, this model of data collection allows the estimation of tendencies only with some distortions. The main source of bias is the fact that certain groups of enterprises may be overrepresented among the respondents: those motivated by: (a) gratitude for the support (b) hope for a next support (c) public relations considerations (d) an urge to complain about the efficiency of the proceedings of the support tender or about some supposed injustice.

No control group. The impact statements of business surveys are justified much stronger if they are based on a comparison of beneficiaries and not supported companies. However, in this particular case it was not feasible to ask a control group of not supported companies.

The willingness of enterprises to participate was relatively low: although the donor organisation had distributed 300 questionnaires among the supported enterprises, only 57 responses have arrived. Due to this small sample size, this could not be a representative survey, but a sufficient number of representatives of the following company strata were included:

• Companies from various size categories ( from small ones up to bigger firms)

• Companies of various sectors (agriculture, industry, and services, including knowledge based companies)

• Companies owned by Hungarians and by foreigners

• Companies of various legal forms (Limited Liability Company, partnership, Shareholder Company, sole proprietorship or other forms).

Table 14. The composition of responding enterprises by size classes

Size class Number of enterprises which replied

The average number of the hierarchy steps of the firms belonging to the given size category

Less than 10 employees 29 1, 6

Between 11 and 50 employees 18 3, 0

Between 51 and 250 employees 5 3, 5



Size class Number of enterprises which replied

The average number of the hierarchy steps of the firms belonging to the given size category

251 to 500 employees 0 -

Over 500 employees 5 4, 8

Total 57 2, 5

Table 15. The composition of responding enterprises by legal form

Legal form Number of enterprises

Sole proprietorship 2

Deposit partnership 4

Limited liability company 41

Shareholder company 8

State owned enterprise 0

General partnership 1

Co-operative 1

Total 57

Table 16. The composition of responding enterprises by sector of their main activity

Classification of replying enterprises Number of enterprises

Agriculture and catering industry 11

Chemical industry 7

Metal industry 6

Production of machinery 9

Electrotechnic and precision engineering 9

Information technology 7

Telecommunications 2

Light industry 3



Classification of replying enterprises Number of enterprises

Construction industry 1

Trade 1

Protection of the environment 1

Total 57

Table 17. Does the company have a foreign owner or co-owner?

Proportion of foreign ownership Number of replying enterprises

There is no foreign co-owner 46

There is, with 15-50 % ownership 7

There is, with 100 % ownership 3

No answer 1

Total 57

The questionnaire 2424 consisted of 74 questions; half of them closed questions, the other half being open questions to which the respondents have answered in writing, with their own words. Such a wide range of open questions has in some respects counterbalanced the small sample size of the survey. These responses were evaluated with the method of content analysis. Due to the high number of the open questions it can be stated that returned responses could be regarded as substitutes or proxies of in-depth interviews. For the above reasons, while the results of the questionnaire based survey had to be handled critically, yet, its data has contributed to the understanding of the impact mechanism of the examined intervention.

2.3.3. Results of the questionnaire-based survey [bib_18]

The professional content of the supported projects. The evaluated support projects have reached a wide range of companies in the industrial, agricultural and service sectors. The professional portfolio of the supported projects has included (a) registration of patents, (b) protection of manufacturing rights, (c) selling of licences abroad, (d) the foundation of a joint venture for utilising a license, (e) to issuing a comprehensive technical publication, (f) organising professional workshops.

The novelty features of the supported projects were ranging from traditional technological developments to high technology investments. About one third of the supported projects have been be characterised by the respondents as high technology development. No correlation could be found between the size of the enterprise and its orientation towards high technology: in every size category the number of the projects representing high technology was about the same. About half of the supported projects were regarded by the beneficiaries as innovations, while other beneficiaries – about one-third of the respondents - have characterised their supported projects as lacking any technological novelty, or technical new feature or innovation.

Volume of financial support. The amount awarded to the beneficiary enterprises has varied widely from small to large scale projects. While companies employing less than 10 persons have received an average of 12 million HUF for their projects (48.000 EUR), enterprises working with more than 10 persons received on average twice as much.

Company growth. The majority of respondent firms were growth oriented: some two-thirds of the respondents

2424For the draft of the questionnaire see the Appendix



have stated that their company had a growth oriented strategy. Innovation was interpreted by the respondents as one of the driving forces of the company’s development. Surprisingly, smaller enterprises planned expansion more frequently than bigger ones, meaning that if the enterprise was bigger, or would have the possibility to grow bigger in the future; this would be very useful for their innovation activities, mainly due to financial and personal reasons. Typical motives for company expansion: "We must improve the personal conditions of our innovative and developing activities." „If we do not extend, we shall be crowded out of the market." The reasoning of those who do not intend to grow: „Our market is restricted and our capital is little." „A larger enterprise would be difficult to control, because, for example, it would not fit into the framework of a family venture."

Innovation and quality. Typical beneficiary enterprises considered themselves to be innovative, moreover, some two thirds of respondents have characterised their firms as “very innovative". Innovative activities have been associated with a high prestige, and many responses have indicated that in launching certain developments the moral factor was more decisive than the financial risk. The majority of respondents have stated that in their companies the prestige and importance of innovative knowledge based work was very high, and so was the moral appreciation of those people who were performing such jobs. However, the financial rewards of employees engaged in development processes were very varied. Many respondents stated that innovative behaviour in a small enterprise depends on conditions like a good leader, a well qualified expert, intention and ambition, or a strong competition.

• Most companies have committed themselves strongly to product and service quality: (a) by continuously innovating their products and services, (b) by introducing quality assurance systems, (c) by establishing research and development departments and (c) by performing technical consultancy work for other firms.

• On the other hand, firms which have demonstrated, according to their own opinion, only modest results in innovation, have explained this by giving the following reasons: (a) their resources were very limited, (b) they could spend on developments only up to a certain percent of their income, or (c) innovations will be introduced only under the pressure of market forces.

Exports. About half of the responding enterprises have pursued export activity at the time of the survey. More than three quarters of the respondents stated that the results of their developments were competitive in the markets of developed countries as well. It is typical of the dynamism and ambition of the vast majority of the firms, that - independently of the fact whether they pursue export activity or not at the moment – they wish to increase their activities on the international markets. As far as export markets are concerned, the EU occupies the first place. In this sample there was no connection between the size of the firm and the extent of its export activity: in each size category about half of the companies were exporting. According to more than three quarters of the respondent firms the results of the developments are competitive on the markets of the developed countries. More than half of the exporting companies have stated that their export activities have increased after the subsidy, but this can rarely be connected with the direct effect of the project.

Effectiveness of the supported projects . Most respondent firms have reached the professional targets of their respective projects, some of them fully, others “up to a satisfactory level". Admittedly, there were unsuccessful or failed projects as well, but their number was insignificant. In the few unsuccessful cases some typical causes of professional failure were the following: „We could not find a suitable co-operation partner (a reference factory, a trading company)", „ We used to have a good partner but that one has gone bankrupt." „The development project was technically successful, but it takes a long time to start production."

Impacts on production and production factors. The turnover of half of the benefited enterprises has increased during the course of their respective projects, in many cases significantly, and the number of those enterprises whose income decreased was insignificant. The number of full time employees has increased in case of one third of the benefited enterprises during the course of the projects, while the number of those enterprises where this number decreased was insignificant. Nearly half of the enterprises stated that the project has directly influenced the number of employees: in most cases – directly or indirectly – it has increased the size of the staff. The projects have significantly improved the networking capabilities of the beneficiary enterprises: about three quarters of the supported companies established relationships with other partner companies and/or institutions during the project. However, only a few respondents have reported about organisational innovations generated by the project.

Spill-over effects . The benefits of the projects appear at the beneficiary firms in the first place: most supported enterprises convert the acquired new know-how to profits by putting a renewed product or service on the market as a result of the development. However, some positive effects “spill over" and reach their clients and buyers



and other elements of the society such as the settlement in which the firm operates.

Efficiency of time management and financial management of the projects . Most beneficiaries have kept the costs of their respective projects within the limits defined in their applications. There were only 4 applicants in the sample who reported that they could not keep to the financial frame of the project. Less than one tenth of the enterprises reported implementation delays compared to the original time schedule. Some 9 enterprises could not keep to the planned time schedule of the expenses. Enterprises working with less than 10 persons have reported on delays more frequently than their bigger counterparts. Some typical explanations of overspending money and not keeping the financial schedule were as follows: „We have received less support than what we have asked for." „The project was only slightly delayed." "We had personal reasons, e.g. childbirth of an employee" „Our external co-operation partner has delayed to deliver." "The donor organisation was late to sign the co-operation contract." „We had to start another subsidised project simultaneously"

Project management culture in bigger companies on the whole was more mature than at smaller firms. Moreover, larger enterprises had more experience in the application procedure; bigger companies have previously won similar subsidies more frequently than smaller ones.

Counterfactuals. According to the self-evaluation of the beneficiaries, without the received support about one third of these projects would have been completed only with delay. Another third of the projects would have been completed only in part: with poorer professional content or in smaller volume or extent.

Figure 8. What would have happened if the donor had not supported the project? Based on a survey of N=57 Hungarian SMEs, beneficiaries of innovation support

Efficiency : client relations and administrative work of the donor organisation . The beneficiaries have reported about a fair relationship between the donor organisation and the applicants: they expressed the opinion that the programmes were satisfactorily planned and organised. The respondents were asked to judge the performance of the donor organisation on an “agree-disagree" scale consisting of five grades. Most respondents have agreed or “fully agreed" with the following statements: (a) “The donor organisation has treated the applicants as equal partners". (b) “There was a fair system of selection." (c) There was a clear system of conditions put down." (d) “The tender motivated us to think about our projects more thoroughly" and (e) “The tender has contributed to the improvement of quality".

Firms implementing high tech developments were demanding more attention from the donor organisation and were more critical and self-conscious of the whole administrative procedure than those firms which had received support for traditional technical solutions.

Recommendations of the beneficiaries to the evaluated subsidy schemes. Not surprisingly, the respondents have unanimously supported the type of subsidy schemes in which they had received support. They think that such subsidy schemes have their legitimate place in the market economy. However, some four-fifths of the respondents would like to see some improvements to it. A significant proportion of respondents have found the financial volume of the subsidy too small.



Recommendations of the beneficiaries about the enterprise support system . Respondents have given their opinions about how to improve the general support system of SMEs in Hungary . In this respect, most respondents have tackled the financial model of the support: (a) some respondents would increase the proportion of non-returnable subsidies. (b) Others would like to increase the proportion of subsidies placed out as risk capital. (c) Again others would increase the proportion of soft loans, with the backing of a credit guarantee. A different set of recommendations was devoted to the administrative procedure of the support schemes: respondents were asking for more mature, simpler, retrospectively controlled multi-step supports and tenders. There were recommendations favouring the segmentation of subsidies according to company size categories.

2.4. Policy level evaluationsSME development policies. In case of policy evaluation the subject of the assessment is an entire policy field including its aims, strategy and its instruments. In particular, SME development policies can be characterised by the range and volume of services supported by them, by the portfolio of preferential credits and subsidies provided for small enterprises and by all accompanying legislative, political and institution development measures.

Template of policy evaluation reports. An important methodological decision is the choice of the structure of policy evaluations, i.e. the definition of its title and of its subtitles. Choosing the template of the report is an important decision because the evaluators demonstrate by this act their opinion about what the real building blocks of the respective policy are. Frequently, the structure of evaluation reports follows a template that has been determined by some guideline issued by the donor agency. Evaluation reports of policies can be structured according to the following principles.

• By policy aims. If the evaluators want to structure their report according to the declared aims of the programme, the report takes one-by-one the aims of the particular SME development programme as they were previously defined in a key document. The attainment level of these aims may be analysed by responding to the following questions. (a) What are the subordinated aims, objectives, preferences into which the main aims can be broken down? (b) What are the instruments that have been utilised for realising these aims and subordinated aims? (c) What are the circumstances and conditions that facilitate and what are the obstacles of reaching the given aims? (d) What results have been obtained in the given period of time? (e) What are the tasks to be solved next.

• By policy instruments. In this case the report evaluates one by one the following policy tools: projects, grant schemes, credit schemes, regulations, measures of institutional development, etc. Frequently the evaluators want to structure the report according to the tools of the examined policy. In this case different chapters of the report are devoted to different implementing institutions, to different steps of legislation or to various co-operation forms with entrepreneurial interest representations, to various support schemes etc. 2525 Compared to the other approaches, this approach is more comfortable for the evaluators, because most source documents are devoted to the implementation of some subsidy scheme or grant scheme or credit scheme. Similarly, it is easy to find interview subjects who are knowledgeable in the provision and administration of some of the above mentioned schemes.

• By evaluation criteria (such as relevance, efficiency, etc.). In this case the policy is treated as if it was a single project and it is being evaluated with the methodology described in the previous chapter.

• Combined structure. It is also possible to structure the report according to a combination of the above possibilities.

The following boxes demonstrate the application of some of the above mentioned template alternatives.

Example for policy evaluation structured by policy aims:

Table 3.22. Box 6.

2525For example: [UNIDO 1996]



Evaluation of the implementation of the decisions of the European Charter of Small Enterprises in the West Balkan states

In 2003 Albania, Bosnia-Herzegovina, Croatia, Macedonia, Serbia and the UN governed province of Kosovo have signed an agreement with the EU aimed at the development of small enterprises. The content of the agreement was to realise the decisions as put down in a key document called „Charter for Small Enterprises" a. This document has defined the following goals for developing the SMEs of the region:

1. Education and training of entrepreneurs

2. Cheaper and simpler establishment of companies

3. Better legislation and regulation

4. Developing the labour force possessing the necessary professional knowledge

5. Electronic taxation and electronic company registration

6. Better utilisation of the advantages of the Common Market

7. Improving the fiscal environment and accessibility to finance for SMEs

8. Improvement of technological capabilities of SMEs

9. Spreading o successful e-business models and improving the quality of enterprise support

10. Stronger and more efficient representation of entrepreneurial interests

In 2006 a comprehensive evaluation was applied using an up-to-date methodology in order to assess the small enterprise development policies of these countries and in particular their results in achieving the above aims during the years 2003-2006 in the countries of the West Balkan area. b

The process of evaluation was organised as follows. In the first wave the West Balkan states have prepared self evaluating national reports by using a template structured by the above 10 aims. These reports were forwarded to independent consultants who have assessed these reports and attached a preliminary numerical rating to the attainment level of each of the 10 aims in case of each participating country. Finally, the governments of these countries were given an opportunity to comment upon the results of the preliminary rating. The ratings were finalised by experts of the European Community and other international organisations, those who had ordered the making of the report.a [EC 2000]b [EC-OECD 2007]

The structure of the evaluation report often simply follows the structure of the key document of the programme evaluated. This is frequently the case if the evaluation precedes the implementation of the programme, i.e. in case of ex ante evaluations. Here the overriding aim of the evaluators is to demonstrate the expected results and impacts of the evaluated SME development measures.

Table 3.23. Box 7.

Evaluation of the SME support measures of the Economic Development Operative Programme for Hungary of 2006

The Economic Development Operational Programme a is one of the executive documents of the New Hungary Development Plan b. This key document is a plan to spend a subsidy portfolio worth 2 and a half billion Euros of supporting projects. The document defines, for the period between 2007 and 2013, the structure of the European Community’s co-financing supports which can be used for enhancing the competitiveness of the Hungarian economy. The document defines a series of development priorities. To each of these priorities the document devotes a specific chapter and allocates a specific sum to it.



Evaluation of the SME support measures of the Economic Development Operative Programme for Hungary of 2006

A particular development priority serves the development of SMEs in an explicit way. The name of this priority is: „Priority No. 2: A complex development of enterprises, with special respect to SMEs".

The Economic Development Operational Programme document underwent, before being accepted, a so-called „ex ante evaluation, in order to obtain answers to the following questions: c

• Is the aid delivery strategy defined in the Programme able to respond to the major challenges facing the regions of Hungary and the sectors of the economy?

• Is the document in compliance with the relevant policies of the European Union , the Hungarian government and the regions of Hungary ?

• Are the goals and priorities defined in the Programme clear and attainable?

• What will be the effects of achieving these aims in quantitative terms?

• Does the document contain adequate indicators?

• Are appropriate executive orders, institutions attached to the Programme ?

In order to respond to the above questions, the evaluators have assessed the programme documents and made interviews with officials of the aid delivery organisations as well as with experts working in the governmental apparatus.

The text of the evaluation report exactly follows the structure of the evaluated document, in other words its chapters tackle the chapters of the Economic Development Operational Programme one by one.

In case of „Priority No. 2: A complex development of enterprises, with special respect to SMEs", both the Operational Programme Document and the Ex Ante Evaluation document analyse the following measures:

• Supporting technological modernisation of enterprises

• Creating working places in disadvantaged areas by way of technological modernisation

• Developing the organisation of companies by spreading up-to-date management methods

• Introducing quality systems and supporting the application of standards

• Introducing e-commerce and other electronic services.

The document analyses the above fields of intervention against the following criteria:

• Are the target groups of the planned support and the professional content defined concretely enough in the document?

• Will the support measure bring significant effects in terms of development policy, employment, the attraction of capital, and innovation?

• Is the support measure scheme tailored to the needs of micro-, small and medium size companies?

• Is the support scheme designed to a large quantity of smaller projects, or rather a fewer number of more costly and more complex projects? Which option is better for the promotion of the effectiveness of the support?

• What conclusions can be drawn from the operation of analogous supporting systems?a [GOP 2006]b [UMFT 2007]c [GOP EX ANTE 2007]



3. Revealing the effects of factors that are exogenous for SME policy3.1. Explaining the business climateThe position of small enterprises is influenced by a wide range of factors, among which SME policy is just one of the many. Business climate as a rule can be explained by a wide range of factors, some of which are beyond the reach of public policies. Such factors are local and natural endowments, social capital, historic events such as wars and other international macroeconomic tendencies that contribute to entry, growth or decline of SMEs in a country or region.

From attribution to contribution. While in case of policy impact assessment the task of the evaluator is to attribute the changes that were experienced to certain measures, interventions, in cases where a wide range of endowment factors are involved, the task of the analyst is rather to determine, to what extent certain factors have contributed to the existing business climate.

Yearbooks analysing “the state of small business" in a given year in a particular country have the explicit or implicit aim of revealing these factors. 2626 Research designs aiming to discover “success factors" or “barriers to growth" of individual small businesses or SME development in general have been relying on the logic of impact assessment.

In all of the above cases the typical assessed factors are not measures of public policies but factors that are typically beyond the reach of politicians. Typical such factors on business development are as follows:

• the business cycle (e.g. inflation or economic depression),

• events of historical or political importance (e.g. privatisation, nationalisation, a war or a deep political change),

• local endowments such as the attractiveness of the settlement for tourists

• the building of major pieces of infrastructure (e.g. a motorway or a harbour or a full scale settlement development activity).

• natural catastrophes

• natural endowments such as raw materials.

In other cases small business development is strongly dependent on the evolution of other sectors of the economy:

• investments of multinational companies into large car assembly plants or shipbuilding factories may become the driving force for the development of hundreds of smaller local subcontractors

• the boom of the tourist industry in a region may offer new market niches for thousands of catering, transport, travel agency and other services.

The above mentioned types of impact mechanisms influencing the business climate have been analysed by a wide portfolio of methods ranging from purely quantitative methods, e.g. by business surveys, by company demography analyses or by analysing databases of balance sheet data [bib_19].

These approaches have been frequently complemented by qualitative methods such as ethnographic analyses and case studies 2727. These instruments have frequently been applied for finding credible explanations to the rise or decline of small businesses in certain settlements or regions, or to reveal specific problems of certain types of firms, e.g. family owned companies. In these cases researchers make interviews with local businessmen whose companies may demonstrate a typical pattern of business development or with persons having a good overview about a wider group of small enterprises such as bankers or experts of interest representations.

2626[GKM – NFGM 1996-2007]

2727[Hugo – Garnsey 2004]



The best research design to explain SME development combine both quantitative and qualitative approaches and information sources. While the usage of business surveys and company demography guarantees the reliability of the results, the qualitative approach is needed to give rich detail, to interpret and to offering deeper understanding of the phenomena.

3.2. Comprehensive business surveysComprehensive business surveys are questionnaire based enterprise surveys that rely on a representative sample of SMEs of a region with a suitably large sample size. The major difference between the sampling strategy of impact assessments and comprehensive business surveys is that the in the former case the sample is extended only to the beneficiaries of some measures and a suitable control group.

Business surveys are designed to offer statistical inference to the enterprise population of a whole region, sector, country, or that of the whole of the European Union . These are costly procedures with the purpose of offering a description of the SME sector in a country or region, mapping the entrepreneurial climate and attributing its success or failure to some factors.

Small business surveys are crucially important information sources for analysing and modelling business cycles. These data collection and analysis activities are statistical services conducted either in regular intervals (e.g. monthly or quarterly) providing time series of basic important indicators about the SME sector of the covered geographical area, or alternatively, ad hoc surveys asking about the impacts of some outstandingly important event (e.g. the introduction of a new currency or a new technology). National surveys are typically conducted by some country-level chamber of commerce, business confederation or economic research institute. International surveys are usually conducted by a consortium of market research organisations, each of these organisations conducting the survey in one particular country.

The range of respondents may consist of members of various chambers and professional associations, suitably complemented by companies appearing in business listings. Business survey s typically differentiate between respondents according to the size of the company, and communicate their numerical findings aggregated to the level of small, medium sized and large businesses. Data may be collected by personal interviews, by telephone, by post or through the Internet. The incoming questionnaires are weighted according to some previously existing frame according to company size, economic sector , geographical area and possibly some other variable , based on tables of official statistical services.

The primary results of business surveys are indicators. The range of indicators may cover the facts for the previous quarter and the expectations for the next quarter. These indicators, as a rule, provide fresh and relevant information about the following issues:

• Turnover, growth, profits, investments, prices, employment and other parameters of business in a given quarter of the year, or month, 2828

• Economic expectations of businesses in terms of orders, costs, inflation and other parameters,

• The spreading of certain new management techniques, innovative procedures and services among companies, e.g. e-business 2929,

• The level of satisfaction of various business segments with SME development policy in general or in particular with the administrative environment.

Business survey s have been used for many decades by governmental institutions, chambers or economic research institutions and have established their methodological traditions in terms of (a) sampling (b) the character and the structure of the questionnaires and (c) the system of the indicators produced by their reports. Surveys covering the EU rely routinely on 5.000 to 10.000 responses. Due to the large number of the sample elements, reliable comparisons of various countries, sectors and company size categories are feasible. 3030 The long series of surveys that have been implemented by the IFO Economic Research Institute of Germany on business climate is an example for such ongoing projects [bib_20].

2828An example of this is the tracing of the monthly business trend by the Canadian Bureau of Statistics in the „The Monthly Wholesale and Retail Trade Survey", Source: [Bérard 2003]2929[EC 2005c]

3030See, for example, [EC 2005c] , i.e. the e-Business W@tch, an annual survey supported by the EC.



Researchers designing business surveys generally prefer the use of random sampling. However, in many cases of telephone surveys, quota sampling is applied, a procedure which does not necessarily follow the rules of random sampling. In this case the internal proportions of the sample are defined according to country, size class of the enterprises, their legal form, the sector they belong to, etc. and various lists companies are used to select the members of the sample.

Researchers of the SME sector often must compare the population of small companies and that of the bigger firms. For this purpose, besides asking a representative sample of SMEs, a sub-sample of bigger companies is also selected which serves as a control group. The resulting stratified sample serves as the basis of calculating the necessary comparative indicators.

3.3. Business demography analysesBusiness demography analyses reveal quantitative trends of business development by using complete registers of companies. Typical outcomes of such analyses are (a) the number of functioning companies in a given year, (b) the number of company births (c) the number of company “deaths", i.e. bankruptcies and liquidations and (d) the “life expectancy" of companies.

Empirical findings 3131 have shown that enterprises are more likely to survive if they are sufficiently large; have been already running for some time; are operated by an experienced entrepreneur, have a sufficient initial capitalization; and are formed in a good business climate. Case studies using regression analyses have shown that higher growth rates are associated with younger firms, better educated entrepreneurs, and with firms being located in regions with low unemployment.

Company demography data can be applied in a wide circle, e.g. in public administration as well as in the course of basic research, market research and the research of economic trends and developments. Such data can be used for multiple purposes:

• Official statistical publications. In most countries a subset of business demography indicators is available in official statistical publications. Such publications routinely contain the number of companies, offering breakdowns for regions, sectors of the economy, size category or legal form of company. government publications demonstrating the results of small enterprise development have used extensively data of company demography published by the national statistical offices, accompanied by data registered by the tax and labour authorities and by SME support organisations. 3232

• Setting the frame for business surveys. Researchers designing business surveys routinely use business demography indicators for describing the internal proportions of the firm population in order to create sampling frames and weighing algorithms.

• Assessing impacts. Time series about the number of companies established or liquidated in certain years, or about the average expected life span of companies offer deep insights into the business climate. Such statistics can be used to make inferences to the impact mechanisms of policies and other factors affecting businesses.

Statistical harmonisation. For several years serious efforts have been made to standardise the production of company demography data across the statistical offices of the member states of the EU and those of the OECD . As a result of this process, company demography data published by the above organisations will become more and more comparable and addable. 3333 The Hungarian Central Statistical Office also participates in this harmonising work. 3434

In Hungary, during the 1990s, at the time of the transformation of the former socialist countries researchers of small enterprise development routinely demonstrated the impacts of SME policy development by using company demographic data. Statistical data about the number of established, registered, operating, and bankrupt enterprises were used to illustrate the effects of the regulatory reforms that were characteristic of that decade [bib_21]. During the first years of market economy, the number of small enterprises was spectacularly growing [bib_22], but later, in a somewhat more consolidated era [bib_23], partly in consequence of the laws of bankruptcy [bib_24], this extensive growth came to a standstill. The availability of detailed company

3131[Parker 2005]

3232[GKM – NFGM 1996-2007]

3333See, for example [Hult 2002] and [Pilat 2002]

3434[Román 2005]



demography data has facilitated the understanding of these trends.

3.4. Case Study “Balaton": The use of small enterprise surveys in regional enterprise development3.4.1. Purpose, genre and method of the research

This case study is based on a business survey implemented in 2002 among entrepreneurs in the region around Lake Balaton in Hungary [bib_25]. The aim of the research was to reveal the local aspects of enterprise development, the relations between enterprises and the local society and the influence exerted by tourism on the development of enterprises. The case study is suitable to demonstrate the application of company demography data in designing a business survey.

Research question s . The purpose of the survey was to answer to the following research questions: What are the special local attributes of entrepreneurial strategy, entrepreneurial culture, social capital and business networks in the settlements around Lake Balaton? What are the characteristics of local ties of businesses in the region? What are the specific features characterising the enterprises of the region? To what extent have policies of local development, regional development, tourism development, and enterprise development been adapted to the above mentioned features of the SME sector in the region? What are the measures of enterprise development that have reached efficiently the small- and medium enterprises? To what extent have local tourist potentials contributed to the competitiveness of the entrepreneurial sector in the region? To what extent and in what ways are local entrepreneurs affiliated to the Balaton Region, and how does this appear in their attitudes, opinions and entrepreneurial behaviour?

Considerations of research design. Since the fundamental questions of the research do not ask for a causal explanation, this task is not a traditional impact assessment. Instead, the task of the evaluation is to reveal the contribution of local endowments, social context and SME development measures to entrepreneurial competitiveness by applying empirical methods. The sociological survey conducted among local entrepreneurs serves partly descriptive purposes: it shows the characteristic features of small enterprises operating around Lake Balaton . However, to a certain extent the research design involves certain features of impact assessment, because (a) on one hand a considerable part of its questions deals with the impact of enterprise support schemes and institutions, (b) on the other hand the study attempts to reveal some impacts of tourism on the regional SME sector. The revealed structure of entrepreneurial opinions and attitudes gives an overview about the business environment in which enterprise development interventions are implemented and affect local small companies.

The research was commissioned by Lake Balaton Development Coordination Agency. This body was created for facilitating the sustainable development of the settlements of Lake Balaton Resort District, to integrate these settlements into European regional structures, to improve the economic competitiveness of the region with special regard to its tourist attraction, to contribute to the development of agriculture, industry and services provided in the region, to support innovation and environment-friendly economy. The support of local enterprises is a suitable means for the realisation of the above targets.

The questionnaire 3535 was organised in five blocks: (a) Economic characteristics of enterprises; (b) Economic networks of enterprises; (c) Social embeddedness of enterprises and entrepreneurs; (d) Entrepreneurial culture; (e) SME support.

• Economic characteristics of enterprises. Within this set of questions attention was paid to the circumstances of the establishment and development of enterprises, to the most important spheres of activity as well as to their interrelations, furthermore, to the relations of employment, to the characteristic legal status of the employees, to questions arising in connection with turnover and income, profitability and the willingness to invest.

• Economic networks of enterprises. This set of questions examined inter-firm relations of the responding enterprises, with special respect to the location of subcontracting, buyer and co-operating partners, with special respect to linkages with companies and other organisations of the Balaton Region. Special attention is paid to co-operation ties within the catering sector and other sectors dependent on tourism.

3535The Appendix contains a detailed outline of this questionnaire.



• Social embeddedness of enterprises and entrepreneurs. In this respect the research examined the roles that entrepreneurs undertook in civil society, the activity which entrepreneurs as economic actors performed in the local society, moreover on the regional and national level. The research also revealed some aspects of social and family relations, i.e. some questions were addressed to the ways the entrepreneur interacts with relatives, friends, and other connections, in order to reveal, what forms of solidarity connects the entrepreneurs as a social actors to the region, to their birthplace and to the settlement where the entrepreneur was living.

• Entrepreneurial culture. In this set of questions the most important element is the examination of the system of norms of the actors of the economy. What role do legal rules and norms of ethical character play in the attitudes of entrepreneurs? What are the entrepreneurs’ visions for the prospects of the future, their individual motivations, and what is the level of their professional and managerial knowledge and skills?

• Support for SMEs. In this set of questions respondents were asked about their experiences with and opinions about subsidy schemes and SME supporting organisations.

The questionnaire-based survey was performed between December 2001 and January 2002.

Sampling. It consisted of telephone interviews with a sample of 1000 entrepreneurs and managers of enterprises around Lake Balaton . 3636 The sample has represented an enterprise population of 25 thousand small- and medium size enterprises operating in the seven small regions of the holiday region Lake Balaton 3737. Sampling was extended to the following small regions:

• Balatonalmádi small region

• Balatonfüred – Veszprém – Ajka small region

• Fonyód small region

• Keszthely – Zalaszentgrót small region

• Marcali – Lengyeltóti – Nagykanizsa small region

• Siófok – Tab small region

• Tapolca small region

The 1000 persons eligible for telephone interviews were selected by compiling a quota sample of the above enterprise population. To create the quotas, four aspects were taken into consideration.

• The small region in which the enterprise is operating. (7 small regions)

• The type of settlement where the premises of the enterprise are situated. ( 2 values: town or village)

• The sector of activity of the enterprise. (8 sectors: Agriculture; Industry; Construction industry; Trade and non-tourist services; Tourism; Transport; Finances an Real Estate; Other)

• The size category of the enterprise. (3 values: 0 or unknown size of the staff; less than 10 people; between 10 and 250 people).

The internal proportions of the 1000-strong quota sample of the enterprises have been identical to the structure of the 25.000-strong enterprise population in terms of (a) the small region in which the firm is operating (b) type of settlement (c) sector of activity. As of the fourth aspect, i.e. the size category of the companies, the following sampling strategy was chosen: bigger companies were over-represented in order to analyse the answers given by their leaders deeper and with a better reliability. The resulting bias was offset with the help of subsequent weighting. As a result, the composition of the sample has deviated from data characterising the region by less than 1 %. 3838

3636Questionnaire based data collection was carried out by Voxinfo Ltd. both on the phone and by way of personal interview.

3737The borders of the holiday resorts do not necessarily correspond to the officially defined borders of the small regions.

3838The information sources used for sampling were as follows. (a) For the definition of quotas and for weighting the distribution of the enterprise population by four aspects the following sources were used: (a) County-level yearbooks published by the Hungarian Central Statistical Office. (b) The address list of enterprises was compiled with the help of the publication named The Company Database published by the Hungarian Central Statistical Office and the company registers of the Chambers of Commerce of Somogy, Veszprém, and



Before the implementation of the questionnaire based survey some qualitative data collection has been done, in order to facilitate the design of the questionnaire and for helping the interpretation of the numerical results. For the above reasons 15 personal in-depth interviews were made with company managers and owners in the very first phase of the research.

3.4.2. Policy context: local embeddedness of SMEs and the role of tourism in enterprise development

The strong interdependence between entrepreneurs and the local community has been realised by both parties for a long time. Most small- and medium enterprises have close connection with the settlement or region in which they are operating, because - regardless of whether they produce products or provide services - most of their clients and co-operation partners come from their direct environment. Typically for smaller companies, their success or failure depends more strongly on the local economic circumstances than on the economic trends of the country as a whole. Simultaneously, the local business climate is a major issue for the competitiveness and attractiveness of settlements: the local quality of life and the development of the local economy depends to a large extent on how enterprises can be motivated (a) to invest, (b) to establish more workplaces, (c) to use public spaces in harmony with the collective interests [bib_26]. In particular, successful settlements have learnt to wake up and support the entrepreneurial inclination latently present in local inhabitants.

For the above reasons, it is one of the biggest challenges enterprise development policies face is, how to adjust their measures to the local circumstances. Parallel to these efforts, regional development policies emphasis the importance of regional competitiveness and the development of the local and regional network of relations of the entrepreneurial sector [bib_27]. It was recognised by the European Community already in the 90’s, that in many cases, the best way of promoting the development of economically weaker regions is to subsidise local projects of enterprises already operating in a region or of firms having the intention to settle there [ bib_28]. Since 2000 research and education on economic transformation has been increasingly focusing on the question of how to integrate enterprise development measures into the framework of local development policies. 3939

Already in previous centuries, central and local governments have faced the challenge of harmonising local development and small business development. The history of enterprise development policies abounds in cases when governments supported the enterprises of a region in various ways, e.g. by settling foreign craftsmen in a region, making enterprise friendly decisions by decreasing tax burdens, by developing the transport infrastructure of a region, or by encouraging the self government of local enterprises by establishing the legal frames of the operation of trade guilds or by delegating certain public tasks to local chambers of commerce and industry.

During the past two decades the following measures were applied to promote regional enterprise development:

• Supporting subcontractor-supplier cooperation in a region [bib_29]: this has been the aim of various SME development measures both in Hungary [bib_30] and in other countries [bib_31];

• Facilitating the creation of local organisations which help the horizontal cooperation of enterprises, the launching of local marketing initiatives, or support the joint appearance of local enterprises on various markets [bib_32];

• Supporting the establishment and development of regional clusters that involve not only enterprises but public and civil actors as well;

• Encouraging the creation of local economic development concepts [bib_33].

The underlying organisational model of these measures is that local governments of regions and settlements have recognised to an increasing extent that they have to transcend classical roles of keeping order and issuing regulations: additionally to these functions they must act as driving forces of local development. For this reason, local governments go beyond the role of being enterprise-friendly but passive hosts of local enterprises. While earlier the generally accepted aim was to maintain and increase the attractiveness of the individual regions, from the 90s the idea of enhancing the competitiveness of settlements and regions has been gaining space. This

Zala counties.3939See [Dallago 1999] and the curriculum of the „Joint Master Course on Comparative Local Development", a Master Course financed by the Erasmus Mundus program of the European Community in 2006-2007. Participants: The University of Trento (Italy), The Corvinus University of Budapest (Hungary), The University of Economics of Ljubljana (Slovenia) and the University of Regensburg (Germany). Trento-Regensburg-Budapest 2006-2007.



approach has been reinforced by the increasing number of those product and service markets, in which not merely enterprises were competing with each other, but rather, the decisive factor of competitiveness was the region in which the goods were produced or the region in which the services have were provided or branded.

In case of tourist services the notion of regional competitiveness and regional marketing is even more accentuated than in case of other goods or services. For this reason, in tourist regions enterprises and settlements are more dependent on each other than in any other regions.

Impacts of tourism. Tourism exerts a significant influence on the receptive region from an economic, social, geographical, settlement development, environmental and cultural point of view. The demand for tourist products depends very much on the seasons of the year and it reacts very sensitively to events such as political tensions, wars, epidemics, and the fluctuation of exchange rates, natural and man-made catastrophes. The development of tourism is typically accompanied by the development of infrastructure and urbanisation, concentration of local and tourist population, local inflation and pollution of the environment. As an influence of tourism, the structure of employment as well as the structure of the receptive community may become rearranged, and the local population may imitate some consumption patterns and other attitudes of the tourist population. The demonstration and the prediction of these influences has been the subject of a large amount of professional literature. 4040 On the other hand, several works deal with how tourism activity affects the tourists themselves. 4141

Small enterprises in tourist regions continuously have to adopt themselves to the fact that the number of guests is basically determined by the accessibility of the region by transport, by the willingness of large multinational chains of hotels, trading and entertainment companies to invest, as well as by their intention to extend their operations vertically. The international literature of tourist small investments deals, in a detailed way, with the strategies of small enterprises which depend, directly or indirectly, on the traffic of tourists. 4242 Tourist enterprises establish a number of workplaces, but most these working places are short lived, often these survive one tourist season only. A rapid decline in the number of tourists can be caused (a) by the deterioration of transport availability, (b) by the appearance of other competing tourist regions offering more advantageous facilities, or, (c) if there is a change in tourist demands and holiday styles. The risk of enterprises embarking on tourism-dependent services is further increased by the fact that most tourist regions are of an „enclave" character, that is, in addition to tourism they offer a very limited number of possibilities of diversifying into other entrepreneurial activities.

In tourist regions the process of enterprise development starts usually with a so-called „non-structured" period. In this hardly regulated phase most of the restaurant-, hotel- pension- and taxi owners as well as street vendors are “entrepreneurs driven by necessity", that is, entrepreneurs who are unable to find other ways of obtaining income. Many of these small enterprises belong to the informal sector.

In the next phase, market conditions are more and more determined by the tourist investments of the appearing stronger enterprises. These companies, as the representatives of the formal economy continue to operate side by side with the smaller, local, and more informal companies, sometimes also cooperating with each other. The weaker stratum of small enterprises, however, is gradually driven out of the market because large enterprises continuously widen their range of products and services, thus depriving the small ones of solvent customers, whereas most small- and medium sized enterprises are not in the position to take the necessary steps in developing their marketing and the quality of their services.

The successful strategy for facing this challenge is to organise the enterprises of a settlement or region into cooperative structures and networks with each other and with the local governments. However, some small enterprises react to this challenge by applying unfair practices of competition, e.g. (a) by withholding important information from tourists or (b) by imposing their services on them by using aggressive or deceptive techniques. If such behaviour is experienced frequently by the tourists, then the phenomenon may damage the reputation and attractiveness of the whole region and the local business climate may deteriorate.

In several countries, in addition to the dominance of trans-national enterprises, entrepreneurial activity connected with tourism is determined by the presence of small local enterprises. Although only a few of them are capable of growing to become a large enterprise, there is a number of examples to demonstrate how small catering firms operating as family businesses at the beginning, have extended and established chains and

4040[Puczkó – Rátz 2002]

4141[Fejős 1998]

4242[Roessingh – Duijnhoven 2004]



networks which originated in some tourist region. 4343

3.4.3. Tourism in the region of Lake Balaton

Hungary depends on tourism to a significant extent. On the basis of the calculations of the so-called Tourism Satellite Account, in 2005, some 8.9 % of the country’s employment could be traced back to activities connected to tourism, as well as 5.6 % of the GDP, 7.9 % of the export, 7.2 % of personal consumption, and 7.4 % of the investments. 4444 Regarding its role in international tourism, Hungary occupies a favourable position. During the years following the year 2000 it was the 12th-15th target country of tourists in the world, while in Europe Hungary was the 8th most popular target country.

The region around Lake Balaton in Hungary is a holiday resort and an administrative category defined by the legal rules № XXI of 1996 "About country development and country planning" and by the legal rule № CXII of 2000 „On the acceptance of the regional plan of Lake Balaton Resort District". The aggregate area of the 164 settlements belonging here is 3.623 square km, which makes up nearly 4 percent of the whole area of the country. The surface of the lake is 600 square km, the length of its coast is 235 km. In 2002, some 2,5 percent of the population of the country was living in this region.

Tourism around the Balaton accounts for altogether one third to one fourth of tourism in Hungary. 4545 The settlements which attract most tourists to Hungary after Budapest are to be found in the Balaton Region. The annual traffic of guests may be estimated as approximately 9,5-10 million people in the Balaton Region; from this amount 20 percent are foreign tourists. Guests spend approximately 55 million guest nights in places of accommodation around the lake. These numbers also include the traffic of guests in non-registered places of accommodation (such as holiday homes owned by institutions or by various companies, non-profit organisations offering accommodation, privately owned cottages, weekend houses and holiday homes let to relatives or friends, real estates owned by foreigners). The so-called commercial accommodations account for only one tenth of the traffic of guests. The majority of places of accommodation of Hungary, about 500 thousand places, are to be found in the Balaton Region.

The surroundings of Balaton cannot be regarded as an economically coherent unit. In particular, settlements of the region that lie further away from the shore are less attractive for tourism than settlements which are situated on the coast.

Following the year 2000, the competitiveness of the Balaton Region has relatively decreased on the international tourist markets, compared to other tourist regions both in Hungary and abroad. 4646 Although the traffic of foreign guests around Lake Balaton has not decreased in the medium term, still an increasing fluctuation of their numbers has been observed. Foreign guests comprise half of the traffic of the guests of the commercial accommodations around Lake Balaton. In certain years, e.g. in 2004 the number of guest nights spent in commercial accommodations decreased by 10%. However, the number of Hungarian guests has significantly increased, in particular the number of guests arriving to places of accommodation with higher quality. Tourism around the Balaton has always had a seasonal character, but during the past several years the tourist season has shrunk to such a short time as one month only.

The main reason why the tourist competitiveness of the region has decreased is that the primary tourist attraction of the lake is still to spend the holidays at the waterside, whereas the corresponding tourist facilities have not yet been developed to successfully marketable quality tourist products. Since 2000 quite a few hotels, thermal baths, sailing boat harbours and cycle paths, horse-riding tourism facilities, entertainment parks, aqua-parks, networks of training paths were built, but a significant part of the tourist infrastructure is not sufficiently up-to-date. The stakeholders of regional and local development have increasingly emphasised the need to focus development efforts on so-called quality tourism rather than serving the needs of mass tourism around the Balaton. Accordingly, in addition to the traditional offers of a waterside holiday resort, services should be diversified in accordance with international trends. This means that the already existing portfolio should be completed by cultural-, green-, wellness- and health tourism, with wine tourism, fishing, elite sports facilities, holiday trips, country- and hunting tourism. The infrastructure in the service of the above listed as well as the facilities and possibilities connected with these should be developed accordingly.

4343[Hitchcock 2000]

4444[WTTC 2007]

4545[BIFT and others 2006]

4646[GKI 2004]



3.4.4. Dependence of local SMEs on tourism

Most of the enterprises around the lake profit, in a more or less direct way, from tourism, but the statement, according to which the whole of the population of the region basically lives on tourism is false. Naturally, it is the catering sector which is most dependent on the extent and structure of the traffic of tourists, but nearly every commercial, entertainment, transporting or financial enterprise as well as trading in real estate is, in one way or another, connected to tourism. It is true even in the case of agricultural enterprises, because farmers growing grapes are also closely tied to tourism through wine tourism and wine festivals, and find nearby markets in the catering sector of the region.

In the Balaton Region approximately 15% of the enterprises offer accommodation: this proportion is 3 times as much as it is in the country as a whole. This proportion has started to decrease during the years following the year 2000. This is partly a consequence of new hotel developments, and many low quality accommodations have been crowded out of the market, which has led to a certain consolidation and concentration. A related problem in the area is that a wide class of informal entrepreneurs provides accommodation by hiding one part or the whole of their income from the tax authorities.

Smaller businesses and those which can operate only during the short summer season cannot provide work to their employees all year long. However, as the tourist season starts, these firms take back their best workers. 4747

In the same time, businesses more exposed to seasonal character employ more people informally, by avoiding the payment of taxes and other social contributions. The prevalence of this type of informal economy is higher than in this region than Hungarian average [bib_34].

Following the year 2000 an increasing number of entrepreneurs have realised the need of networking and the growing importance of local authorities, local governments in the development of tourism. As a consequence, the number of local tourist associations is steadily growing around the lake. In these organisations entrepreneurs, local governments, civil organisations and other local interest groups are jointly performing a regional marketing activity and discuss planned developments.

3.4.5. Company demographic indicators

Statistical assessments of small firms around the lake show a high enterprise density and specialisation. Entrepreneurial opportunities in the Balaton Region have been opened up for a certain part of the local population living at a quite early stage, to a certain extent already before the political changes of 1989, preceding the rise of market economy in Hungary . Consequently, compared to other regions of Hungary, entrepreneurial knowledge and capital has accumulated significantly.

At the time of the research presented in this case study; in the 164 settlements of Lake Balaton Resort District, nearly 25.000 enterprises were operating with a permanent character. The distribution of this enterprise population according to the most important aspects was as follows:

Table 18. The distribution of small- and medium enterprises around Lake Balaton according to their size Balaton Region, 2000

Staff size category Share of companies (%)

Staff 0 or unknown 68

Under 10 persons 29

Between 10 and 250 persons 3

Total 100

Source: County Yearbooks published by the Hungarian Central Statistical Office

4747[BFT 2005]



Table 19. The distribution of small- and medium enterprises around Lake Balaton according to their sector of activity Balaton Region, 2000

Category of activity Share of companies (%)

Agriculture 7

Industry 9

Construction industry 9

Trade and repair 24

Tourism 11

Transport 5

Finance and trade in real estate 24

Other 11

Total 100


Table 20. The distribution of small- and medium enterprises around Lake Balaton according to small regions Balaton Region, 2000

Examined small region Share of companies (%)

Small Region of Balatonalmádi 10

Small Region of Balatonfüred - Veszprém – Ajka 13

Small Region of Fonyód 12

Small Region of Keszthely – Zalaszentgróti 22

Small Region of Marcali – Lengyeltóti – Nagykanizsa 9

Small Region of Siófok – Tab 21

Small Region of Tapolca 13

Total 100


Table 21. The distribution of small- and medium enterprises around Lake Balaton according to the type of settlement where the premises of the enterprises are located Balaton Region, 2000



Type of settlement Share of companies (%)

Town 62

Village 38

Total 100


Enterprise density is much higher in the settlements surrounding Lake Balaton than the country’s average. In 2003, in the settlements of Lake Balaton Resort District the number of registered small- and medium enterprises per one thousand inhabitants was 81, which was the second largest number among the regions of Hungary after Budapest (109). In the summer season an additional 2000 enterprises operate in the region: these are small companies which perform their activities in the region only in the main tourist season. 4848

3.4.6. Results of the questionnaire-based survey

Description of the entrepreneurial sample. In the sample of interviewed entrepreneurs men were overrepresented: 65% of them were men, 35% of them were women. 35% of the respondent small- and medium entrepreneurs have some kind of higher qualification, 62% of them speak some foreign language on a certain level, and nearly one third of the entrepreneurs, preceding the starting of their enterprises, had participated in some kind of professional training needed for running a business.

A surprisingly large proportion of entrepreneurs are in the age group of over 55 years. The willingness of this group to start a business can be explained by the following factors:

• many of them were dismissed by their last employers at the peak of their working abilities,

• low pensions cause problems of survival for many of them,

• at the time of the survey the regulations provided advantageous tax possibilities for pensioners.

The establishment of the company. Most businesses were established with low starting capital, without a business plan, and as a rule, no employees were hired. Businesses typically were started as a response to unemployment or the wish to improve the livelihood of the families. Typically, the market segment targeted by the start-ups was not defined by the entrepreneur’s devotion to his profession but rather by the current market possibilities. Therefore - as it has been pointed out by the survey - the typical small entrepreneur of the Balaton Region around the millennium has operated in more than one sectors, or, to put it another way, he / she was "standing on several feet". One of the interviewed entrepreneurs could be regarded as typical: he was a locksmith by profession, he was selling his own products as well as those of a French partner, moreover, he undertook reparation jobs and, at the same time, he was running a pension of his own by renting out half of his house to tourists.

The motives that have lead to the establishment of enterprises were as follows.

• “Entrepreneurship driven by necessity" . In the examined area nearly half of the businesses (48%) were established to provide income for the family. The number of those who regard themselves as entrepreneurs by necessity was very high. Some 20% of responding entrepreneurs wanted to establish working places for themselves and/or for the members of their families by way of starting a private business. The enterprises which were established due to such motivations have operated, at the time of the survey, typically either without employees or with just a few workers.

• Entrepreneurship motivated by an emerging business opportunity . Entrepreneurs of this type have started a business in order to react to an emerging opportunity such as a demand generated by the tourist sector, gaining access to an innovative product or service, a privatisation wave of state property or other challenges. According to their self-classification, about one third of the enterprises belong to this group. Enterprises which reacted to an opportunity of privatisation were usually medium sized enterprises at the time of the

4848[Oláh 2003]



survey: typically shops, catering establishments or places of entertainment. It was mainly larger companies which referred to the demands of the market, to new ideas, that have played a significant role in their establishment. Entrepreneurs motivated by an occurring opportunity had made serious preparations and had a definite business plan before starting business. Companies of this group have operated their firms with at least 10 employees at the time of the survey.

Financial sources of starting an enterprise. Typically, the responding companies have relied on multiple sources of finance at the time of their establishment. The vast majority of starting entrepreneurs had used their own capital (95%) as one of the sources. Some 27% of the new enterprise owners asked their relatives for credit: this type of finance can be regarded as a kind of transition between privately owned capital and business based credit or support. Among credits, bank credits played the most significant part, 22% of the entrepreneurs applied for it to start their business. This was followed by other kinds of credit, those from acquaintances, business partners, foreign partners, including informal credits. Finally, approximately 10 % of the starting companies had received subsidies from funds managed by various government agencies and foundations.

Dependence on tourism. The survey has demonstrated that a significant part of the entrepreneurs of the region is – directly or indirectly—interested in tourism and in its development. For example, in an in-depth interview an entrepreneur repairing cash machines has stated that according to his experiences in times when there were more tourists, local cash machines needed to be repaired more frequently as well.

The tourist season. A frequent complaint of the entrepreneurs refers to the tourist season being too short and even getting shorter. The subjects of in-depth interviews often complained about other, occasional entrepreneurs who “invade" the holiday areas in the peak tourist season, and leave immediately when the busiest period is over. They reckon that such entrepreneurial behaviour functions like a self fulfilling prophecy, because the lack of services in the after-season months drives away even the rest of the tourists who otherwise would intend to stay.

Entrepreneurial opinions about tourists. There have been complaints in connection with the social composition of tourists. While most tourists have become more demanding in terms of quality, there are also an increasing number of holiday guests who have less money and their demand for quality services is correspondingly low. In the in-depth interviews entrepreneurs have referred to them as young people, most of them Germans, sometimes “unemployed" or even "drug addicts". Entrepreneurs blame such tourists for damaging the infrastructure and scaring away solvent middle class guests. Many recall the "golden age", those times when "Germans used to meet here", that is, the 70s and 80s when tourists from East Germany met their West German relatives at the lake. In those times for Hungarian workers the socialist system of subsidised holidays and holiday homes was still in place, and many entrepreneurs recall that “people spent the money that they had saved during the year, here". Many respondents feel that the quality of services has not developed much since then, though the demand for change can be observed everywhere.

Future prospects. Most entrepreneurs of the sample felt themselves to be successful and the majority of them were optimistic regarding the future of their business. However, many interview subjects talked about their problems and failures as well. Some entrepreneurs have interpreted success as "we can survive with my family, which is a success in itself". The optimism of other respondents was sarcastic, and related to the present situation: they have stated “it can only become better than now". But on the whole, the responses demonstrate an overwhelmingly positive self-evaluation and good prospects for the future of the enterprises of the region and this optimism can be regarded as a significant positive force.

Short term expectations. More than half of the entrepreneurs have reckoned that their businesses would be more successful next year, and only one-tenth expected a decline for the next year. Pessimistic entrepreneurs, those who expected a decline of their business, attributed this – in decreasing order of occurrence - (a) to the general economical conditions, (b) to the expected decline of tourism, (c) to higher taxes, (d) to increased regulatory burdens and (e) their own professional and personal limitations. Optimistic respondents have expected a better year to come as a consequence of the following reasons: (a) investments made during the previous year, (b) harder or more efficient work to be performed, (c) an expected upward trend of tourism. Some respondents have stated that their positive expectations are not based on economic justification, just on the fact that their optimism is a feature of their character. It is interesting to note that the number of those entrepreneurs who thought that tourism would take an upward trend was much higher than the number of those who expected further stagnation.

Intentions to invest. More than half of the entrepreneurs said that during the year following the research (i.e. in 2002), they had the intention to increase their investments. Among those entrepreneurs who regarded themselves to be more successful than others the willingness to invest was considerably higher than that of the average.



Only 6% of the entrepreneurs planned to decrease investments, 12% of the respondents planned to keep investments on the present level, while one quarter of those who answered the questions had no definite ideas concerning the investments of the next year.

Development aims. Most entrepreneurs with the intention to invest planned to do this in connection with launching new products or new business lines of service, or entering new markets. Compared with such plans, the aspect of product or service quality or that of environment friendly production was of secondary interest. Several respondents working in the catering sector stated that significant investments are needed in order to introduce quality assurance and food safety systems that correspond to European Union norms, with special respect to the so-called HACCP food safety quality assurance. The corresponding Hungarian regulations demand that such costly systems must be introduced even in the smallest restaurants and bakeries, as a condition of preserving their permits to operate.

Innovations. Some 12% of the responding entrepreneurs considered themselves to be “outstandingly innovative", and twice as much has thought of themselves as being “more innovative than the average". An overwhelming majority of innovative entrepreneurs stated that they finance fully the innovations to be introduced. When speaking about innovations, 59% of the entrepreneurs reported that they had no opportunity to innovate for economic reasons, in particular due to lack of finances. Administrative constraints to innovation turned out to be the second most important obstacle; these were mentioned by 15% of the responding entrepreneurs. In particular, most respondents that have evaluated themselves as “fairly innovative" or “not innovative" have blamed the lack of financial sources for not implementing their innovative ideas. One tenth of the entrepreneurs thought they would be able to realise their ideas with the help of external support only.

Employing people or subcontracting. While bigger companies prefer traditional forms of employment, smaller enterprises offer full time employment for a much smaller fraction of their co-workers, than bigger ones. Instead, smaller companies prefer to enter with workers into temporal or regularly renewed contracts of assignment or delegate tasks to them as subcontractors. Under the existing regulations this requires less additional labour costs for the entrepreneur, although it offers less security for the worker.

Training. More than half of the responding entrepreneurs have attached special importance to training, both for their employees and for themselves. According to their opinion, (a) training courses leading to the professionalisation of service delivery and production are the most important, which is followed by (b) the necessity for training in business administration and languages, which is an important asset in this tourist region. The most commonly spoken language in the region is German; this is followed by English, Russian and others. Advanced knowledge of languages (or of one language) which is indispensable for the establishment of international business connections is very rare. Several entrepreneurs have maintained that they are not in the position to finance language training. A particularly illuminating in-depth interview made with an entrepreneur has revealed that the family could not rent out the upper floor of their house for foreign tourists because they did not speak foreign languages.

Entrepreneurial knowledge. Most entrepreneurs cannot find their proper way in the legal environment. Typically more qualified acquaintances and family members help them to obtain and to interpret information about regulations and subsidies. Managers of smaller companies want to receive free information about subsidy tenders, moreover, it is their recurring demand that free assistance should be provided in preparing these tender applications. Medium sized enterprises have a greater deal of experience concerning subsidy schemes and tenders, but they emphasise the difficulties to satisfy the formal requirements and complain that the applications are refused if they fail to meeting every detail of the formal requirements.

The study has also touched upon the role of the family of the entrepreneurs in the business, the position of local enterprises within the local society and analysed the relationships with other enterprises.

Family relations of entrepreneurs. Most respondents attach high importance to the family. This is not surprising in view of the fact that 58% of the enterprises regard themselves as family businesses. In 52% of the enterprises children already take part in the business and in two thirds (67%) they expect that the next generation will carry on with it. Among those, who have relatives living in the same settlement, 25 % said that their relatives supported their businesses. The economic role of direct family relations is the highest in agriculture and in the catering business, and is the lowest in the field of industry and of the construction industry. Company size as measured by the number of employees – in accordance with the previous expectations – was significantly different in the family businesses and in the non-family businesses: family businesses worked, as an average, with half as many employees, than non-family businesses did. A typical family business in the region embraced two generations, while the next biggest group of family businesses involved only one generation. The majority



of the responding family enterprises – 52% of them – have claimed that the next generation of the family already helps to perform some tasks in the business, and the family hopes that the children would continue their business.

Inter-generational mobility. Most respondents have stated that the qualification level of the father and of his son or daughter is very similar to each other. However, typically there is no connection between the father’s school qualification and the success of the children. Apparently, the region offers ample opportunities for entrepreneurs with a lower cultural capital, as well. Certain in-depth interviews have indicated that there is wide stratum of entrepreneurs emphasising the positive role of professional mobility. In particular, many respondents have claimed that launching a career or a business in a different profession than that of the father, enjoys high prestige.

Social networks. The survey has also asked about the opinions regarding the importance of the support of family, relatives, direct surroundings, that of acquaintances. Most entrepreneurs primarily believed in personal, family connections. The statements below met a high level of agreement:

• “The role of personal connections is very important in business life."

• “Family constitutes the most important background for business."

• “I prefer making business with people I know."

Company networks. Connections between enterprises of the same settlement are much stronger than between those operating in different settlements of the region. One third of enterprises maintain some kind of business relationship with other companies of the same the settlement. Entrepreneurs having business relations within their own settlement transact two-thirds of their business turnover with local partners as an average. Only one fifth of the enterprises maintain business relations with companies outside the region. Personal connections often enjoy higher priority than business relations - and make the latter more durable. The smaller the size category of the enterprise, the more typical phenomenon is that business connections are constrained to the home settlement where the business is located. In particular, agricultural firms tend to have business partners from the same settlement or from a relatively nearby one, while the business connections of construction, industry, commerce or catering companies characteristically surmount larger distances.

Local identity. Most entrepreneurs have strong bonds with their home towns or villages, often through their families, or through the circle of their friends. Most respondents are economically attached to Lake Balaton and to its surrounding region, and even stronger to the settlement where they live and work. Local embeddedness and bindings are more visible in cases of smaller enterprises than in those of the bigger ones. However, the emotional attachment to Lake Balaton and the surrounding geographic region was significantly weaker than expected. Despite the fact that the respondents clearly accept being stakeholders of the local society, their answers did not bear testimony of an especially strong joint identity with the Balaton Region and local patriotism concerning the whole of the region could also be hardly traced. In particular, most respondents did not feel more attached to the Balaton Region than to the country as a whole. Most respondents have rejected those statements which have referred to a strong feeling of local patriotism, such as „I can be a successful entrepreneur in my native region only" or “I prefer to do business with a person who comes from the Balaton Region."

Certain in-depth interviews have revealed an ambivalent attitude towards the Balaton Region. On the one hand, respondents were aware of the fact that in other, less endowed regions of Hungary they would not have the possibility to be as successful and to have such living standards as they had in the Balaton Region. At the same time - mainly in smaller settlements – respondents nurtured an idealistic picture about distant towns which, according to them, would provide better possibilities for business. It was often heard, that „I could do better with this business in Siófok", or „in Budapest there are better possibilities and this business would prosper more". At the same time, entrepreneurs hope to find business opportunities by developing local connections through local business clubs or local publications. Others prefer to do business with local friends.

Image of the local government. There were many critical remarks concerning the enterprise policy of local governments. A series of structured interviews have revealed that most entrepreneurs have a high level of awareness as tax payers, who expect a responsible entrepreneurial policy from the side of local governments. The connection between enterprises and local governments turned out to be rather ambivalent. Although the subjects of in-depth interviews knew the expression "business friendly local government" and also attached some ideas to its meaning, still they did not think their local government was business friendly - not even those



entrepreneurs who had been supported by their local government. Most respondents could not report on any support provided for them or for any other entrepreneurs by the local government of their settlement. Some settlements provide an exemption from local tax for start up entrepreneurs; others have helped local entrepreneurs by way of purchasing products or services from them. Quite a few entrepreneurs do not trust local authorities, assuming or suspecting some corruption. While the majority of entrepreneurs declared that they knew those legal rules that relate to their businesses, only few of them knew the respective decrees issued by the local government.

Expectations from the local government. Many respondents would expect special tax exemptions as signs of enterprise-friendliness; others would expect local governments to give preference to local entrepreneurs at public procurements against other competitors, a favour which in turn would increase local business tax revenues. Again, others expect the local government to subsidise and regulate the behaviour of local economic actors, in order to (a) support endogenous SMEs (b) exclude cheaters and informal entrepreneurs (c) crowd out those competitors who attempt to get rich in just one tourist season and are just “disturbing the markets".

The role of local government in attracting tourists. Entrepreneurs – primarily in connection with attracting tourists - made considerable demands on the local governments in order to alleviate the problems experienced in the field of tourism. The first and most frequently repeated claim towards the local governments is that they should improve the choice and standard of cultural programs which should contribute, in addition to the natural beauty of the environment, to the tourist attractiveness of the region. The respondents were aware that the Balaton Region is a direct competitor of Mediterranean and exotic destinations, both in case of for Hungarian and foreign tourists. Therefore they stressed that it was not sufficient to depend on the attractiveness of beach tourism, not even in the summer tourist season and wanted to see more cultural events which attract tourists by themselves. Entrepreneurs require local governments to widen he cultural portfolio, organise more festivals, similar to the well known and successful annual festival of the Balaton Region held in the settlement Kapolcs.

The protection of the environment is an important issue, and not only for entrepreneurs in the catering sector. For example, a wine grower entrepreneur has mentioned that his plans to establish a bottling factory could not be realised due to lack of suitable and affordable clean water resources in his settlement.

Participation in nationwide SME support schemes. Only a minority of responding enterprises has taken advantage of existing SME support schemes. At the time of the survey the most favoured forms of support were (a) microcredit and (b) various schemes of investment subsidies. Only some 4% of the interviewed entrepreneurs have obtained microcredit, 4% got investment support, 2% had preferential credit and 2% has obtained non-returnable subsidy or subsidy for creating workplaces. Those who had taken advantage of some financial support were satisfied with the respective procedures on the whole.

Opinions about nationwide SME support schemes. Nearly every respondent had a definite opinion about the existing system of nationwide SME support system. Respondents were asked to rate on a five graded scale, to what extent they agreed with some statements. In particular, most entrepreneurs have agreed with the statements that „Many SMEs do not know how to get to the necessary support" and that „Personal connections play a decisive part in obtaining support." On the other hand, most respondents have disagreed with the following statements: „ The support system is good, but the amount of money being at disposal is not sufficient" and that „The financial support of applicants is good, but administrative simplification and the reduction of administrative burdens would be much more important."

Recommendations to nationwide SME support schemes. Several entrepreneurs would like to increase the number of beneficiaries of SME support schemes; even if this means that the individual volumes of subsidy will be reduced. Most respondents were not in the position to provide the necessary own financial contribution or could absorb only smaller sums of subsidies. At the time of the survey credits of smaller amounts were not available from banks, therefore, in case of smaller scale investments, family resources remained the only alternatives, or they had to give up the idea of developing the firm altogether. Several entrepreneurs had planned to improve the quality of their products or services, but subsequently had to give up the project due to lack of finances and, instead, they decided to choose an extensive way of development, without a perceivable increase in quality.

Non-returnable subsidies vs. credits. Most entrepreneurs prefer non-returnable subsidies to credits, but this is not always the case. If the credit procedures are simple and it is much easier to access the credit than the subsidy, moreover if the volume limits of credit provision are more flexible, than they prefer to borrow.

Information about support possibilities fails to reach many eligible micro- and small ventures. Many



entrepreneurs have learned that they are eligible to apply for some kind of support, but have difficulties in understanding the respective brochures and tender materials even if the information has reached them. SMEs that have applied for subsidies without success typically draw the conclusion that it was not worth to prepare the application material or to have them prepared by experts. Small enterprises must devote significant human and financial resources to compile the application material and the refusal of the application with reference to formal reasons may discourage the applicant.


Chapter 4. 4. Regulatory Policy1. The quality of the legal environment of enterprisesRegulatory policy. Regulation concerns how governments intervene in the economy through laws and other instruments, in pursuit of social, economic and environmental objectives, in order to enhance the functioning of markets or to protect health, safety, or the environment. The policy area of regulatory policy aims at developing good regulations, at establishing a long-term basis for efficient and responsive regulation. The instruments of regulatory policy are administrative simplification, deregulation, regulatory impact assessment; enhancing transparency and communication, the development of alternatives to regulation and revealing and disseminating best practices of regulatory compliance and enforcement.

Regulations are produced, enforced, modified and removed by institutions and processes of the regulatory system. In democratic societies this system is part of the wider governance activity and relies heavily on public consultation.

Institutional economics is a school of economic thought in which the regulatory system, the economic costs of business-to-business, business-to-consumer and business-to government transactions and the proper institutionalisation of property rights are the most important explanatory factors in understanding economic behaviour and economic growth. In this context, the institutions and processes of the regulatory system are crucial in understanding small business development as well. Good institutions and properly formalised codes of behaviour reduce the uncertainty and risk for entrepreneurial behaviour; reduce the transaction costs connected with entrepreneurship. 1

Lawmakers issue rules in order to codify positive norms and values. Such norms are, for example, fair competition, the prohibition of discrimination at work, or the protection of the environment. Typical regulations for companies define mandatory conditions of company registration or permits, in other cases they define limitations for certain groups of enterprises. The aim of imposing regulatory measures on the entrepreneurial sector is, in most cases, to avert non-desired events in the following fields:

• entry and competition of firms on the market;

• quality and security of traded goods;

• environmental effects of the production as well as those of the products, and

• relationships between employees and employers.

Regulatory quality and economic growth . Regulation is an important variable in explaining the economic growth or stagnation of a country. In particular, the quality of regulation has a significant effect on the size of informal economy and indirectly on economic growth. Empirical cross-country comparison shows that in general, countries where regulatory burdens in product markets and labour markets are heavier are characterised by higher level of informality and slower economic growth. Poor regulatory framework creates incentives for firms to avoid compliance by regulations. 2 Heavier regulatory burden and lower quality of the institutional framework not only reduces macroeconomic growth but also increases the macroeconomic volatility in a country. 3 Other empirical studies using company-level data show that firms tend to reduce their formal operations if tax and regulatory burdens increase. However, if the enforcement of regulations and the fairness of courts improve, the growth of formal firms increases. This finding does not hold for informal companies, which are generally unable to take full advantage of the legal framework and the judicial system. 4

The alternatives of regulation. The rationale of issuing business regulations is to harmonise the profit-oriented operation of enterprises with the interests of other stakeholders such as the consumers, employees, or subcontractors of the regulated businesses and generally with the interests of the local community. Regulation,

11 See this in more detail in [Smallbone-Welter 2009]

22[Loayza-Oviedo-Serven 2005]

33[Loayza – Oviedo - Serven 2004]

44[Dabla-Norris and Inchauste 2007]


4. Regulatory Policy

however important, is only one of the possible tools of interest reconciliation. Policy makers may refer to measures which are more practical to apply and are better suited to the particular challenge than regulations. It is the task of public administrations to reconcile the interests of these stakeholders by using various instruments, such as consultations, mediations, delegation of certain tasks to chambers of commerce and professional associations. Central and local governments may positively influence the business environment by financing supported programs, creating institutions for interest reconciliation, supporting the self-regulation of enterprises, supporting the creation of extra-judiciary elected bodies for resolving disputes between companies, by refining the implementation of public procedures regarding ownership rights, warranty and insurance.

Regulatory quality . Regulation is one of the important aspects of the connection between the state and the enterprises. Therefore the examination of the necessity, fairness, simplicity, coherence and business-friendliness of legal rules is a prerequisite of creating a favourable legal environment for investors and for businesses in general. It is the task of the regulatory policy to “regulate the regulations" in order to satisfy the above requirements.

Legislators issuing regulations must find a viable compromise between the interests of the above mentioned stakeholders and the governments must manage the potential conflicts between businesses on the one side and law enforcing government authorities on the other side in such a way that does not limit the efficient operation of enterprises.

While during the 19th and 20th century the notion of quality has become increasingly applied to products and services, quality control of regulations has gained ground only during the last quarter of the 20th century. The procedures of regulatory impact assessment, which serve exactly the purpose of qualifying legal rules with the help of legal and economic analyses, have been introduced relatively recently into the cultures of public administration of developed countries.

Criteria of regulatory quality . During the past two decades, the following criteria of good quality regulation have gained importance. Regulatory rules should satisfy the following expectations:

• Burdens and costs for enterprises resulting from compliance with the regulation, and the public costs resulting from the implementation of the regulation should be justified by the aggregate social usefulness of the regulation.

• Regulations should not lead to a distortion of the market. In particular they should not lead to monopolies or social injustice by imposing a heavier burden on certain selected groups of entrepreneurs or employees.

• The unwanted side effects of a planned regulation should be negligible in order to avoid the necessity of a subsequent modification of the regulation or the need for introducing supplementary rules.

2. Assessing the aggregate impact of the administrative–regulatory environmentThis chapter will show various attempts to evaluate the aggregate regulatory environment of a country. Typical research projects of this kind are based on surveys implemented in a series of countries that enable the researchers to make cross-country comparisons and make inferences to the relationship between (a) the regulatory environment on the one side and (b) economic competitiveness on the other side.

2.1. Regulatory reform initiatives in OECD member statesThe Organisation of Economic Cooperation and Development (OECD ) started its operation in September 1961, with its headquarters in Paris. The members of the organisation are 30 countries, among them the most developed countries of the world. 5 Hungary has been a member since 1996. The member countries of OECD work together to develop policies addressing economic, social and environmental challenges. In particular, OECD maintains departments to facilitate inter-governmental co-operation on Small and Medium-sized Enterprises and Local Economic Development. The organisation promotes democratic governance and rapid

55The 30 OECD member countries are: Australia, Austria , Belgium , Canada, the Czech Republic, Denmark, France, Germany , Greece, Hungary , Iceland, Ireland , Italy, Japan, Korea, Luxembourg , Mexico, the Netherlands, New Zealand, Norway, Poland, Portugal, the Slovak Republic, Spain, Sweden , Switzerland, Turkey, the United Kingdom and the United States . The Commission of the European Communities takes part in the work of the OECD.



economic development by researching and disseminating methods of good governance, by implementing surveys and offering statistical services for the member states and by making recommendations to improve the work of public services worldwide.

Since regulation is one of the most important devices of maintaining the contact between the government and the enterprises, OECD has prepared, already in the 90-ies, proposals for improving regulatory quality in its member states. 6 The organisation has implemented several research projects which have reinforced the finding that in most countries the improvement of the quality of regulations had a positive impact on economic performance. 7 In particular, between 1998 and 2004, OECD experts have assessed the regulatory reform activities of 20 countries. 8 Based on these findings, OECD has issued a number of decisions and proposals concerning good governance and has defined the principles of successful regulatory reforms. 9

The number of OECD member countries that have institutionalised regulatory impact assessment activities in some way or other has continuously increased. In 2004 OECD has recorded altogether 26 countries where RIA has been implemented on some level of government. In several countries RIA activity is supported by certain legal provisions: in the Czech Republic, in Korea and in Mexico a legal rule, in the United States a presidential order has been introduced concerning its application, while in Australia, Austria , Denmark, in the United Kingdom , in Finland , in the Netherlands , in Japan, Canada, Poland, Hungary , Germany , Norway, Italy, Portugal, New Zealand and Sweden governmental decrees prescribe its application. Regulatory impact assessments are regularly carried out in the United States and in Mexico, under the supervision of specialised governmental bodies. In developing countries, however there had been isolated initiatives only. 1010

The organisation annually surveys the situation of small enterprises in the member states. In the resulting yearly reports OECD devotes an important chapter to the analysis of administrative simplifications as one of the most important instruments of enterprise development policy. 1111 In most member states of the OECD the most significant aim of small enterprise development policy is to decrease the administrative burdens. This aim has been served by the following measures:

• Launching initiatives to simplify the language of the regulations;

• Offering technical help to facilitate compliance;

• Making exceptions for small enterprises when issuing certain restrictive regulations;

• Institutionalising one-stop systems of administrative procedures;

• Publishing and disseminating special informative materials for small enterprises.

In the past two decades the governments of OECD member countries have taken to the supervision and simplification of excessive government bureaucracy. Still in OECD member countries, it remains one of the most frequent complaints of citizens and enterprises that public authorities demand too much paperwork with too complicated administration. In 2003, based on one of the surveys made by the OECD, a study was prepared about the best practices to be followed in order to simplify administrative procedures affecting enterprises. 1212

The survey includes case studies from Australia, France, the Netherlands , Mexico, Korea, the United Kingdom , the United State, and others. The examples include both

• bottom-up initiatives where the administrative procedures of various subordinated government authorities are simplified, and

• top-down initiatives where customer friendly measures are initiated at the level of the central government and implemented in the decentralised public agencies.

Projects aimed to simplify administration most frequently choose from the following approaches and models.

66[OECD 1997] and [OECD 2001]

77[OECD 1999]

88E.g.: about Hungary: [OECD 2000]

99E.g.:[OECD 2005a]

1010[OECD 2003]

1111[OECD 2005b]

1212 [OECD 2003]



• Prevention. In certain countries governments focus on preventing the introduction of unnecessary and unreasonable administrative burdens before they come in force. This is an innovative approach compared to the traditional governance style where governments attempt to reduce the existing bureaucracy.

• Use of information technology. In all OECD member countries government make use of the development and spreading of IT, of the possibility of introducing e-government solutions. Information Technology offers practical solutions to diminish paperwork and to contact the authorities quickly and intensively through the Internet.

• One-stop systems of administrative procedures . In nearly all OECD countries institutions are being set up offering a wide range of administrative services for companies and citizens at one location. One-stop systems – also called single window systems – may be institutionalised also as on-line services offered by several authorities having the same target group of companies. Such arrangements may save a lot of time and resources for companies interacting with various Government agencies, which make them very popular. Some examples for public bodies preferring this approach in their e-Government strategies follow.

•

• In the USA all labour related procedures have been integrated into one single Internet portal

• In Great Britain the procedures of all regulations affecting small enterprises have been integrated into one single Internet portal

• In many countries governmental websites have been created with the aim of merging the websites of different departments of the government. For example, the Hungarian government has made considerable efforts to motivate government departments and their subordinated bodies to appear under a uniform image on the Internet when communicating with enterprises and citizens, by merging their specific websites into one single governmental portal. [bib_35]

• Simplification of business licensing. Complicated business licensing causes severe damages because it creates barriers of entry for start-up enterprises. The simplification of business licensing has an outstanding importance in the reduction of administrative burdens, and such measures deliver in a short period of time, spectacular results. However, in some cases this is against the interests of enterprises that are already functioning, and such incumbent companies may lobby the government in order to make licensing rules for new entrants more complicated.

• Time limits set for the authorities. An essential element of administrative burdens is the waiting time from submitting an application to the notification about the decision of the authorities. In an increasing number of countries governments have set deadlines for public bodies in order to accelerate decision making. Moreover, in some countries such provisions are amended with the rule that if the office fails to respond until the time limit, then the request of the customer is automatically granted. 1313 Setting up time limits may result in a considerable decrease of compliance costs for enterprises and citizens, moreover, such measures tend to improve the accountability and the customer-friendliness of governmental bodies.

• Exemptions for small- and medium size enterprises. Impact assessments may play an important role in the rule making process by showing the regulation would impose untenable burdens on smaller companies. There have been examples when in such cases governments have provided exemptions from certain regulations for smaller enterprises.

• The introduction of new organisational models with less bureaucracy. In a number of OECD countries the decrease of administrative burdens is embedded into the organisational reform of the public administration. In certain fields of regulation specialised units were created in government departments with the task of simplifying administrative procedures in the interest of small enterprises. In other countries these units were created on the level of the central government (top-down approach). An alternative cheap and efficient solution is to delegate the task of administrative simplification to independent committees with members of the non-governmental sector and with representatives of academic or entrepreneurial organisations.

• Deregulation . A number of countries have launched deregulation campaigns in order to assess and - if necessary – to phase out hundreds or thousands of existing legal rules which are outdated, unnecessary, or carry contradictions in themselves. A characteristic result of deregulation may be the merging of legal rules.

1313 This is called the „silence is consent" rule.



International experience has shown that campaigns aiming at the reduction of bureaucracy may lead to spectacular results already in the short run. However, launching spectacular campaigns with relatively narrow spectrum involves the risk of diverting the attention from deeper ranging regulatory reforms. Therefore the reduction of bureaucracy should be handled as part of a comprehensive regulatory policy and the issue of the quality of regulations should be continuously kept on the agenda.

2.2. The OECD Economic Regulation IndexInternational organisations have repeatedly launched projects in order to quantify the impacts of regulatory environment on national- and company-level competitiveness Researchers of the OECD have been conducting a remarkable research series since 1999 in order to assess the impact of the national level regulatory framework on GDP. 1414

Data collection. In the first phase of the research an expert survey was constructed in the participating countries. Originally 27 countries answered the questions but later 3 countries dropped out of the sample. The questionnaire of the survey has covered a wide range of different regulatory fields affecting enterprises working on the markets of a wide range of products and services. The questionnaire has asked for both quantitative and qualitative information on more than 1.500 different provisions regulations and administrative procedures. The regulations examined by the questionnaire have ranged from decrees with macroeconomic relevance to sector specific regulations, and it covered acts, legal rules and administrative procedures as well. For each of these regulatory fields the major regulations were qualified by the following method: the main question to the experts was as follows: to what extent do these regulations facilitate, limit or constrain the economic activity of companies, how do these regulations influence the competitiveness of the companies affected by these regulations.

Indicator development . In the second phase of the research a composite indicator was created for each country, the so-called „OECD Economic Regulation Index" expressing the general quality of regulations in that country. These composite indices were created on the level of individual countries, but they were based on primary (elementary) indicators, covering most of the internal economic regulations. 1515 The researchers have developed a metrics expressing the overall level of enterprise-friendliness of the regulatory environment in the respective country. Those countries where a wide range of strict, cumbersome or hostile regulations were in force that were creating obstacles to economic activity have received lower indices. For example, when evaluating the legal environment of the road transport sector, the relative strictness and complexity of the regulation referring to the driving time and rest period of the drivers was also quantified with the help of an appropriate score. By aggregating such elementary indicators across regulations, the experts have created a composite index thus expressing country-level integrated enterprise-friendliness of the regulatory framework.

The specific metrics of the „OECD Economic Regulation Index " was defined in the following way. The entire lack of regulations in a regulatory domain was characterised with „0" value. In case of the United States the value “1" was assigned. All other indices were weighted sums of indices assigned to specific regulations as defined by experts. Thus, for example for the „OECD Economic Regulation Index" Australia and Sweden have obtained the value of 1,3, while Turkey was assigned the value of 3,1 and Italy the value of 3,5.

Econometric calculations. In the third phase of the research statistical calculations were made. First, the „OECD Economic Regulation Index " was correlated with the per capita GDP of the respective countries. It was found that countries characterised with more liberal regulations were on average more successful in terms of economic outputs. Later, in a more sophisticated calculation the „OECD Economic Regulation Index" was used as one of the explanatory variables in a regression model using a set of indicators characterising the responding 23 countries. The regression model has explained the level of per capita GDP of these countries with the help of a series of explanatory variables: these variables were of demographic and socio-economic nature, such as mortality, the level of education and the level of export-orientation of the country. For each country, the values of these variables were taken from the standard databases of the OECD, with the exception of the above mentioned key explanatory variable, i.e. the „OECD Economic Regulation Index". Thus, the regression model has measured the impact of regulatory quality on per capita GDP, while controlling for the above mentioned demographic and socio-economic variables.

Calculation results. The results of the regression model has confirmed that the enterprise-friendly nature of the regulatory environment is closely and positively correlated with the level of per capita GDP. This result is robust

1414[OECD 1999]

1515 On composite indices see: [Lengyel 2002]



in the sense that countries with comparable, similar indices of mortality, education and export-orientation, but with significantly different levels of “OECD Economic Regulation Index" show also differences in their levels of per capita GDP. This result confirms the observation that in less developed countries the regulatory environment limits enterprises much more severely than it does in developed countries.

2.3. The “Doing Business" project of the World Bank2.3.1. Purpose and method

A number of research institutions have embarked on assessing, quantifying and comparing the competitiveness of various countries. 1616 The resulting reports contain survey-based indicators characterising the general economic and political conditions in the participating countries. One of the main goals of these publications is to inform international investors as well as locally operating businesses about the business environment.

The World Bank has published its" Doing Business " report since 2004 every year and it has become one of the most influential country rankings of regulatory competitiveness. The main issue addressed by the report is how regulations in the surveyed 150 to 180 countries affect enterprises and the economy in general. 1717 The research is based on the hypothesis that economic success of a country is not only determined by its macroeconomic policy, but also by the quality of the regulations which affects enterprises and also by the quality of work of those institutions which supervise and enforce the compliance with these rules. Small- and medium sized enterprises are the main target group of the research.

The major information sources of the research are reports of local experts about various fields of regulation in their particular countries. The research evaluates those legal rules which enterprises are most seriously affected by, assesses the work of the implementing government agencies, moreover, it provides information about the dependence of enterprises on business services which they cannot avoid using (such as lawyers, accountants, auditors, etc.).

The database thus created provides a comparative frame for the comparative assessment of governmental regulatory reforms, provides information for planning such reforms, and highlights the good practices of countries that have introduced enterprise-friendly regulations. The database offers sub-indicators for a series of regulatory areas and computes a key composite country-level indicator of environment-friendliness. Since international development aid can be utilised most effectively in a positive regulatory environment and in case of sufficient institutional conditions, these indicators can be used in targeting aid for institutional development. Moreover, the theories about the connection between regulatory framework and economic development can be empirically tested with the help of this research.

The individual yearly reports of the series survey the most fundamental functions / processes in the life of an average enterprise. Thus, in the annual of the year 2004 the regulatory environment of the following business processes / functions were highlighted:

a. the establishment of the enterprise,

b. the employment and dismissal of labour force,

c. the enforcement of the prescriptions put down in contracts,

d. access to credit, and

e. putting an end to the operation of the enterprise.

in 2005 the above analysis was complemented with the regulatory aspects of the following business processes:

f. official registration of business properties

g. protection of investments

In 2006 the regulatory aspects of the following business processes / functions were highlighted:

1616E.g.: [WEF 2007]

1717[WB 2004 - 2006]



h. obtaining permits from government agencies

i. taxation

j. foreign trade.

The following types of indicators were developed to measure the quality of the regulatory environment, the extent of administrative burdens, the complexity of the activities needed to comply with the assessed regulations, time- and cost aspects of complying with the assessed regulations. Some examples for the indicators of the Doing Business reports:

• The number of proceedings needed for the registration of a company

• An index indicating the flexibility of hiring or firing an employee.

• The duration of and cost of having a firm registered

• The costs of enforcing the provisions of a contract

The method of data collection and analysis is identical in the case of each business process / functions examined:

• A central research team elaborates and updates the methodology of the yearly report. A questionnaire is compiled as a tool to evaluate the regulation of the particular business process. This methodology will be distributed to local advisors in all of the surveyed countries. Local advisors in all of the surveyed countries collect the regulations in force that are affecting the assessed business process in the given country.

• The questionnaire is disseminated to the local experts who are proficient in the particular business process / function of the particular country: legal advisors and lawyers engaged in the affairs of enterprises, consultants and judges. These local experts respond to the questionnaires and return them to the central research team. The interaction between the local experts and the central research team consists of several rounds: in each round the preliminary results are analysed and the questionnaire will be refined by introducing new aspects of evaluation. Subsequently, new waves of data collection follow. Thus, for example, the questionnaire covering the regulatory environment of starting a business was refined by adding the following aspect: what is the minimum requirement of starting capital in addition to the already existing indicators. This additional aspect was recommended by legal advisors who called the attention of the researchers to the fact that the requirement of starting capital often represented an effective limitation to entry for start-up companies.

• The outcome of the survey is a set of indicators. These indicators can be compared across business functions / regulatory fields and across countries, thus enabling the researchers to create country rankings.

The indicators are designed in a transparent way which makes their interpretation very straightforward. During the phase of questionnaire development, for each of the above mentioned enterprise functions / processes the researchers define in advance a very unambiguous, clear-cut hypothetic situation, a test case which the companies of every surveyed country are likely to face. This situation will be identified by exactly defining the type of company and the parameters of the test case. The specific questions of the questionnaire relate to the challenge defined with this hypothetic situation.

In 2006, for example, the indicator called „difficulties of starting an enterprise" and its components were based on the following assumption: The test case was that a company had to be registered with the following parameters:

• Limited liability company

• Operating in the most populous city of the country

• Entirely (100 %) possessed by local owners, 5 private persons

• Its starting capital is 10 times of the per capita GDP of the country and it has been paid at the end of 2005 in cash.

• The company intends to pursue general industrial or trading activity, does not export or import, does not



produce specially taxed articles (alcohol, tobacco), and it does not considerably pollute the environment.

• The company rents its premises; it does not possess any real estate.

• The company is not entitled to receive investment subsidy or any other support.

• Following its registration, the company will operate with 50 employees, all of them local inhabitants

• Its annual turnover is at least 100 times of the GDP per capita

• Their company contract does not exceed 10 pages.

Having defined the above type of company, the questionnaire subsequently asks the standard questions about the administrative burdens associated with its establishment. This approach makes the results interpretable in a wide range, furthermore enhances their relevance and comparability.

According to the methodology of the “Doing Business " survey, the indicator "difficulties of starting an enterprise" consists of four components. (1) the number of obligatory proceedings necessary for the registration of the company (interactions with the authorities, with public notary, with lawyer, with accountant, etc.), (2) the duration needed for the registration of the company, (3) the respective costs, (4) and the minimum level of starting capital.

The measurement of the indicator "difficulties of starting an enterprise" is further specified by adopting the following assumptions:

• Every interaction between the founders of the company and external parties (e.g. governmental offices, lawyers, audits, public notaries) is regarded as a proceeding to be counted. Interaction among the founders, leaders and the employees of the company are not taken into consideration as far as this indicator is concerned.

• The founders of the company carry out all proceedings themselves, without the help of accountants and lawyers, unless it is mandatory to involve such a party.

• Proceedings not obligatorily prescribed by the regulation cannot be taken into account.

• If the administrative procedure of company registration allows several possible procedures, then the simplest procedure with the minimum time duration is taken into consideration while calculating the indicator, provided this solution can be employed by a wide range of companies.

• Only those proceedings should be taken into consideration, which are compulsory for each enterprise. Thus, for example, among environment protection regulations only those should be counted in the indicator, which are mandatory for all enterprises. Similarly, proceedings for installing electricity, water, gas, and for arranging waste disposal services are taken into consideration only if there is a separate regulation which makes them compulsory for start up companies.

The above method is applied for every business function / process as listed under (a) to (j). Data collection is completed along these lines – in 2006, for example – in each of the 175 countries taking part in the research.

2.3.2. Selected results of the research

One of the most apparent results of the research is, that the business environment of less developed countries tends to be less enterprise-friendly: companies in these countries have to cope with more resource-consuming regulations than in developed countries. In a wide range of third world countries, the substantial regulatory burden on enterprises certainly contributes to stagnating productivity and economic development, high prevalence of informal economy and corruption.

The indicators of the „Doing Business " project provide information about countries providing advantageous regulatory framework for their enterprises. Such countries are not characterised by lack of regulatory activity, rather, compliance with regulations is less costly and less burdensome in these countries. In Australia, for example, altogether 2 proceedings are needed to start a company, while in Bolivia 15, and in Chad: 12. The parallel indicators expressing the working time needed for starting a company prove the same: while in Toronto, Canada, 2 days are enough to start a company, in the capital of Mozambique, in Maputo, 153 days are needed



for arranging the same issue.

It has been proven by a series of theoretical works that an institutional environment that effectively protects properties and creditors can contribute substantially to long term economic development. 1818 In many developing countries uncertainties surrounding the possession of assets and registering property rights (e.g. those connected with land ownership) has lead to a situation whereby the capital possessed by average people cannot become effectively utilised in small enterprises. Such institutional difficulties may lead to the conservation of economic underdevelopment by themselves. 1919 Businessmen facing the challenge of registering property rights or recovering of debts must waste much more efforts and can expect less success in underdeveloped countries than in developed ones. While in Helsinki 3 proceedings are necessary for the registration of a commercial property, in Abuja (Nigeria) 21 proceedings are needed for the same thing, and the costs of the registration may be as high as 30% of the value of the property itself. While in Tokyo, if the debtor goes bankrupt, the creditor can expect to get more than 90 cents after each dollar of the debt; in Mumbai (Bombay, India) the creditor will get not more than 13 cents in a similar situation.

It has been shown that in many cases standard regulatory solutions which have worked well in developed countries could be transplanted with success into less developed countries. For example, reforms involving (a) the reduction of the number of proceedings necessary for the starting of a company (registration of the company with the statistical agency, with the tax office, with the agency responsible for social insurance, etc.) and (b) introducing electronic procedures to company registration have led to outstanding results in Canada, Singapore, Latvia and Mexico, but in Honduras, Vietnam, Moldavia and Pakistan, as well. This finding contradicts the often repeated argumentation, according to which every region has its peculiarities, and that „one size doesn’t fit all".

By observing selected time series of the sub-indicators of the “Ease of Doing Business " composite indicator, researchers may formulate empirically verified statements about the convergence of regulatory and small business development policies across new EU member states. Cross-country differences of regulatory content and quality between new EU Member States are decreasing. However, in spite of the Europeanisation process these countries have gone through, there are still very substantial deviations in various regulatory fields.

3. Impact assessment of individual regulations3.1. Use, application fields and institutionalisation of RIARegulatory Impact Assessment is an important innovation of governance that has been introduced over the last two decades in an increasing number of countries. In most of the cases, this method is applied as an ex ante assessment of the possible implications of proposed new regulations affecting businesses.

Areas of application of RIA. The most relevant fields of application have been as follows:

• Better Regulation Programmes . Many countries apply RIA as an integral part of long term programmes for checking the quality of planned regulations. In most of these cases the main purpose of preparing RIAs is to improve the regulatory environment for businesses, citizens and consumers, to use RIAs in the consultation process. For example in Ukraine the State Committee for Regulatory Policy and Entrepreneurship (SCURPE) is responsible for preparing an ex ante RIA of every planned regulation. 2020

• Deregulation campaigns . Many countries have launched deregulation programmes and administrative simplification campaigns. Deregulation is another activity where RIA can be successfully applied. If there are doubts about phasing out a regulation or not, a simplified RIA procedure can help with the decision making. In these cases the overriding aim of the activity is to phase out a wide set of unnecessary, outdated regulations. For example, Ukraine has embarked in 2005 in such activity by applying the so-called “Guillotine Principle" to several thousands of regulations issued by the central and local governments. 2121

1818[North 1990]

1919[de Soto 2000]

2020 The legal background of this activity is the Law of Ukraine on Principles of State Regulatory Policy in the Area of Economic Activity (No. 1160-IV, 11.09.2003).2121 The legal basis of this activity was the Decree of the President of Ukraine from 12 May 2005 № 779/2005 ‘On liberalization of enerpreneurial activity and state support of entrepreneurship’.



• Legal harmonisation . Several potential member countries of the European Union have been embarking on extensive or sporadic RIA programmes for facilitating the harmonisation process between European laws and the respective national legislations. In these cases the aim of the RIA activity was to decrease the costs of legal approximation for government, businesses, citizens, and to facilitate the negotiation process with the EU. For example in 2007-2008 the Ministry of Justice of Ukraine has implemented a RIA project [bib_36] in order to reveal the potential impact of introducing the technical legislation of the EU [bib_37].

Institutionalisation of RIA systems is a major task for governments that want to improve the quality of regulatory environment. OECD 2222 has published its ten principles on how to introduce effective RIA

• Maximise political commitment to RIA.

• Allocate responsibilities for RIA programme elements carefully

• Train the regulators

• Use a consistent but flexible analytical method.

• Develop and implement data collection strategies.

• Target RIA efforts

• Integrate RIA with the policy-making process, beginning as early as possible.

• Communicate the results

• Involve the public extensively

• Apply RIA to existing as well as new regulation

When public administrations face the challenge of institutionalising RIA systems, the following major questions and issues must be taken into consideration.

Designation and involvement of authorities implementing the RIA system

• Is there a need for a centre of co-ordination of RIA activities?

• If yes, which part / body of the Government should be responsible?

• How to involve non-governmental organisations?

When / for what type of legislation is a RIA required?

• Only for primary legislation

• For secondary legislation as well

• Only for introducing EU Directives

• For all of the above mentioned areas

Who should prepare a RIA report and under what conditions?

• Prepared by public servants

• Outsourced to external consultants

• A combination of the above (initial RIA by public servant, full RIA by external consultant)

Who should co-operate with whom with the help of a RIA document?

• RIA serves as a co-ordination instrument between various Government agencies (e.g. between line ministries

2222[OECD 2008]



or between central and regional authorities). The preparation of RIA should be co-ordinated across the central ministries of government and other law making institutions

• RIA serves as a co-ordination instrument between Government and stakeholders of the private sector (e.g. between Ministries and company interest representations)

3.2. Conceptual framework of RIAScenario. The course of expected events under a certain hypothesis is called scenario. Impact statements are based on the comparison of various scenarios.

• Baseline scenario. Researchers preparing an ex ante RIA must always consider what would be the consequences of not introducing the examined regulation. This course of events is called the baseline scenario, or sometimes referred to as „business as usual" scenario.

• Counterfactual scenario. Researchers preparing an ex post RIA assess the impacts of the regulation after the measure had been implemented. In this case the hypothetical situation must be analysed that the regulation – which had been actually introduced - had not been introduced. This scenario is often called the „counterfactual", because it is disregarding the fact that the regulation has been implemented. In order to realistically estimate the effects of a regulation, costs, benefits and risks under this scenario are compared with the costs, benefits and risks that have arisen as a consequence of the measure.

The range of affected enterprises. Each RIA must clearly state the scope of those enterprises that are affected by the regulation by giving their characteristic features and offering estimation about the size of this group of enterprises in terms of number of firms, turnover and employment. Most RIAs examine a legal rule for which the range of affected companies is difficult to define in statistical terms, because the group of affected companies does not coincide with traditional categories such as region, sector, or the size class of the enterprise. A legal rule may, for example, refer to such enterprises where the employees are family members, pensioners, etc.... In most countries the statistical enumeration of such enterprises is difficult or impossible.

Business impacts, business responses. Ex ante RIA studies should indicate in advance to what extent the affected enterprises will have to change their business processes and products in order to meet the requirements of the regulation and also, how the examined regulation will affect their income and investments. A possible classification of impacts goes as follows:

• Direct impacts. These are the changes in costs and investments that arise as an immediate consequence of complying with the regulation. Changes in administrative activities, company re-organisations, change of products and technologies that are attributable to the regulation are converted into monetary terms and interpreted as the total of the direct costs arising for the enterprise.

• Indirect impacts. As an indirect impact of the measure, companies may gain or lose markets, may increase / decrease their output, may increase or decrease their prices, may experience changes in the productivity and competitiveness. Indirect impacts are calculated by using some quantitative or verbal models, e.g. for determining how compliance costs affect the output prices of the firm, price elasticity calculations may be applied. Costs arising at different points of time have to be converted into a comparable form, e.g. by projecting them to the current time (discounting). Only those changes must be taken into consideration which are clearly caused by the regulation.

• Distributional effects. In many cases the planners of a regulation want to avoid causing disproportionate costs for certain groups of enterprises. In these cases the aim of impact assessment is not only to reveal the overall or the average cost level caused by the regulation, but also the distribution of the costs among the affected industries, consumers, regions (or countries). As a result of the impact assessment, the planned regulation may be rejected if it threatens to impose extensive burdens on certain groups or sectors.

Typically, the above effects are identified for certain case study companies. The sampling strategy of case study companies must ensure that all different types of affected companies should be represented in the sample: small and big, locally owned and multinational, manufacturing and trade, exporting and importing, etc. The overall and the distributional effects of the regulation should be calculated by using some weighing consideration about the absolute sizes of each of these company groups.

Government regulatory and enforcement costs. The government’s costs in connection with a regulation arise



from planning, drafting, implementing and enforcing the regulation. This includes the development and maintenance of those institutions that are necessary to enforce and control compliance. Institution development costs usually include the wages of hiring and training personnel, the costs of legal procedures against non-compliant companies, as well as the costs of supervision and the granting of permits. Here again, only those additional costs have to be taken into consideration, that are attributable to the introduction of the examined regulation.

Costs and benefits for consumers. Besides companies and government agencies, consumers may also become winners or losers to a certain regulation. As a consequence of the measure, product prices may increase or decrease, product safety may increase. The item social welfare loss refers to losses caused due to the behaviour of some consumers who replace one product with another as a consequence of the regulation. For example, a product-specific regulation may motivate some consumers to purchase a specific product which otherwise - had the regulation not been introduced - they would not have had a preference for. In this case the negative effect that affects the consumers has to be calculated not only in terms of costs, but also in terms of decreasing utility.

Social costs. The category of social costs is a summary of the costs mentioned above. It expresses the price of the regulation for the society as a whole. It covers the costs to the enterprises, those to the government, and those to the consumers, too.

Benefits. Compliance with the requirements of regulations may generate positive effects, which are regarded as benefits. Such are, for example, advantageous changes in the field environment protection, food safety, the safety of industrial products or traffic safety. A decrease in compliance costs can also be regarded as a benefit. For example, if the regulation replaces an earlier, less advantageous regulation, then the decrease of compliance costs may increase the willingness of entrepreneurs to invest and innovate and the resulting increase of competitiveness can also be registered as a positive impact. While the cost effects of the regulation can be calculated by standard means, the monetary quantification of the benefits is often impossible in certain regulatory fields (e.g.: the protection of the environment, food safety or traffic safety). Specific procedures have been developed attaching monetary values to risks, which are closely related to calculations used in insurance mathematics.

3.3. Data collection for regulatory impact assessmentData collection is the most costly and most time consuming activity in the course of an impact assessment project. Research design s are usually seriously constrained by the available data and/or by the resources available for data collection. Therefore the planning of data collection must be performed creatively and a readiness to make compromises in order to arrive to a feasible strategy. The range of available data is the major determinant which primarily defines the choice of the analytic model used for the assessment. For assessing and estimating costs, benefits and risks attributable to a regulation, researchers of social science and policy evaluation have revealed a rich body of information sources and have developed a wide range of data collection methods.

Qualitative methods of data collection. On the other hand, in case of complex issues and problems that are difficult to structure, qualitative methods are more suitable for revealing impact mechanisms and for highlighting entrepreneurial and institutional patterns of behaviour.

• Governmental interviews focus groups and workshops. Every impact assessments starts with the questioning of government officials and experts working for the responsible authority submitting the regulation and with the collection of opinions of those experts that work in other agencies affected by the regulation. The aim of these meetings is to reveal the legal and institutional environment of the regulation in question, to examine the conditions of the introduction as well as those of the continuous implementation and consequences of the planned measure. These “what if" questions about the expected impacts can also be asked in the framework of focus groups and workshops as well.

• Entrepreneurial interviews and consultations. In most impact assessment projects the representatives of the enterprises affected by the regulation are asked about the expected effects of the regulation. It is desirable to develop the collection of these opinions into various forms of consultation or interest reconciliation, and to integrate the feedbacks received from the companies into drafting process of the regulation. The forum of these consultations may be a workshop, a conference based on personal participation or an Internet platform. The resulting co-operation of the regulators and the regulated may result in unexpected and innovative, alternative ways of achieving the aim of the regulation. It is possible to communicate with the target group of the regulation both in a direct and in an indirect way. An example for the latter is the reconciliation of



interests with the help of the representatives of a professional association.

Controlling reliability and authenticity. Information gained from interviews with the affected parties of the regulation has to be controlled from the point of view of authenticity and credibility. This means that the compliance costs announced by the companies (a) have to undergo, on the one hand, an internal coherence and plausibility examination and, (b) on the other hand, they have to be compared with already existing statistical evidence, and thirdly, (c) these claims are to be compared with information coming from independent sources. Impact assessment experts must pay special attention to those costs that are due under any regulatory scenario: such cost items should not be classified as regulatory impacts.

Quantitative methods of data collection are based on the availability of a large body of uniform questionnaires or administrative records. Their advantage is that a wide set of enterprises can be addressed in this way and the reliability of the findings can be characterised by statistical indicators.

• Questionnaire-based surveys. In some countries (e.g. Belgium, France, Australia) impact assessments are relying on surveys covering thousands of enterprises. The questionnaires typically inquire about the administrative burdens imposed on enterprises. The European Union regularly obtains direct feedback from businesses on its legislative proposals or initiatives that have a likely impact on companies with the help of the European Business Test Panel (EBTP). This online survey and consulting tool allows policy makers to collect the opinions of around 3.600 companies of different sizes and sectors located in all EU Member States.

• Analysis of administrative records. Impact assessment can rely on data about beneficiaries of some projects or programmes, or, alternatively, on large sets of micro-data created in business surveys. In other research designs of impact assessment, time series of macro level data is used as the empirical basis in order to draw inference to the consequences of a policy intervention.

3.4. Analytical methods of impact assessmentResearchers of impact assessment projects quite often succeed in collecting sufficient data that enables them to perform certain quantitative calculations. This body of data is the basis of the analysis that results in the identification of the impacts. In its simplest form the analytical steps consist of generalisation, summation and comparison. However, impact assessment may rely on more complex analytical methods.

The selection of the appropriate analytic method can be facilitated by studying impact assessments made in other countries of the same regulation or of a regulation that serves similar aims. In particular, RIAs analyzing the introduction of certain pieces of European legislation should rely on the findings of the equivalent RIAs produced previously in other countries being in similar situations. By using these RIAs as benchmarks, previously hidden details of the regulatory environment can be revealed.

The most frequently applied analytical methods can be classified according to the following typology.

(A) Cost-benefit analysis. For many years cost-benefit assessments were the most wide spread impact assessment methods. Their purpose is to quantify, in the financial sense, the positive and negative consequences of interventions, in particular decisions concerning infrastructure development or concerning the implementation of policy programmes.

The calculation goes as follows. The researchers define a time interval in which the impacts of the decision will appear. For the calculation they need two time series to be available: in case of ex ante analysis this is a forecasting exercise which must produce the time series of costs and the time series of benefits that are attributable to the intervention. These costs and benefits must be quantified in monetary terms. The researchers use various heuristic hypotheses and/or mathematical models to estimate the two time series of costs and benefits and the resulting indicators.

• Cash flows. Then they calculate the time series of cash flows (which for every year equals the difference of costs and benefits).

• Pay-back time. Following this, the researchers calculate the duration of the investment being returned.

• Net Present Value. Subsequently the researchers calculate the net present value (NPV) of the investment. The net present value is calculated by discounting the yearly differences of costs and benefits (i.e. the cash flow



stream) with the help of a previously externally determined discount rate. In other words, for every year of the evaluated time interval, following the planned intervention, the values of these yearly cash flows are projected to the present time with the help of an appropriate discount interest rate. 2323 If the net present value is positive, this indicator can be used for justifying the planned intervention.

• Internal Rate of Return. Additionally, impact assessors frequently use the so-called internal rate of return (IRR) of the investment. The internal rate of return is a calculated discount interest rate which if applied, yields a zero net present value to the investment 2424. If the calculated internal rate of return is negative or low the donor may prefer alternative projects, which yield better interim cash flows.

Following this, a sensitivity analysis is carried out. This is the study of how sensitive the predicted outcomes such as costs, benefits, present values and rates of returns are to changes in the assumptions.

The practical realisation of cost-benefit assessment, however, has its limits, because the credibility of the forecasted costs and benefits may be low. Moreover, there are certain impacts which cannot be expressed in monetary values. Although attaching monetary value to saving a life is an everyday practice in the insurance industry, however it is not ethical to make decisions based on a comparison of values of human lives on the one hand, and other assets on the other hand. Similarly, the monetisation of environmental benefits has its limits. Moreover, cost-benefit analysis applies certain models to forecast costs and benefits to occur in future years, and these models are often debatable. For the above reasons, in recent years quantitative cost-benefit analyses are approximated, substituted or enriched by qualitative analyses, e.g. by comparative case studies.

(B) Cost-effectiveness analysis. The problem of monetising benefits can be avoided, if the researchers apply so-called cost-effectiveness analysis. The starting point of this analysis is that while the costs attributable to an intervention can be more or less precisely quantified, benefits cannot. That is why this approach either (a) compares alternative plans that guarantee the same benefits and chooses from among these the cheapest one, or (b) compares various alternatives characterised by the same cost level and chooses the one which promises the most favourable benefits.

(C) Compliance cost analysis. This type of analysis applies algorithms for computing the costs of complying with the regulation for those units or stakeholders which are affected by the legal rule (companies, households, local governments, banks, etc.). Compliance costs include both administrative costs (i.e. information obligations) and investment costs attributable to the regulation (e.g. a filter to avoid pollution). While compliance cost analysis is a relatively straightforward method to quantify the regulatory burdens, its disadvantage is that it does not take into consideration the indirect and long term economic effects of the regulation.

(D) The Standard Cost Model. This method measures a subset of compliance costs, the so-called administrative costs. In 2003 the governments of certain European countries have established an international network for impact assessment in order to focus on the analysis and decrease of company level administrative burdens. The member countries of the network committed themselves to a uniform methodology of administrative burden measurement, the so-called Standard Cost Model (SCM) . The following countries are parts of the network: Austria , Belgium , the Czech Republic, Denmark, the United Kingdom , Estonia, Finland , France, Ireland , Poland, Latvia, Luxembourg , the Netherlands , Hungary , Germany , Norway, Italy and Sweden . The OECD has also entered the network and has applied the method of the Standard Cost Model to rate its member countries from the point of view of the extent of administrative burdens imposed on enterprises, government agencies and citizens. 2525

The Standard Cost Model (SCM) 2626 serves for the quantification of the administrative burdens imposed on enterprises by regulations. This method can be applied for measuring the impacts of a single legal rule or the aggregate effects of a group of regulations. The method clearly defines, which costs of the enterprises are to be regarded as administrative burdens caused by a specific regulation. According to the cost typology of the SCM, a regulation may cause the following costs to the enterprises: direct financial costs, compliance costs and long term structural costs.

• Direct financial costs are normative obligations of the enterprise to pay a certain sum to the enforcing

2323 See e.g. [Török-Papanek 2004].

2424 See e.g. [Török-Papanek 2004].

2525[OECD 2003]

2626[SCM 2005]



authority such as fees, taxes and procedural duties which have to be paid when applying for a permit.

• Substantive compliance costs are investment costs of developing the company in terms of equipment, infrastructure or organisation, in order to satisfy the requirements of a regulation. For example, an environmental regulation may compel the company to install a filter in order to decrease pollution to a new level. Or a labour safety regulation may compel the firm to improve the physical conditions of labour safety.

• Administrative compliance costs arise when the firm must create a documentation or collect data in order to satisfy the requirements of a regulation. Such obligations arise when the firm must manage and administer the maintenance of an equipment which was previously purchased in order to comply by a regulation. Alternatively, some regulations impose the duty of collecting some information on enterprises, the costs of which are classified in the SCM method as administrative compliance costs.

As of the latter item, the Standard Cost Model measures only those administrative costs which an enterprise must pay only for meeting the demands of the examined regulation. Consequently, regular administrative costs necessary for managing the enterprise, which would be parts of company expenditures even if the regulation was removed, are not included in the administrative burdens calculated by the SCM.

While applying the SCM method for administrative costs, the relevant administrative activities are monetised by collecting and calculating the following cost parameters:

• The unit price of an administrative activity can be calculated from the hourly rates paid for this activity within the enterprise, by taking into consideration also the overhead costs. If the administrative activity is outsourced, the hourly fee paid is applied in the formula.

• Duration, the quantity of time devoted to an administrative activity in one firm.

• Frequency of the administrative activity: how often a given administrative activity has to be carried out annually in one company.

• Size of enterprise population (number of enterprises) affected by the regulation.

With the help of the above parameters the aggregate costs of a given administrative activity can be calculated for all affected firms: the administrative costs arising as the consequence of the regulation by applying the SCM formula:

Yearly administrative costs attributable to the regulation =

= Unit Price x Duration x Frequency x Size of affected enterprise population

An example for the application of the SCM formula: an administrative activity takes 3 working hours (duration), the hourly pay of the administrative employees is 10 EUR (unit price). The activity is carried out in 100.000 enterprises (enterprise population), and it is to be done twice a year (frequency). Thus the cost total of the administrative activity will be: 10 x 3 x 100.000 x 2 = 6.000.000 EUR.

To compute the total burden for the enterprises, the above sum should be complemented by (a) the direct financial costs paid by the affected companies and by (b) the substantive investment that is necessary for being in compliance with the regulation, as estimated for the aggregate of the population.

The basic unit of observation of the SCM method is the so-called “normally efficient business". This refers to a typical enterprise within the target group of the regulation, which is doing its administrative activity with an efficiency that is neither much better, nor much worse than what is characteristic for other enterprises within the target group. “Normally efficient businesses" can be defined and their procedures may be revealed by making expert interviews with stakeholders of the affected company population or with experts of regulatory agencies, consultancies, associations of interest representation or NGOs.

(E) Risk analysis characteristically analyses hazards related to safety, health and environmental issues. This is a method to estimate (a) the level, i.e. the extent of disadvantageous consequences of decisions and (b) their probability, i.e. the likelihood of these unfavourable events to occur. In the context of policy analysis, studies applying risk analysis often compare risks under alternative regulative scenarios, one of these scenarios being



the baseline scenario, i.e. that no regulation is issued at all.

(F) Multicriteria analysis is a structured approach aimed at reaching several different aims or satisfying multiple preferences by choosing one option from various decision alternatives. It is supposed that each option will have consequences against various social, environmental or economic criteria. In order to find the optimal choice, a scoring, ranking or weighting must be defined, which expresses the importance or monetary value attached to each of these criteria. As a rule, these scores are determined with the help of expert surveys or stakeholder surveys. This approach may involve subjective elements or expert judgment.

(G) Sectoral and macroeconomic models. The impact mechanism of certain regulations upon the production, turnover and prices of various economic sectors can be expressed by a system ofeconometric equations. Analogously, it may be straightforward to calculate the impact of some regulations on the supply or demand of some production factors (e.g. on labour force). Sectoral and /or macroeconomical models based on such equations may help to calculate the subsequent indirect effects of increasing or decreasing compliance costs related to a planned regulation. Another class of macroeconomic impact assessment models explain the level of GDP of several countries by using several explanatory variables, where some of these variables are characterising the regulatory behaviour of the government.

3.5. Simplified regulatory impact assessmentsThe Principle of Proportionate Analysis means that no resources should be wasted on RIAs, that deal with less relevant regulations or where the regulation has insignificant impacts. On the other hand, all regulations with significant impacts should be accompanied / preceded by a RIA. To achieve this is the joint responsibility of the Government launching the RIA investigation and of the authors of the RIA.

Therefore, before the person responsible for conducting the RIA examines the steps involved in the process, the following questions should be answered:

• How many options should be explored?

• How deep should be the analysis of the costs, benefits, risks and other impacts such as competition or distributional effects?

• How many people have to be consulted?

• How much care should be given to the issues of compliance and enforcement?

• How stringent should be the review mechanism?

This is where the Principle of Proportionate Analysis comes into effect. The greater the importance or significance of the proposal, the more analysis will be required. The more significant the impacts are likely to be, the deeper the analysis should be, and as a consequence, the wider and costlier will be the data collection efforts.

This means that the amount of information collected, the number of stakeholders consulted and the depth of quantification of impacts will require greater effort. If the proposal concerned has little impact on businesses in general but has major implications for one particular sector, it should still be considered significant. There is no absolute division between what is significant and insignificant for the purposes of RIA. Instead, the authors of the RIA must make a judgement, must evaluate, how significant the introduction of the regulation is going to be. The most significant proposals require a collection of more data, a wider consultation and the involvement of more sophisticated analytic methods in assessing costs, benefits, risks and other impacts such as competition or distributional effects.

What level of analysis is proportionate? As a rule, the RIA should provide decision makers with fact-based evidence to support each proposed policy option for a specified problem. This evidence must cover the costs, benefits and other impacts of the option and have enough detail to support the proposal as it is debated in the legislative institutions of the state or of the entity. It is up to the person responsible for conducting the RIA, to decide on the proportionate level of analysis. It is important to remember that defining the proportionate level of analysis is not necessarily a once-off decision. It should be reviewed in light of stakeholder input as the person responsible for conducting the RIA goes through the process.



If the expected impacts of the regulation are significant, it is advisable to go beyond the level of an Initial RIA and to prepare a Full RIA. While Initial RIAs are short documents and are usually prepared by staff of public administration bodies, most full RIAs are prepared by external experts: consulting or research organisations. In such cases the task must be outsourced to such organisations by the Government agency which needs the RIA.

Depending on the depth of the research efforts invested, one can speak about (a) initial (“preliminary", “simplified", “basic") RIA, (b) partial RIA and (c) full (“in-depth" or “extended") RIA.

Several administrative cultures of RIA have successfully institutionalised the preparation of preliminary or simplified regulatory impact assessments in order to obtain feedback about the expected impacts of a planned regulation at the earliest phase of the policy process. The role of these short documents is

• To formulate preliminary regulatory and non-regulatory options for tackling the problem

• To provide summary information about the legal and institutional background and about the likely impacts of the proposal

• To inform public administration instances, drafters and external stakeholders at the early stage of the policy process

• To highlight availability or lack of data for estimating the impacts

• To highlight if there is a need for conducting a deeper impact assessment, or, on the contrary, there is no such a need, because the likely impacts of the proposal will be negligible.

• To clarify the various stages of the ensuing policy process.

Regarding the content, scope and detail of preliminary or basic RIAs, public administrations have consciously abstained from setting too high standards, in order to facilitate the timely production of these documents. Most guidelines of simplified RIA preparation require the analysts to respond to a list of maximum 10 questions and ask only for some summary information about the size of groups affected by the proposal and about the impacts themselves.

A recent survey 2727 among 10 EU member states has formulated the following question to public administrations: “In your country is there a distinction between a summary and an extended, in-depth impact assessment?" The notion of “analysis depth" appears in the British, the Italian and the Polish RIA practice as well.

Table 4.1. Box 8.

United Kingdom : the three stages of developing a RIA study

The relevant UK methodology for RIA preparation a clearly specifies the various levels of depth of RIA documents. These levels are called “initial", "partial", and "full / final" RIA. They show significant differences in terms of (a) thereliability of impact estimations, (b) the extent of consultation on which the statements are based, and (c) the specific nature of the recommendations.

The characteristic features of these three stages are as follows:

• An initial RIA should inform and ideally accompany the draft regulation to the particular government department. An initial RIA should inform requests for departmental ministerial agreement to a proposal. It should be proportionate to the impact of a proposal and can consist of a rough and ready analysis based on what is already known. The document should be based on existing institutional knowledge and on early/informal consultation. It can consist of a rough and ready analysis based on what the drafters already know. It should include the drafter’s best estimates of the possible risks, benefits and costs, and should identify areas where more information is needed. The contents of an initial RIA are as follows: Provide a clear statement of the policy objectives and the issue. Describe and, where possible, quantify the scale of the

2727[EC 2004b]



United Kingdom : the three stages of developing a RIA study

risk (i.e. the problem to be addressed). Identify regulatory and non-regulatory alternative options, including do nothing/base case. Consider the pros and cons of each option and the fit with existing requirements on the relevant sector. Identify who is affected, including business sectors affected. Identify any issues of equity and fairness. Examine what is already known about the costs and benefits. Identify any potential unintended consequences. Identify distributional impacts. Try to identify markets that may be affected and flag up any potential competition issues. Consider how to secure compliance and whether a review of how it is working is required.

• A partial RIA builds on the initial RIA. It should be submitted with any proposal needing collective agreement from the Prime Minister’s Office, the Cabinet of Ministers or the associated committees. It must also accompany the formal consultation process with the stakeholders. It includes refined policy options on regulation, compliance, monitoring, cost and benefit estimates and risk analysis. An assessment of the impacts on competition and on small firms should be included.

• A full/final RIA builds upon the analysis in the partial RIA which should be updated in the light of consultation and further information and analysis. It should be submitted with clear recommendations to the Ministers and should accompany legislation when it is presented to Parliament. It should also be placed on the website of the responsible Ministry. It must include a signed declaration from the relevant minister that he/she is satisfied that the benefits of a proposal outweigh the costs. The full/final RIA differs from the partial RIA in the first place not in its structure but in the depth and reliability of the information that it is based on, and in the explicit nature of its recommendations. These studies often use some economic / econometric model to forecast the impacts of the planned regulation. The complexity of these studies justify that they should be outsourced to some research or consultancy organisation outside the public administration.

a[RIU UK 2003]

4. Projects and case studies of Regulatory Impact Assessment4.1. Regulatory Impact Assessment in the central organisations of the EUBetter regulation. An important aim of the regulatory policy of the European Union is to develop and promote the methods of good governance among the institutions of the Community and among Member States. Governance concerns the way in which governments use their powers. It is the process of making and implementing decisions in and by public institutions while conducting public affairs, managing public resources, and enforcing regulations and involving stakeholders such as companies, consumers and public authorities. A well-defined component of good governance is the policy area aiming at “better regulation", which consists of actions to generally improve the regulatory environment, to reduce bureaucratic burdens, to design better laws for consumers and businesses. 2828

Regulations are not the only means of implementing policies. In fact, in the European practice of RIA it is one of the major tasks of the analysis to clarify whether the regulation submitted for RIA is needed at all. Besides and instead of issuing new regulations or modifying existing rules, governments may resort to subsidy instruments or to the so-called open methods of coordination which also play a considerable role in this respect. The latter governance strategy consists of (a) collecting good practices of public policies, and best examples of how companies comply with those policies, (b) comparing and analysing them at consultative meetings 2929 in order to find benchmarks (c) disseminating these benchmarks and supporting their applications with the help of subsidy schemes and (d) following the results of the previous efforts in the member states. 3030

2828[EC 2006c]

2929E. g.: in case of SMEs [EC 2006b]

3030[EC 2001]



Small- and medium size enterprises in the EU face an increasingly complicated legal and administrative environment, which has reduced their competitiveness and in some cases has even rendered them relatively weak and vulnerable in the international competition [bib_38]. Therefore, the EU has set the aim of improving the quality of regulations affecting SMEs by way of centrally supported and coordinated actions, including the institutionalisation of regulatory impact assessments and consultations with the concerned interest groups in order to simplify the body of rules affecting SMEs. 3131

Regulatory Impact Assessment, as a tool for improving regulations, has been increasingly applied during the past two decades both in the central organisations of the European Union and in its member states [ bib_39]. Since the 1980s a continuous effort can be observed to assess the impacts of European legislation on small and medium sized companies. 3232 Already in the 90s the impact assessments of several hundreds of drafted directives were issued in publications on paper and later on the Internet. 3333 In practice this means that drafts of regulations are regularly submitted to be tested according to pre-defined criteria.

Involvement of consultancies. In Europe several research and consulting institutions have specialised for the calculation of the costs of following up on regulatory rules. Certain working processes of RIA have been frequently outsourced from the public sphere into the private sphere and the involved private think tanks have brought new methods and extra knowledge to the relevant research. Impact assessments outsourced to external consulting firms by the Directorate General (D.G.) Enterprise of the European Committee usually take 9-12 months to be completed and may cost between EUR 50.000 and 400.000.

Involvement of stakeholders. In addition to relying on expert analyses, regulators frequently meet with representatives of the regulated stakeholders (companies and consumers) on formal and informal interest reconciliation meetings. The task of these consultations is to examine (a) whether the regulation is implemented by the appropriate level of public administration (subsidiarity), and (b) whether the burdens imposed by the regulators on those who are being regulated are not overly exaggerated (proportionality).

Up to 2001, the central organisations of the European Union have prepared impact assessments about planned and issued regulations by using the following criteria:

• Impact on budgets. This form of impact assessment examines the fiscal consequences of the measures.

• Environmental impact assessments. The purpose of these assessments is to reveal the influence of the measures taken by the Union on the environment. Its methodology has been laid down by the Environmental Directorate, but environmental impact assessments have to be carried out by all the other directorates of the EU.

• Business impact assessments. The purpose of these impact assessments is to issue legal rules that burden enterprises within the Union to a reasonable extent only.

Around the year 2000, the demand towards clearly shaped, institutionalised impact assessment systems was growing stronger in the European Union . 3434 Critics of the existing and operating budgetary, environmental and business impact assessment systems have emphasised that these systems had little connection with each other, had become inflexible and bureaucratic. Consequently, the need has been formulated to simplify and to standardise these systems. Moreover, it was suggested that impact assessments should be made mandatory in relation to every important regulatory initiative of the Union, including the ex post impact assessment of each essential legal rule every five year. Regarding impact criteria, the recommendation was made to extend the scope of impact assessments to social impacts as well.

Parallel to the demand for an extended character, requirements concerning the quality of the assessments were also growing. It was claimed that impact assessments should become an organic element of the political process, should reduce the risks of regulatory failures, and should contribute to the transparency of the regulatory reform which aims to establish good governance. Mandatory periodic consultations with entrepreneurial interest groups should be only the first step in introducing an overall regulatory management policy reform in the long run.

3131[EC 2005a]

3232It has become institutionalised under the following names: Regulatory Impact Assessment or Business Impact Assessment (in English) or Fiche d’Impact (in French)3333[EC 1997]

3434[EPC 2001]



RIA in the context of competitiveness . These critical judgements have become more intensified following the Lisbon strategy of 2000, which has laid special emphasis on the competitiveness of enterprises. Against this background, several high level forums of the EU urged the European Committee to address the impact of administrative burdens on competitiveness with the help of impact assessments.

In particular, it has become particularly important to dynamically reduce compliance costs imposed on enterprises by EU regulations. It was understood that if these costs remain high, productivity cannot be improved and the economy cannot grow because the compliance with bureaucratic regulations absorbs the energies and resources of enterprises. Moreover, these compliance costs are often passed over to consumers, thus interfering with the distribution of incomes which further raises questions of fairness. The high level of these costs also reduces the international competitiveness of European companies, especially if overseas competitors can operate in an environment of lower regulatory costs. Extremely high regulatory costs can undermine the success and the rationale of the regulation itself, since they compel many companies to break the rules, with the consequence that law obeying enterprises which dutifully follow the regulations suffer from further competitive disadvantages.

In 2002, following the evaluation of several experimental impact assessment projects, a new method of impact assessment has been introduced in the European Union . 3535 Various Directorates of the Committee of the European Union are now obliged to carry out impact assessments in accordance with a uniform framework and uniform methods. A new framework has been created which allows examining the regulations against the established criteria of economic, social and environmental consequences and confronting the draft rule with the articulated interests of the interest groups affected by the planned regulation. According to the new regulation of impact assessment, first a preliminary impact assessment of the main suggestions is prepared, and an extended, deeper impact assessment is carried out only if the results of the preliminary impact assessment indicate that more detailed assessment is necessary.

In January 2004 the financial ministers of Ireland , the Netherlands , Luxembourg , Great Britain, Austria and Finland put forward a suggestion concerning the further reform of the regulatory system of the European Union . 3636 In this document they urged the elaboration of a standard European method which would be suitable for measuring the administrative burdens caused by regulations. They suggested that, following 2005, the EU should carry out the impact assessment of all new directives by means of a common, approved methodology in order to simplify regulations in a systematic and institutionalised way, both on the level of the European Union as well as in its member states.

Despite continuous reforms, the impact assessment system of the EU is still changing. In 2006, on behalf of the European Committee, an independent board evaluated and summarised the observations and experiences of the Regulatory Impact Assessments having been carried out on behalf of the central organisations of the European Union since 2002, i.e. since the introduction of the new system of impact assessments. 3737 The most important deficiencies were found to be the following:

• It is difficult to acquire data for the completion of the RIA.

• The quantification of indirect impacts and non-evident benefits is rather problematic.

• For the completion of high quality RIAs, expertise and resources are often missing.

• RIAs frequently define alternative policy options that are not adequate or are not realistically evaluated.

• Quite often the procedure of the RIA is not clear enough.

• Consultation with the interest groups is often abandoned.

• Often there is no appropriate coordination between the political decision makers and the staff responsible for implementing the measures.

• It often occurs that no quality control is being carried out for RIA activities.

• It often occurs that a RIA will be prepared only in the latest phase of the decision making process.

3535[EC 2002a] , [EC 2002b] and [EC 2005d]

3636[EC 2004a]

3737[Renda 2006]



Despite ongoing reforms, in 2006 there was still no centralised EU institution for impact assessments, and most of the member states did not have an appointed and adequately authorised organ for the execution of impact assessments. However, a further institutionalisation of regulatory impact assessment is necessary, if the EU wants to reach its declared aims of reducing bureaucratic burdens. In 2006, the European Committee issued a declaration claiming that up to 2012, the administrative burdens of the enterprises caused by EU and member states regulations would be reduced by 25%. According to model calculations, this development would result in an expected 1,5 % increase of the GDP as well as increased investments of up to 150 billion Euros which, in turn would create many new workplaces. 3838

4.2. Regulatory Impact Assessment in Western European member states of the EUThe Member States of the European Union have taken various measures to introduce RIA systems in their regulatory policies. The pioneer of this trend is the government of the UK, but all Western European states have taken some kind of initiatives in order to obtain feedback about the quality of regulatory activity. By doing so, governments have taken steps to develop the dialogue between regulators and those affected by the regulations, to improve the structure of impact assessment methodologies, and to integrate the use of RIAs into the rule making process. Guide books have been published in a number of EU member states on how to prepare regulatory impact assessments 3939 , with special respect to the assessment of the impacts exerted on small enterprises. Several Member States of the EU have launched training projects about impact assessment for public servants.

RIA guides invariably point out that the authenticity of the assessment depends on whether it is based on reliable data that are suitable for making robust, possibly quantitative inferences that even non-professionals understand well. The guides recommend the quantification of the costs and benefits in monetary units wherever possible and the application of rigorous methods. In cases when the benefits originating from the introduction of the regulation cannot be expressed in terms of money, quantification is still desired, e.g. by making statements about the expected number of lives saved, or about expected changes in the volume of pollution, etc. In cases where a complete monetisation of the expected impacts is not possible, evaluators should refer to the most significant cost or benefit items that will change as a result of the regulation.

4.2.1. United Kingdom

In Europe the public administration of the United Kingdom shows the best example for a continuous, institutionalised quality control of regulatory activities. The UK government has been preparing regulatory impact assessments since 1988. 4040 The work is coordinated by the so-called Better Regulation Executive, whose office is part of the Prime Minister’s Office. Its task is to supervise the quality of the regulatory activity of the government, to improve regulatory policy, to develop the methodological devices needed for preparing clear and consistent RIAs and for applying them throughout the policy process.

Impact assessments are carried out not only concerning UK national regulations, but also concerning European Union legal rules affecting public bodies, private organisations (such as small enterprises) or non-governmental organisations.

During the first decade of RIA activity in the U.K., most regulatory impact assessments have analysed regulatory proposals about small- and medium size enterprises 4141 and about competition issues. In recent years, impact assessment has been extended to the policy areas of health care, sustainable development, the enforcement of equal opportunity rights and regional development. Moreover, impact assessments have been used as methodological instruments deregulation, that is, for identifying unnecessary legal rules and for facilitating the process of their withdrawal.

The guidelines of RIA activity have highlighted that efforts invested in regulatory impact assessments should be proportional to the likely effects of the proposals. In those special cases, when the expected effects of a regulation are negligible, there is no need for a detailed RIA. The assessments should pay special attention to the key indicators of compliance cost.

3838[EC 2006a]

3939[BRE 2007]

4040The terms Regulatory Impact Analysis or RIA are also used.

4141[BRE 2005]



An evaluation of recent RIA activities in the UK. Impact assessment activities of the UK government are regularly evaluated by independent experts. In 2006 an evaluation prepared by the National Audit Office 4242

covered the work of four government departments, namely, the Department for Culture, Media and Sport (DCMS), the Department for Trade and Industry (DTI), the Home Office, and that of the Department for Transport (DfT).

The above mentioned Report of the National Audit Office has produced the following findings. In 2005 various organisations of the British government completed ca. 200 regulatory impact assessments to examine the effects of a number of bills (legal drafts). Government departments routinely employ external experts with background in economics and statistics to prepare RIAs. The responsible public servants, in particular the “drafters" of regulations share the relevant legal, administrative and statistical information by using the Intranet network of the particular government department. The final form of the legal rule that is signed by the Minister of that government department which has submitted the proposal must clearly refer to the results of the RIA by stating that “the benefits of the regulation justify its costs". This statement must be justified with the help of the quantitative and qualitative techniques of RIA or, if this is not possible, then the remaining uncertainties must be clearly indicated.

Most legal rules affecting companies are issued by the Department for Trade and Industry. Among them, the above mentioned Report has evaluated the Regulatory Impact Assessments of the following regulations:

• A modification of legal rules concerning labour courts,

• A law on labour relations issued in 2004,

• A law about increasing the minimum wage

and others.

The above mentioned Report of the National Audit Office was based on a large sample of already completed regulatory impact assessments, and all of which have been evaluated against various criteria. A selection of the most important evaluation criteria goes as follows:

• Did the RIA examine all relevant options?

• Did the RIA examine, whether the purposes of the regulation were clear, the intervention was rational?

• Was the RIA conforming to the methodological directives issued by the Better Regulation Unit?

• Did the RIA apply innovative techniques?

• Did the RIA apply quantitative and qualitative assessments, including cost-benefit analyses and sensitivity analyses?

• To what depth has the RIA examined the impacts of the regulation, its enforcement and compliance?

• Was the „afterlife" of the regulation followed in the framework of so-called monitoring or ex post evaluation exercises?

• Were the competition distorting or competition influencing effects of the regulation examined?

• Have consultations taken place with affected stakeholders and have their opinions been taken into consideration in the RIA?

Based on the above findings, the Report has evaluated the individual departments by using the following criteria:

• According to the depth of the administrative culture of impact assessment. Here the evaluators may assess the proactive behaviour of the organisation, as measured e.g. by the time elapsing from the preparation of impact assessment to the legal rule coming into force;

• According to how deeply impact assessments are integrated into political process;

4242[Bourn 2006]



• According to how much stress is laid upon the exploration of the economic consequences of the legal rule.

The evaluators have found that a wide range of RIAs have only met the requirements formally: such analyses were considered to be of a weak quality. On the other hand, a well defined group of RIAs were profoundly integrated into the process of decision making: such RIAs were highly appreciated. In between the two extremes were those, which proved to be useful information but their role in improving regulatory quality was limited.

According to the summarised evaluation of the National Audit Office, the quality of impact assessment activity was rather disappointing in that year. Although the culture of impact assessment preparation has been found highly developed in the UK, the surveyed analyses were, in many cases, not convincing enough, and their presentation was often not quite clear. RIAs were suitable for questioning the necessity of a regulation in certain individual cases only. Apart from this, however, most RIAs have played a very valuable communicative role and have enhanced the clarity of decision making in the particular government departments. RIAs were often prepared too late, so that they were often not in compliance with the political intentions.

The evaluators of the National Audit Office have recommended that RIAs should concentrate more on providing statements about expected economic impacts, instead of just producing statements of purely administrative character. The analyses should more effectively be integrated in the processes of decision-making; they should more clearly reveal the alternatives faced by politics. RIAs should, to a larger extent, rely upon the knowledge and work of external experts and these experts should take bolder initiatives when revealing alternative courses of action for public servants.

4.2.2. Netherlands

In the Netherlands regulatory impact assessments have been carried out since 1985, both for laws and decrees. Impact assessments have been carried out in every government department, an activity which until 1994 was co-ordinated by the Ministry of Justice and after that year - as the interdependence between regulatory quality and competitiveness was increasingly recognised - it became assigned to the Department of Economy.

In the Netherlands , during the first years of impact assessment activity, the major methodological tools were checklists, which were flexibly applied in order to assess the quality of regulations against various criteria. 4343

From 1993 to 2000 the impact assessment method MISTRAL was used, an approach which is focusing at revealing the compliance costs of the regulation but paying not enough attention to the benefits of the measures. The data collecting method of the MISTRAL was based on taking representative samples from paper-based, telephonic, personal or electronic interactions between enterprises and the authorities. These interactions were then assessed in order to determine their demand in terms of resources such as time and money. In case of regulations which have compelled the entrepreneurs to turn to consultants or accountants, the price of such services were interpreted as regulatory burdens and added to the previous items.

MISTRAL was used for pointing out that in the Netherlands, between 1993 and 1998, the administrative burdens imposed on small enterprises by regulations, as expressed in monetary terms, were increased by almost one quarter. Based on these measurement results, the government has undertaken important steps. An important measure was that the company databases used by government agencies were merged, resulting in a single database used by every public authority. Moreover, law enforcing agencies were forbidden to request enterprises to provide data already existing in this database. These authorities had to justify their demands when requesting data from companies.

In 2000 in the Netherlands the internationally harmonised Standard Cost Model (SCM) has replaced the MISTRAL impact assessment method. The new procedure has pointed out compliance costs of regulations, but also tried to take into consideration the benefits of the measure as well.

4.2.3. Belgium

In Belgium a legal rule, introduced in 1988, established a government office (ASA) which was responsible for the simplification of administrative procedures. This agency annually surveys the existing administrative procedures introduced by authorities for companies and households, and measures the time demanded for compliance, as well as the complexity of compliance activity. The arising burdens will be added up and weighed according to the number of those enterprises and households which are affected by the given administrative

4343 [EC 2004b]



procedure, thus getting an index which reflects the total load of administrative burdens.

4.2.4. Germany

In Germany , a uniform work flow was introduced in 2000 for the federal ministries, prescribing that every law and decree must be justified with the help of a regulatory impact assessment (Gesetzesfolgenabschätzung, GFA). These assessments must be structured according to a previously defined template. 4444 However, as the experiences of the first few years have shown, the initiative did not work out well. The public servants working in the relevant ministries were very critical towards the introduction of GFA and have expressed their opinion that the standard methodology was overcomplicated. Regarding the regions of Germany (Bundesländer), although the legal institution of impact assessment has appeared in most departments of the regional governments, until 2005 only a few sample projects of regulatory impact assessment were implemented altogether.

4.2.5. Sweden

In Sweden it was the SimPlex rule of 1998 which first announced that government offices were obliged to make an impact assessment in each case when putting forward a new legal rule or a modification of an existing one. Based on the methodology attached to the rule, the institution issuing the regulation was supposed to answer 12 questions, the majority of which being connected with the expected influence exerted on small- and medium size enterprises.

Since 2002 the Central Chamber of Industry and Commerce of the country evaluates the performance of impact assessment activity and, connected with this, the regulatory environment of enterprises. The 2006 report of the Chamber on regulatory activity has contained a sharp criticism of the government. 4545 According to this report, despite the ambitious plans concerning the simplification of legal rules, the administrative burdens of SMEs have increased in Sweden , while the general quality of regulatory impact assessments remained steadily low.

4.3. Regulatory Impact Assessment in Central and Eastern EuropeFrom the middle of the 90s the governments and the legislative bodies of every Central and Eastern Europe an country have worked on harmonising their legal systems with the collection of legal rules of the European Union , called "Acquis Communautaire". In each potential member country of the EU, plans were prepared for each policy area to introduce the respective European Union regulations. These plans have defined the government agencies responsible for drafting and implementing the necessary modifications and for this activity deadlines were set. These activities were guided by various legal provisions. 4646

For the small- and medium sized enterprises of the candidate countries of the EU, the Europeanising reform of the legal system was important not only for the purpose of operating in a more up-to-date regulatory environment. 4747 Even more importantly, export-oriented enterprises increasingly felt the limitation that their goods and services were allowed to the European market only if these products met the security, health, environmental and other requirements as defined by the rules of the European Union. One of the tasks of legal harmonisation was to develop the legal framework for the enforcement of these rules.

In most candidate countries – including Hungary , Lithuania and later Croatia - the work of legal harmonisation was accompanied by the preparation of impact assessments. The procedure of impact assessments did not follow a uniform methodology or genre, rather, the approach of this applied research has derived methodological ideas (a) partly from the impact assessment activity of the EU operating since 1986 and the resulting „Fiche d’impact" documents, and (b) partly from the experiences of the Better Regulation Office run by the Government of the U.K. since 1988.

In the candidate countries in many cases these impact assessments have considerably contributed to working out a system of argumentation to be applied at the accession negotiations that were preceding the joining of these countries to the EU. Typically, while developing the impact assessment documents, national governments were

4444Gemeinsame Geschäftsordnung der Bundesministerien (GGO), see [Veit 2005].

4545[BSIC 2006]

4646 E.g. in Hungary the 3 years’ plan for the harmonisation of legal rules has been initiated by the 2403/1995 Decree of the Government.

4747 [Tibor 2006]



receiving pressure from two sides: (a) on the one hand, from the side of the organisations representing the interests of those stakeholders (e.g. companies) that were affected by the regulations to be modified and (b) on the other hand from the negotiating partners of the European Union whose task was to facilitate and accelerate the approximation of legal rules, to facilitate and to control their implementation.

Hungary offers an example of how the strategy of impact assessment was devised. In this country the government departments (ministries) in charge of preparing the approximation of legal rules were not in the position of financing large scale detailed impact assessment projects. In fact, their possibilities in this respect were rather limited. The lack of sufficient resources has led to the implementation of relatively cheap impact assessment strategies: government departments have decided either to apply the “comprehensive" or the „in-depth" approach.

“Wide-not-deep" approach. Government departments applying this approach have produced impact assessments for the legal approximation of every regulation of their policy area. However, the source of these impact assessment reports was a meta-research based on the secondary usage of available materials, i.e. existing analyses made previously for other purposes were analysed in the context of legal harmonisation.

Table 4.2. Box 9.

Legal approximation in the field of environment protection

In 1997 an impact assessment project of the Ministry of Environment Protection of Hungary has followed a “comprehensive" strategy [bib_40]. The underlying research was outsourced to an external consultant company.

The consultants have collected, first of all, several hundreds of Hungarian legal rules belonging to the jurisprudence of the Ministry and have associated these Hungarian legal provisions with the corresponding European Union legal sources in environment protection. Following this, the consultants have collected all those applied research results from the preceding decade which have analysed the impacts of the policy pursued in the field of the regulation in question. Furthermore, the consultants have made interviews with all relevant experts and case workers in the Ministry, including those responsible for enforcing the existing regulations and those responsible for international relations. Based on these materials and findings, the consultants have prepared an impact assessment report, which was structured according to the areas to be regulated by the Ministry, i.e. the protection of air, water, soil, the management of waste, wastewater, noise, etc.

Due to the fact that the financial resources allocated for the research were limited, the consultants have made only a secondary analysis of the already existing several hundreds of impact documents, and no attempt was made to meet the representatives of those stakeholders that were affected by the regulations, i.e. the enterprises, the local governments and other institutions did not have opportunity to tell their opinion about the influence the environmental regulation exerted on them,.

„In-depth" strategy. An alternative strategy of impact assessment of legal approximation goes as follows. Experts of the regulating agency choose a typical European directive under their jurisdiction, which has been designated to be transposed into national legislation. Researchers perform a thorough, detailed examination of its expected impacts and draw conclusions to a wider group of directives, in order to learn wider ranging lessons referring to a whole policy area or to the legal and institutional environment of the concerned sector. Such examinations are based on a wide ranging collection of primary data, which may involve a questionnaire based survey, stakeholder meetings and a series of interviews with a relatively wide circle of the representatives of the regulating institutions and the regulated enterprises. Such projects are suitable for the analysis of the legal, institutional, and economical aspects of the act of legal harmonisation.

Table 4.3. Box 10.

Impact assessment of the EU Electromagnetic Compatibility Directive in Hungary

The „in-depth" strategy was followed, for example, by the Ministry of Industry and Trade, in 1996, when this government agency examined the potential impacts of a particular European technical regulation to be introduced in Hungary . The Ministry had the study prepared as a „pilot project" which means that, on the one



Impact assessment of the EU Electromagnetic Compatibility Directive in Hungary

hand, on the basis of the impacts of the specific directive they made conclusions concerning the effects of the introduction of other EU directives, and, on the other hand, on the basis of the methodological experiences resulting from this specific project, they intended to carry out other impact assessments, too.

In this project, the so-called EMC (Electromagnetic Compatibility Directive) a was selected to represent a much wider regulatory area, the so-called New Approach technical regulations of the EU. The EMC directive was selected because its introduction affected the whole system of technical regulation of product markets, including standardisation, conformity assessment, accreditation and metrology. Moreover, the regulation affects a wide range of enterprises [bib_41].

In Hungary as in all post-socialist countries, the implementation of the New Approach technical regulations of the EU have put an end to the situation whereby the safety and environmental compatibility of every electric and electronic product had to be checked and testified by independent accredited conformity assessment bodies. Companies have obtained the right to assess the conformity of their low voltage electric products by themselves. The impact assessment has touched upon sensitive points, because the regulation to be introduced has abolished the monopolistic position of designated bodies such as quality control institutes or laboratories. At the time when the survey was made, these organisations were still owned by the state, but in the course of the coming decade they all became privatised.

The Ministry entrusted an external consultant company to prepare a detailed impact assessment about the introduction of the EMC Directive into Hungarian law [bib_42]. The method of the research was defined by the sample selection instructions and by determining the interview outlines both for interviews to be made with regulators and with the regulated companies.

The success of the research was critically dependent on the selection of the subjects – i.e. the stakeholders - to be interviewed with the help of deep interviews. The survey has covered a wide range of those enterprises of the electronic, electrotechnical and precision engineering sector that have been affected by the planned introduction of the legal rule. Most of these enterprises were successor companies of former large state owned companies that had split into privately owned small or medium sized enterprises during the decade preceding the research, some of them joint ventures.

The guiding principle of the interviews was the same in every case. First, interviewers had to explain the planned modification of the legal rule. After that, the respondents – most of them quality managers of companies - were asked to explain, how the operation, the cost structure and the competitiveness of their enterprise would change if the regulation was introduced.

The main finding of the survey was that the reform of product conformity assessment has affected smaller businesses to a smaller extent, but larger enterprises, exporting firms and enterprises in foreign ownership were affected advantageously.a Electromagnetic compatibility (EMC) is a requirement, demanding that there should be no disturbing electromagnetic effects of electromagnetic appliances on their environment, i.e. on other appliances or on humans.. Its regulation is supported by the EU Council Directive 89/336/EEC of 3 May 1989 on the approximation of the laws of the Member States relating to electromagnetic compability (EMC directive)

Table 4.4. Box 11.

Impact assessment of the EU Low Voltage Directive in Lithuania a

The "in-depth" strategy of impact assessment was used by the Lithuanian government in 2000 when it has initiated a research on the potential institutional and sectoral impacts of introducing European product safety technical regulations in national law.

The pilot study was prepared by external consultants for the Committee for European Integration of the Lithuanian government [bib_43]. The research has revealed what would be the potential impacts of introducing the technical regulations of the EU on Lithuanian producers of low voltage appliances, on organisations performing conformity assessment and on government agencies enforcing technical regulations.



Impact assessment of the EU Low Voltage Directive in Lithuania a

The substantive results of the study have outlined the expected institutional and economic impacts of introducing the European Union’s Low Voltage Directive into Lithuania. The methodological and didactical result of the project was that it gave a blueprint of preparing impact assessment reports for Lithuanian government departments which were responsible for adopting a wide range of European legal rules. The study made recommendations for sampling procedures, for the compilation of questionnaires and defined templates for the optimal choice of report structure, i.e. for the subtitles of the final study. b

In a subsequent methodological paper suggestions were made for revealing the impacts of the entire system of legal harmonisation on one single individual enterprise [bib_44].a The Low Voltage Directive (LVD) 2006/95/EC creates a legal environment for the secure use of low voltage appliances.b At the time of writing this study, the impact assessment of law harmonization of about hundred concrete legal rules and political spheres can be downloaded from the Lithuanian website on European integration in English:

http://www.euro.is.lt/old/showtopitems.php?TopMenuID=165&LangID=2

4.3.1. Impact assessment culture in Hungary

Since 1989 various Hungarian government administrations have made repeated efforts for reforming the regulatory environment. 4848 Regulatory Impact Assessment was one of the tools that were widely recommended for improving the legal environment, but the method was applied only in several isolated projects or short lived campaigns. 4949

For several years, while Hungary was a candidate country of the EU, the efforts to harmonise Hungarian regulations with those of the EU gave a significant impetus to assess the impacts of legal harmonisation 5050. During the 1990s government decrees have been issued which have rendered the performance of ex ante and ex post Regulatory Impact Assessments mandatory for public administration agencies. Other regulations have defined the methodology of regulatory impact assessments and have declared that impact assessments must be accompanied by consultations with the representatives of affected companies and citizen groups. 5151 However, these decrees were never consistently enforced or implemented due to insufficient institutional conditions.

The initiatives of the Ministry of Justice. In Hungary it is the Ministry of Justice which is responsible for regulatory policy, for the simplification of the regulatory framework and for the quality of legal rules. In 2002, within the Ministry of Justice, for a limited period of time, a department was formed, responsible for impact assessments and deregulation. This department has created training materials for public officials 5252 and implemented pilot projects which have assessed the impact of specific legal rules. 5353

The initiatives of the Ministry for Economy. The Ministry for Economy and its legal successors are responsible for creating stable, transparent and predictable regulatory framework and institutional conditions for the operation of companies. The reduction of administrative burdens was one of the most important aims of Hungarian small and medium sized business development policy 5454 since the basic political changes of 1989 following the transformation of the socialist system into a functioning market economy [bib_44]. Hungarian SME development policy has been harmonised with the respective European policies. The principles of this policy were laid down in several Hungarian laws and international documents, e.g. in the influential document entitled „A European Charter for Small Enterprises" which in 2002 has been signed among other EU Member States by the Hungarian government as well. 5555. In various declarations the government has obliged itself to shape the regulatory environment of SMEs in a way that transaction costs can be reduced to a low level both for

4848 [Orbán – Kovácsy 2005]

4949 [OECD 2005c]

5050 [Binning - Futo 1997]

5151 The legal rule of 1987/XI on legislation and the governmental decision of 1994/1008 (Sept.,20) and the governmental orders of 1999/1052 and 2005/1080 (July, 27), 8001/2006 (Jan., 30) IM bullettin.5252[Kovácsy – Orbán – Ovseiko 2004]

5353An example for the impact assessment of an individual legal rule: [Feiler and others 2004]

5454E.g. proclaimed in the „Széchenyi" Enterprise Developing Program. Announced: 1213/2002 (23 Dec.) Government decision.

5555[EC 2000]



transactions between companies and between companies and government agencies 5656, 5757. It is the level of these transaction costs that are central indicators of regulatory impact assessments. Finally, the law concerning small- and medium size enterprises has also made the performance of impact assessments compulsory in cases of modifications of legal rules which affect SMEs. 5858

Consultation. Since the political changes of 1989 interest reconciliation between regulators, consumers, entrepreneurial groups and other stakeholders was regarded as an organic part of the Hungarian legislative process [bib_46] However, the depth and quality of consultations about specific regulations and policies varies widely, depending on which government agency enters into consultation with entrepreneurial interest groups.

One of the tasks of the country’s economic chambers – also put down by law – is, to give expert opinions about drafts of propositions and bills of economic subject. 5959 In 2003 Hungarian government departments have sent 250 materials to the Hungarian Chamber of Commerce and Industry and have asked feedback in form of stakeholder opinions. 6060 The Chamber has established an economic research institute which performs a questionnaire based survey among entrepreneurs several times a year, in order to collect their opinion about the business climate and also about the current changes taking place in the administrative and institutional environment. 6161

Certain online debate forums have also been created to facilitate these consultations. Thus, for example, in the years 2006-2008 the Ministry of Economy and Transport has maintained an online forum on its website for collecting the opinions of entrepreneurs about regulations. 6262

Deregulation refers to the invalidation of those legal rules which have become unnecessary. 6363 After the political changes of 1989 several Hungarian government administrations have implemented extensive deregulation projects which have invalidated thousands of legal rules in two waves: between 1989 and 1991, and between 1995 and 1998. According to data provided by the Ministry of Justice, since 1990, at least seven thousands legal rules: laws, government decrees and ministerial orders have been invalidated. Officially, in 2007 a deregulation programme was in force, according to which ministries and local governments are obliged to periodically update and streamline the body of regulations issued by them. This activity must be based on impact assessment for which the administrative procedures must be created. 6464 In practice, however, deregulation efforts have not reached their aims: on the contrary, between 1990 and 2004 the number of valid Hungarian legal rules has been significantly increased, whereby approximately half of the new rules were modifications of already existing rules. In many cases these modifications were needed due to the fact that the original rules had not been based on a corresponding impact assessment and were issued without proper consideration of their impacts.

In Hungary the legal basis of impact assessment activities is satisfactory, but the institutions entrusted with performing these assessments refer to chronic shortage of resources. For this reason, RIAs are prepared only occasionally. Some isolated impact assessment projects are carried out by various government departments: these are progressive initiatives, but these projects do not represent a critical mass which could have a considerable influence on the entire regulatory environment of enterprises. The ongoing few impact assessment projects are not coordinated and in most cases are not supported by consultations with entrepreneurial interest groups and other interest representations. A powerful political will and a long term view are needed to spread the culture of impact assessment in the Hungarian public administration.

5656[GKM 2006]

5757[Kállay – Kissné – Kőhegyi 2003]

5858SME law: 2004/XXXIV, Paragraph 11.

5959On the basis of the law on the chambers of economy, 1999/CXXI, Paragraph 12 (1) a)

6060The annual executive report on the community decision of the „European Charter for small enterprises" for the year 2004. The report of the Hungarian government can be downloaded from the following website:

http://ec.europa.eu/enterprise/enterprise_policy/charter/2005_charter_docs/report_hungary_2004_hu.pdf6161[Observer and others 2006]

6262„Tuned to business" – A program for making entrepreneural environment more dynamic. This is an interactive website for the collection of regulatory propositions. In 2006-2007 this website is available at the portal: http://www.gkm.gov.hu. The website is divided to the following topics: 1. Simplification of enterprise- and tax administration. 2. Increasing of the legal security of enterprises. 3. Improving the financial conditions of enterprise operation. 4. Assuring fair and transpart competition. 5. Corporate governance. 6. Trade specific suggestions missing from the previous points.6363The expression „deregulation" has recently become also used in the meanings of „liberalisation" and „release".

6464On the supervision of the deregulation of legal rules see government decisions 2358/2002 (Nov., 28) and 104ide6/2003 (May, 28).



4.4. Assessing the administrative environment of enterprises in the USA4.4.1. Regulations reducing bureaucracy and their implementation

In the United States , on the basis of the Paperwork Reduction Act accepted in 1980, all federal offices have to ask the Office of Management and Budget (OMB) for permission, in case they intend to collect data from enterprises or households. 6565 Public agencies obtain such permissions only if it can be justified that the planned data collection is necessary, and those who are affected will not be burdened too much. Based on this rule, during the past 25 years, annual plans have been compiled to restrict the information collecting activity of federal agencies. These bureaucracy reducing plans are based on impact assessments, which estimate the number of hours used for compliance by enterprises or households. Governmental offices are obliged to reduce this index at a rate that is annually defined in advance.

Moreover, it is the responsibility of federal Office of Management and Budget (OMB) to assess bills before they come into force. 6666 The obligation to perform impact assessment and the range of recommended methods that can be applied for such purposes have been stated in a federal decree. 6767 This legal rule makes it compulsory for federal offices to perform economic impact analyses not only for assessing the expected impacts of legal rules but also in case of any other significant measure or decision that has been proposed by federal agencies, covering the policy areas of taxation, environment protection, public health, safety, labour or social protection.

The federal rules on impact assessment are complemented by a further act of regulatory policy which concerns the flexibility of regulations. According to this Act regulators must take into consideration the needs and opinions of small enterprises when planning, issuing and implementing measures. 6868

It is enlightening to analyse the impact assessment methods recommended in the Executive Order 12866 on „Regulatory Planning and Review". According to this rule, every impact assessment consists of three components:

( A ) The justification of the fact that the proposed measure is necessary. A legal rule is needed only if one of the following causes exists: (1) Failure of market mechanisms. Such a failure can be the effect of a natural monopoly, market predomination, the insufficient information of the customers, or the prevalence of substantial externalities. Externalities are effects of a market transaction on other, external stakeholders or on the environment, irrespectively to whether this effect was intended or not intended by the buyers and the sellers of this market transaction. (2) Even if there are no market failures, a regulative intervention must be justified if it influences prices, production and sales quotas, compulsory qualitative norms and limitations to entry of the market.

( B ) The analysis of alternative regulatory approaches or non-regulatory solutions. Federal regulation is needed only in those cases, when a problem cannot be solved in an alternative way. Examples for alternative approaches are court proceedings or various subventions. In case of health, safety or environmental issues it is strongly recommended to demand the achievement of certain essential requirements or levels of results, while it is not recommended to define prescriptive standards and norms in the medical or technical sense. Moreover, it is strongly recommended to segment the regulation which means, that different requirements should be set up for the different layers of the population or for different entrepreneurial groups.

( C ) The analysis of costs and benefits. In case of all examined alternatives costs and benefits must be compared with a so-called baseline scenario, i.e. with the hypothetical situation that the regulation in question has not been / will not be introduced at all. Whenever possible, all costs and benefits have to be expressed in monetary terms. If this is not possible, the impact assessor should refrain from applying artificial solutions. Long term effects which can be expressed in financial terms, should be discounted, that is, their present value must be calculated. The analysis of uncertain outcomes must be accompanied by risk analyses and the level of uncertainty should be clearly stated. Risk should be interpreted as outcomes with known probability distribution, while uncertainty should be interpreted as a situation in which the probability distribution of the possible outcomes is not known.

6565 Paperwork Reduction Act of 1980, Pub. L. No. 96-511, 94 Stat. 2812 (Dec 11 1980).

6666 [Lutter 2001]

6767 Executive Order 12866 of September 30, 1993, „Regulatory Planning and Review"

6868 Regulatory Flexibility Act f 1980, Modified by the „Small Business Regulatory Enforcement Fairness Act" ,accepted in 1996.



Uncertainty may be the result of lack of data, lack of scientific knowledge, or may be the result of the fact, that certain phenomena are basically unpredictable. In case of uncertainty the decree recommends to clearly disclose the lack of information concerning risks, benefits and costs.

The decree supports the usage of the following quantitative methods: simulation, sensitivity analysis 6969, Delphi methods 7070 and meta analysis. 7171 If an economic analysis relies on hidden assumptions, these assumptions should be clearly stated. If several alternative starting hypotheses can be assumed, then it is recommended to perform a sensitivity analysis by using the major starting hypotheses as bases of inference. It has been observed that the impacts of a legal rule are not equally distributed among various social layers with different incomes, with regard to race or gender. Analogously, impacts on businesses may differ according to company size or industrial sector. Therefore, impact statements should be segmented as deep as scientifically feasible. The time pattern of impacts also must be clearly stated, since what is good for one generation, can damage the next one.

The evaluation of benefits is especially complex because when the benefits of legal rules concern health and safety, these values cannot be expressed in monetary terms. It is recommended to express the benefits of such rules by computing the average number of annually saved human lives or casualties.

Costs, however, can be expressed with the help of monetary concepts much more easily. Such are, for example, the compliance costs of the private sector and the administrative costs of the government. Each calculated cost item has to be compared with those costs that would arise if the regulation had not been implemented - but all other conditions would be the same. Certain cost items are transferring the burdens from one stakeholder to another one, but do not increase the total social costs. Such are, for example, insurance costs, which do not reduce, in reality, the social costs of an accident, but transfer them on to the insurance companies. Such transfer costs lead to a redistribution of the burdens attributable to the legal rule. Similarly, if the aim of the analysis is to calculate social costs, then the net price of an item that was purchased for complying with a regulation should be taken into consideration: If, for example, as a consequence of a legal rule it becomes unavoidable to buy some sort of appliance, then the value added tax is not to be calculated as social cost, since this is not a cost appearing on the social level, this being only some kind of redistribution between the purchaser and the state.

The provisions contained in the above quoted legal rules can be regarded as the self limitation of the federal state. Several member states of the USA have their own regulatory impact assessment systems. 7272

4.4.2. Impact of the federal regulations on small enterprises in the USA

In the USA the quality of legislation is evaluated not only by organisations of the government but also by non-governmental organisations, by advisory firms, universities and by research institutions of public policy, in other words, by „think tanks". The independence think tanks is secured by the fact that they do not rely on any support from governmental bodies. The quality of economic analysis on which impact assessments are based, is improved by the fact that the analytical results of competing organisations are published. In comparison, in Europe the institutional background of impact assessment is relatively weaker.

The United States has established the „Small Business Administration " in 1976 with the aim of institutionalising small enterprise development on the level of the federal government. This government agency has a department called „Office of Advocacy" whose task is to reduce the disadvantageous influences exerted on small enterprises by the measures of the federal legislation and regulation. This department not only performs impact assessment research but additionally runs a counselling section for SMEs and represents the interests of small businesses in the rule making process on the federal level. In 2007 the Small Business Administration had 10 regional centres on the territory of the USA with local representations.

Federal government offices of the US regularly publish reports about the cost effects of regulations on American small enterprises. In 2005 the „Office of Advocacy of the Small Business Administration " implemented a

6969Sensitivity analyses examine whether a method is good or not by systematically changing its input parameters and observing their impact on the output.7070Delphi method is a special form of expert survey whereby researchers ask a group of experts who have not met each other personally, to answer a question. In the second round the same question is to be answered but the experts already know the summarised results of the first round.7171Meta analysis is a secondary analysis in which the data of several, earlier surveys and studies are summarised for the purpose of the examination of a research hypothesis.7272For example, for the continuous regulatory reform in New York state which has been running since 1995, the following department of the governor’s office is responsible: Governor’s Office of Regulatory Reform (GORR), see http://www.gorr.state.ny.us



survey among small- and medium size enterprises about the cost effects of federal regulations. 7373 In the report small enterprises are defined as enterprises with less than 20 employees. In the United States these make up 90% of all enterprises.

Policy areas . The report divides federal regulations into four main categories: economic regulations affecting home markets and foreign markets, labour regulations, environmental regulations and the administration of tax regulations. Let us have a more detailed look at the fields of regulation as evaluated in the report.

• Economic regulations. These rules limit or stimulate access to the market, the use of certain inputs or production methods, influence the choice of outputs, the definition of prices and participation in international trade or investments. The following regulators belong here: those concerning quotas and customs, the ones putting limitation to competition, those concerning prices, and those limiting production or employment.

• Labour rules. This group contains rules connected with payments, allowances, the safety of labour and labour hygiene, legal rules concerning human rights as effectuated at the places of work. The study assesses the impacts of 25 different legal rules and orders at length, which embrace most of the regulations in effect at workplaces in the USA in the year 2000. Quite importantly for small businesses, 6 out of these 25 important labour statutes make exceptions for enterprises employing less than 20 people. Such is, for example, the „Age Discrimination in Employment Act" prohibiting discrimination depending on age or the „Americans with Disabilities Act" prohibiting discrimination against the disabled: some provisions of these legal rules are not extended to enterprises with less than 20 employees.

• Measures aimed at the protection of the environment. Cost effects of environmental regulation are published in the environmental impact assessment reports of the U.S. Environmental Protection Agency. This government agency has prepared impact assessments to every important environment protection rule issued before 2004.

• Taxation rules. The report does not calculate or evaluate the tax burden on companies. However, it is concerned with calculating compliance costs in terms US dollar values which are due as a consequence of tax administration, i.e. by spending time on filling out the forms issued by the federal tax administration. Calculating with realistic hourly wages they express, in terms of money, how long the collection of data and the filling of the forms takes for an average enterprise. The above calculations cover the administrative efforts of firms and entrepreneurs devoted to filing profit tax, national insurance and federal personal income taxes.

Sectors. The report divides the American economy into the following five sectors: (a) manufacturing (b) retail and wholesale trade, (c) services, (d) health care, and (e) others.

Cost types. Direct costs attributed to federal legal rules have been divided into two groups: (a) costs to be paid by government agencies and (b) costs to be paid by the enterprises of the private sector. The report deals with the latter only. Let us take, for example, a decree connected to environment protection. In this case the report does not deal with implementation or enforcement costs, that is, with those which increase the expenditures of the executive authority. On the other hand, the report estimates (a) the costs of those additional environmental investments of the enterprise, which were imposed on the enterprise by the legal rule, (b) the costs caused by additional paperwork and (c) the expenses paid out to experts, The report does not touch upon the cost effects of regulations on the level of the member states or settlements. At the same time, it embraces those expenses which have to be paid by consumers as a consequence of enterprises passing on to their customers the extra costs imposed on them by the regulation.

Results. The main numerical result of the research is that compliance costs of SMEs imposed on them by the federal legal rules exceed an annual amount of 1.000 billion dollars, a sum which makes up more than one tenth of the national income.

Table 22. Estimated value of compliance costs caused by federal regulations United States of America, 2004

Field of regulation Estimated annual effect, billion USD

Rules of economical character 591

7373[Crain 2005]



Field of regulation Estimated annual effect, billion USD

Labour rules 106

Environmental rules 221

Taxation rules 195

The total of federal regulations 1.113

Source: [Crain 2005]

Compliance cost s per one employee are dependent on the size of the enterprise and here an inverse relation can be observed. For example, in cases of small enterprises in the manufacturing industry, compliance costs per one employee amount to twice as much of the same expenses of those in a medium sized enterprise. On the other hand, economic and labour related regulations affect medium and larger enterprises more. The reason for this is that small enterprises are, in many cases, exempt from a certain part of labour regulations.

4.5. Case Study “Croatian RIA": Introducing EU technical regulation in Croatia4.5.1. Policy context: legal harmonisation as part of Europeanisation process

Croatia and the EU have signed a Stabilisation and Association Agreement in late 2001. In this document Croatia has committed itself (Article 73) to harmonise its technical legislation with that of the EU within six years after the entry into force of this Agreement at the latest (Article 5). Thus, compared to EU candidate countries, which have committed themselves to introduce free movement of goods legislation upon accession, Croatia had much more time to carry out these changes.

In 2001 a RIA study was prepared as part of a wider impact assessment project initiated by the Ministry of European Integration of Croatia . The project was implemented with the help of foreign consultants [bib_47]. The assessed directive was the so-called Low Voltage Directive (LVD), which regulates the free movement and the safety of low voltage electric equipment within the EU. At the time of the RIA project LVD was not yet introduced into Croatian law, but following its planned introduction, a wide range of Croatian small, medium, and large enterprises were to be affected by this technical regulation.

The Pilot Project approach . The impact analysis of the introduction of the LVD directive was a tool to understand the consequences of the reform in a wider area of technical regulations affecting product markets. The scope of technical legislation was much wider than what was analysed in the RIA study. However, it was expected that the selection of a fairly typical and widely implemented regulation, that of the Low Voltage Directive, would ensure that the study would reveal the most likely occurring benefits, costs, impediments and risks.

Table 4.6. Figure 9.: Simplified scheme of impact mechanism of European legislation for ensuring the free movement of goods a



Vertical legislation for product sectors

Directives on the free movement of:

Low Voltage equipment

Radio equipment and telecommunications terminal equipment

Machinery

Gas Appliances

Lifts

Other product sectors

Individual products may be subject to more than one relevant vertical regulations

Horizontal legislation for setting up

Standardisation

Product markets, e.g. markets of low voltage equipment

Accreditation

Conformity Assessment / testing / inspection

Metrology

Market surveillance

aThis scheme is based on the following two documents: [EC 2003c] and [EC 2003d]

The planned changes in the conformity assessment regime. As in other post-socialist countries, legal harmonisation was expected to abolish the previous rules that had prescribed the mandatory safety testing and certification of every electric equipment by independent bodies. In the previous conformity assessment regime these procedures had to be implemented before putting these products on the market, and were to be applied for all product types, whether home made or imported, whether high or low voltage, whether previously certified in other countries or not. On the other hand, in contrast, the new rules to be introduced were to allow manufacturers to certify their low voltage electric products by themselves, provided they have used the conformity procedures defined in the regulations. In accordance with the institutional model of the EC, standardisation was to be transformed from a function of the government into a private sector activity, and standardisation, certification and accreditation became institutionally separated from each other. In well defined cases the application of standards was not any more mandatory.

The institutional setting. The institutions that protect Croatian consumers from dangerous electronic appliances getting on their market are the customs body, the market surveillance and the certification bodies. The planned reform was to introduce a system, whereby the respective roles and responsibilities of these institutions had to be changed thoroughly. In particular, by abolishing the mandatory certification, the influence of the certification bodies were bound to decrease, and the tasks of the other two organisations (the customs body and the market surveillance) was expected to increase. The cooperation among the above mentioned three institutions had to be strengthened.

4.5.2. Research design



The research design in Croatia was identical to that of a previous Lithuanian project. This was also a pilot project where the impacts of the same EU Directive were used to draw inferences to the impacts of legal harmonisation in the field of technical legislation.

The aims of the Croatian project were the following:

• Exploration of specific impacts. It was required that the study should be a valid impact assessment on its own right. Besides formulating the impacts on the affected enterprises, it should also estimate the implementation costs of legal approximation of EU technical legislation for the Croatian state. In particular, it was expected that the results (a) will facilitate the planning of the government’s budget and (b) will be helpful during the accession negotiations of Croatia with the European Community by providing explanation on when and how the regulation should be introduced or its introduction delayed.

• Spreading of methodology. A parallel aim of the project was to reveal how the EC directives should be introduced and enforced cost efficiently in Croatia , including the analysis of the necessary institution development tasks. The project was implemented in order to illustrate the method of in-depth impact assessment in the case of a fairly typical directive. One of the didactical outcomes of the project was that a book was published in Croatian language about the recommended method of the regulatory impact analysis for the Croatian governmental bodies [bib_48].

• Administrative co-ordination. It was expected that the project (a) will help to coordinate the cooperation of different governmental bodies responsible for legal harmonisation, (b) will demonstrate the feasibility of impact assessment for the relevant government departments and subordinate government agencies and (c) will provide information for enterprises, public institutions and households concerning those changes that the introduction of EC regulations will bring for them.

• International relations. The project was expected to demonstrate towards the European Community that Croatia will carry out the Stabilisation and Association Agreement between the EC and Croatia. 7474 This document has guaranteed for Croatia to benefit from the advantages of the unified European market if, among others, the country adjusts its technical legislation to that of the EU and removes the remaining barriers to trade from its conformity assessment system.

Information sources. The impact assessment study was based on the analysis of documents and on interviews made with all relevant stakeholders: the responsible ministries, the office for consumer protection, the most important certification bodies, the bureau of standards, with chambers of commerce representing the interests of the affected sectors, and with the experts and decision makers of a sample of producing and trading enterprises affected by the regulation. 7575

4.5.3. Results

The economic activity affected. The research has revealed that low voltage products have accounted for more than half of the turnover of Croatian industrial plants producing electric devices, radios, televisions, and communication devices, medical, precision and optical appliances. Among the affected industrial companies there were approximately 10 to 20 larger businesses, employing more than 100 people, and about 100 smaller companies. This industrial sector was in deep recession due to outdated techniques, lack of investments, delayed or unfinished privatisation, and the loss of foreign markets and because of a strong import competition due to relatively open markets. These factors have led to decreasing production and employment in the sector during the years preceding the research.

Impacts. The analysis of the interviews has shown that on the long rum the planned change of technical legislation will have a positive overall influence on the industrial sector producing electric appliances. It was expected that the reform will indirectly contribute to the development of a better industrial structure by facilitating foreign investments and by contributing to a bigger tax income of the government.

The expected winners of the interventions were as follows. As a consequence of legal harmonisation...

• ...the importers of low voltage electronic devices manufactured in the EC will enjoy the benefits of the abolishment of the compulsory qualification.

7474 The Agreement was signed on 29 October, 2001

7575The Appendix contains an outline of the questionnaire sent to the enterprises affected by the legal rule.



• ...the individual consumers as well as households and institutions will enjoy the benefits of livelier competition, provided that the market inspection will reliably remove unsafe products.

• ... Croatian enterprises already producing low voltage electronic devices for EC export, will gain advantages against those local producers who will have to adjust their products and technologies to the new requirements.

The expected losers of the interventions were as follows. As a consequence of legal harmonisation...

• ..., due to the abolishment of mandatory certification of low voltage electronic devices, certification institutions operating in the field of electrotechnic products (there were seven such bodies in Croatia ) will lose most of their income resulting from this activity.

• ...some of those smaller Croatian producers who will not be able to update their technologies to comply with the new safety requirements will be crowded out from their traditional markets.

• ...producers and importers of countries outside the EC, will have to face additional compliance costs, because of the adaptation to the new administrative procedures necessary for conformity assessment.

• ... consumers with preference for cheaper electrotechnic devices can expect their choice becoming limited, because enterprises can be expected to pass some of their additional costs on them.

Table 23. Significance of various impacts of the Low Voltage Directive in Croatia

3 = very strong impact, 2 = medium impact, 1 = low impact and 0 = no impact.

Element Topic Score

Economic significanceNumber of employees affected 2

Number or firms affected 2

Changes in output level 2

Changes in export level 2

Legal and administrative impactsLegal harmonisation 3

Need to establish new institutions 1

Need to reorganise institutional structures

3

Need for training of public servants 2

Technical impactsTechnical adjustment difficulties 1

Lack of knowledge and experience 2

Public attitude Public perception of effects of the measure

2

Increases in costsCosts to industry generally 1

Costs to small businesses 3

Costs to the government 2



3 = very strong impact, 2 = medium impact, 1 = low impact and 0 = no impact.

Element Topic Score

Costs to consumers 1

In the framework of the impact assessment project open workshops were organised where all relevant stakeholders were invited. These occasions were used by the representatives of certification bodies to ask the government and the EC for support. Certification bodies have faced the risk of losing the bulk of their incomes, closing a number of laboratories and dismissing several hundreds of employees.

The report has recommended to take measures in order to alleviate the undesired side affects of the regulation. The recommended form of these measures was to implement well targeted direct subsidy projects and to subsidise consultancy services on behalf of affected SMEs and certification bodies.

4.5.4. Follow-up

Eventually, four years later, in 2005 Croatia has partially harmonised its legislation with the New and Global Approach directives, and LVD was transposed as a part of these measures. The country has adopted legislation aiming at transposing the Acquis on lifts, machinery, gas appliances, low voltage equipment, pressure equipment and simple pressure vessels. However, at the end of 2007 further amendments, horizontal legislation and institutional development of notified bodies still were needed to achieve full alignment with the Acquis.


Chapter 5. 5. Conclusions and recommendationsThe status of evaluation and impact assessment in development policies. Evaluation and impact assessment efforts are applied social research activities which have been adapted and institutionalised according to the needs of donor organizations and public agencies responsible for financing and implementing development policies. For this reason, evaluation and impact assessment efforts must take into consideration (a) the principles of research design in social science, with special respect to methods of data collection and inference (b) the particular organizational traditions of the donor organization or government agency which finances or implements the assessed development measure (c) the nature, aims and instruments of the particular policy area, with special respect to the beneficiary target group.

Planning and implementing evaluation and impact assessment projects. Evaluation and impact assessment efforts should be embedded into a wider managerial and administrative process: these efforts should be part of the policy debate and the resulting studies should lead to policy change. Evaluation and impact assessment activities should be organized as projects and should be planned in advance. Policy makers and donors responsible for planning SME development initiatives should facilitate the feasibility of evaluation and impact assessment efforts by rendering the policy initiatives as transparent as possible and by committing the resources necessary for performing the evaluation or impact assessment. Evaluators must collect data about (a) the conditions prevailing in the SME target group before the measures were taken (baseline data) (b) and about the impacts of the measures during and after the implementation of the measures. Evaluation and impact assessment efforts should be based on appropriate research design, which must be prepared before the measures are taken. The research design which has been developed during the planning phase, should be the basis of any data collection and analysis during the implementation phase.

SME development context. Evaluation and/or impact assessment projects should start with the careful analysis of the assessed initiative/measure. In particular, the nature, targets, intended outcomes, target groups, instruments and professional content of the initiative should be clarified, with special respects to its novelty features as compared with previously existing SME development measures. Moreover, the details of the implementation of the planned measure must be clarified, with special respect to the institutional framework of its implementation.

Focusing on SME development instruments. If SME development measures are to be assessed, the research design of evaluation and impact assessment must be tailored to the specific instruments applied on behalf of SMEs. These instruments may be (a) the introduction of enterprise-friendly regulations facilitating business creation, employment and the access to permits and markets, (b) financial assistance (e.g. microcredit provision, tax relief or subsidy for start-ups), (c) supported business development services (e.g. training, education and provision of advice and consultancy in the field of general entrepreneurship, marketing, export, technology, e-business, in developing business organization or business co-operation patterns such as subcontracting and outsourcing) or (d) institution development (e.g. the creation of science parks, incubators, enterprise zones, cluster organizations or other types of business networks). Each component of the research design of the evaluation and impact assessment effort must correspond to the peculiar features of the above mentioned SME development instruments, of the project organization and of the target group. In particular,

• the research question must be relevant to the assessed measure and must be formulated in terms of the assessed public policy;

• a theoretical hypothesis should be formulated about the impact mechanism (in case of impact assessments) or about the success criteria (in case of evaluations) of the assessed measure;

• the strategy for data collection and sampling must correspond to the organizational framework of project/programme implementation, and must reflect the willingness of the target group members to respond to survey/interview questions; and finally

• when making inferences, the collected data and the applied analytical methods should prepare the grounds of supporting or falsifying the above mentioned theoretical hypothesis about the impact mechanism (in case of impact assessments) or about the success criteria (in case of evaluations) of the assessed SME development measure.


5. Conclusions and recommendations

Local and regional context. The understanding of the local and regional context is essential in the evaluation and impact assessment of SME development measures. Within each country, the rates of enterprise creation, growth and bankruptcy differ markedly across regions and localities. These rates are determined by the natural endowments and entrepreneurial traditions of the region, and by local factors of demography, labour and education. Small business development can be an important driving force in developing disadvantaged or deprived areas. However, evaluators and impact assessors must always take into consideration the specific impediments to entrepreneurship in these communities such as the lack of demand on services, lack of experiences, an under-developed culture of co-operation and a high reliance on social benefits. Projects implemented in localities lagging behind the national average may need a special design and delivery mechanism. In well endowed regions (e.g. in areas with many tourist attractions) the impact and success of SME development measures must be evaluated by properly taking into consideration the local context.

Distinctive features of impact assessment and evaluation. Both evaluation and impact assessment are empirically based comparative analytical exercises used for improving development policies. However, the researchers must establish a clear difference between

• impact assessment on the one side, which should be a value-free activity aimed at finding causal relationships by demonstrating the methodological rigor of positivist analysis and objective social science and

• evaluation on the other side, which cannot be value-free, because the shorter and longer term interests of various stakeholders (e.g. donors, project managers and beneficiaries) can be confronted only if the evaluators rely on certain principles of good governance, and ethics.

Impact assessment is essentially the demonstration of causality between the assessed measure and its impacts on the companies of the target group, a verification of impact mechanisms and the ruling out of alternative explanations. Impact assessments should include an analysis of the counterfactual situation, i.e. the analysis must respond to the question: what will happen (or what would have happened) to the target group if the measure was not (would not have been) taken. Although the counterfactual scenario cannot be observed, indicator values about the counterfactual scenario can be obtained with rigorous methods, e.g. by the following experimental design: by randomly assigning support to a sub-group of eligible applicant SMEs and by randomly denying support to another eligible sub-group of applicant SMEs. However, in most cases of impact assessment, due to lack of data and scarce resources, the counterfactual scenario is assessed by less reliable qualitative methods, e.g. by asking beneficiaries in in-depth interviews about "What do you think, what would happen to your business under the impact of the planned regulation" or "What would have happened to your business if you had not received the subsidy". Impact assessment is a comparative effort in that sense that the indicator values about the actual observed impacts of the measure must be compared with the indicator values describing the impacts under the counterfactual scenario.

Identifying impacts of SME policy measures. In typical circumstances it is extremely difficult if not impossible to isolate the net impact of small business policies, because there are various macroeconomic policies that can influence of entrepreneurship and small business performance in a country. Decisions of public policy on taxation, consumer protection and environmental regulation, immigration and on the institutional framework and procedures of enforcement may strongly influence the birth, survival and growth of SMEs. It is the task of SME policy makers to identify these policy influences on entrepreneurship. Evaluation and impact assessment projects should be key instruments in identifying these interactions between various policy areas and in developing co-operation patterns with other departments of the government.

Evaluation efforts qualify the SME development measure; they determine the value of a project, a programme or a policy, by delivering value judgements about its design and implementation, by analysing the course of SME development actions according to previously defined evaluation criteria (e.g. relevance, efficiency, effectiveness, etc.) Evaluation is a comparative effort in the sense that the actual observed or calculated impacts of the measure must be compared to the expectations of the evaluators which in turn are based on the aims of the measure and on the results of analogous, comparable projects or programmes.

The standards of evaluation and impact assessment activities have been defined by donors and governments, the recommended or prescribed methods are part of these institutional cultures. The EU alone has several dozens of evaluation and impact assessment guidelines and methodologies. Policy makers and donors should continuously work on the establishment and institutionalisation of unified, compatible standards, guidelines and methodologies for obtaining feedback about policy measures, in particular about those affecting SMEs. These standards should rely on good evaluation and impact assessment practices. Researchers should be encouraged to prepare comparative reviews of evaluations and impact assessments of which the subjects are analogous,


5. Conclusions and recommendations

comparable SME development measures in various countries.


Appendix A. 6. Appendices1. Outlines of the questionnaires used in the case studies1.1. Outline of company questionnaire used for SME labour subsidy evaluation (Case Study “SME Employment")1.1.1. Introductory text

„Good morning, I am here on behalf of the SEED Small Enterprise Development Foundation and also on behalf of the OFA National Labour Public Foundation. Your enterprise has applied at OFA for financial support which you have recently received. In our survey we examine the process of the subsidy as well as its impacts. Please, help our work by answering our questions. Naturally, we will handle your answers confidentially, and we are going to keep your anonymity.

1.1.2. Questions

Has your company won any subsidies in one of the tenders announced by the OFA between 1999 and 2003?

What is the legal form of your enterprise? (1 – Ltd.; 2 - Shareholding company; 3 – Cooperative; 4 – 5 – 6: various forms of partnership; 7 – sole proprietorship – other, namely)

What kind of activity does your company pursue? (more than one answer is possible) (1 – farmer; 2 – craftsman , 3 – tradesman; 4 – service provider; 5 - intellectual freelancer, 6 – other, namely)

Which sector does your enterprise belong to? (1 – Industry, mining, energy; 2 – agriculture, forestry, management of water supplies, fishing; 3 – transport, communication; 4 – commerce; 5 – catering; 6 – financial services, insurance; 7 – other services; 8 – public administration; 9 – health service; 10 – culture, education, sciences, church; 11 – other, namely)

The year of the establishment of your business.

What is the most important activity of your business which constitutes most part of your revenue? (in words)

What is the second most important activity of your business, which accounts for the second largest part of you revenue? (in words)

What is the structure of ownership of your enterprise? Please, give data in the order of the proportion of the functioning capital. (1 – Hungarian private persons 2 – the state 3 – local government 4 - foreign citizen or company registered abroad – other enterprise 6 – other owners:)

How many owners does the enterprise have? (…people)

Who are the owners of the enterprise? (1 - the person interviewed; 2 – spouse of the interviewed; 3 – the parent(s) of the interviewed; 4 – the children of the interviewed; 5 – other entrepreneurs; 6 – cooperative members; 7 - other people not enlisted above; 8 – other company, the local government or other institutional owners)

The labour status of the enterprise population at present (at the time of the survey): (a) Number of full time employees; (b) Number of part-time employees; (c) Number of helping family members; (d) Other, namely: …)

The profitability of the enterprise in the year of the application (1 – very profitable; 2 – profitable; 3 – scarcely profitable; 4 – not profitable or; 5 – loss making company)

The profitability of the enterprise last year (1 – very profitable; 2 – profitable; 3 – scarcely profitable; 4 – not profitable or; 5 – loss making company)


6. Appendices

What do you think, how profitable will be your enterprise in three year’s time? (1 – very profitable; 2 – profitable; 3 – scarcely profitable; 4 – not profitable or; 5 – loss making company)

The reach of the activity of the enterprise ( 1 – national, 2 – regional; 3 – restricted to the county, 4 – restricted to the smaller region, or; 5 – restricted to the settlement only)

Has the enterprise worked as a supplier? ( 1 – it has, 2—it has not). If it has: What kind of work did you perform (do you perform) as suppliers? (in words) In what way has the support helped you to become a supplier or in your work as supplier? (in words)

The number of premises belonging to your enterprise ( … pieces)

In your opinion why do entrepreneurs employ new people? (1 – They need manpower for a certain order (project), 2 - manpower is needed for the short run, 3- manpower is needed for the long run; 4 – Increasingly complex tasks cannot be carried out with the present staff; 5 – The expansion of the firm brings along new tasks, 6 – other, namely:)

How can you explain the phenomenon that entrepreneurs are unwilling to employ new people? (1 – the enterprise intends to remain small, 2 – The entrepreneur wants to keep everything under his own control, 3 – the entrepreneur wants to avoid the problems which employment might cause; 4 – there are not enough well qualified candidates for the jobs; 5 – wages are too high ; 6 - taxes are too high; 7 – the administrative procedures of employment are too complicated and time demanding; 8 – the employer is afraid that later, if necessary, it will be difficult to dismiss the new employee; 9 – entrepreneurs are distrustful; 10 – other, namely: …)

Who are your competitors? (in words)

What are the major factors on the market influencing the position of your firm? (more than one answer is possible) ( 1 – decrease of selling prices; 2 – increase of purchasing prices, changes of the costs; 3 – the appearance of new products; 4 –the appearance of new technologies/processes; 5 – the appearance of new competitors; 6 – import competition; 7 – lacking export abilities, 8 – the appearance of new laws regulating operation; 9 - the appearance of new laws concerning taxation; 10 - accession to the EU, 11- other, namely:…)

What is the impact of the accession to the EU on your enterprise? (in words).

The next few questions will be related to the tender and your application submitted.

Which year did you apply?

In how many different years did you win enterprise-development support from OFA between the years of 1999 and 2003?

Please choose the tender which was most important for you and answer, in the following, with reference to that support. Which year have you chosen?...(year)

How did you learn about the tender? (More than one answer is possible) (1 – from a national daily paper; 2 – from a local paper; 3 – from a tender watching service; 4 – from acquaintances; 5 –from the Internet; 6 – from an interest representation, from the chamber; 7 - from an enterprise development foundation, or bureau, 8 – from radio, television; 9 – from another place, namely:)

Since then, from which sources do you gather information about current tenders? (More than one answer is possible) ( 1 – from a national daily paper; 2 – from a local paper; 3 – from a tender watching service; 4 – from acquaintances; 5 – from the Internet; 6 – from an interest representation, from the chamber; 7 – from an enterprise development foundation, or bureau; 8 – from radio, television; 9 - from another place, namely:)

What was the amount of subsidy you applied for? What did you want to spend the support on, from among those listed below? (More than one answer is possible) (1 – purchase of devices; 2 – construction; 3 – purchase of services; 4 – obtaining ISO quality assurance qualification or some other qualification; 5 – training of employees; 6 – counselling or market research; 7 – introduction of new product or technology; 8 – employment of new workforce; 9 – other, namely: )


6. Appendices

What was the name of the programme that you have applied for? (More than one answer can be given) (1 – A programme to improve labour safety; 2 – A programme related to obtain professional knowledge, training, education; 3 – A programme to alleviate liquidity problems; 4 - A programme to hire the first employee; 5 – A programme to support marketing; 6 – Professional advisory programme, 7 – A programme to improve work conditions ; 8 A programme to modernise production; 9 – A programme to develop business environment; 10 A programme to promote the development of technology; 11 – A programme for protecting the environment; 12 – A programme to alleviate the damages caused by flood)

What did you concretely ask support for? (in words)

Did you have to pay back the amount of the support? (1 – subsidy to be repaid or; 2 – non-returnable subsidy)

Was the project related to keeping the employees?(1 yes, 2 – no)

Was the project related to an increase in the number of employees? (If it did: How many new people did you employ as a consequence of the tender? (…people). What was the composition of the group of people whom you employed in consequence of the tender? Were there, among the new employees (More than one can be marked: (a) - registered unemployed (b) - career starter (c) - retrained by labour centres; (d) – unemployed, under the age of 25; (e) unemployed, over the age of 50; (f) – unemployed graduate ; (g) – other, namely: )

If the realisation of the project was related to an increase of the number of employees: What, did you think, the new employees’ job would be, at the time of the application? (in words)

If the realisation of the project was related to an increase in the number of employees: How did you want to choose your co-workers at the time of submitting your application? (1 – by advertisement; 2 - based on family, friends, acquaintance; 3 – on the basis of recommendation of colleagues; 4 – on the basis of recommendation of labour centres; 5 – other, namely)

Did you ask for special advice in the framework of the tender? If you did: what kind of an advice did you want to ask for when you handed in your application? (1 – continuous tracing of the activity; 2 – establishing connections with other enterprises; 3 – management advice; 4 – sales, marketing, advice of market analysis, 4 - the introduction of an environment friendly operation, 6 - quality assurance; 7 - other, namely)

Please rate, according to school grades, at the time of the handing in of your application (a) your experience of writing an application for a tender; (b) your experience of managing a project; (c) your management capacities to implement a project, (d) your financial resources needed to implement a project.

When compiling the application: 1 – did you compile the document yourself, without external help or 2 – did you ask for free external help for the compilation of the documentation 3 – did you ask for the help of a firm specialised in writing applications or 4 – other solution, namely:

If you asked for external help: Why did you do that? (in words) How much did it cost to have the application written? ( …Hungarian Forints)

Have you ever applied to another enterprise-supporting organisation other than the OFA? (1 - yes 2 – no)

If you applied somewhere else, where to? (in words). Have you found any considerable difference between an OFA competition and other competitions? (1 – yes 2 – no) If you have: In what do you see the differences between an OFA competition and the other? (in words)

When did you sign the contract of support: (year, month) When was the programme completed? (when you received the last instalment). (Year, month)

Was there any difference between the support you asked for and which you received? How big was this difference? ( … Hungarian Forints)

If you received less than requested: was the reduction justified? (1 - yes; 2 – no) Which elements of your original programme did you have to drop due to the smaller amount of the support? (in words) Which were the further influences of the fact that you received less than you had expected? (in words)

Did you add to the subsidy from your own sources in order to have the project implemented? (1 – yes, 2 – no) If you did, how much? ( … Hungarian Forints)


6. Appendices

Was there a co-financer other than you in the project? (1- yes, 2 – no).

If you did, who was the co-financer? (1 – bank; 2 – business partner; 3 – other supporting organisation; 4 – the local government; 5 – other, namely) How much did you get from this co-financer? ( … Hungarian Forints)

Apart from securing the financial resources, which were the problems you had to solve in the course of implementing the project? (in words)

If there were other problems: How did you solve them? (in words)

The following questions will be asked to learn about your opinion concerning the subsidy programme and the work of the OFA.

Why was this supported programme necessary for you? (in words)

Has the supported programme solved the problems mentioned? (1 – yes, it has, 2 – no, it has not)

What has changed in your enterprise in consequence of the support? (in words)

Have you succeeded in completing the programme in time? (1 – yes, we have succeeded, 2 – no, we have not succeeded)

If you have not succeeded in completing the programme in time: What was the delay due to? (in words)

Have you succeeded in completing all the activities planned in the program? (1 yes, we have succeeded, 2 – no, we have not succeeded) Why have not you succeeded in completing all the activities in accordance with the plans? (in words)

Have you succeeded in financing the programme according the original plans? (1 yes, we have succeeded, 2 – no, we have not succeeded) Why have not you succeeded in financing the programme in accordance with the plans? (in words)

Did you have, during the implementation of the program, suppliers, subcontractors? (1 – yes there were, 2 there were none) If there were: How satisfied were you with their work? ( 1 – I was not satisfied at all, 2 – I was not satisfied, 3 – I was partly satisfied, partly not, 4 – I was satisfied, 5 – I was fully satisfied) How many percent of the received support did you spend on paying the subcontractors? ( … %)

Please, rate the work of the donor organisation (OFA) between 1-5, according to the following criteria: (a) – The intelligibility of the invitation of the tender; (b) – Providing the applicants with understandable and useful information, advice; (c) - Expertise of the OFA clerks; (d) – The helpfulness of the OFA clerks; (e) – The expertise of the OFA inspectors; (f) – The helpfulness of the OFA inspectors)

What problems have arisen between you and the OFA during your relationship? (in words)

Has the OFA, in your opinion, published the invitation for the tender at the right place? (1- yes 2 - no)

In your opinion where should a tender be made public? (in words) In your opinion has the OFA (a) provided enough opportunities for asking questions in connection with the tender? (b) Was the judgement of the applications and the selection of the winners quick enough? (c) Were the requirements concerning documentation possible to fulfil? (d) Was there enough time to realise the projects? (1 – yes, 2 – no)

In your opinion what programs do Hungarian entrepreneurs need? In what other ways, in your opinion, could the employment ability of Hungarian businesses be improved? (in words) And their competitiveness? (in words)

Did you experience injustice during your application? (1 – yes 2 - no) Please tell us what it was about. (in words)

What are you applying for now? (in words)

Do you see any difference between OFA and other donor organisations? (1 – Yes, 2 - No) If you see any difference, what is it? (in words)

What would you do differently if you were OFA, in tenders of the future? (in words)


6. Appendices

Now I am going to ask you about employment again.

Has the supported project improved your employment possibilities? ( 1 – it has, 2 – it has not) If it has, how? (in words)

Has the support improved, at your company (a) working conditions; (b) labour safety; (c) the qualification of the employees; (d) the qualification of the managers (1 – yes, 2 – no)

Has the support facilitated the work of the employees? (1 – yes, 2 – no) Please tell me in a detailed way, in which areas has there been any improvement. (in words)

Has the support improved the competitiveness of the company? ( 1 – it has, 2 – it has not) Please tell me in a detailed way in which areas has there been any improvement? (in words)

Now I would like to inquire about changes which have occurred since the supported program.

(a) Has the revenue increased since the supported project? (b) Has profitability improved? (c) Was the activity of the company extended? (d) Were new business lines introduced? (e) Have any innovations been realised? ( 1 – yes, in consequence of the tender, 2 – yes, but not in consequence of the tender, 3 – there have not been changes of this kind)

If the activity of the company has been extended as a consequence of the tender, please tell me in details about this new activity (in words)

If there has been innovation as a consequence of the tender, please describe, what kind of innovation do you mean more precisely. (in words)

Has the project had such an impact we have not talked about so far? If it has, what was that? ( 1 – Yes, namely: (in words), 2 – No

Has the tender had a positive influence on the (a) skill of your company to apply for support (b) on the skill of your company concerning planning? ( 1- yes, it has, 2 – no, it has not) Please, justify your answer. (in words)

Has the supported programme had a positive influence on the following phenomena? (a) on the labour situation in the settlement (b) on consumers' supply; (c) on the development of the settlement, on its enterprise friendliness, on its ability to attract capital; (d) on the cultural development of the settlement, (e) on the protection of the environment of the settlement, (f) on cooperation among the enterprises (1 – yes it has, 2 – it has not). Please, tell me what has happened in the above respects. (in words)

What has been sustained up to the present date from the results of the project? (in words)

What do you use, at the moment, the results of the supported project for? (in words)

If the professional content of the project was construction or purchase: Is the building or the purchased device used for the purpose it had been intended for in the program? ( 1 – yes; 2 – no)

If the professional purpose of the project was to obtain ISO or another certification: has the company been certified since the project? ( 1 – yes; 2 – no)

If the professional content of the project was training and education: are those who were trained in the programme still working for the firm? ( 1 – yes; 2 – no)

If the professional content of the project was getting advice or market research, do you still make use of the experience from the required advice / market research? ( 1 – yes, 2 – no)

If the professional content of the project was aimed at the introduction of a new product /or proceedings), has the introduction of the new product / proceeding been realised? ( 1 – yes, 2 – no) Do you still produce it (use those way of proceedings)? (1 – yes; 2 – no)

How long, do you think, these results can be maintained? (1 – they are not maintainable at all; 2 – for a year; 3 – for two years; 4 – for 3-5 years; 5 - for more than five years) Please, give reasons for your answer. (in words)


6. Appendices

If the realisation of the programme was related to an increase in the number of employees: What will happen to this increase connected to the programme applied for in the tender? ( 1 – it is not maintainable at all; 2 – maintainable for one year; 3 – for two years; 4 – for 3-5 years 5 – for more than 5 years.)

Do you plan to establish new working places? (1 – yes; 2 – no)

Is this plan connected to the results of the project? ( 1 – yes, 2 – no). If yes, how? (in words)

What would you do differently today as far as the planning of the project and the compilation of the application documentation are concerned, on the basis of the obtained experience? (in words)

And what would you do differently today concerning the implementation of the project? (in words)

What would have happened if you had not received support? (in words)

Please characterise your enterprise by giving some data of the real estates, car fleet, and producing machinery being in the possession of your firm. Has the enterprise its own office, workshop, shop, company car, truck or van, if yes, which are their quantity specifications? (piece, m)

Name, address, telephone, e-mail, website of the applying enterprise.

The responding person’s gender (1 – male; 2 - female) and age (1 - under 20; 2 - between 20 and 29; 3 - between 30 and 39; 4 - 40 and 49; 5 - 50 and 59; 6 - 60 or older)

Highest school qualification of the responding person ( 1- has not gone to school; 2 – less than 8 elementary classes; 3 – elementary school ; 4 – vocational school, 5 – secondary grammar school and school leaving exam, 6 – college degree or higher technical school, 7 – university degree.

The qualification of the responding person (more than one answer can be given) 1 - Education (e.g.: teachers training, educational science); 2 – Humanities and art (e.g.: history, foreign languages); 3 – Social sciences, economy, law (e.g.: psychology, sociology, economics, business and finance, management); 4 Mathematics, informatics, and other natural sciences (biological sciences, physics, chemistry); 5 - Engineering, technical and civil engineering qualification (e.g.: engineer, engineering technician); 6 - Agriculture and veterinary sciences; 7 - Health service and other social services (e.g.: medical sciences, social attendance); 8 – Services (e.g.: personal services, catering services, accommodation, transport, communal and public health services, security service, health control for the employees))

Name and position of the responding person

The opinion and evaluation of the questioner: What was his impression of the visual appearance of the enterprise? (Premises, the size of the machinery equipment, how up-to-date it is. The number and activity of the employees who were present on the premises. (in words) To what extent were the answers of the interviewed person relevant? Which responses were not credible? (in words)

1.2. Outline of company questionnaire used for SME innovation subsidy evaluation (Case Study “SME Innovation")Questionnaire was sent by post and answered on a voluntary basis

1.2.1. Introductory text

Dear Madame /Sir! Your enterprise received, during the past years, support from the donor agency OMFB for the purpose of implementing a project with an innovative content. These forms of support are now, subsequently evaluated from that viewpoint, whether they have promoted research, development, and innovation activity within the enterprises, and whether they have, as far as their content and realisation is concerned, met the expectations of the enterprises. Please help our work by way of answering the following questions.

1.2.2. Questionnaire

Questions concerning the beneficiary enterprise


6. Appendices

What was the legal form of the applicant enterprise at the time of the tender? (sole proprietorship, partnership, Ltd., shareholding company, economic cooperation, state-owned enterprise, other, namely: )

Is there a foreign co-owner in your company? (yes, his proportion of ownership is ….%, there is no foreign co-owner)

How many full time workers did the company employ at the time of submitting the application? ( less than 10 people, between 11 and 50 people, between 51 and 250 people, between 251 and 500 people, over 500 people)

Do you consider your firm to be a small, medium, or large enterprise? Why?

How many hierarchical steps are there between the leaders and employees of the lowest positions? (in words)

How innovative, innovation-oriented do you regard your firm to be and why? (in words)

What is the role and respect of mental work, of mental contribution to innovative activity within your firm? (in words)

Questions concerning the circumstances and the financing of the application

In which year did you submit the application?

What was the name of the tender for which you have submitted the application? (e.g.: “Tender for the development of export")

In which phase is the project at the moment? (Completed – has not been completed and it is proceeding faster than planned – has not been completed and it is proceeding according to the plans – has not been completed and it is proceeding more slowly than planned)

How much money did your company ask for in the application? ( ….. Hungarian Forints)

How much money did your company receive? ( ….. Hungarian Forints)

Did you succeed to keep expenses within the planned financial frame of the project? (yes, we succeeded – no, we did not succeed) If you did not succeed, please explain, in a few sentences, why not? (in words)

Did you succeed in operating according to the plan of the stages of the utilisation of the money in the project? (yes, we succeeded – no, we did not succeed) If you did not succeed, please explain, in a few sentences, why not? (in words)

Why, do you think it was your firm which has received support? What do you think was missing in others' applications, in those who were not as lucky as you were?

Questions related to the professional content of the project

Please, write down in a few sentences, the professional purpose of the supported project. (in words)

To what extent can the project, for which you received financial support, be regarded as an innovation and why? (in words)

If the supported project is an innovation: is this a product-, or technology- (proceeding) centred innovation? (in words)

Does the innovation embodied in the project facilitate only the work of your firm, or somebody else’s work as well? If somebody else’s, then whose? (in words)

Does your company keep the innovation to itself? If not, through which channels will the innovation embodied in the project will spread further on? (in words)

Competitors, as a rule, try to respond to every innovation by substituting it with a product / service of their own. Which product / service / company is the most serious competitor of the innovation embodied in the project? (in words)


6. Appendices

Is there some organisational change generated by the supported innovation? If yes, what kind of organisational changes, at which companies and in which services? (in words)

Are the further impacts of the innovation by the project? (in words)

With respect to the project as it is now, to what extent have you succeeded in fulfilling the professional aims as put down in your application document? (not at all – to a small extent – more or less – to a satisfactory extent – we have succeeded in completing it fully)

If you succeeded in fulfilling what you had planned in your application only to a small extent: please give your opinion briefly on why your professional purposes have not been fully completed? (in words)

Please write down briefly what you have achieved with the help of the project supported by the OMBF tender.

Impacts of the project

Has the supported project had impacts, consequences and if it has, of what character? (More than one answer is possible: the making of a prototype - increase of market share - increase of revenue - decreased labour costs at the applicant company - decreased labour costs at the clients of the applicant company – decreased total costs at the applicant firm – decreased total costs of the clients of the applying firm - the project closed unsuccessfully - the project closed very unsuccessfully and has damaged the applicant firm – the project did not have any considerable impact)

Have there been any changes in the number of fully employed persons at your company? (Increased – did not change – decreased)

If the number of employees has changed, how is this fact connected to the project? Please give your opinion briefly, in a few sentences. (in words)

Have your employees received some special training in order to be able to implement the project? (yes – no)

Does your project have any results which are still viable? (Yes – no) If it does, which are those? (in words) If not, why not? (in words)

If your project is not finished yet: will it have viable results after it has been closed?

If you project will have viable results: which are those? (in words) If there will not be such results, why not? (in words)

As far as the export activity of your company is concerned, which statement is true in your case? (we do not pursue export activities – earlier we did, but since 19□□ we do not – we pursue export activity – we pursue export activity and plan to expand it covering new markets in the future – we do not pursue export activity yet, but plan to do so)

If you have ever had export activity, please list your target countries. (in words)

How has your revenue changed since the introduction of the project up till now? (It increased by □□ % - it has not changed – it decreased by □□ %)

To what extent has the share of export revenues within the total revenue changed since the start of the project? (It increased by □□ % - it has not changed – it decreased by □□ %)

Do you think the results of your developments are also competitive on the markets of the European Community? (yes – no)

Do you have a quality assurance system? (yes, we have - we have not, but plan to have one in □□□□ – we have not and do not plan to have one, either)

Have you created, in the course of the project, cooperation with other partners? (Yes, the number of our partners acquired in the course of the project is □□, among these □□ possess a quality assurance system – no)

What, do you think, would have happened to your project without the export supporting tender of the OMFB? Please, write down your opinion. (in words)


6. Appendices

Would the innovation embodied by the project have realised without the support? If not, why not? If yes, how much later? What would have been the effects of not having the support (a) on your enterprise (b) on the innovation? (in words)

Opinions about the donor organisation

Please rate the following statements according to what extent you find them to be true. (1 – not at all true, 2 - to a certain extent, 3 - more or less true 4 – satisfactorily true 5 - fully true)

The donor organisation (the OMFB)

…cooperates with us

…is a predictable partner

…developed a relationship of professional character with us

… developed an relationship of administrative character with us

…cooperates with us in an authoritative way

…is a helping partner

…is a business partner

…is a supervising organisation

…lets us work on our own

…wants to know about each phase of the project

…clearly puts down its conditions

…gives us freedom in the implementation of the project

…sets limits for us in the implementation of the project

…keeps continuous contact with us

…regards us as an equal partner

…helps the realisation of the project with ideas

…has delegated professional clerks for this task

…and our company has communicated with each other on equal terms.

The donor organisation has …

… the necessary expertise for the judgement of the applications

…a satisfactory administrative capacity

…a satisfactory quality of administration

…developed an administration which helps to implement the project

The tender…

…makes us think over the project thoroughly

…is fair because we take it for natural that we also have to undertake financial sacrifice in order to implement the project


6. Appendices

…is fair because we take it for natural to pay back the credit

…has defined requirements that are difficult to meet

…has contributed to our possibilities to improve quality

…conforms with up-to-date expectations

…corresponds to European norms

…benefits Hungary as a whole

Opinions about SME innovativeness and about the whole of the SME supporting system

Do you think such a system of tenders is justified in a market economy? Please write down your opinion briefly.

What kind of support / system of tenders would you need? Please write down your opinion briefly.

If you were a supporting organisation, how would you help the Hungarian companies to develop their ability to export, their level of innovation and their quality assurance system and how would you promote the accession to the EU? Please write down your opinion briefly.

Do you think the state should support innovative enterprises? If you think it should, (a) why, (b) how and (c) what kind of objectives of these enterprises should be supported?

How typical, do you think, it is that small- and medium enterprises are innovative? Under what conditions does a small enterprise become innovative?

What do you think, if your enterprise was bigger (or smaller), would it be harmful or useful for its innovative activity? Why?

Do you think the state should support small enterprises (regardless of whether they are innovative or not)? If yes, (a) why, (b) how and (c) what kind of objectives of these enterprises should be supported?

Thank you for your co-operation!

***

1.3. Outline of company questionnaire used for SMEs around Lake Balaton in Hungary (Case Study “Balaton")Questions asked via telephone


„Hello, this is X Y speaking on behalf of Lake Balaton Development Coordination Agency. We are making a sociological survey for the agency about the business climate and about the economic and social relations of enterprises in the region. Please, help our work by answering our questions."

1.3.2. Questions

When was your firm established?

Do you have any companies other than this?

What was the reason of you establishing your firm, and if your firm has undergone a transformation, give the reasons for that, too. (in words)

What is your firm’s most important sphere of activity? (in words)

How many full time employees do you have usually? (Estimation)


6. Appendices

What is the usual legal status of your employees? (employee – subcontractor – contract of assignment – owner – not official – other)

How large is your annual turnover?

Approximately how much was your profit following taxation (your profit) last year?

How much did you spend on investment last year?

In your opinion, compared to this year’s investments, more or less investment would be needed next year? (more –less the same)

Do you think it is possible that you will invest more next year? (yes – no)

What was the direction of your investments last year? (expansion – enlargement of profile – increase of productivity – protection of the environment – other)

What do you think: to what extent is your firm innovative? (Graded on a scale between 1 and 5)

In implementing innovations do you primarily rely upon yourself, or do you take advantage of various supports? (own resources only – also support)

Do you know in your settlement, about a business which has become successful with the help of a good idea? (there is – does not know of such a business)

Do you know, in your settlement a business which has a good idea with good chances to be successful, waiting to be implemented? (there is – does not know any)

What prevents this idea from being implemented? (in words)

Do you have partners (a) within your settlement (b) outside your settlement but within the Balaton Region (c) in the country outside the Balaton Region (d) foreign partners (e) who are large entrepreneurs/ from large enterprises (f) who are medium entrepreneurs/ from medium size enterprises (g) who are small entrepreneurs/ from small enterprises? (Yes – no)

The interviewers asked the following „sub-questions" in each case of the questions (a)-(g):

1. If you have, what is the character of this relation? (suppliers - customers- both - none)

2. How many percent of the turnover do you make with them? (10% - 20% - 30% - 40% - 50% - 60% - 70% - 80% - 90% - 100%)

3. How many percent of these relations is regular cooperation? (not occasional, not restricted to one occasion)? ( 10% - 20% - 30% - 40% - 50% - 60% - 70% - 80% - 90% - 100%)

4. How many percent of these partners is acquaintance / relative, whom you had known before entering into business connection with him/her (10% - 20% - 30% - 40% - 50% - 60% - 70% - 80% - 90% - 100%)

Has your company any interests in tourism? (yes – no) How many percent of the annual turnover can be related to tourism? (10% - 20% - 30% - 40% - 50% - 60% - 70% - 80% - 90% - 100%)

Now I am going to ask you about your personal business and public tasks. Naturally, these data will be worked up anonymously, separated from your telephone number.

Have you any public role, social obligations, commitments, tasks except managing the company? (If you have more, please speak about the one that you consider to be the most important.) If you have, what is it? (MP – local representative – other post at the local government – leader of an association – member of an association – leader of a foundation – member of a foundation – member of some kind of church organisation – other – there is no such task)

What other public tasks do you have? (in words) (Note: Sports organisation, club membership, membership of a social organisation or cooperation, social tasks within a cultural or church organisation, etc.)


6. Appendices

Do you maintain regular non-professional relationship with other entrepreneurs within the region (settlement)? (yes – no)

Please, say some words about this/these connections. (in words) (Note: any regular sponsoring, support, aid provided for civil organisations, churches, societies belong here regardless of the fact whether it comes directly from you or from the enterprise represented by you)

Do you undertake, as a private person or as the leader or the owner of an enterprise, sponsoring or charity activities? (yes – no)

What is this activity? (in words)

How many members of your family and relatives live in the settlement where your enterprise is operating?

Do they participate in building your business connections? (yes – no)

How many members of your family and relatives live outside the given settlement but within the Balaton region?

Do they participate in building your business connections?(yes – no)

How many members of your family and relatives live somewhere else in the country outside your region?


How many members of your family and relatives live abroad?


I am going to read out certain statements now. Please say, in each case, to what extent you agree with the statement. Place your agreement on a five grade scale, and mark five will mean that you fully agree with the statement.

1. Family constitutes the most important background to an enterprise.

2. I can be a successful entrepreneur in my fatherland only.

3. Personal connections play a very important role in the success of a business.

4. Entertainment, sports events, provide an outstanding opportunity to build connections important for the enterprise.

5. I prefer to make business with people I personally know.

6. I prefer to make business with people from the Balaton Region.

In your opinion, will your enterprise be more or less successful next year? (more successful – less successful)

What is the main reason for that? (in words)

In the following, I am going to read a list of statements. Please rate them as you did before, on a scale with five grades, where grade five means that you fully agree with the statement, and grade one means you do not agree with the statement at all.

1. I know the laws, orders and legal rules connected to my enterprise.

2. It is impossible to run a firm by complying with all laws , orders and decrees.

3. Taxation depends too often on skill and luck.

4. Local orders and decrees serve the interests of the local entrepreneurs.

5. I know and make use of the possibilities provided by the local orders.


6. Appendices

6. The orders of the local governments are nothing more than a new burden imposed on entrepreneurs.

Taking everything into consideration, do you regard yourself as a successful businessman? (yes – no)

Has your firm ever applied for some kind of support? (Options: preferential credit – employment support - social support – not-returnable subsidy – energy saving support – regional development support – investment support – other – has never applied for anything)

Does the current system of subsidies, in your opinion, correspond to the needs of the enterprises? (yes – no) (This question must be asked in case of each type of support, for which the interviewed has ever applied.)

Now I am going to read certain statements. Please say, in each case, to what extent you agree with the statement. Place your agreement on a five grade scale, and the mark five will mean that you fully agree with the statement.

1. Tenders are the most suitable solution for the distribution of supports.

2. Personal connections play an important role in obtaining support.

3. Favourable economic environment (reduction of taxes, incubator house, industrial park, shopping street) are more important than financial support.

4. Those companies obtain subsidies which submit the best applications, not those who need it most.

5. Those who judge the applications do not know anything about the situation of entrepreneurs.

6. The system of the distribution of tenders is good, only the money at disposal is little.

7. Many people do not know how to obtain the necessary support.

8. It is more important than financial support that offices reduced the time of administration.

9. The Lake Balaton Development Council promotes the cooperation of the enterprises of the region.

How much subsidy did your company receive last year? (… Hungarian Forints)

Has the local government supported the entrepreneurs of the settlement in any way and if it did, how?

What kind of resources have you used since you started your venture? (More than one answer is possible.) (Own resources – Bank loan – Loan but other than bank loan – The help of relatives and acquaintances – Support from the local government – State support – Foundation support – EC support).

Do you know of the supporting activity of Lake Balaton Development Council? (does not know - knows only a little about it - has been given support - knows somebody who has been given support)

Have you taken advantage of any other kind of support? If you have, what kind of support was that?

The gender of the person who answered the questions (male – female)

What was your highest school qualification when you established your enterprise? (8 classes of elementary school or under – vocational school – secondary grammar school with a school leaving exam – college - university)

Have you participated in some relevant training in the interest of running your enterprise since the time when you established it? (Professional technical or management trainings) (Yes - no)

What kind of training was that? (in words)

What kind of further professional or managerial trainings do you think, you would need?

Apart from your mother tongue which language(s) do you speak? On which level? (basic, intermediate, advanced)


6. Appendices

What was your father’s highest school qualification? (8 classes of elementary school or under – vocational school – secondary school with school leaving exam - college – university)

How many children do you have?

When were you born?

Your family status (single – married – divorced – widow(er)

Is your enterprise a family business? (yes – no)

Does your child participate in the tasks of the business? (yes – no)

Would you like your child to carry on with your business? (yes – no)

Thank you for having helped our work with your answers. If you want to see the summary of the results of this survey, we shall send it to you by post or via e-mail. We wish you further success in your work.

1.4. Outline of company questionnaire used for RIA on technical regulation (Case Study “Croatian RIA")The questions were directly addressed to the subjects or sent to them via mail.


European legal harmonisation in the field of conformity assessment of electrical devices.

Background of the survey

Dear Madame/Sir,

The Ministry of European Integration of the Croatian government prepares a regulatory impact assessment in connection with the planned legal harmonisation of some European Union regulations. In particular, we would like to assess the costs and benefits of introducing new rules on the conformity assessment and testing of electrical devices, and want to learn what impacts this measure would bring for companies. Our survey examines the possible impacts which are likely to emerge following the introduction of the so-called Low Voltage Directive of the EU into the Croatian legal system among enterprises producing and selling electric equipment and also among service providers and conformity assessment institutions and testing laboratories. The introduction of the directive is one of the conditions for Croatia to become part of the unified (single) European market.

The introduction of the directive will fundamentally change the present practice of regulation for putting in electric devices on the market because; following this examination the products in question may enter the market only if they meet the safety requirements of the directive. The conformity of the products with these requirements can be assessed in several ways: (a) by the old method, i.e. by certifying them at the appointed Croatian institutions, but also by several new ways, such as (b) using the certification obtained from an analogous foreign institutions or, (c) in case the producer has satisfactory capacity for such examinations, the producer can also issue the certificate regardless of whether the article has been produced inside or outside the country. The present appointed Croatian conformity assessment institutions will be transformed from law enforcement authorities into bodies entitled to issue certifications. Only such electrical devices can be put on the Croatian market, which rightfully bear the CE sign verifying the conformity of the product with EU requirements. The import of products rightfully carrying the CE sign cannot be prevented.

The survey examines the expected impacts of the above changes.

We would like to receive answers from enterprises which produce, import or export low voltage electrical appliances, that is, devices operating with alternating current (AC) between 50 and 1000 Volt or with direct current (DC) between 75 and 1500 Volt. We have compiled another questionnaire conformity assessment institutions and testing laboratories.


6. Appendices

Your answers will help us to introduce the Low Voltage Directive into Croatian law without causing unnecessary costs but with making the most of the benefits. Your answers will constitute part the impact assessment analysis and will influence the process of legislation. The complete text of the analysis resulting from this survey will be sent to you.

1.4.2. Questions

How many pieces of low voltage electric devices did your company produce in EU countries / import from EU countries / from non-EU countries / export to EU counties / to non-EU countries / sell in Croatia last year?

How many types of low voltage electric devices did your company produce in EU countries / import from EU countries / from non-EU countries / export to EU countries / export to non-EU countries / sell in Croatia? (A device has a different type from other devices if it must undergo separate conformity assessment tests.) Please name the 5 most important types of devices.

At your company how important are those activities which promote the safety of your products? Why? How does this influence the design / development / production / import / export of your products? How does this influence entrepreneurial organisation, the training of your employees? Please estimate the costs which may be led back to activities serving the safety of your products and demonstrate them in a purposefully chosen distribution.

Does your company have a testing capacity in Croatia? If it does, what kind of conformity assessment tests can you perform there, how much did the development of your testing capacity within your company cost, how much are its annual operational expenses? If it doesn’t, are you planning to develop one? If you are, how much will it cost you? What kind of conformity assessment tests do you make in cooperation with specialised outside laboratories?

What additional expenses will it cause you if you have to change your products so that they will meet the requirements of the Low Voltage Directive?

Has your company a quality assurance system? If it has, how does it affect the safety of those low potential products that you produce / import / export / sell in Croatia?

Does your company work as subcontractor or supplier to another firm? If it does, how many percent of your total revenue do these sales amount to? How important factor is the safety of low potential electric products in these subcontractor-purchaser relations or in the cooperative activities?

Does your company have suppliers or subcontractors? If it does, how many percent of your production costs does it amount to? How important factor is the safety of low potential electric products in these subcontractor-purchaser relations or in the cooperative activities?

With regard to the safety of low potential electric products have you ever had connections with (a) government organisations (b) legal courts (c) consumer protecting organisations (d) chambers? If yes, what kind of interaction was / is that? (Complaint, lawsuit, representation of interest, etc.) Please briefly describe a typical case.

What is the magnitude of annual expenses attributed to conformity assessment examinations and obtaining the certificates? What is the composition of these costs according to the following: testing within the company – purchase of standards or having them translated – labour costs – services purchased from laboratories. What is the composition of these costs according to the following: fix costs (e.g.: investment) – annually recurrent costs (e.g.: maintenance, testing)

What is the magnitude of annual expenses attributed to conformity assessment examinations and testing, as projected on one product type?

How many people are employed, at your firm, to perform conformity assessment administration and testing of electric devices? Do you have a department specialised for this job?

Is your firm part of a larger international corporation? If yes, how is the division of labour organised for conformity assessment administration and testing between the local subsidiary company and the foreign departments of the mother company?


6. Appendices

Does your company turn to Croatian laboratories, conformity assessment testing organisations in order to have your products tested and to obtain the necessary certificates? If it does, which are those institutions, how many types of products do you have examined and for how much? Please, divide these estimates according to types of products, types of examinations, and laboratories. Are you satisfied with the accomplishment of these service providers? Why?

Does your company turn to foreign laboratories, conformity assessment testing organs in order to have your products tested and to obtain the necessary certificates? If it does, which are those institutions, how many types of products do you have examined and for how much? Please, divide these estimates according to types of products, types of examination, and laboratories. Are you satisfied with the accomplishment of these service providers? Why?

Are conformity assessment tests easier to complete in Croatia than abroad? What do you think; will the legal harmonisation of the Low Voltage Directive generate a fiercer competition between Croatian laboratories in the future?

Are there any products, produced /imported /exported /sold in Croatia by you, which had to be tested twice, both in Croatia and abroad? How much can you save by not having to have such a product tested again which already rightfully has the sign CE on it?

Has your company had such an experience that a product could not be launched on the market because its import / export / sale was prevented with reference to product safety reasons? If it has, which was that market? Please, describe the case briefly.

Has it ever occurred that your firm had to act according to the requirements of the Low Voltage Directive in the course of export, supply, or purchasing?

Which other technical directives of the European Union concern your products? (Machinery Directive, Electromagnetic Compatibility Directive, etc.)

Do you think that the legal harmonisation of the European Community concerning the conformity assessment of electrical devices will help your company? Which activities of your firm will be especially facilitated or hindered? (production – import – export – selling – investment). What risks does this mean for you and/or for your competitors?

If you had the possibility to delay the introduction of the Low Potential Directive by 2 years, would you make use of this possibility? What would your firm do during this time? Which firms, in your opinion, will be the winners and the losers of the introduction of the directive and why?

General questions about the enterprise: Number of employees – Turnover.

Turnover originating in the production /import / export / local wholesale or retail sales – installation – service of electrical devices.

Events and numbers characteristic of the development or organisational transformation of your firm.

The structure of ownership (Croatian owned – foreign owned) and its changes during the past decade.

2. Definitions - Glossary 1

Term Definition

Administrative cost Administrative costs are defined as the costs incurred by enterprises, the voluntary sector, public authorities and citizens in meeting legal obligations to provide information on their action or production, either to public authorities or to private parties. The

11The definitions in this Glossary are edited versions of the definitions given in basic methodological documents of international organisations such as the EU, the OECD and the World Bank.


6. Appendices

Term Definition

administrative costs consist of two different cost components: the business-as-usual costs and administrative burdens. While the business-as-usual costs correspond to the costs resulting from collecting and processing information which would be done by an entity even in the absence of the legislation, the administrative burdens stem from the part of the process which is done solely because of a legal obligation.

Baseline scenario In the context of an impact assessment the aim of the baseline scenario is to explain how the current situation would evolve without additional public intervention – it is the ‘no policy change’ scenario. A clear baseline scenario also functions as the basis for comparing policy options

Before-after (pretest-post test) control group design A type of experimental design applied in policy research and in other fields of social research such as marketing research. The measure may be a policy, a regulation or a marketing strategy. The target group is exposed to the measure, while the control group is not. Pretest (i.e. before the implementation of the measure) and posttest (after) measurements are made on both groups.

Benefit Compliance with the requirements of regulations may generate positive effects, which are regarded as benefits. Such benefits are, for example, advantageous changes in the field environment protection, food safety, the safety of industrial products or traffic safety. A decrease in compliance costs can also be regarded as a benefit. For example, if the regulation replaces an earlier, less advantageous regulation, then the decrease of compliance costs may increase the willingness of entrepreneurs to invest and innovate and the resulting increase of competitiveness can also be registered as a positive impact.

Beneficiaries The individuals, groups, or organizations, whether targeted or not, that benefit, directly or indirectly, from the development intervention.

Better Regulation Programmes Many countries apply RIA as an integral part of long term programmes for checking and maintaining the quality of planned regulations. In most of these cases the main purpose of preparing RIAs is to improve the regulatory environment for businesses, citizens and consumers, to use RIAs in the consultation process.

Business angel A business angel is an informal investor, usually a successful entrepreneur, who is willing to invest in high-risk, high-growth firms at a very early stage, and adds value by supplying hands-on business advice as well.


6. Appendices

Term Definition

Business climate Business climate is a term which indicates how business development is supported by state, regional and local policies, by local communities, how business-to-business networks and labour relationships facilitate business activities. A good business climate allows businesses to conduct their affairs with minimal interference from authorities, while enabling the access to high quality inputs and to customers at low costs, offering investment possibilities with few risks and higher returns when compared to other places. The key factors used in the measure of business climate include macroeconomic stability, business and income tax levels, workforce availability, energy costs, market size, availability, cost and quality of services, real estate and infrastructure, business friendliness of the regulatory framework, access to financing, capital and incentives.

Business cycle Business cycles are a type of fluctuation found in the aggregate economic activity. A cycle consists of an expansion, i.e. a time period characterised by economic growth, followed by recessions or crises.

Business survey Business surveys are questionnaire based surveys in which a sample of companies is asked about company activities, plans, expectations and their opinion about the business environment. Small business surveys are crucially important information sources for analysing and modelling business cycles

Causality Causality is the relationship between an event (the cause) and a second event (the effect), where the second event is understood as a consequence of the first one. Every impact assessment must cope with the basic problem of causal inference. Since the outcomes under the counterfactual (hypothetical) scenario are not observable, the comparison of the two sets of outcomes under the two scenarios cannot be based on a solid empirical basis. This leads to the so-called fundamental problem of causal inference, i.e. that causality is not directly observable. However, a series of impact assessment methods have been developed to make indirect inferences to causality.

Competition policy Government policy to prevent and reduce the abuse of monopoly power and to ensure fair competition.

Competitiveness The ability and performance of a firm, sub-sector or country to sell and supply goods and/or services in a given market. National level competitiveness is dependent on the set of institutions, policies, and factors that determine the level of productivity of a country. The term may also be applied to markets, where it is used to refer to the extent to which the market structure may be regarded as perfectly


6. Appendices

Term Definition

competitive. This usage has nothing to do with the extent to which individual firms are competitive.

Compliance cost An expenditure of time or money in conforming to government requirements such as legislation or regulation.

Control group In program evaluation, the group that does not participate in the program. In the context of experimental design of impact assessment in the framework of SME policy: A group of companies that has not been exposed to the policy intervention is called a control group, if its composition is comparable and similar to the group which was exposed to the policy intervention. The two groups are compared in order to clarify, whether the outcomes can be attributed to the intervention, i.e. whether a causal mechanism can be identified.

Cost-benefit analysis Cost-benefit analysis is a technique designed to determine the impacts of a project or plan by quantifying and monetising both its costs and benefits. Its main steps are as follows. (1) The researcher lists all expected benefits and costs, and calculates (or estimates) their expected magnitude (in physical terms). (2) Lists the expected time-path of the impacts. (3) Express the value of the impacts in monetary terms for each time period (e.g. year). (4) Calculates the discounted value of costs occurring in the future by using a standard discount rate (e.g. the central interest rate) (5) Calculates the discounted value of benefits by the same method. (6) Calculates the net present value by subtracting the above two values from each other. (7) Indicates which margins of error or uncertainty need to be taken into account.

Cost-effectiveness analysis Cost-effectiveness analysis is a technique designed to determine the impacts of a project or plan by quantifying its costs, but not expressing its benefits in monetary terms. The starting point of this analysis is that while the costs attributable to an intervention can be more or less precisely quantified, benefits cannot. That is why this approach either (a) compares alternative plans that guarantee the same benefits and chooses from among these the cheapest one, or (b) compares various alternatives characterised by the same cost level and chooses the one which promises the most favourable benefits.

Counterfactual A hypothetical situation, condition, course of events or a scenario which is running contrary to the facts. In ex post impact assessment a counterfactual scenario is the likely course of events that would have happened if the analysed measure (an implemented policy or regulation) had not been taken. This scenario is called


6. Appendices

Term Definition

counterfactual, because it is disregarding the fact that in reality the measure has been implemented..

Dependent variable The variable to be explained in a multiple regression model. In the context of impact assessment, an indicator which shows, to what extent the implementation of the measure successful or efficient.

Deregulation The removal or simplification of government rules and regulations that constrain the operation of market forces. Deregulation programmes and campaigns attempt to keep the necessary laws against fraud, pollution or dangerous food, while eliminating or reducing government control of how business is done. Deregulation is an activity where RIA can be successfully applied. If there are doubts about phasing out a regulation or not, a simplified RIA procedure can help in decision making.

Difference-in-differences method The impact of a policy on an outcome can be estimated by computing a double difference, one over time (before-after) and one across subjects (between beneficiaries and non beneficiaries).

Direct impacts These are the changes in costs and investments that arise as an immediate consequence of complying with the regulation. Changes in administrative activities, company re-organisations, and change of products and technologies that are attributable to the regulation are converted into monetary terms and interpreted as the total of the direct costs arising for the enterprise.

Discount rate In the context of cost-benefit analysis, the future costs and benefits are discounted (reduced) because it is assumed that they are not worth as much to people today, because people would rather have money and benefits now than later. In order to put everything into today's values a discount rate is applied to future values. The choice of a discount rate makes a big difference to the outcome of a cost-benefit analysis.

Distributional effects The compliance costs and the benefits of a policy intervention are unevenly distributed among the affected industries, consumers, regions. In the context of impact assessment, distributional effects are criteria showing the extent to which a policy design will result in disproportionate impacts on different regions, sectors, or households

Doing Business A project of the World Bank providing objective measures of business regulations for local firms in 183 economies and selected cities at the sub national level. By gathering and analyzing comprehensive quantitative data to compare business regulation environments across economies and over time, it offers


6. Appendices

Term Definition

country rankings since 2004.

Dual economy A dual economy is the existence of two separate economic sectors within one country, divided by different levels of development, technology, and different patterns of demand.

Effectiveness The extent to which the development intervention’s objectives were achieved, or are expected to be achieved, taking into account their relative importance.

Endogenous variables In simultaneous equations models, variables that are determined by the equations in the system.

Europeanisation process In the context of the European integration process: the institutionalization of a distinctly European political system.

Evaluation The systematic and objective assessment of an on-going or completed project, programme or policy, its design, implementation and results. The aim is to determine the relevance and fulfilment of objectives, development efficiency, effectiveness, impact and sustainability. An evaluation should provide information that is credible and useful, enabling the incorporation of lessons learned into the decision– making process of both recipients and donors. Evaluation also refers to the process of determining the worth or significance of an activity, policy or program. An assessment, as systematic and objective as possible, of a planned, on-going, or completed development intervention.

Ex-ante evaluation An evaluation that is performed before implementation of a development intervention.

Ex-post evaluation Evaluation of a development intervention after it has been completed.

Experimental design A type of research design in which the conditions of a program or project (also called as „treatment") are controlled by the researcher whereby subjects are randomly assigned to treatment group or to the group which does not receive the treatment (the so-called control group). The control group is a valid comparison group since there should be no systematic difference between their characteristics and those of the treatment group. Observations or measurements are made on both groups in order to reveal the impacts of the treatment. The purpose of experimental design is to rule out alternative causes, leaving only the actual treatment that is the real cause.

External evaluation The evaluation of a development intervention


6. Appendices

Term Definition

conducted by entities and/or individuals outside the donor and implementing organizations.

Externality In economics, an externality (or transaction spillover) is a cost to be paid or a benefit enjoyed by a party (a „third person") who did not agree to the action that causes the cost or benefit. These costs of benefits are not appearing in, are not transmitted through the prices of the action that causes the externality. A benefit in this case is called a positive externality or external benefit, while a cost is called a negative externality or external cost. Many negative externalities (also called "external costs" or "external diseconomies") are related to the environmental consequences of production.

Fiscal and monetary policies Fiscal policies are government decisions usually relating to taxation, customs and government spending. Monetary policy is the regulation of the supply of money and interest rates, controlled by the National Bank.

Full RIA A detailed Regulatory Impact Assessment study with reliable findings and recommendations, based on a wide range of information sources such as consultation. As a rule, full RIAs are not prepared by Government officials, but are outsourced to external consultants, researchers or to academic organisations.

Governance In the context of public administration, governance is what a government does, i.e. the management power and policy by using the government as an instrument.

Impact Positive and negative, primary and secondary long-term effects produced by a development intervention, directly or indirectly, intended or unintended. Theoretically, an inference about the existence of an impact of an intervention can be made only by comparing two scenarios: (a) the one in which the assessed measure was taken and (b) another scenario in which the measure was not taken. One of the above scenarios is counterfactual.

Impact Assessment A process that prepares evidence for political decision-makers on the advantages and disadvantages of possible policy options by assessing their potential consequences. Analyses, explanations about the expected of certain social-political interventions, policies and measures, based on empirical evidence.

Independent variable In regression analysis, a variable that is used to explain variation in the dependent variable.

Indicator Quantitative or qualitative factor or variable that provides a simple and reliable means to measure


6. Appendices

Term Definition

achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor.

Indirect impacts Indirect impacts are developments caused by a project or policy intervention, but unlike direct effects, occur (a) either later in time or (b) further away along the causal chain or (c) are farther removed in distance or (d) as a result of a complex causal pathway. A regulation may have un-intended indirect effects such as a group of companies gaining or losing markets, or experiencing changes in productivity and competitiveness. The direct consequence of a simplified customs procedure can be the reduction of transaction costs of exporting firms, which may trigger the indirect effect of increased trade between certain countries.

Initial RIA An initial RIA (Regulatory Impact Assessment) is a relatively short document prepared at an early stage of the policy process. It can consist of a rough and quick analysis based on what is already known, including the best available estimates of the possible risks, benefits and costs. The document helps to identify areas where more information is needed.

Internal Rate of Return The internal rate of return is a calculated discount interest rate which if applied, yields a zero net present value to the investment. If the calculated internal rate of return is negative or low the donor may prefer alternative projects, which yield better interim cash flows.

Interrupted Time Series Design A research design using the time series of a key variable for a time interval which starts well before the date of the intervention and ends well after that date.

Legal harmonisation Cooperation between governments to make laws more uniform and coherent. The European Community has institutionalised legal harmonisation in order to achieve uniformity in laws of member states in various policy fields, e.g. to facilitate free trade or to protect fundamental rights of citizens. Several potential member countries of the European Union have been embarking on extensive or sporadic RIA programmes for facilitating the harmonisation process between European laws and the respective national legislations.

Logical Framework Management tool used to improve the design of interventions, most often at the project level. It involves identifying strategic elements (inputs, outputs, outcomes, impact) and their causal relationships, indicators, and the assumptions or risks that may influence success and failure. It thus facilitates planning, execution and evaluation of a


6. Appendices

Term Definition

development intervention.

Macro-econometric models Econometric models, usually a set of equations, using macro level statistical data. These models are designed to evaluate macro-economic and sectoral impacts of economic or environmental policies.

Market failures Lack of efficiency in the allocation of goods on a particular market of products or services. Scenarios where individuals' pursuit of pure self-interest leads to results that are not efficient from the point of view of the society. Market failures are often associated with (a) biased or asymmetric information, (b) non-competitive markets or (c) costs to be paid by third parties (externalities). The existence of a market failure is often used as a justification for government intervention in a particular market.

Monitoring A continuing function that uses systematic collection of data on specified indicators. It provides project management and the main stakeholders of an ongoing development intervention with indications of the extent of progress and achievement of objectives and progress in the use of allocated funds.

Multicriteria analysis Evaluation technique whereby various aspects and dimensions of positive and negative impacts are combined and aggregated into a single framework to allow easier comparison of scenarios. It allows to present impacts that are a mixture of qualitative, quantitative and monetary data, and where there are varying degrees of certainty. Multicriteria analysis usually will be implemented by defining a scoring, ranking or weighting system which expresses the importance or monetary value attached to each of these criteria.

Net Present Value The difference between the sum of discounted benefits and the sum of discounted costs of a policy option is called its net present value. Can be calculated if both costs and benefits are monetizable and easy to forecast for the full analysis period.

One-stop systems of administrative procedures One-stop systems – also called single window systems – are client service offices of authorities, offering a wide range of administrative services for companies and citizens at one location or on-line. Frequently, several authorities that have the same target group of companies jointly offer these services.

Partial RIA A partial RIA is a Regulatory Impact Assessment document building on the initial RIA, but its sophistication and level of detail does not reach that of the full RIA. It includes refined policy options on regulation, compliance, monitoring, cost and benefit


6. Appendices

Term Definition

estimates and risk analysis. An assessment of the impacts on competition and on small firms should be included.

Performance of a development intervention The degree to which a development intervention or a development partner operates according to specific criteria/standards/ guidelines or achieves results in accordance with stated goals or plans.

Performance measurement A system for assessing performance of development interventions against stated goals.

PHARE The Phare programme is one of the three pre-accession instruments financed by the European Union to assist the applicant countries of Central and Eastern Europe in their preparations for joining the European Union. Originally created in 1989 as the Poland and Hungary: Assistance for Restructuring their Economies (PHARE) programme, Phare has expanded from Poland and Hungary to cover ten countries.

Pilot Project A pilot project is generally a project which is designed as a test or trial to demonstrate the effectiveness of a full program.

Policy A policy is typically described as a deliberate plan of action to guide decisions and achieve rational outcome(s). The term may apply to government, private sector organizations and groups, and individuals. Policies are guided by strategies, involving aims and priorities. Policy instruments include rule making (e.g. legislation) , projects and programmes.

Policy area The term is used to describe a specific set of interventions, laws, regulations, programmes, projects which are implemented in order to serve an overarching aim, e.g. economic development, social inclusion, reduction of unemployment, public finances, etc. In typical public administrations the responsibilities of a ministry cover one or more policy areas.

Process Evaluation A type of policy or project evaluation, whereby the identification of impacts are not the priority focus of the evaluation exercise. Instead, the major aim of the researchers is to qualify the design and the process of policy / project implementation.

Programme Organised set of financial, organisational and human interventions mobilised to achieve an objective or set of objectives in a given period. As a rule, programmes are smaller or larger aggregates of more or less homogenous projects serving similar purposes. E.g. various individual sub-projects can be aggregated into


6. Appendices

Term Definition

projects or into programmes facilitating development in a specific sector. The aggregate of projects affecting a specific sector, or having identical objectives can be called a programme.

Probabilistic impact The notion of impact can also be defined in probabilistic settings in the following manner: the occurrence of an event (the cause) increases the probability of another event (the impact), whereby the strength of the causal relationship is measured by the increase of this probability.

Project Non divisible operation, delimited in terms of schedule and budget, and placed under the responsibility of an operator. Examples for projects: holding a training course, building a bridge, distributing a pre-determined amount of mosquito nets in malaria infected region.

Project Evaluation Evaluation of an individual development intervention designed to achieve specific objectives within specified resources and implementation schedules, often within the framework of a broader program. Project evaluation is a phase of project cycle management, which in turn is a method of aid delivery at international and national donor organisations. It is a type of micro level inquiry about the economic and administrative rationality and about the success or failure of a project. Project evaluation is the implementation of a detailed system of criteria which has been developed at donor organisations.

Qualitative methods Qualitative method is a family of inference methods, whereby the empirical basis of the research is spoken or written text, participant observation. It is opposed to quantitative methods whereby the data to be analysed is of numerical / statistical nature. Examples of quantitative methods are semi-structured interviews, open interviews, focus group interviews, participant observation, discourse analysis and document analysis.

Quantitative methods Quantitative methods are inference methods whereby the data to be analysed is of numerical / statistical nature. In the context of impact assessment the quantitative techniques can be broadly categorized into experimental, quasi-experimental, and regression-based techniques.

Questionnaire-based surveys A survey is a method for collecting quantitative information about items in a population. A questionnaire is a survey instrument consisting of a series of questions for the purpose of gathering information from respondents.

Regression discontinuity design Nonexperimental evaluation method. It is adequate for programs that use a continuous index to rank potential


6. Appendices

Term Definition

beneficiaries and that have a threshold along the index that determines whether potential beneficiaries receive the program or not. The cut-off threshold for program eligibility provides a dividing point between the treatment and comparison groups.

Regression In statistics, regression analysis includes any techniques for modelling and analyzing several variables, when the focus is on the relationship between a dependent variable and one or more independent variables. It helps us understand how the typical value of the dependent variable changes when any one of the independent variables is varied, while the other independent variables are held fixed. A bivariate linear regression line has an equation of the form Y = a + bX, where X is the explanatory variable and Y is the dependent variable. The slope of the line is b, and a is the intercept (the value of y when x = 0). A multivariate regression has several independent variables.

Regulatory Impact Assessment (RIA) RIA is a fact based analysis which is used as a systematic decision tool in public administrations in order to examine and measure the likely benefits, costs, risks, and competition effects, distributional effects of new or existing regulation.

Regulatory policy Regulation concerns how governments intervene in the economy through laws and other instruments, in pursuit of social, economic and environmental objectives, in order to enhance the functioning of markets or to protect health, safety, or the environment. The policy area of regulatory policy aims at developing good regulations, at establishing a long-term basis for efficient and responsive regulation. The instruments of regulatory policy are administrative simplification, deregulation, regulatory impact assessment; enhancing transparency and communication, the development of alternatives to regulation and revealing and disseminating best practices of regulatory compliance and enforcement.

Regulatory quality The quality of a regulation is determined by the following conditions. The regulation should serve clear policy goals. The regulation should be effective in achieving the above goals. The regulation should possess a legal basis and be consistent with other regulations and policies. The regulation should be clear, simple, and practical for users. The necessity and the effectiveness of the regulation should be empirically verified. The regulation should produce benefits that justify its costs, by taking into consideration economic, environmental and social effects and the distribution of these effects across society. Compliance with the regulation should be


6. Appendices

Term Definition

possible with as little compliance costs as possible. Enforcement of the regulation should be possible with as little enforcement costs as possible. The regulation should be compatible as far as possible with competition, trade and investment-facilitating principles at domestic and international levels and should distort the market to the smallest possible extent.

Research design The term "research design" in social science refers to how a researcher puts a research study together to answer a question or a set of questions. Research designs are composed of the following four major components: (a) the research question (2) the underlying theory (3) the data and the sampling strategy for collecting the data (4) and the use of data in order to make inferences, i.e. a plan that shows how the researcher expects to use his/her evidence to make inferences.

Risk analysis/ Risk assessment Risk assessment is the determination of quantitative or qualitative value of risk related to a concrete situation and a recognized threat (also called hazard). Risk analysis characteristically analyses hazards related to safety, health and environmental issues. This is a method to estimate (a) the level, i.e. the extent of disadvantageous consequences of decisions and (b) their probability, i.e. the likelihood of these unfavourable events to occur. In the context of development policies, a detailed examination of risks to be mitigated by the intervention and an analysis of the potential unwanted and negative consequences to human life, health, property, or the environment posed by development interventions.

Scenario The course of expected events under a certain hypothesis is called scenario. Impact statements are based on the comparison of various scenarios.

Sensitivity analysis In the context of impact assessment, sensitivity analysis addresses the impact of different assumptions on the effectiveness of policy options. It can be used to explore how the impacts of the analysed change in response to variations in key parameters and how they interact.

Small and medium sized enterprises (SMEs) Official definition of the EU:

1. A micro enterprise has less than 10 employees, its turnover does not exceed 2 million EUR and its balance sheet total does not exceed 2 million EUR.

2. A small enterprise has less than 50 employees, its turnover does not exceed 10 million EUR and its balance sheet total does not exceed 10 million EUR.


6. Appendices

Term Definition

3. A medium sized enterprise has less than 250 employees, its turnover does not exceed 50 million EUR and its balance sheet total does not exceed 43 million EUR.

SME development policy A policy area developing the SME sector by improving its legal environment, by facilitating the access of small businesses to markets, moreover to capital, know how, premises and other factors.

Social capital Social capital is a measure of social embeddedness of economic actors. In the context of entrepreneurship social capital is the ability of entrepreneurs to utilise the necessary resources such as network contacts to start and/or run a business. It is assumed that the social network can provide financial capital, information, potential employees, or access to clients.

Social cost In the context of impact assessment, a cost item expressing the cost of the regulation for the society as a whole. It covers the costs to the enterprises, those to the government, and those to the consumers as well.

Social embeddedness of businesses A state of businesses being located or secured within a larger social entity or context, , the extent to which company behaviour is determined not only by economic considerations (e.g. maximization of profits) but also by social aspects. The economic life of a firm or market is territorially embedded in its peculiar social and cultural relations: in place-specific characteristics, infrastructure, operating environments, and conditions of production.

Spillover effects See Externality

Stakeholders Agencies, organisations, groups or individuals who have a direct or indirect interest in the development intervention or its evaluation.

Standard Cost Model (SCM) The Standard Cost Model (SCM) is a method for determining the administrative burdens for businesses imposed by regulation.

Sustainability The continuation of benefits from a development intervention after the project or development assistance has been completed. The probability of continued long-term benefits. The resilience to risk of the net benefit flows over time.

Target Group The intended beneficiaries (individuals, households, groups, firms) of an intervention. An intervention may have more than one target group. This term should be distinguished from "population" in the statistical sense. In the particular case of SME development the target


6. Appendices

Term Definition

population can be selected as a well defined subgroup of small businesses, such as start-ups, potential subcontractors, innovative firms, companies in lagging behind settlements, etc.

Theory-based evaluation In the context of policy evaluation, theory-based evaluations explain the mechanisms of the determining or causal factors and the logical sequence judged important for success of a policy intervention. Failure to be clear about the causal sequence by which a policy is expected to work can result in well intentioned policies being misplaced, and outcomes that are contrary to those that were anticipated.

Top-down initiatives In the context of organizational theory, top-down initiatives are initiated by an executive, decision maker or at the level of the central government and implemented by lower levels in the hierarchy, e.g. in the decentralised public agencies. It is opposite to bottom-up initiatives.

.

Treatment group In program evaluation, the group that participates in the program. The group which does not participate, is called the control group.

3. References[Australian Government 1999] “Regulatory Performance Indicators - A Guide for Departments and Agencies" Australian Government (1999).

[Bartus and others 2005] Bartus Tamás, Lannert Judit, Dr. Moksony Ferenc, Németh Szilvia, Papp Z. Attila, Szántó Zoltán: A fejlesztéspolitikai intézkedések társadalmi hatásainak vizsgálata" Manuscript, TÁRKI Budapest, 2005. – In Hungarian. Translated title: “Assesment of social impacts of development policy measures"

[Bateman 2000] Milford Bateman: „Neo-Liberalism, SME Development and the Role of Business Support Centres in the Transition Economies of Central and Eastern Europe" Small Business Economics, 2000 Volume 14 p. 275–298.

[Bérard 2003] “Dealing with misclassified units in repeated business surveys: the experience of the redesigned Canadian Monthly Wholesale and Retail Trade Survey" Hélène Bérard (Statistics Canada). SSC Annual Meeting, Proceedings of the Survey Methods Section, June 2003.

[BFT 2005] Balaton Fejlesztési Tanács: „A Balaton régió fejlesztési stratégiája 2007 - 2013" Készítette: Vital Pro Kft. 2005. december 12. – Lake Balaton Development Council: “Development strategy in the Balaton Region 2007 – 2013" Prepared by Vital Pro Ltd.

[BIFT and others 2006] „LIFE Balaton Project. Integrált döntés támogató rendszer megvalósítása a fenntartható turizmus elérése érdekében a balatoni régióban" – “LIFE Balaton Project. Implementation of an integrated decision support system in the Balaton Region in order to achieve a sustainable tourism" by Lake Balaton Development Coordination Agency, the Department of Regional Geography at ELTE TTK, the Association of NGOs in the Balaton Region, and the Geonardo Ltd., 2006.


6. Appendices

[Binning-Futo 1997] Kenneth Binning - Futó Péter: „A jogharmonizációs szabályozás gazdasági hatásvizsgálata“ Statisztikai Szemle, 1997. 2. sz. 101 - 114. p. In Hungarian. (Title in English: Economic impact assessment of regulations for legal harmonisation)

[Bourn 2006] Bourn, John: “Evaluation of Regulatory Impact Assessments “2005 - 06. National Audit Office, London, 2006, 43 oldal

[BRE 2005] U.K. Better Regulation Team Small Business Service (2005 november): “Small Firms Impact Test Guidance For Policy Makers" London. 7 oldal.

[BRE 2007] The suggested methodology of RIA can be downloaded from the website of the UK Department for BERR “Better Regulation Executive" :

http://www.cabinetoffice.gov.uk/regulation/ria/ria_guidance/index.asp

[BSIC 2006] “Red Tape for Business in Sweden. Regulation Indicator for 2006" Board of Swedish Industry and Commerce for Better Regulation (NNR) August 2006, Stockholm, 32 oldal.

[Conlin-Stirrat 2008] Sean Conlin and Roderick L. Stirrat: Current Challenges in Development Evaluation. Appeared in: Evaluation. Special issue European Evaluation Society Conference. Volume 14. No. 2. April 2008.

[Crain 2005] Crain, W. Mark: „The Impact of Regulatory Costs on Small Firms", Lafayette College, Easton, Pennsylvania, 2005.

[Dabla-Norrisa and Inchauste 2007] Informality and Regulations: What Drives Firm Growth? Era Dabla-Norris and Gabriela Inchauste. IMF Working Paper. Middle East and Central Asia and IMF Institute. May 2007

[Dallago 1999] Bruno Dallago: „Context and Policies for the Transformation and Growth of SMES. The Case of Hungary with Russian Implications." Research paper prepared within the framework of the UNU-Wider Project. Helsinki 1999, 89 p.

[de Soto 2000] de Soto, Hernando: „The Mystery of Capital. Why Capitalism Triumphs in the West and Fails Everywhere Else" (New York: Random House, 2000); 243 pages.

[EAR 2005] European Agency for Reconstruction, Programming and Quality Assurance Division, Evaluation Unit: “Evaluation Guidelines" Revision 1 – May 2005. EAR/2005/30/04. downloadable in the summer 2007:

www.ear.eu.int/publications/main/documents/RevisedEvaluationGuidelines.pdf

[EC 1997] "The European Commission's Business Impact Assessment System" Published as part of the "Improving the Quality of Legislation for Business" series. Brussels, 1997. Directorate General XXII, p. 579.

[EC 2000] “European Charter for small enterprises". Annex III of the Conclusions of the Presidency of the Santa Maria Da Feira European Council of 19 and 20 June 2000.

[EC 2001] „European Governance – a White Paper" Commission of The European Communities. Brussels, 25.7.2001. COM(2001) 428 final.

[EC 2002a] Commission of the European Communities (May 2002): „Communication from the Commission on Impact Assessment" COM(2002) 276 final, Brussels, May 6 2002, 19 pages.

[EC 2002b] Enterprise Directorate - General, European Commission (2002): „Business impact assessment pilot project - Final report – Lessons learned and the way forward", Enterprise Papers, No 9., 2002, 27 oldal

[EC 2003a] „Jobs, Jobs, Jobs. Creating more employment in Europe" Report of the Employment Taskforce chaired by Wim Kok. November 2003. downloadable in the summer of 2007: http://ec.europa.eu/employment_social/employment_strategy/pdf/etf_en.pdf

[EC 2003b] European Commission: “Business demography in Europe. Results for 10 Member States and Norway. Data 1997–2000" Luxembourg: Office for Official Publications of the European Communities, 2003. ISBN 92 - 894 - 5708 – 2


6. Appendices

[EC 2003c] “Main Administrative Structures Required for Implementing the Acquis". Informal working document issued by DG Enlargement of the European Commission in April 2003.

[EC 2003d] “Guidelines on the Application of Directive 2006/95/EC (Electrical Equipment Designed for Use Within Certain Voltage Limits)". European Commission, August 2007.

[EC 2004a] „Advancing regulatory reform in Europe - A joint statement of the Irish, Dutch, Luxembourg, UK, Austrian and Finnish Presidencies of the European Union" 7 December 2004, 11 oldal

[EC 2004b] „A Comparative Analysis of Regulatory Impact Assessment in 10 EU Countries" A Report Prepared for the EU Directors of Better Regulation Group. Dublin, May 2004. Italian, Irish and Dutch Presidencies of the Council of the European Union, 199 oldal.

[EC 2004c] European Commission: “Project Cycle Management Guidelines" Brussels, 2004. 158 pages. Title of series: „Aid Delivery Methods"

[EC 2005a] “Better Regulation for Growth and Jobs in the European Union" Communication of the European Commission, 2005.

[EC 2005b] „The Multiannual Programme for Enterprise and Entrepreneurship - Council Decision (2000/819/EC) of 20 December 2000 and amended by the Decision (1776/2005/EC) of the European Parliament and of the Council of 28 September 2005 on a multiannual programme for enterprise and entrepreneurship" document of the EU programme, 2005

[EC 2005c] European Commission: “The European e - Business Report. 2005 edition. A portrait of e - business in 10 sectors of the EU Economy. 4th Synthesis report of the eBusiness W@tch" Luxembourg 2005 November.

[EC 2005d] Impact Assessment Guidelines. European Commission SEC(2005) 791. 15 June 2005

[EC 2006a] Press release Nr IP/06/1562 of the European Commission Brussels, November 14 2006.

[EC 2006b] European Commission: “European Charter for Small Enterprises - 2006 good practice selection" Luxembourg: Office for Official Publications of the European Communities. 49 pages.

[EC 2006c] Better Regulation - simply explained. European Commission. Luxembourg: Office for Official Publications of the European Communities, 2006. ISBN 92-79-02465-5

[EC-OECD 2007] SME Policy Index 2007. Report on the Implementation of the European Charter for Small Enterprises in the Western Balkans. Prepared by the European Commission Directorate General for Enterprise and Industry and the OECD Investment Compact for South East Europe. In consultation with: the European Bank for Reconstruction and Development and the European Training Foundation. No indication of the place of publishing, 2007.

[EPC 2001] „Regulatory Impact Analysis: Improving the Quality of EU Regulatory Activity" Prepared by the European Policy Center. September 2001.

[Feiler and others 2004] „Az üvegházhatású gázok kibocsátási egységeinek kereskedelméről szóló törvényjavaslat előzetes hatásvizsgálata" Feiler József, Kelemen Ágnes, Dr. Kiss Rita, Dr. Kovácsy Zsombor, Orbán Krisztián. Budapest, GKM - IM - KVVM, November 2004, manuscript, 40 pages. – “Preliminary impact assessment of the bill on the trading of greenhouse gas emission quotas".

[Fejős 1998] Fejős Zoltán (editor): “A turizmus mint kulturális rendszer" Néprajzi Múzeum Budapest 1998. – “Tourism as a cultural system" Museum of Ethnography Budapest 1998.

[Figyelő and others 2006] „KKV Körkép – 2006. április" – “Overview of the SMEs – April 2006" Joint research of Figyelő, the Research Institute of Economics and Enterprises of the Hungarian Chamber of Commerce and Industry and Volksbank. Questionnaire and research report on http://www.gvi.hu

[GKI 2004] „A Balaton - térség nemzetgazdasági - szintű jövedelemtermelő képességének vizsgálata" – “Assessment of the profit earning capacity of the Balaton Region on the level of the national economy" Edited by Dr. Molnár László. GKI Economic Research Co. April 2004.


6. Appendices

[GKM - NFGM 1996 - 2007] „A kis - és középvállalkozások helyzete" Éves jelentések 1996-2007. Kiadta: Gazdasági és Közlekedési Minisztérium és Nemzeti Fejlesztési és Gazdasági Minisztérium. Szerkesztette: Kállay László és szerzőtársai, Kőhegyi Kálmán, Kissné Kovács Eszter and Maszlag Ludmilla. – The state of small and medium enterprises in Hungary. 2003-2004 annual reports 1996-2007" Edited by: Kállay László and co-authors, Kőhegyi Kálmán, Kissné Kovács Eszter and Maszlag Ludmilla. Published by the Ministry for Economy and Transport and by the Ministry of National Development and Economy of Hungary.

[GKM 2006] Gazdasági és Közlekedési Minisztérium: „A kis - és középvállalkozások fejlesztésének koncepciója 2007 - 2013. Tervezet" - “Development concept for small and medium enterprises" Manuscript, Budapest December 2006. 55 pages.

[GOP 2006] “Gazdaságfejlesztési Operatív Program". – “Economic Development Operational Program" CCI-number: 2007HU161PO001. Government of Hungary. Passed by the Hungarian Government on November 29 2006.

[GOP EX ANTE 2007] A Gazdaságfejlesztési Operatív Programme ex ante értékelése. – Ex ante evaluation of the Economic Development Operational Program. by the joint consortium of PricewaterhouseCoopers - KPMG - Fitzpatrick Associates Economic Consultants Ltd 2007. The document can be downloaded from http://www.nfu.hu.

[Hitchcock 2000] Hitchcock, Michael: “Ethnicity and Tourism Entrepreneurship in Java and Bali" Current Issues in Tourism Vol. 3, No.3, 2000.

[House 1999] Values in evaluation and social research. Author: Ernest R. House. SAGE, 1999, ISBN 0761911553, 9780761911555. 152 pages.

[Hugo – Garnsey 2004] Oliver Hugo & Elizabeth Garnsey: From Obstacle to Opportunity: Problem-Solving and Competence Creation in New Firms. Centre for Technology Management Working Paper Series, No: 2004/03, August 2004. University of Cambridge, U.K.

[Hult 2002] “Development of statistics on business demography in the European Union - a progress report" Merja Hult, EUROSTAT. Presentation at the International Roundtable on Business Survey Frames conferences, Lisbon, 2002.

[Kállay 2003] László Kállay: “Microfinance in Hungary – Opportunities and Impediments" in: Lengyel Imre (ed.): Knowledge Transfer, Small and Medium Sized Enterprises and Regional Development in Hungary. JATE Press Szeged 2003. p. 96-110.

[Kállay - Kissné - Kőhegyi 2003] Kállay László, Kissné Kovács Eszter, Kőhegyi Kálmán: „Piaci környezet, szabályozás és vállalkozásösztönzés" – “Market environment, regulation and enterprise incentives" prepared for the following research: „Gazdasági versenyképesség: helyzetkép és az állami beavatkozás lehetőségei" Manuscript, Budapest August 2003. 54 pages.

[Kállay 2005] Kállay László: A tranzakciós költségek: optimum, méretgazdaságosság, egyensúly. Doktori disszertáció. Szeged, 2005.

[Kane and others 2007] Kane, T - Holmes, Kim R - O'Grady, Mary Anastasia: „2007 Index of Economic Freedom. The Link Between Economic Opportunity and Prosperity" The Heritage Foundation, Washington, 2007.

[King - Keohane - Verba 1994] King, Gary – Keohane, Robert O. – Verba, Sidney: Designing Social Inquiry. Scientific Inference in Qualitative Research. Princeton, New Jersey: Princeton University Press, 1994.

[Kovácsy – Orbán - Ovseiko 2004] Dr. Kovácsy Zsombor, Orbán Krisztián: „A szabályozás hatásvizsgálata. Módszertani útmutató" A „Társadalmi hatások vizsgálata" c. fejezetet írta: Pavel Ovseiko. Megjelent a BM Közigazgatásszervezési és Közszolgálati Hivatalának gondozásában, a „Módszertani Füzetek" sorozat keretében. Budapest, 2004. – “Social Impact Assesment" chapter by Pavel Ovseiko in “Regulatory Impact Assessment. A methodological guidebook"

[Lengyel 2002] Lengyel György (szerk.): „Indikátorok és elemzések" – “Indicators and analyses" Manuscript, 127 pages, Budapesti Közgazdasági és Államigazgatási Egyetem, Budapest 2002.


6. Appendices

[Loayza – Oviedo - Serven 2004] Regulation and Macroeconomic Performance. Norman Loayza, Ana Maria Oviedo and Luis Serven. September 2004. World Bank Policy Research Working Paper No. 3469.

[Loayza-Oviedo-Serven 2005] The Impact of Regulation on Growth and Informality: Cross-Country Evidence. Norman V. Loayza, Ana Maria Oviedo, Luis Serven. Published by the AEI-BROOKINGS JOINT CENTER FOR REGULATORY STUDIES in May 2005.

[Lutter 2001] Lutter, Randall: „Study on Economic Analysis of Regulation in the U.S. (What Lessons for the European Commission?)" - Report to the Enterprise Directorate General, European Commission for the Workshop on Impact Assessment in Member States. AEI - Brookings Joint Center for Regulatory Studies, Brussels, 2001, 23 pages.

[Mohr 1995] Mohr, Lawrence B: „Impact analysis for programme evaluation" SAGE publications, Chicago 1988 and 1995.

[Moksony 1999] Moksony Ferenc: „Gondolatok és adatok. Társadalomtudományi elméletek empirikus ellenőrzése" Budapest: Osiris Kiadó, 1999. – “Thoughts and data. Empirical examination of social science theories".

[Nolan 2004] Entrepreneurship and Local Economic Development: Programme and Policy Recommendations. Author: Alistair Nolan: OECD, Paris, April 2004. ISBN Number: 9264199799

[North 1990] North, Douglass. C.: „Institutions, Institutional Change and Economic Performance" Cambridge: Cambridge University Press, 1990.

[OECD 1997] OECD: „Assessing the Impacts of Proposed Laws and Regulations", SIGMA PAPER No. 13. OECD, Paris, 1997, 92 pages.

[OECD 1999] OECD Economic Outlook, Chapter VII: “Cross - Country Patterns of Product Market Regulation," Paris: OECD, 1999, pp. 179 - 189.

[OECD 2000] “OECD Reviews of Regulatory Reform - Regulatory Reform in Hungary. OECD July 2000.

[OECD 2001] OECD: „Improving Policy Instruments through Impact Assesment", SIGMA PAPER No. 31., OECD, Paris, 2001. 52 pages.

[OECD 2003] OECD: „From Red Tape to Smart Tape. Administrative Simplification in OECD Countries" OECD Publishing. Paris, 2003, 261 pages.

[OECD 2005a] “OECD Guiding Principles for Regulatory Quality and Performance" Decision of the OECD Council April 2005.

[OECD 2005b] OECD: „SME and Entrepreneurship Outlook" OECD, Paris, 2005

[OECD 2005c] OECD: „Regulatory Impact Analysis in OECD Countries. Challenges for developing countries" Ed.: Delia Rodrigo, published by OECD, 33 pages, Dhaka, Bangladesh. June 2005.

[Oláh 2003] Oláh Miklós: „Egy rendhagyó régió. A Balaton kiemelt üdülőkörzet, mint kiemelt turisztikai térség strukturális problémáiról röviden a legújabb kutatások eredményeinek tükrében", Balatonfüred, August 2003. – “A special region. Summarising the structural problems of Lake Balaton Resort District, with the new research results in mind"

[OECD 2007] OECD Framework for the Evaluation of SME and Entrepreneurship Policies and Programmes. Authors: Dr. Jonathan Potter and Prof. David Storey. OECD Paris 2007. p. 126.

[OECD 2008] Building an Institutional Framework for Regulatory Impact Analysis (RIA): Guidance for Policy Makers. OECD Regulatory Policy Division Directorate for Public Governance and Territorial Development. Paris 2008.

[OECD 2010] Competitiveness and Private Sector Development: Egypt 2010. Business Climate Development Strategy. OECD Paris 2010.


6. Appendices

[Orbán - Kovácsy 2005] Orbán Krisztián és Kovácsy Zsombor: „Recent Regulatory Reform Activities in Hungary" Manuscript, Ministry for Justice, Budapest 2005.

[Oldsman – Hallberg 2002] Eric Oldsman and Kris Hallberg: Framework for Evaluating the Impact of Small Enterprise Initiatives. World Bank 2002.

[Parker 2005] Parker, Simon: The Economics of Entrepreneurship: What We Know and What We Don’t. Foundations and Trends in Entrepreneurship, Vol. 1, No. 1, 2005

[Peredy 2007] „A hazai K+F és innováció finanszírozási lehetőségei a Gazdasági és Közlekedési Minisztérium szempontjából: alapelvek, források, programok" – “Local research and innovation financing possibilities of the Hungarian Ministry for Economy and Transport" presentation of Dr. Peredy Zoltán (GKM Innovációs és K+F Főosztály), Szeged, 2007.

[Pilat 2002] “Work on enterprise demography at the OECD –an update" Dirk Pilat, Organisation for Economic Co - operation and Development.. Presented at the International Roundtable on Business Survey Frames conferences, Lisbon, 2002.

[Puczkó – Rácz 2002] Puczkó László - Rátz Tamara: „A turizmus hatásai" Budapest, 2002. Aula Kiadó Kft (3rd edition) – “The effects of tourism"

[Purdon – Lessof – Woodfield – Bryson 2001] Research Methods for Policy Evaluation. Department for Work and Pensions. Research Working Paper No 2. By Susan Purdon, Carli Lessof, Kandy Woodfield and Caroline Bryson. National Centre for Social Research, U.K. 2001.

[Radaelli 2003] Radaelli, Claudio M.: „Getting to Grips with the Notion of Quality in the Diffusion of Regulatory Impact Assessment in Europe" Paper Prepared for the Conference on Regulatory Impact Assessment. Manchester, UK, 26 and 27 November 2003.

[Renda 2006] Renda, Andrea: „Impact Assessment in the EU: the State of the Art and the Art of the State" AmCham EU, March 2006 Brussels, 18 pages.

[RIU UK 2003] “Better Policy Making: A Guide to Regulatory Impact Assessment". Regulatory Impact Unit, Cabinet Office of the Government of Great Britain. Publication date: January 2003.

[Roessingh – Duijnhoven 2004] Roessingh, Carel and Duijnhoven, Hanneke: „Small Entrepreneurs and Shifting Identities: The Case of Tourism in Puerto Plata (Northern Dominican Republic)" Vrije Universiteit Amsterdam, Faculty of Social Sciences, Department of Culture, Organisation and Management, The Netherlands. Journal of Tourism and Cultural Change Vol. 2, No. 3, 2004.

[Román 2005] Román Zoltán: „A demográfia újabb ága: a vállalatdemográfia" Magyar Tudomány, 2005/3 p.297. – “The new branch of demography: enterprise demography"

[Schumpeter 1961] The theory of economic development : an inquiry into profits, capital, credit, interest, and the business cycle. By J.A Schumpeter, New York 1961.

[Schweke 2000] Curbing Business Subsidy Competition: Does the European Union Have an Answer? Working Paper by William Schweke, September 2000, Corporation for Enterprise Development,

Durham, North Caroline.

[SCM 2005] „International Standard Cost Model Manual – Measuring and reducing administrative burdens for businesses" Published by the Secreteriate of the International SCM Network. October 2005, 63 pages, no indication of the place of publishing. Dowloadable from http://www.administrative - burdens.com/ and the OECD website.

[Smallbone-Welter 2009] Smallbone, David- Welter, Friederike: Entrepreneurship and Small Business Development in Post-Socialist Economies. Routledge 2009.

[Szántó 1992]. Szántó Zoltán 1992. Az „értékmentes" társadalomtudományok eszméje. In Becskeházi Attila – Fekete László – Szántó Zoltán: A tudomány normativitása. Széchenyi Füzetek VI. Budapest, Széchenyi Szakkollégium. (Zoltan Szanto: The idea of a value-free social science. Paper appeared in the volume


6. Appendices

„Normativeness of science", edited by Becskeházi Attila – Fekete László – Szántó Zoltán, Budapest, 1992.)

[Szirmai 2003] Péter Szirmai: „The Capital-less Capitalism: review on Hungarian Small Enterprises." Published in: Lengyel Imre (ed.): Knowledge Transfer, Small and Medium Sized Enterprises and Regional Development in Hungary. JATE Press Szeged 2003. p. 122-129.

[Szirmai 2005] Peter Szirmai and Krisztian Csapo: “A Hungarian Experience of Entrepreneurship Teaching: Fostering Student Enterprises". Paper presented at the 28th ISBE (Institute for Small Business & Entrepreneurship) UK National Conference. 1-3 November 2005 – Blackpool.

[Tibor 2006] Ágnes TIBOR: „Small Enterprises in Hungary and the Joining the European Union" Published in: Rencontres de St. Gall, 2006. Wildhaus-St. Gallen, September 2006.

[Török - Papanek 2004] „Az EU tagországok innováció - és kkv politikájának kapcsolódása" – “Connection of the innovation and SME-policies of the EU member states" Leaders of the research: Török Ádám and Papanek Gábor. Magyar Vállalatgazdasági Kutatásokért Alapítvány and GKI Gazdaságkutató Rt. Budapest, August 2004.

[UMFT 2007] „Új Magyarország Fejlesztési Terv. Magyarország Nemzeti Stratégiai Referenciakerete. 2007–2013. Foglalkoztatás és növekedés." Az Európai Bizottság döntésének dátuma: 2007. május 7. (In English: New Hungary Development Plan. National Development Plan for Hungary, 2007-2013).

[UNIDO 1996] „SME - A Comparative Analysis of SME Strategies, Policies and Programmes in Central European Initiative Countries" Country reports of the following states: Austria, the Czech Republic, Italy, Poland, Hungary, Rumania, Slovakia. UNIDO United Nations Industrial Development Organisation 1996. Ther reports can be accessed online at:

http://www.unido.org

[Veit 2005] Veit, Sylvia: „Entpolitisierung staatlicher Regulierungsprozesse durch Gesetzesfolgenabschätzungen?“ FoJuS Diskussionspapiere Nr. 3/2005. 23 pages.

[WB 2003] World Bank Operations Evaluation Department: “The First 30 Years" Edited by Patrick G. Grasso, Sulaiman S. Wasty, Rachel V. Weaving. 2003 The World Bank, Washington, D.C. 198 pages. Downloadable in the summer of 2007:

http://www.worldbank.org/oed

[WB 2004] Monitoring and Evaluation: Some Tools, Methods & Approaches. World Bank, Operations Evaluation Department, Knowledge Programs and Evaluation Capacity Development Group (OEDKE). Published by The World Bank, Washington, D.C., 2004.

[WB 2004 - 2006] „Doing Business in 2004 – Understanding Regulation". „Doing Business in 2005 – Removing Obstacles to Growth". ’Doing Business in 2006 – Creating Jobs" Published by the World Bank.

[WB 2008] Doing Business 2009. Published September 2008 by Palgrave Macmillan , World Bank ISBN: 0-8213-7609-8.

[WB 2007] World Bank Independent Evaluation Group (IEG): “Evaluation Tools & Approaches" downloadable in the summer of 2007:

http://www.worldbank.org/oed/oed_approach_summary.html.

[WEF 2007] „The Global Competitiveness Report 2006 - 2007" World Economic Forum. Palgrave Macmillan, ISBN: 1 - 4039 - 9636 - 9

[WTTC 2007] „Hungary: Travel & Tourism - Navigating the Path Ahead" The 2007 Travel & Tourism Economic Research. Published by the World Travel & Tourism Council (WTTC) and Accenture. Downloadable from: http://wttc.org.

4. Research, consultancy and educational projects mentioned in the text


6. Appendices

Legal harmonisation impact assessment of the EC directives concerning technology policy in Hungary, through the example of the EMC (electro magnetic compatibility of products) directive. Client: the Hungarian Ministry for Economy. Project implemented by The Public Policy Unit (London) and the Ipargazdasági Kutató és Tanácsadó Ltd. (Budapest) Sponsored by: The Know How Fund (UK). Budapest 1998.

Legal harmonisation impact assessment of the EC directives concerning technology policy in Lithuania through the example of the EMC (electro magnetic compatibility of products) directive. Client: EU Integration Committee of Lithuania. Executors: European Institute of Public Administration (Luxembourg) and the Ipargazdasági Kutató és Tanácsadó Ltd. (Budapest). Financed by: PHARE. Budapest - Vilnius 2000.

Evaluation of PHARE sponsored SME projects in ten countries including Hungary. Client: Evaluation Unit of the European Committee. Executors: RDH / LDK consultancy consortium. Brussels and 10 supported countries, 1999.

Local financing of small enterprises. Joint British – Hungarian – Slovenian – Macedonian research. The research was coordinated by the University of Wolverhampton. Financed by: PHARE ACE program. Wolverhampton - Budapest 2000.

Legal harmonisation impact assessment of the EC directives concerning technology policy in Croatia through the example of the LVD (low voltage directive). Client: Croatian Ministry for European Integration. Executors: P-E International Consulting (UK) and the Ipargazdasági Kutató és Tanácsadó Ltd. (Budapest). Financed by: DFID (Department for International Development, UK) Budapest - Zagreb 2001.

Evaluation of support constructions for the innovation projects of small enterprises. Client: Országos Műszaki Fejlesztési Bizottság. Budapest 2000.

Sociological Survey of the SMEs around Lake Balaton. Client: Lake Balaton Development Council. Executor: Voxinfo Ltd. Budapest – Siófok – Balatonfüred 2001.

Evaluation Study of the Impact of the Hungarian Micro-Credit Programme. Client: the European Union in the „Framework Contract AMS/451 – Lot N° 10" project. Executed by Pohl Consulting and Associates. Munich.

Entrepreneurial incubators in the EU and in Hungary. Client: the Hungarian Ministry for Economy. Executor: SEED Kisvállalkozás-fejlesztési Alapítvány. Budapest 2002.

Regional Clusters in the EU and in Hungary. Client: the Hungarian Ministry for Economy. Implemented by SEED Kisvállalkozás-fejlesztési Alapítvány. Budapest 2003.

Evaluation of employment support for small enterprises Client: OFA Országos Foglalkoztatási Közalapítvány. Executor: SEED Kisvállalkozás-fejlesztési Alapítvány. Budapest 2004.

E-Commerce as a tool of SME Development. Evaluation of the PHARE project. The evaluation was executed by Europe Ltd., the client was the Hungarian National Office for Development (Nemzeti Fejlesztési Hivatal). Budapest 2006.

Joint Master Course on Comparative Local Development. Joint Master course of four universities, financed by the Erasmus Mundus programme of the EU. Participants: University of Trento (Italy), Corvinus University Budapest (Hungary), Ljubljana University of Economics (Slovenia) and University of Regensburg (Germany). Trento-Regensburg-Budapest 2006-2007.

IndexAapproach, 102Austria, Error: Reference source not found, 73, 84, 90

BBalaton, 58, 58, 58, 58, 59, 62, 62, 62, 62, 62, 63, 63, 63, 63, 63, 64, 64, 64, 64, 64, 64, 65, 65, 68, 68, 69Baseline scenario, 81


6. Appendices

Before-after regression, 17Belgium, Error: Reference source not found, 84, 93, 93Better Regulation Programmes, 79Business survey, 44, 56, 56Business surveys, 56

Ccausality, 5, 108, 127Central and Eastern Europe, 20, 20, 20, 21, 22, 94, 94Community Innovation System, 44Competition and consumer protection policies, 23competitiveness, 6, 20, 24, 24, 28, 29, 29, 30, 30, 30, 30, 30, 31, 34, 34, 38, 38, 40, 41, 42, 42, 42, 43, 44, 53, 58, 58, 60, 60, 60, 60, 61, 62, 62, 75, 75, 76, 89, 90, 93Compliance cost, 84, 102Cost-benefit analysis, 83Cost-effectiveness analysis, 84Counterfactual, 81Croatia, 94, 102, 102, 102, 104, 104, 104, 104, 105, 105, 106

DDeregulation, 74, 79, 98design, 3designs, 5, 6, 10Difference-in-difference method, 15Direct impacts, 81Distributional effects, 81Doing Business, 76, 77, 78, 78, 79

Ee-business services, 28Education policies, 23Effectiveness, 7, 26, 28, 46, 50Efficiency, 7, 26, 28, 46, 51, 51Entrepreneurship, 22, 65, 65, 79EU Electromagnetic Compatibility Directive, 95EU Low Voltage Directive, 96European Business Test Panel, 83European Committee, 89, 90, 91European Council, 30European Innovation Scoreboard, 44, 44European Union, 18, 25, 25, 27, 54, 56, 67, 83, 88, 89, 89, 90, 90, 90, 91, 91, 94, 94, 95Europeanisation process, 22, 79, 102Evaluation, 1, 6, 7, 7, 8, 14, 24, 27, 27, 28, 52, 53, 53, 54evaluations, 8

FFinland, 73, 84, 90Fiscal and monetary policies, 22

GGermany, 56, Error: Reference source not found, 73, 84, 94, 94


6. Appendices

HHungarian State Technical Development Committee, 45Hungary, 13, 18, 25, 25, 26, 27, 28, 28, 29, 32, 40, 43, 43, 45, 45, 52, 53, 53, 54, 57, 58, 62, 62, 62, 63, 65, 68, Error: Reference source not found, 72, 73, 84, Error: Reference source not found, 94, 95, 95, 95, 95, 96, 97, 97, 97, 98

IImpact, 7Increases in costs, 105Indicator development, 75Indirect impacts, 81Intermediaries, 10Internal Rate of Return, 84Ireland, Error: Reference source not found, 84, 90

LLegal harmonisation, 80, 105Lisbon Programme, 44Lithuania, 94Logical Framework, 26Luxembourg, Error: Reference source not found, 84, 90

MMonitoring, 42Multicriteria analysis, 86

NNational Public Foundation, Error: Reference source not foundNet Present Value, 83Netherlands, 73, 73, 84, 90, 93, 93, 93, 93

OOECD, 19, 57, 72, 72, Error: Reference source not found, 73, Error: Reference source not found, Error:Reference source not found, Error: Reference source not found, Error: Reference source not found, 73, Error: Reference source not found, 73, Error: Reference source not found, 73, Error: Referencesource not found, 74, 74, 74, 75, Error: Reference source not found, 75, 84, Error: Reference sourcenot foundOECD Economic Regulation Index, 75, 75Office of Advocacy of the Small Business Administration, 100Office of Management and Budget, 99, 99

PPay-back time, 83PHARE, 27, 27, 28policy, 6, 97Policy, 29, 43, 52, 60, 71, 101, 102procedures, 74Programme, 25, 43, 53, 53, 54, 54, 54, 54, 54Project, 24, 51, 102Public Foundation for Employment, 31


6. Appendices

QQuantitative methods, 83

RRegression discontinuity design, 17, 17Regression model with statistical controls, 17Regulatory policy, 23Regulatory quality, 71, 72regulatory quality, 72Relevance, 7, 26, 28, 45Research design, 12, 82, 103Research question, 58Research questions, 5, 6

SScenario, 81scenario, 81Small Business Administration, 100SME development policy, 12, 19, 20, 30, 31, 97social capital, 19Social cost, 82Spill-over effects, 50Stabilisation and Association Agreement, 102, 104Standard Cost Model (SCM), 84Strategy, 43Summary Innovation Index, 44, 45Sustainability, 7, 29, 38, 46Sweden, Error: Reference source not found, 73, 75, 84, 94, 94, 94

TTechnical impacts, 105Time Series Design, 14, 15Tourism, 61, 62, 62, 62, 64Treatment group, 12

UUkraine, 79, Error: Reference source not found, 79, Error: Reference source not foundUnited Kingdom, Error: Reference source not found, 73, 73, 84, 87, 91, 91United States, Error: Reference source not found, 73, 75, 99, 100, 100, 101

WWest Balkan states, 53, 53World Bank, 19, 47, 76, 76


BIBLIOGRÁFIANoteAuthor’s publications referred to in the text

The bibliographical data of publications authored and co-authored are in the order of their appearance in the book. Studies and consultancy documents that were not published in print, e.g. studies that were published only on the Internet, are marked with an asterisk ( *)

[bib_1] Peter Futo. An evaluation tool for assessing e-Inclusion policies, programmes and projects. Document created in the framework of the „ICT for ALL" Project Budapest 10. September 2008. Compiled by Peter Futo, Corvinus University, Budapest. With the contribution of consortium members of the „ICT for ALL" Project: ASM (Poland), ICIE (Italy), EP (Greece) and INSO (UK).

[bib_2] Péter Futó and László Kállay. Эмансипация и кризис: развитие малого и среднего бизнеса в Венгрии (1945-1990- е гг.) Футо, Петер, Каллаи, Ласло. Рубеж (альманах социальных исследований). (Translation of title: Emancipation and crisis. The development of small and medium size business in Hungary 1945-1990.). 1997. 10-11. С. 87-115.

[bib_3] Péter Futó. “The role of credit guarantees in financing Hungarian small businesses" Small Enterprise Development. June 1997. 34 - 40.

[bib_4] Péter Futó, Milford Bateman, Taki Fiti, and Hedvika Usenik. “The local financial system and sustainable SME development in South - East Europe. Lessons from Hungary, Macedonia and Slovenia" Appeared as Chapter 4 of the following book: Small Enterprise Development in South - East Europe. Policies for Sustainable Growth. Editors: W. Bartlett, M. Bateman, M. Vehovec. 2002. Kluwer Academic Publishers. Boston - Dordrecht - London. 83 - 125.

[bib_5] *Péter Futó and Gábor Klein. „A Magyarországra kiszervezett üzleti folyamatok és üzleti szolgáltató központok. A befektetés-ösztönzés, a munkahelyteremtés és a munkahely-megőrzés feltételeinek elemzése a szektorban." Tanulmány a Gazdasági és Közlekedési Minisztérium megbízásából, Title in English: Business Processes Outsourcing and Shared Service Centres in Hungary. An analysis of investment promotion, job creation and job retention. 2007 június 12.

[bib_6] Peter Futo. SME in the Hungarian Public Procurement Process. Paper presented to the 3rd International Conference "Enterprise in Transition". 1999. Split.

[bib_7] Péter Futó. „A vállalati hitelképesség megállapításának mennyiségi módszerei a tőkés bankoknál". Title in English: Quantitative methods for assessing the creditworthiness of companies in commercial banks. 1990. szept. Pénzügyi Szemle. In Hungarian. 593-609.

[bib_8] Péter Futó and Anita Ihász. „Kis- és középvállalkozások értékelését szolgáló diagnosztika". Title in English: Diagnostics for the evaluation of SMEs. 2001. 4. sz. Vezetéstudomány. In Hungarian. 46-53.

[bib_9] *Péter Futó. „Micro-level evaluation of micro-credits" Manuscript, 24 pages, consultancy document. Prepared in 2002 for the„Evaluation Study of the Impact of the Micro-Credit Programme" in the EU financed „Framework Contract AMS/451 – Lot N° 10" project. The evaluation project was implemented by Pohl Consulting and Associates. 2002. Munich.

[bib_10] *Milford Bateman. “An Evaluation of Phare SME programmes in Hungary". Manuscript, consultancy document September 1999. Local Consultant: Péter Futó. Downloadable in the summer of 2007 from: www.ec.europa.eu/europeaid/evaluation/reports/cards/951508_hun.pdf . September 1999.

[bib_11] Péter Futó, Kinga Hanthy, Pál Lányi, András Mihály, and Anikó Soltész. „"A szociális gazdaság jelene és jövője Magyarországon", Title in English: Social economy in Hungary . 2005. Nemzeti Felnőttképzési Intézet (National Institute for Adult Education) as part of the „Felnőttképzési Füzetek“ series. Budapest, In Hungarian. 131 p.


BIBLIOGRÁFIA

[bib_12] Péter Futó. „Kis- és középvállalkozások munkaügyi támogatásának kérdőíves hatásvizsgálata", Title in English: Questionnaire-based impact assessment of the employment subsidy scheme provided for SMEs. 2007 április. Európai Tükör. In Hungarian.

[bib_13] Péter Futó. „A vállalkozás - fejlesztési politikák összehangolása", Title in English: European harmonisation of enterprise development policies. 2002/5. Európai Tükör. In Hungarian.

[bib_14] Anikó Soltész, Péter Futó, Pál Lányi, Susan Kutor, and Judit Dobák. „A vállalkozói inkubációs rendszer formái, nemzetközi és hazai működésének gyakorlata, a hazai továbbfejlesztés modelljei és az állami szerepvállalás módszerei", Title in English: International and local practice of the enterpreneurial incubation system, further development models in Hungary and the role of the state . 2002. SEED Kisvállalkozás - fejlesztési Alapítvány. Budapest, In Hungarian.

[bib_15] *Péter Futó. „Regionális klaszterek és a létesítésükre, működtetésükre vonatkozó politikák az Európai Unióban" Manuscript, Title in English: Regional clusters in the EU and policies for establishing and operating them. 2003. SEED Alapítvány. Budapest, In Hungarian.

[bib_16] *Péter Futó, Anikó Soltész, and Pál Lányi. „Regionális klaszterek és a létesítésükre, működtetésükre vonatkozó politikák Magyarországon" Manuscript, Title in English: Regional clusters in Hungary and the policies for establishing and operating them. 2003. SEED Alapítvány. Budapest, In Hungarian.

[bib_17] *Péter Futó, László Kállay, and Anikó Soltész. „Értékelés az OMFB kis - és középvállalkozás - támogatási pályázati tevékenységéről (1995 - 1998)” Manuscript, final document of the consultancy project completed for the Országos Műszaki Fejlesztési Bizottság, Title in English: Evaluation of the tenders subsidizing SME innovations offered by the National Committee for Technical Development. July 2000. Budapest in Hungarian.

[bib_18] Péter Futó. „A kisvállalatok támogatott innovációs - fejlesztési projektjei, 1995és 1999 között", Title in English: Supported innovation – development projects of small enterprises between 1995 and 1999 . 2001. 6. sz. Statisztikai Szemle. In Hungarian. 522 - 535.

[bib_19] Péter Futó. „Iparvállalati viselkedéstípusok mérlegadatok alapján"., Explaining the behavior of industrial firms on the basis of balance sheet data. 1990 julius. Statisztikai Szemle. In Hungarian. 578-596.

[bib_20] Péter Futó. "Az IFO felmérések kézikönyve" Book review, Title in English: Review of the Guidebook of the surveys implemented by IFO Institute of Economic Research. October 1991. Statisztikai Szemle. In Hungarian.

[bib_21] Péter Futó, Paul Hoggett, and László Kállay. „Small Firms and Economic Transformation in Hungary" Innovation Vol. 10, No. 2. 1997. 171-184.

[bib_22] Péter Futó and László Kállay. „A kis - és középvállalati szektor kialakulása", Title in English: Development of the SME sector. September 1994. Statisztikai Szemle. In Hungarian. 647 - 665.

[bib_23] Péter Futó and László Kállay. „A kis - és középvállalati szektor elterjedése és szerkezete", Title in English: The evolution and the structure of the SME sector. 1994 október. Statisztikai Szemle. In Hungarian. 722 - 738.

[bib_24] Péter Futó. „A csődök mérhető kisérőjelenségei", Title in English: Measurable symptoms accompanying bankruptcy. 1994. február. Statisztikai Szemle. In Hungarian. 125-139.

[bib_25] Péter Futó and Koppány Kalmár. „A Balaton környéki kis - és középvállalkozások és a regionális vállalkozásfejlesztés", Title in English: SMEs around Lake Balaton and regional enterprise development. 2002. július - augusztus. Comitatus. In Hungarian. 72-86.

[bib_26] Péter Futó. „Pest romló városrészei és a hiányzó anti-szlam politika", Title in English: Declining zones in the town of Pest and the missing anti-slum policy. 1997/. 7-8. sz. Falu, Város, Régió. In Hungarian. 12-16.

[bib_27] Péter Futó, Ilona Pálné Kovács, and Tamás Fleischer. "Governance in Regional and Environmental Policies in Hungary: Challenges of Europeanisation and Adaptation." Fifth chapter of the book:


BIBLIOGRÁFIA

„Adapting to EU Multi-Level Governance: Regional and Environmental Policies in Cohesion and CEE Countries". Editors: Christos Parakevopoulos, Panagiogitis Getimis and Nicholas Rees., 301 pages. ISBN 0 7546 4533 9. 2006. ASHGATE Publishing House.

[bib_28] Péter Futó. „Kis - és középvállalkozások az Európai Unióban - helyzetkép a szektorról és a közösségi KKV politikáról" Manuscript, Title in English: SMEs in the EU – overview of the sector and the SME policy of the community. March 2001. Prepared for the Small Enterprise Development Institute of the Budapest Univesity of Economics. In Hungarian.

[bib_29] Péter Futó. „Alvállalkozók – beszállítók nemzetközi összehasonlításban" The article was republished in "Vállalkozók és vállalkozói hajlandóság", ed.: Lengyel György, BKE Szociológia és Szociálpolitika Tanszék, 1996. p. 205-225. . Title in English: Subcontractorts – suppliers in the international comparison. 1995. 2. sz. Statisztikai Szemle. In Hungarian. 155 - 165..

[bib_30] Péter Futó. “Supporting subcontracting linkages in Hungary" Small Enterprise Development. 1998. 4. 46-55.

[bib_31] Eszter Bod, Donát Bonifert, and Péter Futó. „A háttéripari kapcsolatok intézményes fejlesztése – nemzetközi tapasztalatok", Title in English: Institutional development of supplier industry linkages – international experiences. 2000. No. 3. Európai Tükör. In Hungarian. 71-87.

[bib_32] Péter Futó and Emese Hurton. „A vállalatok közti kapcsolatokról", Title in English: A typology of inter-company linkages. 1998. 1. sz. Vezetéstudomány. In Hungarian. 10-22.

[bib_33] Péter Futó and Zsuzsa Szeszler. „A településfejlesztési koncepció elkészítésének módszerei az EU - ban és Magyarországon", Title in English: Methods for creating a settlement development concept in the EU and in Hungary. 2003. TÖOSZ (Association of Local Governments, Települési Önkormányzatok Országos Szövetsége) as one of the JÓTÁR methodological guidebooks. Budapest, In Hungarian.

[bib_34] Mária Dezsériné Major, Péter Futó, and László Kállay. „Mekkora az informális gazdaság Magyarországon?", Title in English: What is the size of the informal economy in Hungary? . 1994/5. Budapesti Negyed. In Hungarian.

[bib_35] *Péter Futó, Ádám Tardos, Karajánnisz Manolisz, and Máté Homoki. „Az elektronikus hálózati kommunikáció társadalmi, gazdasági és kormányzati hasznosítása Magyarországon", Manuscript, Title in English: Social, economic and governmental utilization of broadband communication in Hungary. 2005. Ariosz Ltd., Downloadable in the summer of 2007 from the website of the Hungarian Ministry for Economy. Budapest in Hungarian.

[bib_36] Péter Futó Dr.. A termékek szabad áramlására vonatkozó EU-s direktívák fogadtatása ukrán gépipari cégek körében: egy hatásvizsgálat eredményei. In Hungarian. (Title in English: The reception of European Free Movement Directives among companies of the mechanical industry in Ukraine.) Paper presented to the 2008 Hungarian Conference of Industrial Economics (Ipar- és Vállalatgazdasági Konferencia) , Szeged, 2008 October 29-30. Paper to be published in 2009 „Európai Tükör" , a periodical of the Prime Minister’s Office of Hungary.

[bib_37] Péter *Futó Dr., Oleh Myroshnichenko, and Bohdan Senchuk. Regulatory impact analysis of the introduction of the EC Low Voltage directive into Ukrainian legislation. In 2009 the study can be downloaded: http://ueplac.kiev.ua/downloads/ria/lvd_full_ria_for_ukraine.pdf. 10 April 2008. Prepared by: Ukrainian-European Policy and Legal Advice Centre (UEPLAC). Leader of RIA Team: Darius Žeruolis, Senior Legal Approximation Adviser. Kiev.

[bib_38] Péter Futó Dr.. „Kis - és középvállalkozási politika az EU tagállamokban és régiókban", Title in English: SME policy of the EU member states and regions. 1999. 1. sz. Európai Tükör. In Hungarian. 77-89.

[bib_39] Péter Futó. „Az Európai Unió kis - és középvállalkozási szektorának helyzete és a közösségi KKV - politika", Title in English: The situation of the SME sector in the EU and the SME policy of the community. 1998. december. Európai Tükör. In Hungarian. 18-39..

[bib_40] Péter Futó. „Jogharmonizáció a környezetvédelemben", Title in English: Legal harmonisation in environmental protection. 1998. 3. sz. Európai Tükör. In Hungarian. 31-45.


BIBLIOGRÁFIA

[bib_41] Kenneth Binning and Péter Futó. „A jogharmonizációs szabályozás gazdasági hatásvizsgálata“, Title in English: Economic impact assessment of regulations for legal harmonisation. 1997. 2. sz. Statisztikai Szemle. In Hungarian. 101 - 114.

[bib_42] *Péter Futó, Donát Bonifert, Eszter Parajdi, and Binning Kenneth. „Az elektromágneses összeférhetőségre vonatkozó jogszabályok EU harmonizációjának hatása az elektronikai, elektrotechnikai és műszeripari vállalatokra" Manuscript, The research was sponsored by the Know How Fund (UK), Title in English: Impact of the EU harmonisation of the laws regarding electro magnetic compatibility on electronic, electrotechnic and precision engineering companies. September 1996. Ipargazdasági Kutató és Tanácsadó Ltd.. Budapest. 108 pages.

[bib_43] * Péter Futó and Agne Seselgyte. „The Impacts of a «Free Movement of Goods» Directive. Pilot Study. Regulatory Impact Analysis of the Introduction of the Low Voltage Directive into Lithuanian Law", Sponsored by the PHARE Program. Manuscript, Downloadable froml: http://www.euro.lt/Pov_tyrimai/docs/12.pdf. . May 2000. Ipargazdasági Kutató és Tanácsadó Ltd.. Vilnius.

[bib_44] Péter Futó. „Az EU jogharmonizáció hatása az egyes vállalatokra - az átvilágítás módszertana", Title in English: Impact of the EU legal harmonisation on individual companies – the methodology of impact assessment. June 2002. Vezetéstudomány. In Hungarian.

[bib_45] Péter Futó and László Kállay. „A kisvállalkozás-fejlesztési politika és eszközei" Struktúrák – Szervezetek – Stratégiák., Title in English: Small enterprise development policy and its instruments. 1994/4. Ipargazdasági Szemle. In Hungarian.

[bib_46] Péter Futó and Magdolna Knáb. „Vállalkozói érdekcsoportok",Title in English: Entrepreneurial interest groups. January 1996. Statisztikai Szemle. In Hungarian. 35-54.

[bib_47] * Péter Futó, Ivona Štritof, and Toni Lukšić. „Regulatory Impact Analysis of the Introduction of the Low Voltage Directive (73/23/EEC) into the Croatian Law - A Case Study of the Harmonisation of EU Technical Legislation. Pilot Study" Manuscript, Sponsored by DFID (UK Foreign Office). 2001. Zagreb.

[bib_48] P. Futó, J. Cuculić, T. Lukšić, and I. Štritof. „Metodološki priručnik za izradu analise učinka uvođenja propisa Europske unije. Ministarstvo za europske integracije" RH.ISBN 953 - 97806 - 9 – 1., Title in English: Methodological guidelines of economic impact assessment of regulation in legal harmonisation. 2002. . Zagreb In Croatian.
