1

Permanent advisory bodies in Flanders (Belgium): an analysis of

member satisfaction of advice production and use across 9 strategic

advisory councils

Ellen Fobé*, Marleen Brans*, Diederik Vancoppenolle° & Jan Van Damme*

* University of Leuven, Public Management Institute, Belgium

° Flemish Government, Child and Family Agency, Belgium

Abstract Belgium, and the Flemish Government in particular, has a long tradition of funding a relatively large number of advisory bodies of varying constitution, such as advisory councils, ad-hoc committees, expert groups, citizen consultation forums etc. Policy-making in Flanders has thus traditionally been characterized by a high density of advisory bodies with varying impacts and status. These advisory bodies tend to be highly integrated into the official policy-making cycle. In addition, the advisory landscape tends to make more use of experience-based expertise that a wide range of societal stakeholders can provide than of academic expert opinion. This is not surprising in a consensus-based political system with neo-corporatist traits. Recently, the Flemish government has reshaped its advisory landscape in response to three concerns: the political will to restore political primacy in policy making, the desire to control the growing competition of advice and advisory bodies, and the need for securing societal support for policy interventions. In this paper, we analyze the defining aspects of the new strategic advice councils in Flanders such as their age, size, membership mix, and legacy. We also explore the relationship of these factors with participant satisfaction about the advisory process and the use of advice by policy makers. For this, we will make use of the results of a 2009 survey with the members of the new strategic advisory system, which had a response rate of 72.4%. The survey was conducted in nine out of the twelve strategic advisory councils in Flanders and provides material for a comprehensive analysis of Flanders’ formalized system of permanent advisory bodies.

Introduction Research shows that different countries have adopted increasingly diverse types of public consultation mechanisms (Lowndes et al., 2001; OECD, 2001; Barker, 1979; Bellone & Goerl, 1992). Finland, The Netherlands and the UK for example, have a strong, but diverse tradition of participation by citizens or stakeholders in public policy (Van Damme & Brans, 2008). Citizen and stakeholder participation and consultation takes place in permanent or ad-hoc advisory committees, through expert opinion groups, citizen consultation forums etc. Belgium, and the Flemish Government in particular, fund a relatively large number of permanent advisory bodies of varying constitution. In a recent study the OECD stated that there are about 250 advisory bodies at the Belgian federal level and 46 councils at the regional level (OECD, 2010). Additionally, a large number of other forums for participation have been created, amounting to more than 600 councils and committees (De Standaard, 03.11.2011). This extensive amount of public consultation mechanisms is not surprising in a consensus-based political system with neo-corporatist traits, where

2

traditional stakeholder groups possess policy-making powers (Lehmbruch & Schmitter, 1977; Cawson, 1982). Of course, the involvement of stakeholders in public policy-making varies to different degrees and across policy domains. Often stakeholders are consulted during the policy process, thereby providing additional policy information, specific practitioners’ insights, or feedback on policies (OECD, 2001).

Despite their importance as public consultation mechanisms in Belgium and in Flanders, very little research has been conducted on advisory bodies. In Flanders there is no longstanding policy analytical research tradition on this topic as in e.g. The Netherlands (Halffman & Hoppe, 2004) or the U.S. (Santos & Chess, 2003; Lavertu et al., 2011). And although advisory councils have been the topic of research in Flanders from a comparative politics perspective (Dewachter, 1995; De Winter, 2009), this has not provided insight into how these councils are organized, how advice comes to play and what the influence of the advice on public policy might be. The paper tries to fill this analytical gap by providing insight into the councils’ most important characteristics, membership satisfaction on the internal advisory process, as well as the role and impact of advisory councils on public policies in Flanders. For this, the article predominantly draws on a survey that was conducted with the members of the new strategic advisory system in Flanders, three years after the advisory decree was implemented. The survey was conducted in nine out of the twelve strategic advisory councils and had a response rate of 72,4%. We review some key findings on Flanders’ formalized system of advisory bodies.

The results show that there are important differences between the strategic advisory councils in Flanders, with regard to for example budget or staff size. The councils thus start off with different capacities for advising government though their members in general appreciate the advisory process itself. They have for example a positive perception on the extent to which they feel involved during the advisory process, on its transparency and decision mode. The members also are generally satisfied about the quality of the advice their councils produce. The advice itself however, only has a very limited influence on policy according to the councils’ members. This might prove to be problematic, given that influencing policy is a crucial condition in regarding advisory councils as mechanisms for creating better policies and generating a broader societal support.

The paper is structured as follows: First, the context for conducting policy advice is discussed. Second, the research approach (survey) is dealt with in more detail. Then, the main findings of the study are presented. We analyzed the defining characteristics of the new strategic advice councils in Flanders. These pertain to elements such as the councils’ age, size, membership mix, and legacy. We also explore participant satisfaction about the advisory process and the use of advice by policy makers. Last but not least, some concluding remarks are made.

The context for advising governments While policy problems become increasingly complex or ‘wicked’ (Rittel and Webber, 1973), “the capacities essential for coping with [these] critical problems are changing very slowly (…)” (Dror, 2007:84). To better support a more complex, dynamic and uncertain policy process, attention for evidence based policy has (re)emerged recently within governments (Davies et al., 1999; Pawson, 2002; Shaxson, 2005;

3

Bowen & Zwi, 2005; Nutley et al., 2009; Boswell, 2009). Evidence based policy, or evidence informed policy (Bowen & Zwi, 2005), aims to increase the effectiveness and efficiency of policies (Howlett, 2008). Its relevance also stems from a heightened accountability awareness of policy-makers, due to increased public scrutiny and the need for stronger societal support for policies (Sanderson, 2002; Sharp, 2005). In a complex policy environment, governments make use of very different sources and types of evidence, such as performance analyses, policy evaluations, impact assessments, citizen panels, foresight exercises, permanent or ad-hoc policy advisory committees, etc. Moreover, research shows that different countries have adopted increasingly diverse types of public consultation mechanisms (Lowndes et al., 2001; OECD, 2001; Bellone & Goerl, 1992). To counter a reduced policy analytical capacity (Painter & Pierre, 2005) of governments we can also distinguish a dual trend toward professionalization of policy making on the one hand, and interactiveness on the other hand. The former aims to improve policies’ effectiveness through a better identification and formulation of policy goals and instruments by policy analysts. The latter pertains to garnering stronger support for policies via a closer involvement of citizens or stakeholder groups. While a trend toward professionalization can be considered as highly instrumental to policy-making, a stronger call for interactiveness seems to be inspired by more subjective, democratic considerations (Brans & Vancoppenolle, 2005). Both the arguments of efficiency and effectiveness, and of policy support are valid in a context of increased complexities and higher uncertainties. These different developments however, may also lead to tensions in the policy making process since they increasingly call upon the role of objective policy analysts in the policy process, while at the same time bestowing an important position on citizens and stakeholders (Brans & Vancoppenolle, 2005).

Both elements in the policy advisory context are of relevance. Given the extensive diversity in sources of evidence for policy, policy advice has become “more contested and competitive” (Halligan, 1995). In another paper, we already questioned the possibility for a single source of evidence to influence policy in a more competitive policy making arena (Fobé & Brans, 2012). We will not treat this in detail here, yet suffice it to note that advisory bodies provide only one of many different sources of evidence for policy.

The dual movement toward policy analytical professionalization and interactiveness for better policies and wider policy support, can also be distinguished within the broader public consultation and advisory body literature (OECD, 2001; McLaverty, 2002; Halffman & Hoppe, 2004; Papadopoulos & Warin, 2007). As Peters and Barker argue, advisory bodies are implemented for very specific reasons:

“[D]emocracy has now acquired a more continuous character; and receiving advice helps governments to appear more open and democratic. (…) [G]overnments may [also] accept or seek out advice simply because they want to make the right decisions” (1993).

The appearance of being more open and democratic provides policy makers with stronger policy support. And making the right decisions, involves acquiring information and expertise to increase policies’ efficiency and effectiveness. These motives for the implementation of advisory bodies are part of a rationalist and instrumentalist approach to policy making and even democracy (Mayer et al., 2005). Higher efficiency and effectiveness and stronger support strengthens the policy-

4

making process by improving policy formulation capacities and facilitating policy implementation. Of course, other possible motives for the implementation of advisory bodies exist. Suffice it to refer to substantive democratic arrangements for empowerment of civil society actors, or in a less noble fashion to strategic and symbolic participation. In the case of strategic motives, the involvement of advisory bodies in policy-making gives governments at least the appearance of being welcoming toward and responsive to knowledge and insights provided by various actors (Peters & Barker, 1993). This strategic dimension of advice production is related to the “political nature of policy making” (Schön & Rein, 1994:14). Governments may thus seek policy advise to legitimize policy decisions that have already been decided upon (Sabatier, 1978; Weiss, 1986; Boswell, 2009).

Apart from identifying the motives behind designing advisory bodies at one point of time, it is also important to understand the historical embedment of advisory bodies. Advisory councils often remain part of the policy-making process because they have been for a long time and are not easily abolished. Institutionalized advisory bodies have for long played an important role in consensus-based political systems with neo-corporatist traits (Lehmbruch & Schmitter, 1977). Policy-making in policy domains such as education, welfare or health, and socio-economic policies is traditionally dominated by few but key stakeholders (Schmitter, 1984; Scholten, 1987; Van den Brande, 1987; De Winter & Dumont, 2003; Van Waarden & Lehmbruch, 2004). They consult with one another in formal and institutionalized advisory bodies and make bipartite or tripartite decisions in their respective policy domains (Ebbinghaus, 2006). These advisory bodies tend to be highly integrated into the official policy-making cycle (Bulmer, 1993). Of course, the involvement of stakeholders in public policy-making varies to different degrees and across different policy domains in systems with neo-corporatist traits. In certain policy domains, policy formulation is very much dominated by tripartite or bipartite corporatism, where key stakeholders are endowed with formal policy-making powers. In other domains, the power of strong traditional stakeholders is played out in informal and formal consultations at one or more instances of the policy cycle. (OECD, 2001). This consultation often proceeds via permanent or ad hoc advisory bodies.

In the Belgian/Flemish advisory tradition, the consultation of stakeholders is historically and typically organized through formally institutionalized advisory councils. This is not surprising in a political system with neo-corporatist traits. Due to their continuingly growing number, the need for reform, transparency and simplification in the advisory council system has recently acquired a place on the government’s agenda (see OECD, 2010). At the regional level in Belgium some reforms in the advisory system have already been implemented. Among these is an important rationalization of the advisory bodies in Flanders. The basis for this reform was a 2000 report to the Flemish government which made reference of no less than 126 advisory bodies providing advice directly to policy-makers. This list of what were deemed as “the most essential advisory councils” did not even include those advisory bodies which were directly situated within the central Flemish policy administration or its executive agencies (Stroobants & Victor, 2000). Responding to calls for political primacy, more transparency, and to concerns over strengthening societal support for policy, the Flemish government reshaped its advisory landscape by decree in 2003. Additionally, the advisory system reform in itself was part of a broader reorganization of the Flemish policy administration aimed at increasing the efficiency and effectiveness of public policies in Flanders.

5

In 2006, the advisory system reform was actually implemented and today, a diverse set of stakeholder groups, citizens and/or scientific experts interact with each other within 12 strategic advisory councils. The table below points out the main policy domains within which these advisory council are active.

Figure 1 – 12 strategic advisory councils in the Flemish policy making process

Council Policy domain

Minaraad Environment & Nature

MORA Mobility

SARC Culture, Sports, Youth & Media

SARiV International relations

SARO Spatial planning

Vlaamse Woonraad Housing

VLABEST Administrative Affairs

VLOR Education

VRWI Science, Technology & Innovation

SERV General Policy, Employment, (Social-)Economy, Energy

SALV Agriculture, Fishing

SAR WGG Wellbeing, Public Health, Family

Assessing the success of advisory bodies We apply an instrumental perspective on advisory councils, and consider them important mechanisms for governments to improve the quality of policy decisions and strengthen support for their policies. In this context, Halligan points out two significant developments in the conduct of advising governments, i.e. the broadening of sources of advice and the expansion of the advisory function toward non-policy actors (Halligan, 1995). The inclusion in policy advisory systems of more sources and actors for advice can also be linked to a general, dual trend toward professionalization and interactiveness.

More precisely, advisory bodies provide different types of knowledge and insights for the policy making process, varying from objective research results to experience-based knowledge to subjective ideas and opinions (Head, 2010). Advisory councils may be constituted entirely of scientists (see for example Jasanoff, 1994; Lavertu et al., 2011), however, they can also include other actors, such as stakeholders, interest groups representatives, individual citizens or political analysts (Halffman & Hoppe, 2004; Papadopoulos & Warin, 2007). The rather classical distinction between the use of objective, scientific analysis in light of better policies (professionalization), and the inclusion of subjective opinions and interests by stakeholders leading to broader societal support (interactiveness), needs to be nuanced. Pielke (2010) refers to the fact that scientists not only provide objective analysis as ‘science arbiter’ in the policy

6

process, thereby improving the quality of policy decisions. They can also play a more ‘political’ role as ‘issue advocate’ in situations where policy problems are highly uncertain on a scientific level and strongly contested on the societal level (Pielke, 2010; Rice, 2011). Moreover, citizens, stakeholders and/or interest group representatives may provide policymakers with important insights, based on their own experience and expertise (Street, 1993; Topf, 1993). Given that “he who does, knows” (Bulmer, 1993:37), this experience-based knowledge can also improve the effectiveness and efficiency of government policies.

Facts as well as opinions are indeed closely related to each other and policies usually come to play through a combination of both (Peters & Barker, 1993). But neither the formulation of better policies nor the broadening of societal support for policies seems exclusively dependent upon the inclusion of one type of actor or the other. The success of these mechanisms at enabling better policies and increasing policy support is extensively discussed in the literature on interactive policies. Several authors argue that certain characteristics contribute to an important extent to this success (Sterne & Zagon, 1997; Webler & Tuler, 2000; Beierle, 2000; OECD, 2001; Beierle & Cayford, 2002; Santos & Chess, 2003; Rowe & Frewer, 2004; Papadopoulos & Warin, 2007). Beierle and Cayford for example, refer to the general ‘intensity’ of the consultation process (2002), stating that more intense participatory mechanisms are also more successful. Hence, participation mechanisms which are permanent and institutionalized, have a fixed membership and apply specific advisory procedures, achieve better substantive and process-related results (Beierle & Cayford, 2002). Other authors refer to similar elements to achieve this positive output result. Among others, Papadopoulos and Warin (2007), as well as Webler and Tuler (2000) make a distinction in their respective frameworks, between characteristics on the input side and elements of the process side of advising contributing to the success of participatory mechanisms. The former refers to factors such as the degree of institutionalization, and the extent to which membership is diverse or representative, while the latter mentions elements such as transparency of the advisory process, facilitation of discussions, or the nature of certain process rules and procedures (Webler & Tuler, 2000; Beierle, 2000; Papadopoulos & Warin, 2007; Edelenbos et al., 2009). Participation mechanisms are then labeled as successful if the advice they provide is not only substantially robust and relevant for policy, but also strongly supported by their members (Beierle & Cayford, 2002). Based on this, we argue that intense participation mechanisms are better at improving the general quality of policy decisions as well as strengthening support for policies (Murswieck, 1993; Beierle, 2002; Mayer et al., 2005; Brans et al., 2010).

The figure schematically captures the above mentioned process leading to successful participation mechanisms. This mode of reasoning implicitly assumes, however, that the advice produced by successful participation mechanisms is actually used by policy makers in the policy making process. And as many authors have already discussed, it would be quite easy to assume that the relationship between evidence and policy were that self-evident (Weiss, 1978; 1980; 1982; Caplan, 1978; Davies et al., 1999; Kirkhart, 2000; Neilson, 2001; Amara et al., 2004; Koontz, 2005; Fobé & Brans, 2012).

7

Advisory context

Policy making context

Figure 2 – Successful participation mechanisms (based on Beierle & Cayford, 2002)

Research focus and approach Our research was commissioned by the Flemish government, who agreed to an evaluation of its newly created strategic advisory system in 2009, three years after the decree establishing these councils was implemented. In this paper, we focus on advisory councils in Flanders and their possibilities for success in creating better policies and ensuring a broad policy support. Several elements are of importance here:

1. Our analysis will first discuss certain defining characteristics on the input side of the strategic advice system in Flanders, such as the councils legal framework, tasks, age, membership characteristics and legacy. Based on this, we discuss the capacity of the advisory system in general to produce policy advice.

2. We then look at the perception of members on several input and process related elements, such as the position of the council in the policy process, the time for advice production, transparency and facilitation of the process of advising, and the degree of involvement of members in the advisory process. We rely on the results of the survey we conducted.

3. Third, we explore whether the policy advice is of substantial quality and/or based on broad support, as well as to what extent the advice provided by the strategic advisory councils was used in the policy making process. Here also, we present the councils’ members perceptions resulting from of the survey we conducted.

Next to an extensive document analysis and several interviews the analysis makes use of the results of a survey carried out in 2009. The survey was conducted to obtain a clear and broad view within a relatively limited time period of different aspects of the advisory process and the councils’ members’ perception and satisfaction about that process. It was carried out with nine out of the twelve strategic advisory councils in Flanders. At the time of the research, only eleven of the current twelve councils had already been formally established. Nine out of these eleven active advisory councils agreed to cooperate with the research and distribute the

Advice production:

- Input

- Process

Quality of advice:

- Substantive quality

- Broadly supported

Effects:

- Better policy decisions

- Higher policy support

8

survey among their members. Two strategic advice councils decided not to participate. The nine participating councils represent three quarters of the total number of strategic advisory councils in Flanders, and stand for an equal share of members in these councils.

The content of the survey was decided upon via a process of internal and external validation. A document analysis and 13 elite interviews in two councils were carried out first. Based on this, a draft of the survey was made and sent to several actors within the policy advisory field. Their comments helped complete the survey, which was then distributed to all members of the participating councils.

On June 10, 2009, the survey was distributed on paper and via email to a total of 203 members of the nine participating advisory councils in Flanders. Five weeks later, the survey’s response rate was 68,5%. The members of those advisory councils with a less than average response rate were sent a reminder via email. The final response rate on July 22, 2009 was 72,4% In total, 147 out of 203 members had filled in the survey. The electronic version of the survey was filled in by 59% of the councils’ members, 41% sent in the paper forms. Additionally, the response rates of the individual councils all surpass 62%, while for some councils the response rate even amounts to more than 80%. The figure below, points out the total number of members in the strategic advice councils and their individual response rates.

Figure 3 – The participating councils, their total number of members and individual response rates

Council Policy domain Members

(#)

Response

(%)

Minaraad Environment & Nature 24 65%

MORA Mobility 28 62%

SARC Culture, Sports, Youth & Media 13 85%

SARiV International relations 20 75%

SARO Spatial planning 20 65%

Vlaamse Woonraad Housing 20 75%

VLABEST Administrative Affairs 14 77%

VLOR Education 39 86%

VRWI Science, Technology & Innovation 25 68%

Due to the overall high response rate, we may state that the results of the survey are widely supported throughout the policy advisory system. However, some remarks on the interpretation of the results discussed below have to be made here. First, the survey only relates to the perceptions and satisfaction of members of the advisory councils. The survey did not include political actors (ministers, members of parliament) as principals of these advisory councils, nor members of the councils’ secretariats. We could assume that the perceptions of these actors differ to some extent from those of the councils’ members, for example regarding the quality or the

9

influence of the advice. Second, the councils all have a general assembly of which the members could fill in the survey. Some advisory bodies also consist of subassemblies with often different members than those in the general council. These members were not included in the survey. As is the case with political actors, it is also possible that these members have a different perception on the working of their advisory council. However, it seemed more relevant to limit the research to members of the general assembly only and not include the members of subassemblies for several reasons: There are only a few councils containing subassemblies, the general assembly makes the most important advisory decisions for the council and the latter is a feature shared by all councils, allowing for comparative observations.

The results of our document analysis and survey are presented below. First, the main characteristics of the Flemish policy advisory system are elaborated upon. Then, the producing of policy advice is assessed from a members’ perspective. Here, the perceptions of members on elements of the advisory process within the councils are discussed. Then, we look at aspects relating to the perceived quality and to the use and influence of the policy advice.

Main characteristics the Flemish policy advisory system The 2003 reform of the Flemish advisory system reorganized the official advisory system in Flanders. In particular, existing advisory bodies were reinstalled as strategic advisory councils, and new strategic advisory councils were created in policy domains where no such permanent structures existed yet or through the merging of various committees and councils. The strategic advisory councils are permanent in that they have been established by decree in their respective policy domains. Via separate decrees these councils are also appointed specific advisory tasks within the policy making process in Flanders. More precisely, the formal role of the strategic advisory councils in the policy making process comes after the policy formulation phase: When policy decisions have been agreed upon in principle between the Flemish government coalition partners, ministers are required to solicit advice with the council(s) in their policy domain. The councils then advise governments on these policy proposals predominantly on the basis of consensus between their members, although some councils include the possibility to incorporate one or more minority standpoints in their advice as well. Policy makers are free to diverge from the advice they have been required to solicit. Thus, the strategic advisory councils do not provide policy advice that is binding. Next to the ministers’ mandatory solicitation of advice on policy decisions after a first principal agreement between the coalition partners, the strategic advisory councils are also entitled to give advice on their own account. The councils can indeed take the initiative of advising government earlier in the policy cycle.

For their activities, the twelve advisory councils receive an annual working budget. In 2011 this budget amounted to an average of 1.4 million euros per council. There are, however, strong differences between the councils. The four largest councils annually receive more than 1.2 million euros each, while the four smallest councils have an annual working budget of 291.000 euros or less. The volume of the working budget is linked to the councils’ membership size, but predominantly related to the tasks that have been assigned to them. These tasks are for some councils broader than advising the Flemish government. They also relate to the implementation of projects and the execution of research in the councils’ respective policy domains. If this is the

10

case, as is for example with the education council, the annual working budget is significantly increased in comparison to councils that only function as advisory bodies. The councils’ staff or secretariat is responsible for writing up the advice and facilitating the advisory process. It also communicates with the minister, the relevant governmental departments or agencies, parliament, other stakeholders as well as the public in general. The councils’ staff size is related to the budget received by government and the different tasks which have been allocated to them.

Figure 4 – Comparison of main characteristics of Flemish advisory councils

Council Year Budget* Staff

size

Members

Specific membership**

President

Council for Environment & Nature

1995/

2009

1.170 7 24 20R + 4A/L Chosen R

Mobility Council 2007 424 5 28 24R + 3A + 1L Appointed L

Council for Culture, Sports, Youth & Media

2008 172 3 13 9R + 4 A/L Chosen A/L

Council on International Relations

2008 340 3 20 10R+10A/L Chosen A/L

Council on Spatial Planning

2008 303 1 20 15R + 5A/L Chosen A/L

Flemish Housing Council

2007 247 1 20 15R + 5A/L Chosen A/L

Council on Administrative Affairs

2007 188 1 14 4R + 10A/L Chosen A/L

Education Council

1990

/2006

2.570 26 39 32R + 7L Appointed

extra

Science, Technology & Innovation Council

1993

/2010

840 6 25 13R + 12A/L*** Chosen A/L

(*) Budget for 2009 presented in EUR x1.000 (**) Number of members representing interest and stakeholder groups (R), advising as independent

actors from the academic community (A) or participating as lay/individual experts or stakeholders (L). (***) The Flemish council on Science, Technology and Innovation includes among its 12 individual and scientific experts also the heads of five Flemish government administration agencies.

11

The councils provide policy advice based on the consultation of a specific number of members. Membership size differs, but on average the councils contain about 20 members. The councils’ membership is to a certain extent fixed by decree: The decrees establishing each council do not state who will be confirmed by government as a member, but they do hold the specific and total number of members in each council. More precisely, each decree contains details on the number of members representing specified stakeholder groups, as well as on the number of individual members having a scientific or lay background. Candidates have to apply for membership and are then instated by government. Each council also has a council president who can be chosen from among the council’s members or who is appointed by the government.

The figure compares the nine participating councils along the above mentioned characteristics. It shows the year the councils were established (or reinstated), their working budget, staff size and total numbers of members. Additionally, the figure also includes the distribution of members as representatives or societal organizations or individual experts/stakeholders, and the way the council president is named.

From the overview it is clear that even though a reorganization of the advisory system has been put in place, significant differences between the councils exist. Firstly, it is important to note that the current advisory system in Flanders is relatively new, since the reform took effect in 2006. Eight of the strategic advisory councils were indeed newly created in 2006, their operation taking effect (most often) one or two years later. Six of these eight newly created councils participated in our research. At the time of our research, they had been active in their respective policy domains for only a short period of time, namely 2 years. Of the four already existing councils, three participated in the research. On average, these three councils had been active for about 15 years in conducting advice for government. When the reform took place, they were reinstated as strategic advisory councils, and although their membership was slightly altered or expanded, their staff and budget remained roughly the same. Importantly, this ensured continuity in the functioning of these councils and the way advice was produced. In this context, we also refer to these established councils’ legacy as advisory bodies. Not only do the already existing councils receive higher government grants and possess of a fully equipped and experienced staff, they can also count more on a strong and long standing reputation of providing sound advice for governments. The councils that were newly created still had to establish and profile themselves as advisory bodies within their respective policy domains as well as vis-à-vis their political principals. We would therefore argue that, already at the start of the establishment of the new advisory system in Flanders, the different strategic advisory councils possessed varying capacities to fulfill their task of formulating policy advice.

Additionally, with regard to membership, the figure shows that lawmakers gave a high preference to the representation of stakeholders and interest groups in the advisory councils. These members often have experience-based knowledge in their policy domain, related to their role as service users or as co-producers of policies. This experience-based knowledge has been awarded a higher importance in the advisory process in comparison to insights provided by individual stakeholders or through academic expertise. Although the advisory system reform included the possibility to create exclusively scientific advisory councils, this has not been the case. And it is only in the council on Administrative Affairs that academics as such play a leading

12

role in advising governments via this type of advisory body. The emphasis in the strategic advisory system in Flanders was thus clearly placed on representation and input from societal actors. As mentioned earlier, this is of course not surprising in a system with neo-corporatist traits. It needs to be noted however, that the other advisory councils do actively look for (academic) expertise during the advisory process and often turn to of a ‘pool’ of scientists to provide them with relevant information. This information then often serves as the scientific background based on which the discussion between stakeholders and interest group representatives takes place. Following the emphasis on inclusion of stakeholders and interest group representatives in the strategic advisory councils we could also assume that the policy advice they produce (by consensus) will be based on (broad) societal support. As we will see later on, this may have an effect on the extent to which these advisory councils are able to generate support for policies.

Perceptions on the production of policy advice Now that we have established an overview of the defining characteristics of the strategic advisory councils in Flanders, we treat the various aspects of the advisory process itself in more detail. Below, the perception of the advisory councils’ members regarding the input and process side of the advice is discussed. We rely on the results of our survey and look at elements such as the council’s position in the policy process, the time for advice production, the degree of facilitation of the advisory process, its transparency and specific voting procedures were questioned.

First, at the input side of advice production we discuss two important elements on which the advisory councils themselves have no influence. More precisely we asked the members of the advisory councils to indicate 1) how often the advice question comes at the right time in the policy process, and 2) how often there is enough time for advice production. Both elements are determined by government itself, and equal to all advisory councils: The question for advice by the councils’ political principal is mandatory after the first decision has been principally agreed upon by the government coalition. The advisory councils then have a maximum number of four weeks to provide government with an advice on the topic.

The relative distribution of the members’ perceptions regarding these two questions are presented here. We can see that almost 60% of members perceive the advice as not coming at the right time in the policy making process, although the distribution of answers is quite even. The survey does not indicate, however, whether the question for advice then comes too early or rather too late in the policy process. We can also point to statistically significant differences between the councils1: Members of the Science, Technology & Innovation Council indicate more often that the advice does come at the right time of the policy process, while the members of the Council on Spatial Planning, the Flemish Housing Council and the Council on Administrative Affairs answer significantly less that this is the case. Interestingly to note is that the latter three councils are newly established in their policy domains and have limited working budgets and staff compared to the other councils.

1 Non-parametric test with p=0.0001

13

Figure 5 – Member perception on the moment of and time for advice production

Question for advice comes at the right time in the policy making process. (N=141)

There is enough time for the production of advice after the question has been received. (N=144)

Never 4,26% 4,86%

Almost never

14,18% 40,97%

Sometimes 41,84% 34,03%

Often 21,28% 13,10%

Almost always

15,60% 5,56%

Always 2,04% 1,39%

The councils apply different strategies to counteract the relatively late moment at which they are asked to formulate policy advice. The Education Council as well as the Council for Science, Technology & Innovation, for example, proactively consult with their political principal (the minister or his/her personal cabinet members2) during the policy making process. The councils hereby try to have the minister ask for advice prior to the required moment in the policy process. Intense consultation requires of course the necessary capacity within each council secretariat to follow up certain dossiers. Consultation with their political principal would also allow the councils to gather more information in advance, i.e. before receiving a question for advice. This allows them to have more time during the official advisory period. More precisely, the council only has four weeks to produce the advice after the question for advice has been received. During this time, the council secretariat first has to gather relevant information on the topic and distribute this to the members. They (or actors delegated from within the same organization) first come together in a preliminary meeting where they discuss the topic properly, then the council secretariat writes up a draft of the advice, which finally has to be established predominantly by consensus and approved at a meeting by the general council.

As the results of the survey show, almost 80% of the members indicate that they seldom have sufficient time for the formulation of policy advice, while only 13% indicates that this is often the case. Only 7% is truly positive with regard to this question and perceives there to be enough time for the production of advice. Here also, there are statistically significant differences between the councils3. Yet again,

2 A ministerial cabinet comprises the staff of personal advisors who are hired when a minister takes office and

who are not part of the administrative hierarchy. They assist the minister in identifying and formulating problems, in outlining policy, and in everyday decision-making. They are spoils in that they come and go with their minister. 3 Non-parametric test with p=0.0001

14

members of the Innovation Council perceive the time for advice production significantly more often to be enough, while members of the Housing and Spatial Planning Council indicate to greater extent that this is not the case. From the interviews we indeed assumed that in general, the time for formulating the policy advice would be perceived as insufficient. The councils themselves do not only look for increased consultation with the minister to prevent time for advice to become a problem. They also take up policy advice production at their own initiative. In this way, time for advice production is not limited to four weeks and advice can be formulated even before decisions have been principally decided upon between government coalition members. It can allow the councils to proactively place a topic on the policy agenda, or provide advice during the early stages of the policy formulation phase.

The organization of the internal advisory process, is considered to be important in light of the success of advisory bodies. According to various authors, several factors can contribute to this success (Webler & Tuler, 2000; Beierle, 2000; Papadopoulos & Warin, 2007; Edelenbos et al., 2009). Below, we further discuss some of these elements situated at the process side of advice production, such as the participation by the councils’ members, the degree of facilitation of the advisory process, its transparency and specific voting procedures. It needs to be noted that the councils are able to decide upon the application of these elements themselves and thus have the possibility to improve their own performance if necessary.

Figure 6 – Perceptions regarding the input of members during the advisory process

I feel strongly involved during the advisory process (N=144)

I am satisfied about the extent to which my input is being considered during the advisory process (N=144)

Absolutely not 0,69% 1,39%

No 2,03% 5,56%

Not really, no 9,03% 8,33%

Neither positive nor negative

13,89% 12,50%

A bit, yes 36,81% 47,92%

Yes 29,86% 19,44%

Absolutely 7,64% 4,86%

15

A first set of elements we discuss here, is the extent to which the members of the councils feel strongly involved during the advisory process and are satisfied about the consideration during this process of their own input. This is important with regard to the active participation of members in the council and thus the way policy advice is produced: through a search for consensus, based on the input of multiple and often very different actors. Overall, both factors can be seen as positive aspects of the advisory process. About 70% of the councils’ members feel strongly involved and the same percentage of members is satisfied about the way their own input is being considered. Only 10% of the total number of members in the advisory councils, does not feel strongly involved during the advisory process, and about 15% indicate not to be satisfied with the way their input is being considered. Even though the figures provide quite a positive view on this subject, we emphasize the importance of addressing possible problems beforehand. Members who do not feel involved in the production of policy advice in their council, may no longer actively participate during the advisory process, or even terminate their participation in the council in general. This can possibly generate a negative influence on the quality of the policy advice and the support for it.

Figure 7 – Perceptions on transparency and voting procedures within the councils

I have a clear view on the advisory process in my council (N=142)

I agree with the way my council decides upon the advice (N=144)

Absolutely not 0% 0%

No 2,11% 1,39%

Not really, no 5,63% 4,17%

Neither positive nor negative

12,68% 9,72%

A bit, yes 33,10% 34,03%

Yes 34,51% 36,81%

Absolutely 11,97% 13,89%

A second set of elements presented, are factors relating to the internal advisory process. We questioned to what extent the overall advisory process was perceived by the members as transparent. The production of policy advice during the aforementioned four weeks goes through several phases which are more or less fixed and applied in each dossier to roughly the same extent. A clear view on these phases can provide the members with a sense of ownership. In this context we also discuss to what extent the members agree on voting procedures for advice production in the council. As mentioned already, most advisory councils provide

16

advice based on a consensus between their members. The Environment and Nature Council and the Education Council include the possibility of adding one or more minority standpoints in their final advice, although looking for consensus is always considered. It needs to be noted, that we did not ask the members which type of decision making system they preferred. The question rather focused on the extent to which they agreed with the way advice was decided upon in their own council.

The figure presents the relative distribution of answers across the advisory councils. As was the case with the other elements of the advisory process, the transparency of the process and the way the councils decide on the advice is generally appreciated positively by the members. Around 45-50% even strongly appreciates these elements, while only 7% of the total number of members in the advisory councils indicates not to have a clear view on the advisory process and about 5% does not agree with the way advice is decided upon in their council. Additionally, when we consider the answers for each council separately, there are differences among member perceptions on the topic of transparency of the advisory process. And although the score for each council on a scale from 1 to 7 (1 being highly negative, and 7 being highly positive) is relatively high, i.e. 5,28, these differences are statistically significant4. The members of the Scientific Council, the Council for Nature and Environment and the Council on Administrative Affairs indicate relatively more that they perceive the advisory process to be transparent. On the other hand, and yet again, the Flemish Housing Council’s members indicate that this is much less the case.

Figure 8 – Average scores on transparency of the advisory process

Council Average score

(from 1-7)

Council for Environment & Nature 5,80

Mobility Council 5,38

Council for Culture, Sports, Youth & Media 5,10

Council on International Relations 5

Council on Spatial Planning 4,85

Flemish Housing Council 4,53

Council on Administrative Affairs 5,7

Education Council 5,34

Science, Technology & Innovation Council 5,64

A last factor we discuss in this part of the paper, relates to the role of the councils’ staff and president as facilitators of advice production in their respective councils. The council secretariat provides the relevant information to the members on the topic

4 Non-parametric test with p=0.0211

17

of discussion and writes up the final advice. In getting to this final advice, the secretariat and the council president are responsible for facilitating the discussions between the members. They are also assumed to mediate between the members during these discussions and strive for a consensus on the advice. We asked the members to what extent the discussions in the council were structured well by the president, and the council president added to the production of the advice. We also asked to what extent the council staff provided adequate relevant information for the members to generate the advice, and the council staff added to the proceeding of the advisory process. We regarded these four elements as part of the same scale on ‘facilitation of policy advice’ and present the overall score in the figure below. The result on a scale from 1 to 7 (1 being highly negative, and 7 being highly positive) shows that the councils’ members have a very positive perception of the way advice is facilitated by both the secretariat and the president of each council.

Figure 9 – Perceived facilitation of policy advice

Scale Average score

(from 1-7)

Facilitation of policy advice 5,74

Items*

The discussions in the council were structured well by the president of the council

The council president added to the production of the advice

The council staff provided adequate relevant information for the members to generate the advice

The council staff added to the proceeding of the advisory process

(*) Chronbach’s alpha raw score = 0.74

In general, we can conclude that the factors pertaining to the production of advice within the councils are predominantly positively appreciated by their members. We would argue that this is relevant in light of the substantial quality of the advice. Additionally, and possibly more important, a general positive perception about the advisory process would contribute to the support for this advice by its members, as well as to the functioning of the advisory councils in general. We need to note, however, that there is to some extent margin for improvement on this subject, given that about 10% of members do not have a clear view on or indicate to feel little involved during the advisory process. These aspects of internal organization of the advisory process can be (easily) addressed by the strategic advisory councils themselves.

Member perspectives on the quality and influence of policy advice Quality advice, i.e. policy advice that is substantively relevant and supported by different stakeholders, can contribute to better policies and wider policy support

18

(Murswieck, 1993; Beierle, 2002; Mayer et al., 2005). From the analysis above, we can conclude that, as far as the input side is concerned, strategic advisory councils start off with different capacities to advise governments. The process side of the advisory context showed, however, that the strategic advisory councils’ members perceive the councils to function considerably well – even though there are significant differences between the councils. In this part of the paper, we focus on how members perceive the quality and influence of the policy advice they produce. We then discuss the consequences these perceptions might have for the advisory councils to contribute to better policies and wider policy support.

To assess the members’ perception on the quality of the advice and the support for it, we asked them to what extent they agreed to the policy advice being 1) evidence based, 2) well-structured and readable, and 3) strongly supported. The results show that the members have a positive appreciation of the substantive and technical quality of their councils’ advice, as well as of the societal support this advice relies on. The figure shows the average scores on a scale from 1 to 7 (1 indicates a very low and 7 a very high degree of agreement with the statements). It also shows to what extent the members are satisfied about the strategic level of the advice their councils produce. These questions pertain to the formulation of advice based on a question from the minister, or initiated on the councils’ own account. For these items, the average scores are lower. Additionally, we note a 10% difference between the two levels of satisfaction on the strategic level of the advice. The members indicate that they are more satisfied about the strategic level in the advice they produce on their own initiative, compared to questions of advice coming from the minister.

Figure 10 – Perceived quality and support base of policy advice

Questions N Average score (from 1-7)

The policy advice relies on a broad evidence base 145 5,78

The policy advice is well structured and readable 144 5,71

The policy advice is supported among societal stakeholders 138 5,35

I am satisfied about the strategic level of the advice when it is produced after a question from the minister was received

141 4,26

I am satisfied about the strategic level of the advice when this is produced on my council’s own initiative

141 4,95

Next, we look at how members perceive the influence of the advice they produce within the strategic advisory councils. Interestingly, the scores for these items are reasonably low in comparison to other items in this papers. The construction of a Likert scale5 for various items treating this topic, leads to an average score of 2,69 (based on scores ranging from 1-6, where 1 indicates that the advice never

5 Cronbach’s alpha raw score = 0.82

19

influences policy and a score of 6 indicates that this is always the case). In general, the members of the strategic advisory councils seem to be quite skeptical about the different ways the advisory councils are able to influence the policy making process. We questioned several items, pertaining to an instrumental influence of the policy advice, but also to a possible agenda-setting influence and a more strategic influence on policy or policy makers (see for example Greene, 1988; Weiss, 1991; Kirkhart, 2000; Bekkers et al., 2004).

Instrumental influence or direct influence refers to advice being directly used by policy makers in the policy process. Our scale of instrumental influence6 contains two variables. More precisely, we asked the respondents to indicate 1) how often the advice leads to a change in the implementation of policies, and 2) how often it changes policy decisions.

We consider an agenda-setting influence7 of advice taking place when it leads to 1) a political discussion, 2) attention by the media, as well as 3) a debate within society in general.

Third, advice can be used (possibly even misused) in a more political context. Here it serves as a means 1) to either legitimize the government’s own policies or, from the viewpoint of the political opposition, 2) to argue for a change in policies.

The discussion treats the perceptions of the advisory councils’ members on these different elements in more detail. The figure below shows the scores of the councils separately.

We can see that the members perceive the advice which they produce, only very occasionally to have an influence on policy and policy makers. On a scale from 1 (never) to 6 (always) the average scores are 2,85 for instrumental influence, 2,34 for an agenda-setting influence, and 3,19 with regard to a strategic influence. Additionally, there are statistically significant differences8 between the various councils. The Council for Science and Innovation has significantly higher scores on the three types of influence, although relatively speaking, the scores remain quite low. A bit surprising perhaps are the low scores which the members of the Council on Administrative Affairs attributed to the various types of influence generated by the advice they produce. Perhaps these members, which predominantly have a scientific background, are even more critical of government compared to their counterparts in other councils, who predominantly represent stakeholder organizations or interest groups.

It also needs to be noticed that compared to all the other questions in the survey, the elements pertaining to influence generated the lowest average response rates among respondents. For every question in these items, about 20% of respondents indicated that they did not know how often the advice influences policy. The degree to which the councils’ members responded to these items or indicated not being able to answer the question, also differs strongly. For example, only 40% of respondents in the Council on International Relations answered the items on instrumental influence, while the two other types of influence also generated low response rates.

6 Cronbach’s alpha raw score = 0.82

7 Cronbach’s alpha raw score = 0.81

8 Non-parametric test with p<0.0001 (instrumental influence); p=0.0008 (agenda-setting influence); p=0.0006

(strategic influence)

20

The members of other councils attributed to a larger extent scores to the questions regarding the different types of influence.

Figure 11 – Member perception on three types of influence of the policy advice

Council Instrumental

Influence

Agenda-setting

Influence

Strategic

Influence*

N (%) Score

(1-6)

N (%) Score (1-6)

N (%) Score (1-6)

Council for Environment & Nature

85 2,79 95 2,67 80 3,06

Mobility Council 67 2,88 89 2,52 72 3,15

Council for Culture, Sports, Youth & Media

100 2,95 82 1,70 82 3

Council on International Relations

40 2,33 47 2,14 53 3,13

Council on Spatial Planning

85 2,14 85 2,21 77 3

Flemish Housing Council 87 2,35 67 1,90 40 3,5

Council on Administrative Affairs

90 1,94 80 1,83 90 2,11

Education Council 87 3,26 87 2,42 87 3,19

Science, Technology & Innovation Council

100 3,82 93 2,79 93 4,38

(*) Cronbach’s alpha raw score for the above two elements was quite low, i.e. 0.52; in the figure we only present the scores for legitimizing policy

Successful advisory councils in Flanders…or not? So are these advisory councils instrumental in creating better policies and generating policy support? When we take the perceived quality and support of the policy advice into consideration, and then look back at the perception of members regarding the influence of their advice on policy, some remarks can be made.

We consider the reasoning that qualitative and supported policy advice leads to qualitative policy decisions as well as stronger support for policies (Murswieck, 1993; Beierle, 2002). This is based on the assumption that policy advice will be actually used by policy makers in the policy process. Several authors have already shown that the link between evidence and policy is not that self-evident (Weiss, 1978; 1980; 1982; Caplan, 1978; Davies et al., 1999; Kirkhart, 2000; Neilson, 2001; Amara et al.,

21

2004; Koontz, 2005; Fobé & Brans, 2012). Based on the insights in the literature, the application of an instrumental perspective on the establishment of advisory councils already needs to be approached with reservation. In light of the above results, however, we argue that the instrumental goals of government for establishing policy advisory councils will not be reached in Flanders. We state this especially referring to the assumption that advisory councils are established to generate stronger support for policies. After all, the results of our survey show that the qualitative perception of policy advice is high, but that members do not perceive this advice as being sufficiently used by policy makers. This negative perception on the use of policy advice by policy makers, will possibly not result in higher support for policy decisions. Based on the members’ perceptions, we could even argue that the establishment of advisory councils in Flanders may counteract the goal of generating more support for policies and have a negative effect.

Conclusion In this paper, we looked at the Flemish advisory councils as means established to create better policies and generate a broader societal support for them. Several characteristics pertaining to the functioning of these advisory councils were assessed. The analysis was based largely on a survey we conducted in 2009 with the members of the new strategic advisory system in Flanders. These councils resulted from a 2003 reform. Twelve permanent strategic advisory councils were established and allocated a specific working budget. They were also assigned one or more staff members and constituted of a more or less fixed membership mix between representatives of stakeholder groups, individual stakeholders and scientists. Even at the start of the new advice system, there were already large differences between the councils’ capacities to produce policy advice. However, as the analysis showed, the members at least perceived the process of advice production in their advisory councils considerably positive. Additionally, the councils’ members were satisfied with the quality of the advice they produced. Notwithstanding a positive appreciation of the advisory process itself, the members perceive the policy influence of their advice as very limited. This might pose a problem when considering the instrumental motives for the establishment of advisory councils by government.

In the literature on consultation mechanisms, several authors argue that quality advice can lead to better policies and stronger support. Implicitly this argument assumes that quality policy advice actually be used by policy makers. Influencing policy is, therefore, a crucial (but implicit) condition when regarding advisory councils as mechanisms for creating better policies and generating a broader societal support. We noted, however, that the strategic advisory councils’ members are quite skeptical about the various ways in which the advice which they produce influences policy or policy makers. We first take into account here that we have only looked at the perception of the members of the councils, and make note of the fact that based on these results we can say little about the goal of actually creating better policies. For this, more research is required as well as the inclusion of political actors in the analysis. However, these results do lead us to question whether the establishment of the new advisory system in Flanders will lead to stronger societal support. More importantly, we even point out the risk of negatively affecting the support for policies, due to the fact that advice by predominantly stakeholder-based advisory councils is perceived to be ignored by policy makers. As mentioned already, the strategic

22

advisory system in Flanders was also established over concerns to restore political primacy in the policy process, and there seems to be a tradeoff between the latter and an instrumental perspective on policy. We conclude by stating that the management of these advisory council members’ expectations by policy makers has to be considered equally important as the actual management of the advisory process itself. Additionally, it would be interesting for future research to assess policy makers’ perspectives on this topic. An analysis of their expectations about advisory councils, the way they perceive the quality of the policy advice and the influence of the advice on policy, may produce some interesting and possibly surprising results.

References Amara, N., Ouimet, M., & Landry, R. (2004). New evidence on instrumental, conceptual and

symbolic utilization of university research in government agencies. Science Communication, 26(1), pp.75-106.

Beierle, T., (2002), The quality of stakeholder-based decisions. Risk Analysis, 22, 739-748 Beierle T. & Cayford, J., (2002), Democracy in practive. Public participation in environmental

decisions. Washington D.C.: Resources for the future. Bekkers, V., Fenger, M., Homburg, V., & Putters, K., (2004), Doorwerking van strategische

beleidsadvisering. Erasmus University Rotterdam & University of Tilburg. Bellone, C.C. en Goerl F.F., (1992), Reconciling Public Entrepreneurship and Democracy in

Public Administration Review, 52, 2. Boswell, C., (2009), The Political Uses of Expert Knowledge. Immigration Policy and Social

Research. Edinburgh: University of Edinburgh. Bowen, S. & Zwi, A. (2005) Pathways to “Evidence-Informed” Policy and Practice: A

Framework for Action, Public Library of Science, 2(7), pp.600-605. Brans, M. & Vancoppenolle, D., (2005), Policy-making reforms and civil service: an

exploration of agendas and consequences. In: Painter, M. & Pierre, J., (Eds.). Challenges to State Policy Capacity: Global Trends and Comparative Perspectives. Basingstoke: Palgrave Macmillan, pp. 164-184.

Brans, M., Van Damme, J. & Gaskell, J., (2010), Balancing expertise, societal input and political control in the production of policy advice. Education Councils in Europe. Brussels: Eunec. 218p.

Bulmer, M., (1993), The Royal Commission and Departmental Committee in the British policy-making process. In Peters, G. & Barker, A. (Eds.), Advising West European Governments, Inquiries, Expertise and Public Policy, pp.37-49. University of Pittsburgh Press.

Cawson, A., (1982), Corporatism and welfare : social policy and state intervention in Britain, London: Heinemann.

Caplan, N., (1979), The Two-Communities Theory and knowledge utilization. American Behavorial Scientist, 22, pp.459-470.Barker A., (1979), Public Participation in Britain. A Classified Bibliography. London.

Davies, H., Nutley, S. & Smith, P., (1999), Editorial: what works? The role of evidence in public sector policy and practice, Public Money & Management, 19,1, pp. 3-5.

De Standaard, 03.11.2011. N-VA vraagt einde wildgroei adviesraden. Dewachter, W., (1995), Besluitvorming in politiek België, Leuven: Acco. De Winter, L. & Dumont, P. (2003), Belgium: Delegation and accountability under

partitocratic rule. In Strom, Müller, Bergman (Eds.), Delegation and accountability in parliamentary democracies..

De Winter, L., (2009), “Belgium: Insider pressure groups in an outsider Parliament” In Norton, P., (Ed.) Parliaments & pressure groups in Western Europe, London: Cass.

23

Dror, Y., (2007), Crafting the past on the future: realistic visions and futuristic nightmares? In Schuyt et al. (Eds.) The thinking state?, (pp.83-97), The Hague, Scientific Council for Government Policy.Greene, J.G., (1988), Stakeholder participation and utilization in program evaluation, Evaluation Review, 12, pp.91-116.Halffman, W. & Hoppe, R., (2004), “Science/policy boundaries: a changing division of labour in Dutch expert policy advice”. In Maasse, S. & Weingart, P. (Eds.), Scientific Expertise and Political Decision Making, Dordrecht: Kluwer.

Ebbinghaus, B., (2006), Reforming Bismarckian Corporatism: The Changing Role of Social Partnership in Continental EuropePaper presented at the Conference: A long good bye to Bismarck? The Politics of Welfare Reform in Continental Europe. Minda de Gunzburg Center for European Studies, Harvard University, June 16-17.

Edelenbos, J., Klok, P.-J. & Van Tatenhove, J., (2009), The Institutional Embedding of Interactive Policy Making Insights From a Comparative Research Based on Eight Interactive Projects in the Netherlands. The American Review of Public Administration, 39, 2,pp.125-148.

Fobé E., & Brans, M., (2012), The use of policy-oriented foresight in Flemish policy making: A tentative explanation of the factors influencing knowledge utilization. Paper presented at the XVI IRSPM Conference in Rome, Italy from 11-13 April 2012.

Halligan, J., (1995), “Policy advice and the public service” In Peters, G. & Savoie, D. (Eds.), Governance in a changing environment, Quebec: Canadian Centre for Management Development.

Head, B.W., (2010). ‘Evidence-based Policy: Principles and Requirements’, in Productivity Commission, Strengthening Evidence Based Policy in the Australian Federation, Volume 1: Proceedings, Roundtable Proceedings, Canberra: Productivity Commission, pp. 13-26.

Howlett, M., (2008), Enhanced Policy Analytical Capacity as a Prerequisite for Effective Evidence-Based Policy-Making: Theory, Concepts and Lessons from the Canadian Case. Paper prepared at the International Research Symposium on Public Management XII, from 26-28.03.2008 in Brisbane, Australia.

Jasanoff, S., 1994, The fifth branch. Science advisors as policy makers. Cambrigde: Harvard University Press, 302p.

Kirkhart, K.E., (2000), Reconceptualizing evaluation use: an integrated theory of influence. Directions for Evaluation, 88, pp.5-22.

Koontz, T., (2005), We Finished the Plan, So Now What? Impacts of Collaborative Stakeholder Participation on Land Use Policy. Policy Studies Journal, 33,3, pp.459-481.

Lavertu, S., Walters, D. & Weiming, D., (2012), Scientific Expertise and the Balance of Political Interests: MEDCAC and Medicare Coverage Decisions. JPART, 22, pp.55-81.

Lembruch, G. & Schmitter, F. 1977, Patterns of corporatist policy-making Sage, London. Lowndes, V., Pratchett, L., & Stoker, G. 2001, "Trends in public participation: Part 1. Local

government perspectives", Public administration, 79[1], pp. 205-222. Mayer,I., Edelenbos,J., Monnikhof,R., (2005), Interactive Policy Development: Undermining

or Sustaining Democracy? Public Administration, 83, 179-199 McLaverty, P., (2002). Public participation and innovations in community governance.

Aldershot: Ashgate. Murswieck, A., (1993), Policy advice in federal Germany. In Peters, G. & Barker, A. (Eds.),

Advising West European Governments, Inquiries, Expertise and Public Policy (pp.87-97) University of Pittsburgh Press

Neilson, S., (2001), IDRC-supported research and its influence on public policy. Knowledge utilization and public policy processes: a literature review. Ottawa: The International Development Research Centre, 53p.

Nutley, S., Davies, H.. & Walter, I., (2009), Evidence for policy and decision-making: a practical guide. Argyrous, G. (ed.). University of New South Wales Press.

OECD, (2010), Better Regulation in Europe - Belgium. Paris: OECD.

24

OECD, (2001), Citizens as partners. Information, consultation and public participation in policy-making. Paris: OECD.

Painter, M. & Pierre, J., Challenges to State Policy Capacity: Global Trends and Comparative Perspectives. Basingstoke: Palgrave Macmillan.

Papadopoulos, Y. & Warin, P., (2007), Are innovative, participatory and deliberative procedures in policy making democratic and effective? European Journal of Political Research, 46, 445-472

Pawson, R. (2002). Evidence-based policy: The promise of ‘Realist Synthesis’. Evaluation, 8,3, pp.340–358.

Peters G. & Barker, A. (1993b), Introduction. Governments, information, advice and policy-making. In Peters, G. & Barker, A. (Eds.), Advising West European Governments, Inquiries, Expertise and Public Policy (pp.1-19) University of Pittsburgh Press.

Pielke, R. jr. 2010 The Honest Broker: Making Sense of Science in Policy and Politics. Cambridge. Cambridge University Press.

Rice, J., (2011), Advocacy science and fisheries decision-making. ICES Journal of Marine Science, 68(10), pp.2007–2012.

Rittel, H. & Webber, M. 1973 "Dilemmas in a General Theory of Planning," Policy Sciences, 4, pp. 155-159.

Rowe, G. & Frewer, L.J., (2004), Evaluating Public-Participation Exercises: A Research Agenda. Science, Technology, & Human Values,29, 512-557Sanderson, I., (2002), Evaluation, policy learning and evidence-based policy-making. Public Administration, 80, pp.1-22.

Sabatier, P., (1978), The Acquisition and Utilization of Technical Information by Administrative Agencies. Administrative Science Quarterly, 23, pp.396-417.

Santos, S. & Chess, C., (2003), Evaluating citizen advisory boards: the importance of theory and participant-based criteria and practical implications, Risk Analysis, 23.

Schmitter, P., (1984), EU working paper n° 106. Neo-corporatism and the state, San Domenico di Fiesole: Badia Fiesolana.

Scholten, I., (1987), “Corporatism and the neo-liberal backlash in the Netherlands”, in Scholten, I. (Ed.), Political Stability and Neo-corporatism. London: Sage.

Schön, D. & Rein, M., (1994), Frame reflection : toward the resolution of intractable policy controversies, New York: Basicbooks, 247p.

Sharp, C. (2005), The improvement of public sector delivery: Supporting evidence based practice through action research. Edinburgh: Scottish Executive Social Research.

Shaxson, L (2005) ‘Is your evidence robust enough? Questions for policy makers and practitioners’ Evidence and Policy: A Journal of Research, Debate and Practice, Vol. 1, No. 1, pp.101-111.Schmitter, P., (1982), “Reflections on Neo-Corporatism”, in Lehmbruch, G. & Schmitter, P. (Eds.), Patterns of corporatist policy-making, London: Sage.

Sterne, P. & Zagon, S., (1997), Public consultation guide. Changing the relationship between government and Canadians. Quebec, Canadian Centre for Management Development, 91p.

Street, J., (1993), Policy advice in an established official advice structure. AIDS Advice through the British Department of Health. In Peters, G. & Barker, A. (Eds.), Advising West European Governments, Inquiries, Expertise and Public Policy, pp.151-164. Pittsburgh: University of Pittsburgh Press

Stroobants, E. & Victor, L., (2000), Beter bestuur – een visie op een transparant organisatiemodel voor de Vlaamse administratie. Brussel: Ministerie van de Vlaamse Gemeenschap.

Topf, R., (1993), Conclusion. Advice to governments – Some theoretical and practical issues. In Peters, G. & Barker, A. (Eds.), Advising West European Governments, Inquiries, Expertise and Public Policy, pp.182-228. Pittsburgh. University of Pittsburgh Press

Van Damme, J. & Brans, M., (2008), Interactief beleid van de Vlaamse overheid. Een internationale verkenning naar visie, beleid en praktijken. Leuven: Steunpunt Bestuurlijke Organisatie Vlaanderen.

25

Van Den Brande, A., (1987), “Neo-corporatism and functional-integral power in Belgium” In Scholten, I. (Ed.), Political Stability and Neo-corporatism. London: Sage.

Van Waarden, F. & Lehmbruch, G., (2004), Renegotiating the Welfare State. Flexible adjustment through corporatist concertation, London: Routledge.

Webler, T. & Tuler, S., (2000), Fairness and competence in citizen participation. Theoretical

reflections from a case study. Administration & Society, 32, 566-595 Weiss, C.H., (1978), Improving the linkage between sociale research and public policy. In

Lynn, L.E. (Ed.) Knowledge and policy: the uncertain connection, (pp.23-81), Washington D.C., National Academy of Sciences

Weiss, C.H., (1980), Knowledge creep and decision accretion. Knowledge, Creation, Diffusion, Utilization, 1, 381-404

Weiss, C.H., (1982), Policy research in the context of diffuse decision making. The Journal of Higher Education, 53, 619-639

Weiss, C.H., (1986), Research and policy-making: A limited partnership. In Heller, F., (Ed.). The use and abuse of social science. Londen: Sage. pp.214-235.

Weiss, C.H. (1991). Policy research: data, ideas, or arguments. In P. Wagner, C. H. Weiss, B. Wittrock & H. Wollman (Eds.), Social science and modern states: national experiences and theoretical crossroads (pp. 307-332). Cambridge: Cambridge Univeristy Press.