second-order auditing practices

Systemic Practice and Action Research, Vol. 14, No. 2, 2001

1094-429X/ 01/ 0400-0157$19.50/ 0 2001 Plenum Publishing Corporation

157

Second-Order Auditing Practices1

Alfonso Reyes2

This paper describes the process that was followed at the Colombian National AuditOffice, during 3.5 years, to develop new auditing practices, called second-orderauditing. The purpose of the second-order control is to detect possible faults in theinternal control mechanisms of the audited entities. General and particular modelsfor second-order audit are presented, along with a method to carry out this type ofauditing. Finally, a particular application of second-order auditing carried out in aColombian national institution is presented.

KEY WORDS: second-order auditing; second-order control; unfolding of complexity;organizational learning.

1. INTRODUCTION

The 1991 constitutional reform opened up a space to start a process of modern-izing the Colombian state. Since then, many different national institutions havebegun profound changes to their organizations based on the new Constitution.

This administrative modernization of state organizations has, to a greatextent, been prompted by article 209 of the Constitution, which states that “. . .the administrative function is at the service of the general interests . . .” and itsdevelopment should be based on principles of equality, impartiality, and trans-parency, by decentralization, delegation, and distribution of functions.

The aforementioned Constitution also established, as an indispensable con-dition of efficient public administration, the development of internal control sys-tems in each of the public offices. To this effect, article 269 of the Constitutionsays, “In public organizations, the corresponding authorities must design andapply, according to the nature of each function, methods and procedures of inter-nal control, according to what is mandated by law . . . .”

The legal development of this constitutional norm has included variouslaws and decrees. Nevertheless, in spite of the constitutional mandate and its

1Translated from Spanish by Rebeca Donoso.2 Department of Industrial Engineering, Universidad de los Andes, Bogota, Colombia.

Reyes158

corresponding legal development, there were very few public organizations thatmanaged to develop internal control systems that allowed them to supervise thefulfillment of obligations autonomously. And the process of modernization ofColombia’s public administration will not be completed until these internal con-trol systems are operating extensively throughout all of the public offices.

The constituents’ desire to accelerate the development of effective self-con-trol systems in public organizations led to a debate on the traditional role of theNational Audit Office in Colombia (Contralorıa General de la Republica; CGR).It was concluded that the CGR’s mission could no longer be just detecting irreg-ularities in the use of national resources by public organizations, but should bemaking sure that the public entities made an effective use of these resources.

From this perspective, the CGR’s primary role should be to ensure the qual-ity of the internal control systems in each public entity, or to control these enti-ties’ control systems. In other words, this meant exercising second-order control(see paper by Espejo in this issue).

This change in the CGR’s mission would provide it a direct benefit in thesense that it could practice a second-order control more effectively with theinfrastructure that it had at that time. This would be possible because the vastmajority of the internal problems in each public institution could be detected andsolved more quickly within the institution itself. As a consequence of this, themanagement of all the public offices that it had to audit would also improve.

The auditing teams would go from a police-like function to a facilitatingfunction in the processof improving the internal control systems of each stateentity, without neglecting their investigating responsibilities whenever necessary.Thus, the CGR would no longer assess the fulfillment of its objectives from thepoint of view of the amount of money that it had been able to recover in differentlawsuits, but from the point of view of the effectiveness of the public entitiesthat it had audited.

This document describes the process that was followed at the CGR, during3.5 years, to develop new auditing practices, called second-order auditing, thatwould enable the fulfillment of the objectives.

2. HISTORICAL CONTEXT

The practice of public control began in Colombia with the first Law Courtsestablished by the Spaniards who came to America in the 16th Century. From1511, when the Santo Domingo Royal Audience Law Court was created, until1898, when the Bogota Court of Accounts was reinstated, the objective of fiscalcontrol was to examine the accounts of all the employees who collected andinvested public funds. These entities also had the power to bestow sanctionson those employees who were responsible for transgressions or offenses in themanagement of these funds.

Second-Order Auditing Practices 159

In 1923, the work done by the Kemmerer Mission was the first and mostimmediate predecessor of the present CGR. A few years later, in 1931, theCGR began to control the budget applications, revenues and expenditures, andto supervise the accounting of public offices. At this point its administrativestructure began to grow.

In 1945, through a constitutional reform, the CGR was given the functionof supervision of the management of all public offices. In the development ofthis legislative act, Law 58 (1946), determining the statutes of public control,was passed.

The third phase in the evolution of public control, from the Kemmerer Mis-sion was Law 20 (1975), which established that public control required an anal-ysis (a prior and perceptive analysis) of all the transactions and activities per-formed by public offices. When this law was passed the traditional examination,explanation, and expiration of the accounts ceased to exist.

In order to fulfill this fiscal control, the CGR grew even more and estab-lished an office in each public entity, from where its employees could personallysupervise what was going on. This control extended to anyone who, in any cir-cumstance, had to manage the nation’s public resources or funds. Likewise, itwas given the ability to sue in case of theft of public funds.

At that moment, fiscal control was done through two types of auditing prac-tices: financial and operational. The former consisted of “an objective, system-atic, professional, and independent examination of the operations according touniversally accepted accounting and auditing norms,” and the latter included “acritical and systematic revision designed to assess the effectiveness, efficiency,and economy with which the public funds have been used.”

This model of fiscal control went into a crisis at the beginning of the sev-enties because the CGR offices in each public entity became the bottleneck forpublic administration as they had to go over each contract before it was signed.

The control function focused on the collection of large amounts of data,which referred to events or situations of the past and, on registering them,after some analysis, in extensive reports that were sent to the General Con-troller through a large chain of bureaucratic intermediaries. Naturally, when thesereports were made public they affected the news and public opinion for a while,but their usefulness was limited. First, the fact that the reports referred to pastevents did not preclude that these same events would not occur again because,in the meantime, the public entity under scrutiny had changed. Second, the factthat the reports were centralized in the main office made it more difficult for theparticular entities to adopt decisions or to effect changes once they had detecteda fault. Third, the audited entity did not enter into a learning process to improveits own internal control mechanisms.

To avoid this serious administrative obstacle, the new 1991 Constitutionmodified fiscal control by totally eliminating prior and perceptive control and

Reyes160

substituting it for an a posteriori and selective auditing. To this effect Law 42(1993) was issued. This is the law that regulates fiscal control in Colombia atpresent.

An a posteriori and selective control pretends

(a) to determine to what extent the results obtained by the audited entitiesfulfil the objectives proposed by their planning bureaus,

(b) to issue a statement on the financial viability and result of the operations,clarifying whether they comply or not with established norms.

(c) to verify if the activities that have been performed with public fundscomply with the rules that regulate them,

(d) to evaluate the environmental cost of the controlled entities, and(e) to assess the efficiency, economy, efficacy, and equity with which the

public resources are managed.

In tune with this new type of a posteriori and selective fiscal control, theconstituents established that all public offices had to develop their own internalcontrol systems. This led to a new modification of the personnel layout of theCGR because now it no longer made sense to have an office in each publicentity. The number of employees was, thus, drastically reduced, from 17,000 toapproximately 5000.

The creation of internal control systems in each public entity made it nec-essary for the constituents to be more explicit about the nature of these changes,given the fact that the CGR would seemingly no longer be in control. This appar-ent lack of clarity in the legislation has led to different interpretations of theConstitution based on the aforementioned premises. In fact, in many cases, thecreation of internal control systems has been interpreted as a mere delegation ofthe CGR control function so that each entity can have prior and “total” controlover all hiring. This and other interpretations of what the constituents had inmind, when they expressed the idea of internal control, has meant that, in actualpractice, only 20% of all public entities have made an effort to develop effectiveinternal control mechanisms.

The change in fiscal control established by the new Constitution not onlyeliminated the offices of the CGR in each public entity, but also led to a profounddebate over its reason for existence and its mission with respect to all other publicentities. This debate arose from the constitutional mandate to give autonomy toall public entities with respect to regulation and control. Therefore, if each entitysubject to control was to have its own internal control system, what was theexternal control that the CGR had to exercise?

However, and in spite of the enormous effort to change, the CGR wasunable to abandon its character of police-like and supervising entity, after somany decades of this kind of practice. In spite of the fact that it no longer hada large number of employees and that its function was selective, the focus of


its auditory function continued to be that of an extensive search for ill-managedpublic resources in those entities under its scrutiny. This is why the CGR’s effi-ciency continued to be measured in terms of the money that it recovered as aresult of its investigations. This indicator, of course, led to the paradox that theCGR was more efficient insofar as the public entities were more corrupt!

This identity crisis was reflected in the auditing process. According to theestablished norms, the audit had to be integral, that is, it had to include fiscalas well as physical, legal, and management aspects. Likewise, its reports had toinclude economy, efficiency, efficacy, and environmental impact of the resourcesused by the audited entities. In practice, this led to the creation in the CGR ofoperative units for each type of audit and the individual reports were then broughttogether to produce the integral audit report.

On the other hand, the evaluation of the internal control systems of publicoffices was limited to a verification of the existence of an office of internal con-trol and to the fulfillment of the characteristics that, according to the legislation,this office must have.

To resolve this identity crisis, it became necessary to rethink the purpose ofthe auditing function. This meant a new understanding of the meaning of con-trolling the activities performed by another entity. This gave birth to the projectof institutional strengthening of the CGR and, in particular, to the developmentof new auditing practices which were more in line with a new way of under-standing how the CGR could exercise control,

3. SECOND-ORDER AUDIT

The new Colombian Constitution of 1991 decreed that all public entitiesshould design and operate internal control systems that would allow them toregulate themselves. In line with this, it is the CGR’s job to exercise externalcontrol over these entities to ensure that the quality of their self-control systemsis appropriate. The objective of the second-order control is to detect possiblefaults in the internal control mechanisms. The challenge is to develop this typeof control by incorporating a new type of audit: the second-order audit.

This type of audit seeks to encourage an organizational learning process(Argyris et al., 1978; Senge, 1992; Espejo et al., 1996) within the audited entityand within the CGR as well. This learning process is distributed and varies quali-tatively along a continuum that has, at one end, those entities that have not devel-oped internal control systems and, at the other end, entities that have advancedself-control systems. In the first case, the CGR’s audit facilitates the develop-ment of an internal control system in the audited entity. In the second case, theCGR audit is concentrated on the quality control of the existing internal controlmechanisms.

Understanding second-order audit as part of a learning process within an

Reyes162

organization has direct consequences for the structure required to support it. Inparticular, it has four characteristics that contrast with the traditional auditingpractices.

• It facilitates a process of internal reflection within each entity. Therefore,the auditing teams had to include employees of the audited public entity.This manner of integrating the auditing teams facilitates the establish-ment of cooperative relationships based on trust, increases the potentialof the auditing process by using the knowledge and relationships that theemployees of the audited entity have with respect to their own organi-zation, and encourages a learning process within the audited entity (seepaper by Espejo in this issue).

• The primary sources of the information for the audit would be semistruc-tured interviews and discussion workshops.

• The audit reports must be incorporated into the learning circuits of theaudited entity. This means that they will be discussed with the managersof the audited entity, who have decision-making capacity and who canpromote the necessary actions to act upon the recommendations of thesaid reports. These audit reports will not be distant and impersonal doc-uments sent to the audited entity, but an input into a dialogue betweenadministrators of both entities, the auditors and the audited, at the appro-priate structural levels. The objective of this dialogue is to generate com-mitments on the part of the administrators of the audited entity so that anyfaults that might have been detected during the process can be corrected.Such commitments include an agreement about the mechanisms that willbe employed to ensure that the commitments have been totally fulfilled.

• The objective of each audit is not to pinpoint a particular employee, butto observe the relationships between the different actors (internally andexternally) who were participating in the realization of the purpose ofeach entity.

In short, second-order auditing is part of an organizational learning processbetween two entities (auditor and audited), based on the development of trustand cooperative relationships where the desired objective is to produce a com-mitment on the part of the audited entity through dialogues at the appropriateorganizational levels. This process encourages the development of appropriateinternal control mechanisms and can be supported by a decentralized organiza-tional structure that could be strengthened by technological tools that facilitatethe coordination between mixed auditing teams.

In practice, to carry out this type of auditing it was necessary to build appro-priate communication channels that would overcome geographical, institutional,and hierarchical barriers common to the more traditional audits. To achieve thisgoal, three working groups were developed.


3.1. Auditing Group

These were made up by employees of the CGR and by employees of theaudited entity (if possible, belonging to its internal control office). The size ofthese groups was, on the average, eight employees, four from each entity. Therewas an auditing group in each entity, which was responsible for the operationaldevelopment of the audit.

3.2. Technical Group

It was composed of middle-level employees of the CGR and by a projectconsultant. This team had to supply the necessary logistics so that the auditingteam could fulfill its purpose.

3.3. Integration Group

There was an integration group for each audit. These groups were consti-tuted by the coordinator of the auditing group in the CGR, the head of the internalcontrol office in the audited entity, and an external consultant. Each integrationgroup was responsible for the quality control of the work performed by the audit-ing group. Likewise, it solved coordination problems within the auditing processin the audited entity. Three meetings were held during the auditing process, andat these meetings the progress reports (presented in advance by the chairs of theauditing groups) were discussed. There were written minutes of these meetingsto record the agreements that had been reached.

In the development of each audit, the three groups interacted recurrentlyamong themselves and with relevant officers of the auditing and audited entities.This can be best seen in the drawing in Fig. 1.

It is possible to have many mixed teams operating simultaneously in dif-ferent parts of the organization and in different parts of the country. In that case,the integral character of the audit would not be driven by the sum of the parts (ofall the reports that were produced) but by a global vision of the audited entity.

To maintain coherence between all of these audit teams is a complex job thatcould be made easier by using computer technology. The latest developments inthis field point toward computer tools that support team efforts. We are referringhere to CSCW (Computer Support Cooperative Work), which could become thebest support tool for the auditing processes of the CGR.

The general model of second-order audit basically consisted of four stages:perform observations, model the organization, compare the observations withcommon archetypes of structural problems, and generate agreements with theaudited entity. The diagram in Fig. 2 illustrates this generic model of the auditprocess.

Reyes164

Fig. 1. Structural support of second-order audit.

The archetypes mentioned here refer to recurrent structural problems thatwere identified during the development of organizational audits. To express thesestructural archetypes in a coherent way, a graphical language was developed.The initial archetypes were taken as hypotheses that had to be validated for anyparticular organizational audit. In this way, it was possible to have a dynamicset of structural archetypes (Espejo, 1997).

Fig. 2. A general model of the second-order auditing process.


Fig. 3. A detailed model of the second-order auditing process.

In methodological terms each auditing group used a set of diagnostic toolssuch as the declaration of identity, the unfolding of complexity, and the analysisof discretion (Espejo et al., 1996) taken from the Viplan Method.3

Taking into account these diagnostic tools, a more detailed model of theorganizational auditing process was developed. This model can be appreciatedin the diagram in Fig. 3.

A complex training program was designed and developed to prepare theauditing groups in the use of these new auditing practices. An explanation ofthis training program can be found in the paper by Roberto Zarama in this issue.

A more detailed explanation of each one of the activities of these new audit-ing practices is given in the next section.

4. APPLYING SECOND-ORDER AUDIT

The new auditing practices applied to the audited entity strove to detectfaults within the internal control system. To find the foci of lack of control itwas decided to use, mainly, the Viable System Model (VSM) (Beer, 1979, 1981,1985) and the VIPLAN Method (Espejo et al., 1999). This method provides threetools that can be used as diagnostic tools as such and that help to elaborate themodel of the Viable System for a focused organization. These tools are declara-tion of identity, unfolding of complexity, and study of distribution of discretion.

4.1. Declaration of Identity

If you model an organization by means of a black box to represent theprocesses that transform input (resources like, for instance, raw materials) into

3 Viplan is a computer-supported learning system developed by Raul Espejo. It presents a method-ology for applying the VSM.

Reyes166

the goods or services that it provides, it is possible to identify six constitutiveelements within this process. These elements can be thus identified with themnemonic TASCOI.

• Transformation: The process of conversion of raw materials or input intogoods and services. It can be expressed generically as: Given X (input)to produce Y (goods and services) through Z (producing activities) toachieve purpose W.

• Actors: The agents producing the transformation.• Suppliers: The providers of resources required for the transformation.• Clients: Those receiving the products or services of the transformation.• Owners: Those who have a holistic vision of the organization and the

capacity to modify the transformation.• Interveners: Those who, being external agents to the organization, can

directly affect the transformation with their actions.

In this way, the declaration of identity allows us to establish the necessaryproducing activities or primary activities of the organization and its relevantstakeholders through a process of analysis. Likewise, it points toward the maingenerators of complexity for the organization.

4.2. Unfolding of Complexity

Based on the declaration of identity of an organization it is possible to estab-lish its primary activities. A grouping of these activities in different logical levelsallows an initial view of the structure of the system.4 If we draw this structure,we have a tree-like diagram in which each level presents the necessary and logi-cal activities to bring about the purpose of the organization at that level. Eachactivity on each level similarly can be subdivided into primary (sub) activities.This diagram does not represent a hierarchy, but simply relationships or inclu-sion of activities at different logical levels. As we descend our tree from top tobottom these activities will be increasingly disaggregated. This is why this iscalled unfolding of complexity.

Some organizations can group their activities regionally. When this is thecase, the regional organization can be included as part of the unfolding of com-plexity.

The definition of unfolding of complexity should be clear enough to estab-lish that this is a study tool and not a photographic representation of the organi-zation that is being scrutinized. It is simply a way of representing graphically the

4 Here we mean by structure a set of resources and relations that recurrently constitute the primaryactivities of an organization (Espejo et al., 1996).


Fig. 4. Unfolding of complexity of a System of Higher Education.

way to organize the primary activities of this particular organization. This meansthat it is possible to elaborate many unfoldings of complexity for the same orga-nization and that each may focus on different aspects of the relationship withother parts of the organization.

The diagrams in Figures 4 and 5 illustrate an unfolding of complexity fora system of higher education. On the first level (from bottom to top) we findthe activities performed by universities and polytechnic institutes. With respectto formal education, these institutions offer their services in three cycles andaccording to their certified capacity. At the same time, these institutions are over-seen by a regional institution that oversees the correct performance of highereducation in the corresponding region.

4.3. Study of Distribution of Discretion

The purpose of this tool is to discuss the relationship that exists betweenthe primary activities, as identified through the previous tools, and the supportactivities or regulatory functions that are necessary to perform those activitieseffectively. Therefore, it is necessary to identify initially which would be thesesupport activities in a level of disaggregation equivalent to the last logical levelof the unfolding of complexity.

Once these support activities have been identified, the method suggests thata matrix be constructed to analyze their relationship with the primary activities.The columns in the matrix would correspond to the support activities, and thelines to the primary activities.

This matrix allows us to relate each support activity with a primary activity

Reyes

168

Fig. 5. Unfolding of complexity of a System of Higher Education.


at each logical level of the unfolding of complexity. The criterion for analysiswill be the discretion defined as delegation plus acting capacity. Therefore, thepoint that appears at the intersection is interpreted as meaning that this primaryactivity (line) has the ability (discretion) to perform (with its own resource) therespective support activity (column). Another form of interpretation is sayingthat the corresponding support activity (column) is decentralized in the respectiveprimary (line) activity.

In this manner, it is possible to analyze for each support activity the actualand desired level of discretion in each one of the primary activities. This matrixis also a good guideline to decide the degree of centralization or decentralizationwithin an organization because it focuses attention on the level of discretion thateach primary activity must have with respect to the support activities that sustainit. Please note that this also allows us to qualify the debate about centralizationor decentralization in an organization, in a methodical way. For example, basedon the nature of the resources it may be sensible to centralize some supportactivities and to decentralize others, partially or completely.

Whether or not something is sensible depends entirely on the relationshipthat one wishes to have between activities (primary or support) as well as thelevel of technology that one wishes to use. An appropriate use of technology canhelp to decentralize functionally a support activity while keeping the resourcescentralized. For example, if the legal advice activity is a support activity thatrequires highly trained professionals, it is possible to concentrate this resource(trained personnel) in the organization’s central offices at the same time as func-tionally decentralizing it through video conferencing and electronic mail sys-tems, supported by appropriate adjustments to communications.

Figure 6 illustrates an example of a matrix showing a distribution of dis-cretion for a higher education system.

4.4. Diagnosing Control Problems

Apart from the three tools mentioned previously, the auditors who wereusing these new practices also used the VSM as a diagnostic tool. However, itwas very important to emphasize that the system in focus had to be determinedcarefully in order to make good use of the model. For example, it did not makesense to make a viable model of a mnistry since a ministry is an entity thatdefines policies and is not viable in itself. To clarify this, second-order auditdistinguished four types of state, or public entities.

4.4.1. Policy Regulating EntitiesA policy regulating entity is not an autonomous entity. It is an entity whose

reason for existing is justified as long as it forms part of a larger system thatcontains it. Its aggregate value consists in its capacity to define and maintain the

Reyes170

Fig. 6. An example of a discretionality matrix for a higher education system.

identity of the system through the definition of rules and policies. It is charac-teristic of these entities that the necessary actions to execute these policies arecarried out by other entities that also form part of the larger system. This is pre-cisely the case of the ministries in the Colombian State. Each ministry definesand maintains the identity of one sector. For example, the Ministry of Healthdoes this with the health sector and the Ministry of Justice does this with thejustice sector.


From this perspective, a second-order audit of these entities focuses onobserving how effectively they are going through the process of defining andmaintaining the identity of their sector. This implies studying in detail at leastthree aspects: first, that the policies that are being generated are being effec-tively developed by the entities within the sector that have been charged withtheir implementation; second, that the generating process of these policies shouldproduce, as a result, the implementation of the policies, that is, the recognition ofthe executing capacity of the entities within each sector and, at the same time, aresponse to the development program of the government at that time; and third,that these policies are legitimate, that is, that they are supported by a majorityof the citizens who will be affected by their implementation.

With respect to the first aspect, and as part of the auditing, it was impor-tant to observe the monitoring mechanisms that each entity was using to ensurethe correct implementation of the policies by the entities within the sector. Itis possible that this monitoring function was not being executed by the policyregulating entity, but by another entity within the sector (this would be the caseof the superintendence). In ths case, the policy regulating entity should have amechanism to ensure that the monitoring mechanism was being done effectively.In any case, deficiencies in these mechanisms would easily lead to irrelevantpolicies, or, in the case of a ministry, to legislation that would not be obeyed.

With respect to the second aspect, the auditing had to focus on observingthe process through which the policies were agreed upon at the government leveland at the level of the entities that would have to execute them. In this respect,the main function of the policy regulating entity is the design and supervision ofthis mechanism for agreement. If the government entities failed to be involved,the policies would find no support in the legislature and would probably neverbecome national laws. If the entities charged with the implementation of thepolicies failed to be involved, it could be a tacit signal that their capacity toimplement them was not being recognized. Again, these faults could lead toirrelevant policies or, in the case of a ministry, to legislation that could not beput into practice.

With respect to the third aspect, the audit focused on observing the mech-anisms that the policy regulating entity had to measure the degree to which thepolicies were accepted or rejected by the public in general. Faults in, or lackof these mechanisms, could mean that policies or laws were being generatedwithout public support.

4.4.2. Function Regulating EntitiesA function regulating entity is not an autonomous entity. As with the policy

regulating entities, its reason for being has meaning insofar as it forms part ofa larger system that contains it. Its aggregate value stems from the fact thatit supports the primary activities executed by the system. For this reason, the

Reyes172

organizational audit of this entity focused on observing the effectiveness of itssupport function vis-a-vis the purpose of the system that contained it.

These support functions could be classified as control, monitoring, or coor-dination functions. Any regulating entity could be executing one or more of thesefunctions. For example, in Colombia a superintendence executes primarily mon-itoring functions over other entities of the sector to which it belongs, while theNational Planning Office (NPO) executes all three functions at national level.To observe its effectiveness it became necessary to understand and to analyzethe communication mechanisms through which the NPO communicated with theentities that it supported in its sector.

4.4.3. Primary EntitiesPrimary entities are entities that can be studied with the criteria of viability

of the Viable System Model (Beer, 1979, 1981, 1985).

4.4.4. Mixed EntitiesMixed entities are those that, at the same time, execute both primary and

support activities. There may exist dysfunctional mixed entities. These are enti-ties that, based on their identity, should be regulating policies or functions withintheir sector but that find themselves executing primary activities within their sec-tor. Likewise, there can be entities that are legitimately executing both types ofactivities.

The second-order audit initially had to determine if the entity was functionalor dysfunctional according to the previous criteria. Once this question had beenanswered the criteria for policy regulating, function regulating, or primary enti-ties could be applicable.

By using these diagnostic tools, it is possible to identify some struc-tural problems in the organizations. Second-order auditing practices identifyfour types of structural archetypal problems: Identity archetypes, Cohesionarchetypes, Performance archetypes, and Citizenship archetypes (Espejo, 1997).These archetypes are explained elsewhere (see paper by Espejo et al., in thisissue).

4.5. Generating Agreements

We mentioned before that a second-order audit report is not an end in itselfbut simply a means to encourage improvement agreements between the entityand the CGR. The parts of this agreement might be of three types.

• Operational: When there were changes that the audited entity could effecton its own, in no more than 6 months, without modifying its internalregulations.

• Strategic: When the required changes involved a change in the internal


regulations of the entity, but not in the legislation. These changes wouldtake between 6 months and 1 year, and they were changes in strategy toachieve a purpose.

• Normative: When it was necessary to effect changes that went beyondthe competencies of the top managers of the entity and a legal or con-stitutional reform was necessary. In this case, the CGR would open upinstances to discuss these changes with the appropriate agents to see howthey could be effected.

Finally, each part of the agreement obtained from the audited entity is away to monitor the development of all other parts of the general agreement.

5. AN APPLICATION OF SECOND-ORDER AUDITING5

In this section we present a specific application of the second-order auditingpractices described in this paper. This auditing was performed at Colciencias, aColombian government public entity whose main scope is the development ofscience and technology in the country. To show the results of this auditing wehave decided to follow the sequence of the methodological tools described in thisarticle, even though the process, in practice, was not linear but iterative. Becauseof the limitations of space, we have chosen to show only the main results of theauditing; a more detailed explanation of the case is given in CGR (1998c).

5.1. Declaration of Identity

Colciencias was originally created at the end of the 50s as an entity ascribedto the Ministry of Education. To build its identity, Colciencias has been working,in the short term, to create the proper conditions to turn science and technologyinto an economic and sociocultural factor for the development of the country.In that sense, the goals pursued by Colciencias are promoting the integration ofscience and technology into the national life, increasing the competitiveness ofthe county’s productive sector in the context of an open economy, and improvingthe population’s welfare and quality of life.

For this purpose, it actually formulates and coordinates the development ofscience and technology policies and strategies; promotes scientific and techno-logic development; promotes the spread, popularization, and social appropriationof knowledge; and observes, quantifies, and follows up the country’s develop-ment of science and technology.

The stakeholders in these processes are Colciencias’ staff, individual

5 This auditing was carried out by an audit team coordinated at Colciencias by Juan Jose PlataCaviedes and at the CGR by Ruth Maribel Cuadrado Torres.

Reyes174

researchers, national program councils, research centers, universities, some gov-ernmental entities, international cooperation organizations, and nongovernmentalorganizations. In 1997, the budget handled by Colciencias was US $33,700,000.

5.2. Unfolding of Complexity

According to the classification of public entities of second-order auditing,Colciencias is a policy and function regulating entity. For this reason, to under-stand its mission it is necessary to observe its activities as part of the NationalSystem of Science and Technology (NSST).

In Colciencias’ complexity unfolding, five mission-based activities areacknowledged: (a) policy and strategy formulation, (b) promotion of scientificand technological development, (c) spreading and appropriation of social knowl-edge, (d) monitoring of scientific development, and (e) acquisition of resources.In Fig. 7, three levels of this unfolding are shown.

The activity of promoting science and technology consumes 82.8% of theentity’s total budget, 9.4% of the budget is devoted to spreading activities, and5.3% is directed to policy formulation. Finally, the activity of monitoring sci-entific development represents only 2.5% of the total budget.6 From this, it isapparent that, ironically, Colciencias does not devote resources to the activity ofobtaining them (0.3% of its total budget). For this reason this activity appearsin the unfolding of complexity diagram as a dashed line.

On the other hand, Colciencias is the NSST technical Secretary. Figure 8shows an unfolding of complexity for the NSST. The primary activities of thissystem are to formulate science and technology policies in the country, to consol-idate the research capacity of the Colombian scientific community, to institution-alize science and technology in the country, to regionalize and internationalizescience and technology, and to increase the social appropriation of knowledge.

5.3. Diagnostic and Agreements

Once the second-order auditing team performed and presented a final di-agnostic to Colciencias’ board of directors, its members agreed to concentrateColciencias future actions on four aspects:

• improving the mechanisms for policy formulation in science and technol-ogy, its promotion, implementation, and monitoring,

• developing new mechanisms for acquiring resources in order to imple-ment such policies;

6 These figures correspond to an average calculated over 3 years from 1995 to 1997. Data are takenfrom the Financial Division of Colciencias.

Second-Order A

uditing Practices

175Fig. 7. Unfolding of complexity of a System of Higher Education.

Reyes

176

Fig. 8. National System of Science and Technology: unfolding of complexity.


• implementing mechanisms for building a local capacity in science andtechnology; and

• improving the management of science and technology programs.

5.3.1. Policy FormulationColciencias is accomplishing a dual purpose. On the one hand, it is main-

taining its identity as a governmental agency supporting the development of anational capacity in science and technology, along with its incorporation into thenational culture. On the other, it is acting as a technical secretariat of the NSST,that is, as a regulating entity of the National System of Science and Technology.In this context, Colciencias’s raison d’etre, as well as its institutional viability,are related to the vitality and growth of the whole NSST; the entity’s social val-idation is given by the national capacity to carry out scientific and technologicaldevelopments which will be incorporated into the productive sector and society.These two purposes suggest that the specific development of research projectsshould be in the hands of particular research groups and institutions, leavingColciencias out of these kinds of operational activities.

This point also suggests that Colciencias should assume a stronger coordi-nation activity in order to give coherence to the myriad of projects that will becarried out by the members of the NSST. In addition, Colciencias has to developand implement a continuous observation system that allows the NSST to mea-sure the impact of the policies generated. These monitoring mechanisms have torecognize the autonomy of the research groups and institutions in carrying outtheir projects.

Finally, the mechanism of policy making in science and technology has tobe inserted into both the mechanisms for formulating the national, regional, andlocal development programs of the country and the formulation of the nationaleducational policies.

5.3.2. ResourcesAccording to Colciencias’ espoused identity, one of its mission-based activ-

ities is the gathering of resources. However, there is very little capacity in Col-ciencias to carry out this activity. Therefore, it is necessary to increase this capac-ity and to diversify the sources of these resources. One of the mechanisms thathas been very successful in collecting funds is tax exemptions of donations.However, it is important to develop a more aggressive campaign to promote themechanism in companies of all sectors.

5.3.3. Generating a Local Capacity for Science and TechnologyThe development of a local capacity for science and technology has as a

prerequisite the political and administrative decentralization of the country. Inrecent years Colombia has been moving in that direction. However, the localcoordination mechanisms that allow the incorporation of science and technol-

Reyes178

ogy development into regions are still very weak. This has left open a challeng-ing question: Which new communication and coordination mechanisms couldbe promoted to align regional dynamics with the national programs in scienceand technology?

The activities carried out to address the demand for supporting science andtechnology are oriented to give an answer at a global level, according to criteriaof quality, relevance, and efficiency. Resource assignment based on merits isgenerally accepted, but in a natural way it promotes a concentration of resources,precisely where they abound, increasing the gap among regions. For this reasonit is necessary to think about new strategies involving criteria and mechanismsthat help to overcome the actual regional unbalances.

A more flexible management with a stronger setting in local and regionaldynamics would allow the construction of local and regional identities for sci-ence and technology and would motivate a better dynamic and capacity fornational and international competition.

5.3.4. Administrative Procedures and Science and Technology ManagementThe demand for research funding has increased in recent years. Taking

only 2 years, it is found that from 364 requests in 1995, there were 816 in1996, that is, an increase of 124%. Although this growth rate has tended todecrease, overall the demand is still increasing and the human resources support-ing this activity in Colciencias have remained stable. This situation has raised adilemma between increasing Colciencias’ personnel and designing mechanismsto increase its capacity to work through others. Taking into account the weakeconomic situation of the country, it is clear that the second choice is the onethat has to be followed.

A consequence of this unbalanced situation is reflected in the average timethat a research proposal has to take in order to be approved by Colciencias.Despite the efforts to improve it, today it is still an average of 4 months, thereforeit is necessary to redesign this administrative process so that the research groupsor institutions themselves are involved in filtering up the proposals.

Taking into account that Colciencias is simultaneously a policy and functionregulating entity, some of its support activities come into conflict because theyare accountable to different systems. This is the case for the activities carriedout by the systems division. In fact, this division is in charge of four main func-tions: (a) evaluating and monitoring research proposals regarding informationtechnology—in this respect the division is carrying out a mission-based activityof Colciencias; (b) giving technological support to the management of researchproposals—in this respect the division is acting as a support activity of Colcien-cias; (c) developing the information system for the National System of Scienceand Technology—in this respect the division is carrying out a support activityfor the NSST; and (d) participating in the definition of policies for the develop-


ment of science and technology in the country—in this respect the division iscarrying out a primary activity of the NSST.

All of these functions, however, have been assigned to the systems divisionin recent years without making a structural reform of Colciencias. The divisionstill only has the personnel and the structure of a typical supporting division ofa public entity.

6. A FINAL COMMENT

From 1996 to 1998, 52 second-order audits were carried out in the samenumber of national public institutions. During the course of this process, second-order auditing practices were developed to the final state shown in this paper.At the end of the process a set of agreements was developed for each one of theaudited institutions. Some of the final reports showing the diagnostics and theagreements were published in a collection of books edited by the ContralorıaGeneral de la Republica (CGR, 1998a).

With respect to second-order audits as a learning process, we found that theexisting structure of the CGR did not support the integration of different audits.The fact that the CGR’s structure was based on sectors, and each of them wasstructured in units specialized in a certain type of control (financial, legal, physi-cal, managerial, etc.), led to a situation where different audit groups were formedto audit different aspects of the same entity. In some cases, this not only gen-erated an unnecessary duplicity of efforts (when requesting similar informationfor different purposes), but also made it difficult for any employee of the CGRto relate the findings of each audit group.

To expedite the process of constructing an integral vision of each entity,the role of the “responsible auditor” was created. This person came from a spe-cific section and was responsible for a group of entities, regardless of the typeof audit. In this way, the responsible auditor would form the audit groups thathe/ she thought appropriate to execute one or many audits in the entities underhis/ her responsibility. In this manner, this person could develop a more generalunderstanding of the problems, strengths, and weaknesses of the audited entities.

Similarly, the responsible auditors of a group of entities belonging to a sec-tor would meet periodically to conform “a sector’s integration group.” Thesegroups would have the responsibility of planning the audits to be executed inthe following year as well as negotiating the resources that all of them wouldshare because they were all part of the same sector. In turn, this group wouldhave the possibility of developing a shared understanding of the problems of thesector formed by these entities, as it shared audit experiences in each one of theentities.

In this way it was possible not only to meet the purpose of the organiza-tional audit of developing a process of organizational learning between the CGR

Reyes180

and the audited entities, but also to develop a learning process inside the CGR toconstruct a better understanding of the different sectors that make up the Colom-bian State. This last result was named the State of the State (CGR, 1998b), andit was published in a book by the CGR.

REFERENCES

Argirys, C., and Schon, D. (1978). Organizational Learning: A Theory of Action Perspective,Addison–Wesley, Reading, M.A.

Beer, S. (1959). Cybernetics and Management, English University Press, London.Beer, S. (1979). The Heart of the Enterprise, John Wiley & Sons, Chichester.Beer, S. (1981). Brain of the Firm, John Wiley & Sons, Chichester.Beer, S. (1985). Diagnosing the System for Organization, John Wiley & Sons, Chichester.Checkland, P. (1981). Systems Thinking: Systems Practice, John Wiley & Sons, Chichester.Espejo, R. (1997). Structural Archetypes, Syncho Ltd., internal document.Espejo, R., and Harnden, R. (1989). The Viable System Model: Interpretations and Applications of

Stafford Beer’s VSM, John Wiley and Sons, Chichester.Espejo, R., Schumman, W., Schwaninger, M., and Bilello, U. (1996). Organizational Transformation

and Learning: A Cybernetic Approach to Management, John Wiley and Sons, Chichester.CGR (1998a). La Construccion de un nuevo discurso de control, Contralorıa General de la Republica

de Colombia, Bogota.CGR (1998b). El Estado del Estado, Contralorıa General de la Republica de Colombia, Bogota.CGR (1998c). Colciencias: Ciencia con conciencia, Contralorıa General de la Republica de Colom-

bia, Bogota.Senge, P. (1990). The Fifth Discipline, Doubleday, New York.Von Foerster, H. (1996). Cybernetics of Cybernetics, Future Systems, Minneapolis.

second-order auditing practices

Documents