knowing and understandingunstats.un.org/unsd/dnss/docs-nqaf/nqaf english...the two sides of a coin,...

Knowing and Understanding

BPS “Statistical Quality Assurance Framework” (Stat – QAF)

National Bureaucratic Reformation Program

(2010 – 2014)

Project Management Unit Statistical Capacity Building

ChangE & Reform for Developing of StAtisticS (CERDAS)

Prepared by. Wiwiek Arumwaty

iii

Statistical Quality Assurance Framework (Stat‐QAF) 2011

Introduction

Based on the Law No. 16 of 1997 and Peraturan Pemerintah (Government Regulation) No. 86 of 2007, Badan Pusat Statistik (BPS‐Statistics Indonesia) has strengthened its status as an government institution which has a mandate to undertake all statistical activities of the National Statistics System (NSS) of Indonesia. A reliable statistical information system will include not only the quantity or variety statistical data produced but also the quality. Nowadays, the National Statistical Offices (NSOs) will have to focus more on data quality to meet the increasing demand. Statistical data which are not in line with the needs, inaccurate, incoherent, out of date, difficult to access, and difficult to understand would reduce the credibility of BPS as the main source of objective and reliable of statistical data. Through the Bureaucratic Reform Program (BRP), which has been launched since 2010, BPS will continue improving its data quality. The Quality Assurance Framework (QAF) is a framework designed to guarantee the quality of data produced. The existence of QAF in BPS is one of the goals to be achieved in the BPS‐BRP. Changes created by the overall government BRP are intended to bring all government institutions/agencies well‐managed, especially in providing public services. As data quality is a major pillar in the BPS BRP, producing various types of high quality statistical data has become a challenge for BPS. At the same time, this challenge is an opportunity for BPS to show its capacity as an institution which having a mandate to implement all statistical activities in Indonesia. In accordance which stated previously, the various statistical data which have already been produced should be followed by an increase in their quality.

iv


The Statistical QAF (Stat–QAF) as one of the statistical reform pillars is more than a series of activities to enhance the data quality. It supports the “BPS Strategic Plan” due to data quality improvement. More specifically, the Stat‐QAF itself is a complete description on how to manage data quality, especially in terms of identifying problems and solutions, as part of monitoring and evaluating implementation. As data quality has become a priority, a mechanism to monitor the implementation of quality processes will be developed. Furthermore, the Stat‐QAF will used as a framework designed to produce a standard service level in statistical activities. This role of Stat‐QAF, which has been practiced in several countries, will be fully implemented in Indonesia soon. Additionally, the Stat‐QAF could be used to determine the hierarchy of management system and related statistical procedures. In the future, the Stat‐QAF will be used to compare NSOs performances of various countries in conducting statistical activities. The implementation of the BPS Stat‐QAF is one of ultimate goals in supporting the BPS‐BRP In response to the challenge for data quality, the year of 2011 has been set as the year of implementation of the activities to improve the quality of BPS statistical data (milestone year). With the spirit of mutual concern, togetherness, and firm desire to change, BPS now has to move from the conventional to the modern era in carrying out its statistical activities. Even though there will be a new unit which will specifically deal with managing the data quality, the implementation of data quality improvement will be the responsibility of all of BPS staffs. This booklet is prepared to describe the link between BPS vision, mission, core values, and codes of practice to the BPS Strategic

v


Plan in producing statistical data. It also shows the relation between both the quantity and the quality of statistical data, like the two sides of a coin, which having equal probability to be presented and analyzed further. I am sure that we are all aware of the importance of data quality and optimistic that we will do our best to continuously improve the quality of statistical data produce in our own respective area of responsibility.

Jakarta, January 2011

Chief Statistician, BPS‐Statistics Indonesia

Dr. Rusman Heriawan

vi


Statistical Regulation No. 39 of 2010 of the Chief Statistician.

Declared in Jakarta, on July 14, 2010

Vision: "The agent of trustworthy statistical data for all" Mission: ‐ To strengthen the constitutional and operational

foundations of the statistical institutions in conducting efficient and effective statistical activities.

‐ To create competent and professional statisticians, supported by the latest information technology for statistical advancement in Indonesia.

‐ To increase the implementation of classification standards, concepts and definitions, measurement, and statistics code of practice, as a universal statistical activities.

‐ To increase the quality of statistical information services for all.

‐ To increase coordination, integration and synchronization of statistical activities, conducted by public and private institutions, within an effective and efficient framework of the National Statistical System (NSS).

Core Values:

Core values are guiding principles that all individuals (in an organization) believe and practice in their work environment and form dominant characteristics of the organization. For BPS, core values are a set of principles that have enriched the belief and represent code of ethic of all BPS staffs in carrying out their duties. The BPS's core values are "Professional, Integrity, and Trusteeship."

vii


a. Professionalism Professionalism is expected to be possessed by all BPS staffs in carrying out their tasks. It includes: ‐ Competent: have the suitable expertise in the task performed;

‐ Effective: have the optimum impact; ‐ Efficient: do the tasks productively using the allocated resources;

‐ Innovative: ability to perform the tasks using better ways.

‐ Systematic: well‐organized, do the job in proper procedure and mechanism.

b. Integrity: Integrity is an attitude and work behavior that must be possessed by all BPS staffs. It includes: ‐ Dedication: has a strong loyalty to the profession and institution.

‐ Discipline: carry out the tasks in accordance to the rules and procedures.

‐ Consistent: word and action go together. ‐ Transparent: respect the ideas, suggestions, opinions, feedbacks, and criticisms from co‐workers.

‐ Accountable: responsible for the works performed.

c. Trustworthy (Amanah): Trustworthy is an attitude that must be possessed by all BPS staffs. It consists of: ‐ Reliable: carry out tasks in accordance with the given guidance and produce the quality outputs timely.

‐ Honest: perform all works in good manner and do not deviate from the principles of morality;

viii


‐ Sincere: do the tasks unconditionally, avoid conflicts of interest (personal and group), and dedicate all tasks for the sake of the nation.

‐ Fair: be just and respect the rights of others. Code of Practice The following list presents the BPS Code of Practice.

a. Professional Independence: from policy, regulatory, or administrative ministries and agencies, and from political and other external interference in producing and disseminating statistical data.

b. Confidentiality: the privacy of data providers and the confidentiality of the information they provide are guaranteed.

c. Impartiality and Accessibility: statistics are compiled on a non‐partisan basis and can be accessed and utilized by all parties.

d. Appropriate Statistical Procedures: statistics produced by applying the norms, standards, procedures, and criteria commonly applied in all statistical activities.

e. Uses and Interpretation: statistics are presented in a form that facilitates correct uses and proper interpretation.

f. Objectivity: statistics are compiled on an objective basis reflecting the real situation.

g. Relevance: statistics must meet the needs of users. h. Accuracy: statistics must accurately portray reality. i. Timeliness and Current: statistics must be available in a

timely manner and up to date. j. Consistency: statistics which are consistent over time

and space.

ix


k. Accessibility and Affordability: statistics should be accessible on an impartial basis and affordable.

l. Clarity: statistics should be presented in a clear and understandable form.

m. Non‐Excessive Burden on Respondents: statistics should be produced without excessive burden on respondents.

x


List of Content: ‐ Preface ‐ BPS’ Regulatory Framework (Vision, Mission, Core Values and Code of Practice)

1. A Glance of BPS official Statistics 1.1. Background 1.2. Scope of Statistical Activities 1.3. Statistical Data as a Core Business 1.4. The Role of Monitoring and Evaluation

2. Quality‐Assurance of Statistical Data 2.1. Reason on Quality Improvement (Concepts, Role and

Benefits) 2.2. Quality Assurance as a Statistical Process 2.3. Quality Assurance as a Framework (QAF) 2.4. Quality Elements versus Code of Practices in Statistics

3. Performance Indicator, Quality Unit, and Environmental Influence 3.1 Key Performances Indicators 3.2 Statistical Dashboard 3.3 The Quality Assurance Unit 3.4 Environmental Influence on Quality Improvement

Program 4. BPS Internal Strategy and Policy

4.1 Emphasizing more on Quality Aspects. 4.2 Managing Quality on National Statistical Systems 4.3 Setting SOP and Implementing Quick wins 4.4 Mapping of BPS Statistical Activities

xi


4.5 Constructing BPS Stat‐QAF and Its Guidelines

1


1 A Glance of BPS Official Statistics

1.1 Background Global demands on the quality of statistics issue, especially those produced by the NSO (National Statistics Office), is prominent. The United Nations Declaration by United Nations Statistics Commission in 1994 introduced a concept known as the "UN Fundamental Principles of Official Statistics" which are intended for NSO for all countries for building, maintaining, and developing various statistical products. Quality guarantees security and reassures various parties in the implementation and use of statistical data. Several countries have undertaken preliminary testing for preparing the implementation of Quality Assurance Framework including the European Union, Canada, Australia, Finland, China, Japan, and Korea.

In response to those demands BPS which having mandate in organizing Indonesia official statistics believe that quality of statistical activities has become an important pillar in the bureaucratic reform process. Moreover, the need for good data on social as well as economic has increasing, that ideally it should be followed by the improving in quality. Quality should become an integral part of the daily statistical activities in all working units. There has to be a real objective, and targets to be achieved. In the first stage of the reforms, program that has been carried out are determining Quick Win and reviewing Standard Operational Procedure (SOP). This is the first step in monitoring the process of awakening statistical data quality in BPS. To see how the quality of statistical data conducted among

2


all working units in BPS, there is a need of detailed mapping of statistical activities, including the process of identifying the data produced. Statistical map is a form of statistical constellation defining in the process of identifying, knowing/ learning, and understanding how to compile diverse BPS statistical data periodically, as outlined in the various development programs. Statistical maps will include structure of statistics, its character or specifications, as well as statistical studies of the integration process. All of these individual statistical records will serve as a basis on making a problem diagnosis occurred, including the quality. Based on the availability of the basic information in BPS’ metadata, a method will be developed to preparing a map of statistical activity, respectively. This information will be functioned as an instrument in monitoring and evaluation, how the goals and targets are achieved. The main target of statistical activities reform should not only consider the quantity but also quality. Quality assure the worthiness of statistical data starting from data input (data collection), data processing, final result (output) and the outcomes. It is important to remember that good information is derived from data that is processed by using the concepts and scientific procedures correctly and appropriately. Encouragement on improving the various quantity statistical data as well quality must be benefit to public. In other word, it is achieved if the data are built in accordance with user needs and assured the quality. As for the changes, it is necessary to use measuring tool called Key Performance Indicators (KPI). The instrument is a universal tool which could be used to present the comparison of statistical data quality among the countries. With the establishment of good

3


BPS statistical performance, is expected to achieve the ISO (International Standards Organization/ISO) which describing fundamental quality management systems.

1.2 Scope of BPS Statistical Activities

a. Vision, Mission and Strategy Vision is an expectation or an ideal goal of BPS to be achieved in carrying out the functions and mission of the statistics development programs. The vision outlined in the form of more concrete follow‐up, especially on the third objective of BPS in "Improving the application of classification standards, concepts and definitions, measurement and statistical code of practice which is universal used broadly. The strategy in the future is not only evaluating the statistical data for quality assurance but also looking at quality as a need of public demand.

b. Strategic Plan (RENSTRA)

A very close relation between vision, mission, core values and codes of practice foreshadowed statistical development process, at both national and local level (provincial and district/city). The link is expressed in a more real pace to BPS as set forth in the Strategic Plan (Strategic Plan) of BPS. Even RPJM (Medium Term National Development Plan) year 2010‐2014 has put strengthening data systems and statistical information as part of national development management support. In a way, more than half of a century, BPS with all the professional supporting personnel has produced various statistical data each with its own advantages and weaknesses.

Major problems facing BPS in the collection of statistical

4


data is the awareness of respondents to provide the correct information. On the other hand, the problems can also be caused by errors of both internally and externally. Internal problems that need attention include BPS management role is not optimum in monitoring and evaluating the quality of statistical data. In developing and maintaining statistics, BPS not supposed to think only about the quantity but also the quality since the quality has increasingly become the needs of data users.

Improved data quality becomes one of strategic objectives to be achieved by BPS. Within the next 5 (five) years, BPS will seek reforms to the development of comprehensiveness in statistics. The improvement will be carried out in different lines of activities such as: improving the quality of Human Resources (HR), and accelerating the needs of the Information and Communication Technology (ICT). In order that the National Statistics System (NSS) becomes reliable, effective and efficient, statistical development program designed to increase the availability of reliable statistical information and the "quality" of its product as well.

c. Statistics Development Program

As an effort for the success of national development in the field of statistics, BPS has arranged a strategy in the field of quality improvement and data diversity. The quality has become a priority in addition to increasing in statistical variability that is relatively well organized. Data quality dimensions consist of elements of accuracy, relevant, timely manner, easily accessible, coherent, and interpretability. Even with the very specific BPS strategic objectives is "the availability of comprehensive data,

5


accurate, and timely manner throughout the national development, also to improve kind of services for the data users. Through the program STATCAP‐CERDAS (Statistical Capacity Building ‐ ChangE and Reform for Development of StAtisticS), proclaimed in a 4 (four) pillars of BPS‐BRP used as a reference to BPS statistical activities during the year 2010‐2014 which includes the components of i) Improving data quality ii) builder and improving Human Resources (HR) quality iii) Strengthening role of Information and Communication Technology (ICT) and its infrastructure and iv) Strengthening role of institution in relating to data sources and data community. The four pillars will related to each other and may have a causal relationship, either directly or indirectly.

d. Products (Outputs) of statistics

BPS is the highest authority of government agency which having a mandate to carry out activities in all official statistics; thus fulfilling the needs of quality of the data produced becomes a necessity. Statistical data produced must meet the needs of potential users. Meanwhile, data collected should not also be a burden to the respondent. The statistical process of data collecting, processing up to the data producing would have to put more attention to quality at every stage of activities.

Historically, official statistics of BPS usually refers to the need of government, although there are some cases where some statistical activities conducted due to fulfill special requests. In addition, the official statistical information produced by other government institutions such as the

6


Ministry of Finance, Central Bank (Bank of Indonesia), Ministry of Agriculture, Ministry of Mines, Ministry of Health, Ministry of Education, Ministry of Social Affairs and others, in accordance with their interests and capacities, respectively. The statistical data produced by a variety of BPS’ subject matters area have similar role, although constructed by using with various measurement approaches. Meanwhile, the statistical data generated by other government agencies also has a strategic role in providing the Indonesia official statistics. Therefore, the development of statistics should be able to integrate and harmonize the various statistical products into a system of official statistics as the National Statistical System (NSS). It is necessary to be quote that official statistical data produced should be in an assured condition.

"Official statistics is a statistical system, comprised of essential and high‐quality statistics that are produced regularly or at sufficiently frequent intervals, and are nationally representative. The available statistics must in practice Reflect the user's needs and priorities, and adhere to international standards and recommendations on the compilation of statistics” (adapted from Statistics of Finland)

1.3 Statistical Data as a BPS Core Business

According to the government act, the statistical data produced by BPS namely basic data, practically very diverse. Various type of data produced both economic as well as social statistics to meet the demands of society, particularly government. The types of statistical activities conducted in general covered of censuses, surveys, administrative data

7


collections, and compilation (such as National Accounts and Statistical Analysis). The main focus in the agenda of statistical reform is improving the quality of “core statistical data” such as Indonesia System of National Accounts, Agricultural (Food crop and estate crops), Manufacturing, Foreign Trade (Export‐Import), Prices and inflation, House hold survey (SUSENAS), and labor force survey (SAKERNAS) which mostly concentrated on economic aspects. While other statistical data prepared to support the needs of other statistical areas (such as National Account Statistics). The methodology and statistical analysis are also given priority to be evaluated and improved. "Code of practice in statistics" is a commitment of all BPS staffs in carrying out statistical activities to produce a variety of data output, based on statistical knowledge and experience. So far, the development of statistical data usually still focused on quantity aspects, but in the future the quality will also take precedence, to keep the output balance. On the other hand, improvement of statistical data quality due to increasing of user demands, both internal and external country. The statistical data quality is any of data that has minimal errors so that the information delivered will not become misleading to users. Not all of data referred as statistical‐data since some of them are presented in aggregate statistical format by using certain statistical measurements such as total, average, ratio, proportion/percentage, and deviation. Some of them are followed by the visual presentation in the form of tables, charts and graphs. Statistical Data, as underlined in BPS Vision and Mission, is a statistical output generated by the

8


BPS staffs based on the principles fundamental of statistical organization, core values and code of practice applies.

Within the scheme of Bureaucratic Reform Program (BPS‐BRP), not all BPS statistical data produced will be elaborated and developed. It will be prioritized on the 7 kinds of output as mentioned above. From all of them particularly the compilation of System of National Accounts of Indonesia (SNNI) needs to get more attention as a strategic data that have not all been utilized fully. System of National Accounts (SNA) is a comprehensive instrument which consists of various statistics that can be used to measure the economic performance of a country. It has a function on selecting, collecting, compiling and presenting various macroeconomic statistics, which are presented in aggregate form. Vary of other statistics produced by BPS other than those described above also has an important role, for the potential users. The statistical data will also be monitored and evaluated either in results and processes during the data collection process. Although in the short term the data‐quality monitoring is only aimed at the certain statistics but in the long term priority of all data‐quality statistics should also be monitored, periodically. Ideally, in the preparation of statistical data should consider the balance between quantity and quality for which good data can be generated. Furthermore, to improve efficiency, the statistics that is not needed, will be reconsidered its continuation.

9


1.4 Role of Monitoring and Evaluation

Principally, Monitoring and Evaluation (MONEV) is a part of a research activity whose function is to monitor and evaluate any activities in order to get the feedback of the running process. This monitoring is part of the management and administrative processes to measure the level of accountability, and efficiency of the statistical activities. From this research it will be understood the process and the effectiveness of the implementation of data quality management on various kinds and levels of statistical data produced by BPS. The development of statistical data should not solely focus on quantity but also need to consider the quality aspects, so that there is a proper balance in the delivery of statistical information. For the purpose of monitoring and maintaining the quality assurance program, a special working unit urged/need to be formed with the support of statistical professional staffs. They are responsible to develop and to monitor all level of statistical activities. Instruments for monitoring will be designed in accordance with the needs and the results which are reported regularly. Availability of quality monitoring devices, providing profound implications on the availability of statistical data, which automatically includes the structure and status of data generated both in terms of how detailed the data, and its coverage. Through these devices, the public could know the quality management framework that is applied and developed by the BPS. Practically, although having a very important role MONEV is often ignored, because there is lack of awareness of its benefits. MONEV is rarely used, but as a matter of fact it is a

10


very useful instrument to align planning and execution. It has appropriate methods to assess the process and the result. Awareness of all staffs is needed on the implementing statistical activities to build and maintain the appropriate procedures will keep the better in the administrative systems. The process of documenting, maintaining and evaluating must be committed of that unit and need to be carried out continuously.

From the process it will be known: a. How the quality of statistical data can be monitored on a

regular basis b. Understanding of the importance of quality and to

intervene in case of any uncommon thing happened, c. Impact of quality on statistical data generated, d. Pattern of data quality related to other data quality.

Monitoring and evaluating the quality of statistical data produced which stable conducted, continuously, consistent and in orderly manner, will lead BPS maintaining statistical quality culture in performing the task in the statistical data development.

11


2 Statistical Data Quality Assurance

2.1 Understanding Some Aspects of Quality "Quality" has a very broad meaning, which is generally defined as levels of accuracy or the merits of an object / condition. It is realized that during these various programs of statistical activities carried out are still prioritizing the quantity, and the quality of statistical data sometimes is forgotten. It should be noted that there are some activities that also taken into account as part of the quality improvement. Quality is an assuring of a statistics product, which would keep protection and certainty to the users. On the other hand, because of the quality aspects cannot be separated from the data input, this is also the guarantee for the parties’ data providers (respondents). The quality has an important role in building confidence in the statistical products produced. A quality simply can be defined as a condition which is free from error. Or in other words the quality is also defined as quality content, which explains about the merits of a product/condition, with the aim is to give the signal for a better condition. Similarly, with the issue of statistical data quality as is an integral part the statistical output produced by BPS. In addition to pursuing the achievement of quantity, quality at various levels of statistical process needs also to be taken into account. To get a good data, a balance condition between quantity and quality is needed to be developed further. Quality can be monitored through a chain of statistical activities that are relatively long way and interrelated. It is started from the process of planning, collecting, processing,

12


presenting, analyzing and even in the dissemination level. Therefore it is necessary to develop an instrument that can be used to monitor how the quality can be formed by inter related of various statistical data, over time and across regions. Running a huge of statistical activities in a relatively long period with a high pressure target sometimes causing the meaning of quality is forgotten.

Development of data‐quality statistics is not only being done on each Subject Matter area, but also compiled at the institutional level (BPS). Thus, the construction of quality cannot built separately, it has to be supported by other factors like professional staffs, Infrastructures, Information and Communication Technology (ICT), budget constraint, Regulations, and Management. Although it is not in plenary, the relationship between working unit in producing the statistics, gives an indication that the process under coordination and integration of statistical activities in the environment BPS is being implemented.

2.2 Assuring of Quality The issue of quality assurance framework although seems as something new but implicitly it is an integral part of the quantity, as described previously. The quantity and quality are two interrelated elements. In practice the quality cannot be identified without the existence of quantity data. For BPS, in every implementation of statistical activities also need to be considered aspects of quality, though sometimes it is overlooked quality monitoring. Therefore, in the future development of statistical quality should become a priority and in line with the development of quantity, instead of having more meaningful data according to the user demand.

13


a. Assuring of Input Data/information

The input data is assured in data acquisition that includes the source of the data/information obtained. That is whether it is done through censuses or surveys (primary data), data derived from other parties (secondary data), or raw/processed data from existing administrative records. The willingness of others to provide information in an honest and truthful is the key to success in maintaining the quality. It has to be considered that the excessiveness of the questions could be a burden to respondents also will affect the quality of the data obtained.

b. Assuring for Statistical Process

Is guaranteed in the statistical process from planning to the results obtained, the stages include planning, preparation of statistical design, data collection, data processing, presentation and data analysis. Specifically in the data processing process includes the classification / reclassification, data entry (validation, and verification), justification and tabulation. The process in transforming the input to be data output requires expertise and precision, which also influence the quality of statistical data produced.

c. Assuring of Quality Product (Output)

The quality of input data will affect the quality of output, since the input change becomes output through the process of transformation. Establishing the quality of statistical data is a reflection of responsibility of the BPS management, due to statistical data construction starting from identification (how the information/data obtained), until data presentation (output). At the end, level of

14


quality assurance is highly depending on the statistical processes which assure as the statistical activities chain. Through these three stages of assurance, the quality of statistical data generated will be well maintained because it is always controlled at every stage of process.

2.3 Statistical Quality Assurance Framework (Stat‐QAF)

Statistical‐QAF is a working instrument that was previously introduced by the United Nations, to assess the quality of statistical products produced by the NSO. The device related to the United Nation's Fundamental Principles of Official Statistics (10 items), the European Statistics Code of Practices (15 items) and the IMF's Data Quality Framework (5 items). The device functions as a management system to monitor and evaluate the statistical process, which stages from collection of data/information to the presentation of results (outputs). Each NSO recommended building and developing the instrument with reference to international standards. With these tools, the establishment of data‐generated statistics can be ascertained in a safe condition to further utilization. Results of measurement of this data quality should be submitted in the form of systematic reporting on statistical data produced.

Stat‐QAF itself is not just a series of quality improvement stages that to be implemented and developed, but will be emphasized more on systems and procedures placed on every step of statistical activities. Thus, Stat‐QAF is an image of the management system by following the procedure, which must be understood and implemented by the entire of BPS’ staffs, particularly those who involve in the preparation of statistical products. Development of Stat‐QAF

15


will benefit to BPS, especially in improving the statistician professionalism as stating at the code of practice.

The BPS defines the quality of statistical output data in simple form and will be suitable and beneficial for users. With reference to what has happened in other countries, and also international institutions, the definition of the quality dimension of the BPS data output is as follows:

a. Relevance • Refers to the degree to which outputs meet current

and potential users’ needs, to which they shed light on the issues of most important to users.

• A high degree or relevance implies all statistics that are needed are produced, no statistics that are not needed are produced, and concepts and classifications take account of user needs and international standards.

b. Accuracy • Refers to the degree to which outputs accurately and

reliably portray reality, to which the data actually measure the phenomena they are designed to measure.

• Usually characterized in terms of error in statistical estimates and is traditionally decomposed into sampling and non‐sampling errors. The latter are often further described by source of error, such as non‐response.

• The degree of consistency of estimates over time, often referred to as reliability, is an aspect of accuracy.

16


c. Timeliness and Punctuality

• Timeliness refers to the length of time between the availability of the output and the event or phenomenon they describe. The shorter the time period, the more timely the output.

• Punctuality refers to the difference between the date of the data are first released and the scheduled target date for release as announced in an official release calendar, or as laid down by regulations, or as previously agreed with partners.

d. Interpretability • Reflects the extent to which outputs are presented in a

clear and understandable form. • It is determined by the availability of metadata,

supplementary information and support services that are required for users to fully understand the outputs and make appropriate use of them.

e. Accessibility • Refers to the ease with which users are able to access

the data. • It is includes the ease with which users are able to

determine the data exist, the suitability and form of the access media, the cost of access, and impartiality of access procedures, in other words absence of access preferences.

f. Coherence • Refers to the degree different outputs can be

successfully brought together and used in combination. It depends upon the degree the outputs use similar concepts and definitions.

17


g. Comparability

• It is same as coherence but used in reference to outputs that contain the same data items but that differs in time period, region, or any other relevant domain.

h. Trustworthiness • Refers to the degree of confidence that users place in

outputs based simply on their image of BPS, i.e. our brand image.

• Determined in part by the integrity of the production process. However, the most important aspect is trust in the objectivity of the data. This means that the data are perceived to be produced professionally in accordance with appropriate statistical standards, also that policies and practices are transparent, in particular, data are not manipulated, nor is their release timed in response to political pressure.

The most important aspect is the confidence in the objectivity of the data, because the trustworthiness will bring better reputation and high credibility of the BPS. Objectivity may also be interpreted as if the data obtained and presented in a professional manner must follow the applicable statistical standards, particularly in reference to policies and practices. Data should not be manipulated, and if published it is in response to the needs of user, and not because of political pressure. Thus, Stat‐QAF as the quality framework will serve as a clearing unit/house where the statistical data will be cleared before being disseminated. The process includes the designing of working formats, preparation of technical manuals (guidelines), making the framework example (template), staff training and stage of implementation.

18


2.4 Quality and Code of Practice in Statistics Quality plays an important role in the construction and development of statistical data, where all stages in the preparation of the statistical data from individual level to the level of institutions (collective), in accordance with the conditions and capacities on each stage. The elements of quality set by the BPS include 8 dimensions, as mention above. The various dimensions of quality can be achieved through participation and support of BPS staffs by using the existing BPS statistical code of practice.

"The code of practice" is a set of guidelines that are designed with the intention of guiding the attitudes and behavior of members of an organization/institution in accordance with its vision and mission of. As a universal guide, the code of practice should be a reference for all members to form a professional attitude in carrying out the task. Since it involves the interests of many parties, especially for users, the preparation of this code shall be based on an agreement, and must be provisional. The statistical code of practice established by BPS includes 13 elements, are: i) independent, ii) confidentiality, iii) impartial and the results may be utilized by all parties, iv) the norms, standards, procedures and criteria used, v) correct utilization and interpretation, vi) objectively, vii) relevance, viii) accurate, ix) timely manner, and up to date x ) consistent xi) accessible easily, and affordable xii) understandable, and xiii) less burden on respondents.

19


3. Performance Indicator, Working Unit on Quality and Environmental Influence

3.1 Key Performance Indicators Key Performance Indicators (KPI) is the basic measurement of management information that serves a tool used to monitor and evaluate the quality of statistical data, including the improvement efforts. As the prior signal, these key indicators can provide important information to the users on overall conclusions about the quality of statistical data (output) produced by BPS which fitness for used.

In its development, assessment of data quality becomes an important issue, because it ensures that the statistical process is implemented according to its plans and objectives. Quality indicator provides information on the products derived from statistical process by using the standard concepts and scientific procedures. Although not all dimensions of data quality can be measured, for the very least this indicator can give an indication about the quality of each of production on statistical data.

The indicator covers the measurement of 8 dimensions of quality of various activities and its products especially for statistical data provided by BPS. Thus, it is desirable to have a representative measure of performance statistics by selecting and using variables that can represent each quality dimension. Selected indicators of quality dimensions will be displayed in line with quantitative information‐statistical data output, and will be presented on the media named "Statistical Dashboard"

20


3.2 Statistical Dashboard as Media Control Panel Statistics A "Dashboard" is a media used to monitor and evaluate the development of various statistical data produced which served as a panel presentation. In a simple sense, a quality dashboard is defined as a form of statistical panel display on a screen with its measurement. By using high technology, the various statistical products either in quantity and quality can be monitored regularly.

The "Dash board” can also serve as a shopping window to see the output of statistical data produced by BPS and readily for public access. Not all data‐statistics need to be shown, but only the selected statistical data needs to be displayed. It is because of the purpose to protect confidential data as well as the individual data. Updating and improvement of data should also be included in the information.

Through these media ones can know the condition and status of data in the very recent condition, including coverage, and more detail information. This media is important for decision makers in particular the high level managers of BPS to obtain the data status and its condition, soon. Moreover if necessary to intervene when there are problems raise on the processing statistical data .

3.3 Quality Assurance Unit The principle of total quality management (TQM) asserts that quality must be the concern of all BPS staff involve. They should contribute to improving quality as their own initiatives or through BPS re‐engineering project. In normal circumstance one would not think that quality is his/her

21


problem, however there are always important things in daily work which relate to quality

A special unit required to monitor and evaluate both the quality and quantity of statistical data produced by BPS. This unit supported by internal auditor will function as a “clearing house” with the following tasks: (i) to design a quality monitoring form, ii) to manage the quality form, (iii) to develop a quality reporting mechanism and procedures, (iv) to build a quality control panel indicators (quality dashboard), and (v) to conduct quality awareness training courses. All of these tasks indicate the importance the role and the function of the Statistical Quality Assurance Unit (Stat‐QAU).

In a short term, the Stat‐QAU is responsible for building, maintaining, and developing the quality infrastructure as defined in the Stat‐QAF. Quality monitoring consists of planning, survey instruments development, data collection, data processing, data presentation and analysis, as well as dissemination. Thus, quality improvement becomes an integral part of statistical activities in all BPS units. Quality management should and need to be undertaken continuously. Specifically, the functions of Stat‐QAU include:

a. Preparing proposals for quality improvement b. Developing and maintaining quality instruments c. Conducting training of quality awareness d. Managing quality assessment e. Improving the performance of quality assurance f. Constructing, and managing the "Statistical Quality

Dashboard" g. Maintaining the QAF instruments.

22


3.4 Environmental Impacts The existence and viability of BPS as an organization, in carrying out its vision and mission depends on the (internal and external) influence which sometime unpredictable. The impact could be positive or negative depending on the issues being raised. The internal influence includes willingness to change the mindset of BPS staffs and the work culture. Other internal factors are the quality of the raw data, budget limitation, and time constraint. While external factors include: data users, respondents, data duplication, and quality of the secondary data sources, working environment.

The success of BPS to overcome both internal and external influences requires an appropriate strategy by considering the potency, limitation, challenges and opportunities. The policy and strategy must be transparent and flexible by referring to the long‐term of national development program, particularly in the field of statistics. Practically, Stat‐QAF implementation process will be implemented in some stages by considering the two aspects as mentioned above. The development of Stat‐QAF should be created not only at the central level (head quarters) but also at the provincial, and district levels. Finally, BPS Stat‐QAF should be developed at a country level (national) by covering other government and private institutions dealing with statistics (namely as, National Quality Assurance Framework).

23


4. BPS Strategies and Policies on Quality 4.1 More Emphasis on Quality.

After establishing core values, code of ethics, and code of practice, further steps that need to be developed by BPS are the strategies and policies to improve its performance in conducting its core business. In carrying out its mission, BPS has been producing a variety of statistical products in accordance with the current needs and demands. Routine activities which are carried out to achieve the set targets lead to the low attention paid to the quality aspect. For a number of years, the statistical development has been focusing on producing a more variety of statistical data. Hence, it is a high time for BPS to pay attention to improving the quality of statistical data it produces. In fact, the quality plays an important role in providing assurance on the statistical data which will be used by users for various purposes.

4.2 Improving Management of National Statistical System The demand on higher quality of statistical data becomes more prominent due to an increasing need of potential users. Quality improvement means better and regular data monitoring, management and evaluation by trained professionals in a special unit within BPS. Stat‐QAF development should be the responsibility of all BPS staff because it will strengthen the BPS reputation and credibility as an institution which has the mandate to be the main player in implementing the National Statistical System (NSS). Various official statistics issued by the other government agencies should also be integrated with BPS statistical data to portray the development of Indonesia.

24


4.3 Setting Standard Operational Procedure and Implementing Quick‐Wins Phase I (2009‐2010) of the BPS Reform has reached its two main targets, are i) mapping the SOP (Standard Operational Procedures) and ii) accelerating statistical activities ("Quick‐Wins"). SOP document is intended to identify statistical process undertaken by all Subject Matter (SM) areas in conducting their statistical activities starting from planning to data presentation, including a description of linkages with other parties (both within BPS and with other agencies). While "Quick‐Wins" is the acceleration of the current statistical activities, including acceleration in the planning, the budget provision, the data collection, the data processing, the data presentation, and the publication and the delivery of information to the public (press‐releases).

4.4 Mapping BPS Statistical Activities The target to be achieved in Phase II of the BPS Reform (2010‐2014) is mapping and evaluating all statistical activities. The results of this activity will be used as a basis for development and monitoring of Stat‐QAF. Statistical data mapping activities will start from the elaboration process of the existing meta‐data BPS and continuing with the process of documentation. The activities will include Recognizing/Identifying, Classifying, Structuring, Processing and Computing, Analyzing, Reporting and Publishing. These activities are intended to create a detailed and complete documentation on BPS products. The activities are not limited to the national level but will also be conducted at the provincial and district/city levels. The process of recording statistical activities, resulting in statistical data, is to balance the data quantity with data quality. The result is also useful for developing the management of statistical data quality.

25


4.5 Building BPS Stat‐QAF and Guideline Monitoring quality, as part of Stat‐QAF activities, is important and urgent, because the availability of quality statistical data should be the responsibility of all BPS employees. Quality statistical data are statistics which can meet the standard rules, according to the existing criteria of statistical data quality dimensions, and at the same time can serve the needs of the community. Thus, the Stat‐QAF as the quality framework will serve as a clearing house where a variety of statistical data will be evaluated, arranged, described and organized before distribution. The process includes designing working formats, preparing technical manuals (guidelines), making the framework of sample (template), training the staff, and implementation. The Stat‐QAF will develop a technical guidance for quality management.

A new unit, equipped with professionals who are qualified in the field of quality, will be established to monitor and evaluate the quality of statistical data periodically. In addition, the unit will prepare and establish "Key Performance Indicators (KPIs)" as comparative measures of performance. These KPIs will be inputted in the planned "BPS Statistical Dashboard". As a statistical control panel, BPS Statistical Dashboard will contain the institutional profile with a range of products produced and their characteristics. This will also enable us to see the relationship between various statistical products either directly or indirectly in a system of statistical data. This will then lead BPS to attain its ultimate goal in building the "Consolidated and Integrated Statistics"

Based on experience, the BPS needs to look for opportunities to gain a professional recognition at the international level, for instance in the form of the ISO (International Standards Organization) certificate. This ISO Certification is the world

26


recognition of the achievement in managing statistical activities, using the international standards and concepts. Definitively, the attainment of the ISO certificate by BPS means that the statistical activities carried out and developed by BPS has been able to meet the standards of quality management. Standardization is intended to let other parties know the flow of statistical activities clearly. Thus, it is important to develop a guide book describing the history of how various kinds of statistical data were generated, so that if any errors occur, they could be detected and directly corrected.

In sum, equipped with the spirit of statistical reform, the development of statistical data at BPS should cover both quantity and quality aspects. Quality assurance will be an "entry point" for BPS to become more credible and accountable at the regional, national, and international levels. Starting from its Vision and Mission that are based on core values, code of ethic, and code of practice, it is possible for BPS to obtain the ISO certificate in a few years, as a form of recognition of the statistical profession.

The processed diagram in connecting BPS’ Vision and Mission to achieving the international Certification (ISO)

CODE OF PRACTICE

Project

PROGRAM PROGRAM

PROGRAM PROGRAM

STRATEGIC PLAN (RENSTRA)

Project

Project Project

ProjectProject Project

Project

VISION MISSION CORE VALUES

INPUT OUTPUTTRANSFORMATION INTERNATIONAL STANDARD ORGANIZATION

DASHBOARD

‐ DATA (QUANTITY) ‐ DATA (QUALITY)

QUALITY ASSURANCE FRAMEWORK

KEY PERFORMACE INDICATORS

27


References:

1. Michael J. Colledge. “BPS Quality Assurance Framework (Draft, Version 2.1)”. 2010. Jakarta

2. Statistics Canada. “Statistics Canada’s Quality Assurance Framework”. 2002. Ottawa

3. Statistics Canada. “Statistics Canada Quality Guidelines”. 2003. Ottawa

4. EUROSTAT .Quality Work and Quality Assurance within Statistics”. 1999. Brussels.

5. UK National Statistics “Guidelines for Measuring Statistical Quality (version 3.1)”.

6. Statistics Finland. Handbooks on“Quality Guidelines for Official Statistics (2nd Revised Edition)”. 2007. Helsinski.

knowing and understandingunstats.un.org/unsd/dnss/docs-nqaf/nqaf english...the two sides of a coin,...

Documents