1 the south african statistical quality assessment framework (sasqaf) presentation made at the...
Post on 22-Dec-2015
214 Views
Preview:
TRANSCRIPT
1
The South African Statistical Quality Assessment Framework (SASQAF)
Presentation made at the Conference on Data Quality for International Organisations
Helsinki, Finland, 6–7 May 2010
Seble Worku National Statistics System Division
Statistics South Africa
2
Background• The broader National Statistics System (NSS) is characterized
by three gapsCapacity gap
- Insufficient statistical skills- Inappropriate placement of skilled personnel
Quality gap- Preponderant usage of data of uncertain or
unknown qualityInformation gap
- Insufficient supply of statistical information- Insufficient supply of data on second
economy
• To address the Quality Gap the Statistician General has gazetted through parliament the South African Statistical Quality Assessment Framework (SASQAF) on the 23rd September 2009
• SASQAF quality requirements must be met by all statistics producers in the NSS to qualify as “Official Statistics”.
3
The Statistics Act No. 6 of 1999• Stats SA is governed by the Statistics Act (No. 6 of
1999)• The Stats Act covers “Official” Statistics and “Other”
statistics• Implies, Stats Act covers all statistics produced that
informs policy, planning and monitoring of government performance.
• Section 14(7) of the Act, empowers the Statistician-General to “designate as official statistics any statistics or class of statistics” produced by Stats SA or any organ of state Required that a rational, sustainable and
transparent framework for assessing the quality of those statistics being developed
South African Statistical Quality Assessment Framework (SASQAF) has been developed for this purpose
4
Definition
Official statistics’ definition is statutory – see Statistics Act [No. 6 of 1999]
Practical criteria of official statistics – Must be used in the public domain– Are from organs of state and other agencies that are
partners in the National Statistics System [NSS]– Are sustainable– Have met quality criteria as defined by the Statistician-
General [SASQAF]
National statistics’ definition is implicitly statutory
Official statistics are statistics designated as official statistics by the Statistician-General within the provisions of the Statistics Act
National statistics are statistics not designated as official Statistics by the Statistician-General
5
Structure of the framework
• Provides a structure for the assessment of statistical products based on 1. Dimensions of quality,2. Indicators, 3. Standards,4. Benchmarks
• Each of the 8 quality dimensions consists of number of indicators
• Within the indicators a number of benchmarks are identified relating to a 4-point scale;
1. Quality statistics (4) 2. Acceptable statistics (3) 3. Questionable statistics (2) 4. Poor statistics (1)
6
Relevance
Accuracy
Coherence
TimelinessInterpretability
Accessibility
Meeting realneeds of clients
Correctly describes phenomena it is designed to measure
Info available atdesired referencepoint
Ease of obtaininginfo from agency
Availability of supplementaryinfo and metadata
Harmonisation of different infowithin broadanalyticaland temporalframework
Integrity
Free from political interference:Adherence to objectivity, professionalism, transparency,ethical standards
Methodologicalsoundness
Sound methodologies:•International standards and guidelines - good practice•Agreed practices•Dataset-specific
The Dimensions of Quality
SASQAF
7
Indicators and Standards
1st Edition 2nd Edition
Dimension Indicators Indicators Standards
Prerequisites of Quality 7 8 21
Relevance 7 5 5Accuracy 7 7 36Timeliness 5 4 10
Accessibility 13 12 13
Interpretability 3 3 3
Comparability and Coherence
5 5 12
Methodological soundness
6 6 14
Integrity 6 6 6
Quality 59 56 120
8
Quality dimension Description
0. Prerequisites of quality Refers to the institutional and organisational conditions that have an impact on data quality.
Key components IndicatorAssessment Levels
Quality Statistics Acceptable Statistics
Questionable Statistics
Poor Statistics
Level 4 Level 3 Level 2 Level 1
Legal and institutional environment (including Memoranda of Understanding (MoUs) or Service Level Agreements (SLAs)Privacy and confidentialityResources are commensurate with the needs of statistical programmesQuality is the cornerstone of statistical work
The responsibility for producing statistics is clearly specified.
The responsibility for producing statistics is explicitly specified through a legal framework.
The responsibility for producing statistics is specified through a legal framework.
The responsibility for producing statistics is implied through a legal framework.
The responsibility for producing statistics is not specified.
Standards and policies are in place to promote consistency of methods and results.
All standards and policies are in place to promote consistency of methods and results, and are adhered to.
The majority of standards are in place to promote consistency of methods and results.
Some standards are in place to promote consistency of methods and results.
No standards are in place to promote consistency of methods and results.
Data sharing and coordination among data-producing agencies is clearly specified and adhered to.
Data sharing and coordination among data-producing agencies is explicitly specified through a legal framework.
Data sharing and coordination among data-producing agencies is specified through a legal framework.
Data sharing and coordination among data-producing agencies is implied through a legal framework.
Data sharing and coordination among data-producing agencies is not specified.
Layout of the framework
9
Quality Standards
•Indicator:
The responsibility of producing statistics is clearly specified.
•Standard:A legal arrangement exists that explicitly mandates the production of statistics.
•Benchmarks:Quality statistics: A law or legal arrangement
exists that explicitly provides the mandate for the production of statistics;
Poor statistics: No arrangement exists.
10
Purpose of the framework
SASQAF provides an universal framework for assessing the quality of statistics within Stats SA and the NSS
•Reviews within the NSS: certification of data
• Self-assessment by data-producing agencies: rate own performance
• Assessment by data users based on quality declaration: brings trust in data
• Assessment by international agencies: ease international comparison
11
Purpose of the framework
•SASQAF provides an universal framework for assessing the quality of statistics within Stats SA and the NSS
•Reviews within the NSS: certification of data
• Self-assessment by data-producing agencies: rate own performance
•Assessment by data users based on quality declaration: brings trust in data
•Assessment by international agencies: ease international comparison
12
ConclusionAdherence to SASQAF will
• encourage compliance to the agreed standards, procedures and guidelines resulting in improvement of quality, closing the quality
gap
• ensure that more statistics are certified as official closing the supply gap
• assist the users in assessing the quality of data and products promoting transparency
• ensure that all published products include statements about data quality (quality declaration) informing users about the quality of data and products
• ensure that more statistical products produced in the NSS are declared as “fit for use”
13
Thank you
top related