nik - polish supreme audit office - nku.cz · from theory to everyday practice dr wieslaw karlinski...

Experience of NIK in using CAATs in audit

Experience of NIK in using CAATs in the audit process

from theory to everyday practice

dr Wieslaw Karlinski Department of Audit Methodology & Professional Development

Deputy Director

Praha, September 2012

NIK - Polish Supreme Audit Office

Experience of NIK in using CAATs in audit 2

At this conference we are exchanging of experiences with use AMS systems in our institutions. Most of the institutions audited by us has also their own experience and their own information systems of processes management.

As a result, much of the information about the audited process is stored in databases and supported by IT systems.

Therefore, I would like to focus on how this information can be effectivele used and present the experience of NIK in this regard.

Introduction


Data - information collected or shared by SAI

Metadata - information about the data available in IT systems of audited entities

Methods - data binding with controlled processes, allowing to develop algorithms for the analysis

Tools - CAATs used to collect and analyze data

Staff - auditors who knows how to use these tools

Conditions for the effective use of CAATs

What are the Polish conditions and our experience in this area?


NIK AMS

„production”

database

analytical

database

central

government

registers and

databases

IT systems of

audited units

Data resources used in the audit process

Internal External

• The analytical database of NIK is independent of the „production” database • Databases of audited units are always more complex and detailed,

regardless of the availability of central databases


o Introduced in 2009, contains

• List of entities subject to NIK audits

• Basic information (financial and non-financial) about these units based on central registers and databases

• Key informationabout their IT systems

o Currently covers about 90 000 entities, including

• 3 200 state budget entities

• 3 100 local government administration entities

Practical application:

• Drafting the audit plan

• Selecting particular entities to be audited

• Preparing the support of CAATs

NIK analytical database


1. Public registers (e.g. schools, healthcare units)

2. Financial plans and reports of all state budget entities

3. Financial reports of local government administration entities

4. Branch data warehouses (e.g. concerning taxes)

5. Central accounting systems (e.g. National Information System for the UE projects monitoring)

All the above mentioned data are used for updating the analytical database and for the actual analyses.

Only data derived from the sources no. 4 and 5 may be processed for the direct transactional data analysis and sampling.

Central government registers and databases


Possible situations:

• Homogeneous IT systems (e.g. tax-system)

• IT systems defined by the legal regulations (e.g. accounting

system)

• IT systems covering analogical aspects due to similar practice

and responsibilities of auditees

• Unique IT systems

A typical situation in the Polish setting

• Multiple IT systems created by different providers

• Lack of uniform standards of data storage and reporting

• Difficulties in acquiring useful data from IT systems

IT systems of audited units


More than 200 different IT systems, of which:

• 10 covers about 62% of all installations

• 50 covers about 85% of all installations

IT accounting systems in budgetary units

0

50

100

150

200

250

300

350

1 3 5 7 9 11 13 15 17 19 21 23 25 27 29 31 33 35 37 39 41 43 45 47 49

nu

mb

er

of

inst

alla

tio

ns

particular systems (the first fifty)

Experience of NIK in using CAATs in audit

MS Office

NIK Auditor Assistant (PK)

ACL (ver.8 & ver.9)

„production” database SterNIK

CAATs used in NIK

NIK Auditor Assistant (PK)

analytical database

system of the law information

Central

applications

On-site

applications

„mobile auditor”


• Tool developed in NIK in 2003-2012 (latest version – 5.2)

• Supports various types of audit in:

• statistical and non-statistical sampling

• data transformation and analysis

• automatic analysis of accounting data

• step-by-step financial audit examination

• Installed on laptops of all auditors

• In 2011 we developed an English version which was made available to the SAI of Denmark (Rigsrevisionen)

NIK Auditor Assistant

Why did we choose to design our own tool

• It is presisely fits our needs and is user-(i.e. auditor-)friendly

• It may be constantly upgraded

• It applies well-recognized statistical methods

• We are not limited by the commercial software license

• It may be used by other public entities free of charge


o Sampling stages • sample size calculation

• random data rearrangement

• sample selection with statistical and no statistical methods

• evaluation – using statistical and no statistical selection

o Sampling methods • Attribute sampling (fixed sample size, sequential, discovery sampling)

• Variable sampling (unstratified and stratified MPU, difference and ratio estimation, cluster sampling)

• PPS (MUS, Hartley-Rao estimation)

o Supports different data distributions • Normal, t-Student, Poisson’s, binomial, hipergeometric

NIK Auditor Assistant - sampling

Virtually all sampling method used in the audit are supported


NIK Auditor Assistant – other functions

Data Acquisition

acquisition from text reports

cryptographic hash function

anonimization

coding

Data

Transformation

joining

selection

grouping

sorting

Typical Audit Analysis

classification

stratification

aging

basic statistics

gaps

duplicates

Analysis of Accounts

budget ary classification

standariza-tion

„FK-scan” selection

„FK-scan” analysis

contra entries

Step-by-step Examination

defining the population

sampling

audit question-

naire

evaluation

reporting

summary statistics


„FK-scan” methodology – selection

Incorrect or incomplete data

• lack of mandatory elements

• vague description or no

description of the operation

• incorrect date

• conflict of dates

Incorrect accounting periods

• incorrect year

• incorrect month

• accounting date exceeds the date

of the report

Possible errors in compliance or

reliability

• incorrect paragraphs (budget

classification)

• duplicates or missing data

• accounting skip clearing accounts

• improper classification of fixed

assets

• improper classification of

current/capital expenditure

Entries atypical in terms of dates or

accounts

• made on a non-working day

• violation of the natural sequence of

dates

• comments on payments

• approved after Dec 31st

• accounting for non-standard

accounts


„FK-scan” methodology – data analysis

o Non-typical logins of the operator

o Expenses exceeding the financial plan

o Turnover on selected accounts (including seldom used or sensitive)

o Conformity of periodic budget reports with the accounting books

o Analysis of accounts in association with e-banking


o Participation of auditors fluent speakers of CAATs (including the ACL) is insufficient, and the number of licenses ACL is minimal (30)

o Our own tools (NIK Auditor Assistant) partially solved both problems

o In addition, we created a special team of CAATs support

• Local level: 2-3 persons in every NIK unit

• Central level: 4 persons in Department of Audit Methodology

• Coordinated by dr Karlinski

o Central team focuses on the development of methodology and provide solutions (e.g. new scripts)

o Local teams support the auditors and provide branch-specific solutions

o Each audit we try to obtain information about IT systems operating in the audited units to update our metadata database

Personnel and organizational solutions


o Selection of units scheduled for audit

o CAATs in financial audit

o CATTs in other types of audit (e.g. testing the effectiveness

of enforcement of the traffic tickets issued by the police)

Examples of practical application of CAATs


SELECTED with:

• exclusions

• inclusions (mandatory audit)

• random PPS selection with the assumption that we

examine 20% of expenditures in this part of the budget

Selection of units scheduled for audit

Ministry of Justice

1

2

3

4

5

6

7

8

9

10

11

Court of Appeals (n=11)

The Hundred Court

(n=45) The District Court

(n=320)

Case: State budget execution in the part „courts”

USED SOURCES:

• analytical database and last year audit results

• NIK Auditor Assistant as tools


CAATs in financial audit – audit model

Acquisition of data from local accounting systems

Standarization

„FK-scan”

MUS

Reports and statistics

Simple random selection

Highest amount

Electronic dataset Non-electronic (paper) dataset


o Every year NIK carries out about 200 financial audits

o Almost all auditees possess electronic accounting systems

o In each case we use statistical sampling and if possible – automatic

analysis of the accounting records

o The major problem is acquiring data from the accounting systems

o In each audit :

• „NIK Auditor Assistant” (PK) is used as the basic tool

• ACL or other tools in particular situations (e.g. to convert *.pdf data files)

CAATs in financial audit – typical statistics


o Full financial audit performed in 193 auditees of wchich 189 (98%)

possessed a computer accounting system

o Data from accounting systems received in 183 cases (97%)

o Automated analyses „FK-scan” performed in over 150 cases

o MUS sampling used in 183 cases

o Simple random selection used in 10 cases

o Average examined sample size per unit – 157 (30 500 in total)

It seems, that despite the difficulties with acquiring accounting

records, CAATs facilitate financial audit in everyday practice of NIK

auditors

CAATs in financial audit – last year statistics


CAATs in other types of audit

Case:

effectiveness of enforcement of the credited mandates

issued by the police

• Offence

• Mandate payment

citizen

• Credited mandate punishment

Policeman

• Mandates evidence

• Collection of payments

• Issuing writs of execution

Governor (voivod)

• Execution

Inland Revenue -

7 days

~5 mln mandates/year

16 voivods

~350 police

commands ~400 Inland

Revenues

database database

database/papier

• achieved in 2011 income ~ 750 mln PLN

• backlog at the end of 2011 ~ 609 mln PLN

• the limitation period due - 3 years


Audited units – 8 governors (1/2), 16 police commands, 16 Inland

Revenues

Single database governor - more than 1 million records

Scope of the study

timeliness of submission documents and registration

timeliness of payments

complete data sets

relationships between databases of different actors

the efficiency and effectiveness of enforcement

limitation

selection of the sample to be tested in the different categories of cases

CAATs in credited mandates audit


o The use of CAATs in the financial audit is in the NIK almost complete. For other types of audits, the situation is much more complicated.

o Nor does it seem reasonable that the CAATs used every day and in every audit. It is important to apply it where it can bring good results.

o On the other hand, it is worth using CAATs even where this is not necessary, because it is a condition of the acquisition of appropriate practice for auditors.

o The problem for the external auditor is able to obtain the data in electronic form.

o But the biggest problem is the ability to link the data with audited processes.

Conclusions


NIK - Polish Supreme Audit Office

THANK YOU FOR YOUR ATTENTION

[email protected]

nik - polish supreme audit office - nku.cz · from theory to everyday practice dr wieslaw karlinski...

Documents