transferring auditors’ internal control evaluation knowledge to management

17
Transferring auditors’ internal control evaluation knowledge to management q Chuleeporn Changchit a , Clyde W. Holsapple b, * , Ralph E. Viator c a Department of Management Sciences, Henry B. Tippie College of Business, W284 John Pappajohn Business Building, University of Iowa, Iowa City, IA 52242-1000, USA b School of Management, Carol M. Gatton College of Business and Economics, University of Kentucky, Lexington, KY 40506-0034, USA c Area of Accountancy, College of Business Administration, Texas Tech University, Lubbock, TX 79409, USA Abstract Due to both regulatory and competitive forces, attention to an organization’s internal controls has increased significantly in the 1990s. Although management is ultimately responsible for ensuring internal controls are adequate, managers often lack knowledge of internal control concepts. This study reports on an experiment testing an expert system developed to facilitate the transfer of internal control knowledge to management. Experimental results indicate that expert systems are viable aids for transferring internal control knowledge to managers whose work experience is outside of accounting and control systems. q 2001 Elsevier Science Ltd. All rights reserved. Keywords: Intelligent system; Expert system; Management; Internal controls; Knowledge transfer; Learning theory 1. Introduction Internal control is defined as a process designed to provide reasonable assurance regarding the achievement of objectives in the following categories: reliability of finan- cial reporting, effectiveness and efficiency of operations, and compliance with applicable laws and regulation (AICPA, 1996). Due to both regulatory and competitive forces, attention to an organization’s internal controls has increased significantly in the 1990s (Weber, 1999). Both accounting firms and researchers have devoted significant effort to the development of decision aids, especially expert systems, to assist auditors in internal control evaluations (Bailey, Duke, Gerlach, Ko, Meservy & Whinston, 1985; Boritz, 1985; Gal, 1985; Cummings & Apostolou, 1987; Cummings, Lauer & Baker, 1988; O’Leary & Watkins, 1989; Graham, Damens & Ness, 1991; Vinze, Karan & Murthy, 1991). Researchers have long contended that expert systems could be used not only as decision aids but also to train non-expert users (Holsapple & Whinston, 1986; Biggs, Messier & Hansen, 1987; Borthick & West, 1987; Gal & Steinbart, 1992; Odem & Dorr, 1995). Prior studies found that subjects who practiced making decisions with the aid of an expert system were better and quicker at reaching deci- sions than subjects who practiced without the support of the expert system (Oz, 1989; Murphy, 1990; Eining & Dorr, 1991; Fedorowicz, Oz & Berger, 1992). However, there are no previous reported studies of an expert system to facilitate the transfer of internal control evaluation knowl- edge to management. There are several reasons why managers are likely to perform internal control evaluations. First, establishment and supervision of internal control systems are the responsibility of management, not external auditors (COSO, 1992). Second, external auditors do not have the immediate and detailed insight into the operation of the internal control systems that management does. And third, the evaluation of internal control systems should be treated as a continuous rather than a discrete process, so that any weaknesses can be prevented or detected as soon as possible. Even though the literature has reported that an expert system can facilitate the transfer of internal control evalua- tion knowledge to novice subjects (Libby, 1995; Lieb & Gillease, 1996; Weber, 1999), all such studies used either auditors or accounting students as subjects. Without conducting the research, we are reluctant to assume that having managers as subjects will yield the same positive result. There are significant reasons why managers who lack training in accounting systems may be less likely to Expert Systems with Applications 20 (2001) 275–291 PERGAMON Expert Systems with Applications 0957-4174/01/$ - see front matter q 2001 Elsevier Science Ltd. All rights reserved. PII: S0957-4174(00)00066-X www.elsevier.com/locate/eswa q Data Availability: Data is available from the authors; however, requests for data must be specific regarding intended use. * Corresponding author. Tel.: 11-606-257-5236; fax: 11-606-257-8031. E-mail addresses: [email protected] (C. Changchit), [email protected] (C.W. Holsapple), [email protected] (R.E. Viator).

Upload: independent

Post on 15-May-2023

0 views

Category:

Documents


0 download

TRANSCRIPT

Transferring auditors' internal control evaluation knowledge tomanagementq

Chuleeporn Changchita, Clyde W. Holsappleb,*, Ralph E. Viatorc

aDepartment of Management Sciences, Henry B. Tippie College of Business, W284 John Pappajohn Business Building, University of Iowa, Iowa City,

IA 52242-1000, USAbSchool of Management, Carol M. Gatton College of Business and Economics, University of Kentucky, Lexington, KY 40506-0034, USA

cArea of Accountancy, College of Business Administration, Texas Tech University, Lubbock, TX 79409, USA

Abstract

Due to both regulatory and competitive forces, attention to an organization's internal controls has increased signi®cantly in the 1990s.

Although management is ultimately responsible for ensuring internal controls are adequate, managers often lack knowledge of internal

control concepts. This study reports on an experiment testing an expert system developed to facilitate the transfer of internal control

knowledge to management. Experimental results indicate that expert systems are viable aids for transferring internal control knowledge

to managers whose work experience is outside of accounting and control systems. q 2001 Elsevier Science Ltd. All rights reserved.

Keywords: Intelligent system; Expert system; Management; Internal controls; Knowledge transfer; Learning theory

1. Introduction

Internal control is de®ned as a process designed to

provide reasonable assurance regarding the achievement

of objectives in the following categories: reliability of ®nan-

cial reporting, effectiveness and ef®ciency of operations,

and compliance with applicable laws and regulation

(AICPA, 1996). Due to both regulatory and competitive

forces, attention to an organization's internal controls has

increased signi®cantly in the 1990s (Weber, 1999). Both

accounting ®rms and researchers have devoted signi®cant

effort to the development of decision aids, especially expert

systems, to assist auditors in internal control evaluations

(Bailey, Duke, Gerlach, Ko, Meservy & Whinston, 1985;

Boritz, 1985; Gal, 1985; Cummings & Apostolou, 1987;

Cummings, Lauer & Baker, 1988; O'Leary & Watkins,

1989; Graham, Damens & Ness, 1991; Vinze, Karan &

Murthy, 1991).

Researchers have long contended that expert systems

could be used not only as decision aids but also to train

non-expert users (Holsapple & Whinston, 1986; Biggs,

Messier & Hansen, 1987; Borthick & West, 1987; Gal &

Steinbart, 1992; Odem & Dorr, 1995). Prior studies found

that subjects who practiced making decisions with the aid of

an expert system were better and quicker at reaching deci-

sions than subjects who practiced without the support of the

expert system (Oz, 1989; Murphy, 1990; Eining & Dorr,

1991; Fedorowicz, Oz & Berger, 1992). However, there

are no previous reported studies of an expert system to

facilitate the transfer of internal control evaluation knowl-

edge to management. There are several reasons why

managers are likely to perform internal control evaluations.

First, establishment and supervision of internal control

systems are the responsibility of management, not external

auditors (COSO, 1992). Second, external auditors do not

have the immediate and detailed insight into the operation

of the internal control systems that management does. And

third, the evaluation of internal control systems should be

treated as a continuous rather than a discrete process, so that

any weaknesses can be prevented or detected as soon as

possible.

Even though the literature has reported that an expert

system can facilitate the transfer of internal control evalua-

tion knowledge to novice subjects (Libby, 1995; Lieb &

Gillease, 1996; Weber, 1999), all such studies used either

auditors or accounting students as subjects. Without

conducting the research, we are reluctant to assume that

having managers as subjects will yield the same positive

result. There are signi®cant reasons why managers who

lack training in accounting systems may be less likely to

Expert Systems with Applications 20 (2001) 275±291PERGAMON

Expert Systemswith Applications

0957-4174/01/$ - see front matter q 2001 Elsevier Science Ltd. All rights reserved.

PII: S0957-4174(00)00066-X

www.elsevier.com/locate/eswa

q Data Availability: Data is available from the authors; however, requests

for data must be speci®c regarding intended use.

* Corresponding author. Tel.: 11-606-257-5236; fax: 11-606-257-8031.

E-mail addresses: [email protected] (C. Changchit),

[email protected] (C.W. Holsapple), [email protected] (R.E. Viator).

acquire internal control evaluation knowledge. First,

managers, including corporate internal auditors, come

from diverse backgrounds, such as operations and informa-

tion systems training, and do not necessarily have an

accounting educational background (Viator & Curtis,

1998). Second, differences in educational background

have been directly linked to differences in knowledge struc-

ture (Curtis & Viator, forthcoming). Studies have shown

that a mismatch between knowledge structure and task

structure can have a detrimental effect on performance

(Nelson, Libby & Bonner, 1995), and differences in knowl-

edge structure are systematically associated with the quality

of performance in reviewing of internal control systems

(Curtis & Viator, forthcoming). Given variations in educa-

tional background, work experience, and knowledge struc-

tures, the transfer of internal control evaluation knowledge

appears to be less likely for managers, as compared to exter-

nal auditors or accounting student novices. Third, managers

may not be willing to learn the concept of the internal

controls, or may not feel comfortable using expert systems.

For example, they may provide incorrect inputs, not under-

stand system outputs, ®nd it dif®cult to work with expert

systems, or resist considering the advice it gives. This

research examines whether managers' use of expert systems

in the evaluation of internal controls can overcome the

barriers identi®ed above, resulting in the transfer of internal

control knowledge to those managers.

2. Background and hypotheses

One of the most fundamental learning concepts is acqui-

sition of knowledge (Cormier & Hagman, 1987). Informa-

tion processing theory uses the computer as a model for

human learning (Miller, 1956). Like the computer, the

human mind takes in information, performs operations to

change its form and content, stores and locates it, and gener-

ates responses to it. Thus, processing involves gathering and

representing information (encoding), holding information

(retention), and getting at the information when needed

(retrieval). Several decision aids were developed to facili-

tate the transfer of the knowledge of one source to another.

In this study, we are concerned with the impact of using

expert systems to facilitate such learning. The following six

hypotheses were designed to examine the value of the

system as follows:

H1: The improvement in accuracy scores of participants

trained with an internal control evaluation expert system

is higher that the improvement in accuracy scores of

participants with no decision aid.

Hypothesis H1 examines whether a participant's accu-

racy in detecting internal control weaknesses improves

after being trained with an expert system. Accuracy of deci-

sion-making is examined as a measure of the system's effec-

tiveness (Sharda, Barr & McDonnell, 1988; Eining & Dorr,

1991). In order to examine the effectiveness of the system,

we compared participants' improvement (Session I vs

Session IV) in the accuracy scores between the Expert

System (ES) group and No Decision Aid (NDA) group.

We hypothesized that, on average, the ES group's improve-

ment in accuracy scores will be higher than the NDA

group's improvement.

H2: The improvement in decision time of participants

trained with an internal control evaluation expert system

is higher than the improvement in decision time of parti-

cipants with no decision aid.

Hypothesis H2 examines whether a participant's decision

time in accurately detecting internal control weaknesses is

reduced after being trained with an expert system. The time

used to make correct decisions (i.e., time per accuracy score)

was examined as a measure of the system's ef®ciency. In

order to measure the improvement in decision time, we

compare the times used by participants in both groups in

accurately detected internal control weaknesses. We hypothe-

sized that, on average, the ES group's improvement in deci-

sion time will be higher than the NDA group's improvement.

Hypotheses H3a, H3b, H4a, and H4b are based on the

Technology Acceptance Model (TAM). The TAM provides

a basis for explaining the determinants of computer accep-

tance and user behavior (Davis, Bagozzi & Warshaw, 1989).

The model suggests that computer usage is determined by

two criteria: perceived usefulness and perceived ease of use.

Perceived usefulness is de®ned as the prospective users'

subjective probability that using a speci®c application

system will increase his or her job performance within an

organizational context. Perceived ease of use is de®ned as

the degree to which the prospective user expects the target

system to be free of effort (Davis et al., 1989).

H3a: Participants trained with an internal control evalua-

tion expert system are more satis®ed with their accuracy

than participants with no decision aid.

H3b: Participants trained with an internal control evalua-

tion expert system are more satis®ed with their speed than

participants with no decision aid.

Hypotheses H3a and H3b examine participants'

perceived usefulness of the expert system in improving

their performances in detecting internal control weaknesses.

The hypotheses test whether there were differences in satis-

faction with performances between participants practicing

with an expert system (ES) and those with no decision aid

(NDA). We hypothesized that, on average, participants in

the ES group would be more satis®ed with their accuracy

and speed than participants in the NDA group.

H4a: Participants trained with an internal control evalua-

tion expert system perceive that performing tasks requires

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291276

less effort, compared to participants with no decision aid.

H4b: Participants trained with an internal control evalua-

tion expert system perceive that performing tasks is more

interesting, compared to participants with no decision aid.

Hypotheses H4a and H4b examine participants'

perceived usefulness of the expert system in improving

their attitudes toward the internal control evaluation tasks.

The hypotheses test whether there were differences in parti-

cipants' perceptions of the task between the ES group and

the NDA group. We hypothesized that, on average, partici-

pants in the ES group would perceive that the evaluation of

internal controls requires less effort and is more interesting

than participants in the NDA group.

3. Research methodology

To examine the impact of using an expert system to facil-

itate the transfer of the auditor's internal control evaluation

knowledge to management, an experiment was conducted.

The research design consisted of two treatment groups,

three covariates, and six dependent variables. The two treat-

ment groups were subjects with an expert system (ES) and

those with no decision aid (NDA). Three covariates were

measured, including the initial accuracy score from Session

I, percentage of management duties, and years of work. The

six dependent variables included subjects' accuracy, deci-

sion time, satisfaction with accuracy, satisfaction with

speed, perception of task dif®culty, and perception of task

attractiveness.

3.1. Subjects

Because the main purpose of this study is to investigate

whether an expert system can be devised to help managers

more ef®ciently and/or more effectively detect potential

weaknesses of internal control systems in their organiza-

tions, the subjects were persons with managerial responsi-

bilities. These managers were drawn from client ®rms of

three international accounting ®rms. Lists of potential

subjects were obtained from these accounting ®rms. One

hundred and forty subjects were randomly selected from

those lists (one hundred for the experimental group and

forty for a control group). To gain subjects' cooperation,

the invitation letter explained clearly the importance of

the study and assured con®dentiality. All subjects were

told that they would receive a summary of this study.

The experiment was conducted in an isolated, controlled

environment provided by college laboratory and conference

rooms. There were 50 participants in the experimental group

(50% participation rate) and 15 in the control group (37.5%

participation rate). Each participant served as an internal

control decision maker. Even though some participants

hold positions in accounting and ®nancial areas, telephone

interviews were conducted with these participants to ensure

that they had neither a background in internal control

concepts nor prior hands-on experience with an expert

system. Tables 1±3 show the participants' industries, educa-

tion, average years of work, average percentage of manage-

ment duties, and diversity of management roles.

3.2. Internal control case studies

Four case studies (A, B, C, and D) were generated from

the manipulation of several cues for detecting the potential

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 277

Table 1

Distribution of subjects across industry

Industries No. of subjects

Consultant 12

Legal 8

Manufacturing 8

Government 11

Insurance 3

Banking 2

Construction 2

Distribution 2

Mining 2

Transportation 2

Education 3

Finance 3

Others 7

Total 65

Table 3

Participants' titles

Title No. of subjects

Finance and Accounting

Manager

9

General Manager 7

Supervisor 10

Director 5

Senior Analyst 4

Assistant Manager 6

Vice President 2

Tax Manager 2

Administrative Manager 6

Legal Manager 2

Chief Accountant 2

Engineer 2

System Analyst 2

Lawyer 2

Marketing Manager 2

Operating Manager 1

Joint Venture Manager 1

Total 65

Table 2

Participants' demographics

Education 55 Bachelor, 10 Master

Average years of work 5.46

Average percentage of management duties 53.7

weaknesses in internal control systems. These cues were

obtained from a review of auditing texts, accounting texts,

and input from accounting professors and experienced audi-

tors. (See Appendix B) The scenario in each case dealt with

the adequacy of internal control over a company's sales and

collection cycle. Also, each case included background infor-

mation about the ®ctitious company and a partial organiza-

tion chart. Each case contained ten potential weaknesses in

the internal control system. Three experienced auditors and

three managers were asked to pilot test these cases to ensure

their similarity with respect to the degree of dif®culty in

detecting the potential internal control weaknesses. Revi-

sions to the cases were made based on feedback provided

(see Appendix C).

3.3. Experimental task

The experiment consisted of four sessions. Prior to the

experiment, the four cases were randomly assigned to the

sessions, yielding the result of Case A being assigned to

Session IV, Case B to Session I, Case C to Session III,

and Case D to Session II. In each session, participants

were asked to play the role of a manager trying to detect

the potential weaknesses of the internal control system

described in the case. The maximum time allowed for

each session was two hours.

In Session I, participants in both groups performed the

case without any decision aid. Because no participant had a

background in internal control concepts, a booklet contain-

ing an overview of basic internal control activities (Appen-

dix A) and a list of possible internal control weaknesses

(Appendix B) were provided in this session. In Sessions II

and III, participants in the NDA group performed the second

and third case studies without the expert system, while parti-

cipants in the ES group were allowed to use the expert

system as a decision aid in detecting internal control weak-

nesses in the assigned case. In Session IV, the expert system

was removed, participants in both groups performed the

fourth case study without any decision aid. A booklet

containing an overview of basic internal control activities

and a list of possible internal control weaknesses were

provided again in this session.

For each session, the time required for decision-making

was recorded and an accuracy score was calculated. In addi-

tion, questionnaires were given at the end of each session to

measure participants' satisfaction with their performances

as well as their perceptions of effort and interest in the task.

This questionnaire was also used to gather data about their

demographics.

A performance-based reward system was used here to

induce optimal participant behavior and reduce problems

due to guessing (Smith, 1982). Participants were told that

their performances across the series of cases would be used

to calculate two scores: an accuracy score and a time per

accuracy score. They were informed that prizes would be

given to those participants attaining the highest accuracy

scores and to those attaining the best time per accuracy

scores. Participants appeared to be genuinely interested in

garnering prizes. In addition, as practicing managers who

agreed to devote the many hours required to this study, their

own senses of commitment and professionalism were

factors in securing their efforts within the experiment.

In arriving at an accuracy score, all points related to

correctly identi®ed weaknesses were summed. In addition,

to prevent the subjects from trying to detect weaknesses by

guessing, they were informed at the beginning of the experi-

ment that one-third of all points for inaccurately identi®ed

weaknesses will be subtracted as the penalty for guessing.

Non-response for a potential weakness results in neither

addition nor subtraction.

3.4. Dependent variables

Two experienced auditors and two accounting professors

evaluated the list of internal control weaknesses previously

described. Each evaluator assigned a score to each weakness

using a scale of 0±10, based on the degree of importance of

each weakness (0Ðthe least important, 10Ðvery impor-

tant). The average of the scores for each weakness (i.e.,

divided by four) was assigned as its importance. In order

to examine the impact of using the expert system to facil-

itate the transfer of knowledge, the improvement in accu-

racy was compared across the two treatment groups:

participants in the ES group versus participants in the

NDA group.

The time used to make decisions accurately (i.e., Time

per Accuracy Score) was examined as a measure of ef®-

ciency. The time per accuracy score was computed as

follows:

Time Used to Perform Task

Accuracy Score

For example, Mr. A correctly detects 5 weaknesses and

incorrectly identi®es 5 weaknesses. The time used to do the

case is 40 min. Assuming that each weakness has 3 points

associated with it. Mr. A's accuracy score is

10� ((5 £ 3) 2 (5 £ 3)/3) and his time per accuracy score

is 40/10, or 4 min per accuracy score.

Four questions (detailed in Appendix D) were asked to

investigate participant satisfaction and perception of the

task. These questionnaires were pretested with ten MBA

students to ensure their clarity and construct validity.

1. On a scale of 1 (very unsatisfactory) to 7 (very satisfac-

tory), how satis®ed were you with your accuracy in

answering the case study?

2. On a scale of 1 (very unsatisfactory) to 7 (very satisfac-

tory), how satis®ed were you with your speed in answer-

ing the case study?

3. On a scale of 1 (very dif®cult) to 7 (very easy), how

dif®cult was it to do the case study?

4. On a scale of 1 (very boring) to 7 (very interesting), how

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291278

interesting was the task you performed?

3.5. Decision aid

A prototype expert system to train management in inter-

nal control evaluation was constructed. The knowledge of

the system was acquired from several sources including an

experienced auditor, who served as the expert for the

system, auditing texts, accounting texts, and the COSO

framework. The expert was asked to describe, in detail

and step-by-step, the techniques and processes he uses in

evaluating an internal control system. The reasons for each

decision were also acquired in an attempt to develop an

expert system that would be able to emulate both the

expert's knowledge and his reasoning behavior.

Because the system was aimed at training managers in

internal control evaluation, additional features, other than

those included in previously reported expert systems, were

integrated into the system for easy understanding and use by

persons who are not familiar with internal control concepts.

For instance, the system incorporated several diagrams to

direct users to the next steps of detecting internal control

weaknesses. Also, the system prompted users for entering

data relevant to detecting the internal control weaknesses. A

menu was also used to reduce the possibility of typing

errors. The system reports each weakness identi®ed and

why the weakness found is considered important, not just

the adequacy of the overall internal control system within

the organization. Figs. 1±3 present examples of diagrams,

input screens with menus, and system output.

Once a prototype expert system was developed, the

expert was asked to review the expertise that had been

captured in the rule sets. Test cases were used during the

validity review.

4. Results and discussion

In order to assess the homogeneity of subjects' internal

control knowledge between the experimental group and the

control group, an ANOVA was performed on the ®rst

session results to test for differences in Accuracy Score

and in Time per Accuracy Score between the groups. At

an alpha level of 0.05, no signi®cant difference was found

between the groups' initial levels of internal control knowl-

edge.

The experimental data were analyzed as a completely

randomized design with a one-way treatment structure

using the Analysis of Covariance (ANCOVA) technique.

The StatVieww system was used to perform the test and

each participant was an experimental unit. Hypotheses H1

and H2 were tested by calculating performance differences

between Session IV and Session I, for each participant.

Tests of hypotheses H3a, H3b, H4a, and H4b were based

on participants' responses to the questionnaire items at the

end of Session IV. The ANCOVA factors included experi-

mental group and three covariates: initial score, percentage

of management duties, and years of work experience.

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 279

Fig. 1. Diagram for checking the existence of proper authorization of sales orders, sales invoices, credit memo, and changes in payment conditions memos.

Table 4 presents the ANCOVA results testing H1, regard-

ing participants' accuracy in detecting potential weaknesses

in internal control systems after being trained with the

expert system. The results show that the improvement in

accuracy for participants in the ES group was signi®cantly

higher than those in the NDA group (11.50 vs 3.67, respec-

tively, with p , 0.008). Regarding the covariates, only the

initial score was found to have an effect to the improvement

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291280

Fig. 2. A menu for selecting the person required to approve sales invoices.

Fig. 3. Report of weakness found for improper authorization of sales invoices of transactions and accounts.

of participants' accuracy (p , 0.02). Analysis of the beta

coef®cients indicated that participants with lower initial

scores had a greater improvement than participants with

higher initial scores.

In order to ensure the homogeneity of participant's accu-

racy score at the beginning of the experiment, an F-test was

used to compare initial accuracy scores between the two

groups. No signi®cant difference was found (p . 0.1).

Hypothesis H2 examined whether there was an impact on

participants' speed in accurately detecting potential weak-

nesses in internal control systems after being trained with

the expert system. Because some participants obtained a 0

accuracy score, an ANOVA could not be run directly to test

if there was a signi®cant difference in the mean of time per

accuracy score between the sessions (i.e., the time cannot be

divided by zero). In order to solve this problem, a positive

theoretical minimum score (20) was added to each partici-

pant's accuracy score. The result is referred to as an adjusted

time per accuracy score. The results are summarized in

Table 5 and indicate that participants in the ES group had

signi®cantly greater speed (i.e., the improvement in their

performances in Session IV vs Session I) in accurately

detecting internal control weaknesses than participants in

the NDA group (3.67 vs 1.18, respectively, with

p , 0.0001). This result suggests that using the expert

system as a training tool did affect participants' ef®ciency

in a positive direction.

For the covariates, as with the accuracy score, only the

initial decision time was found to affect the improvement

of participants' decision time (p , 0.0001). Analysis of

the beta coef®cients indicated that participants with

lower initial decision time had a greater improvement

than participants with higher initial decision time. In

order to ensure the homogeneity of participant's decision

time at the beginning of the experiment, an F-test was

used to compare their initial speeds. No signi®cant differ-

ence was found (p . 0.2).

Hypotheses H3a and H3b examined the difference

between the ES group and the NDA group regarding parti-

cipants' satisfaction with performance. The results are

summarized in Tables 6 and 7. On average, participants in

the ES group were more satis®ed with their accuracy, but

the difference between groups was not statistically signi®-

cant (3.94 vs 3.00, respectively, p . 0.1). However, the

result shows that participants in the ES group were signi®-

cantly more satis®ed with their speed than participants in the

NDA group (4.04 vs 2.87, respectively, p , 0.05).

Hypotheses H4a and H4b examined the difference

between the ES and the NDA group regarding participants'

perceptions of the task. The results are summarized in

Tables 8 and 9 and indicate that there was no signi®cant

difference between the ES and the NDA group regarding

attitude on the dif®culty of the task (3.65 vs 2.93, respec-

tively, p . 0.17). These ®ndings might be attributed to

participants in the ES group using the expert system in

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 281

Table 5

Improvement in decision time

Source DF F-value P-valuea

Panel A. Analysis of covariance

Experimental group 1 6.205 , 0.0001

Initial score in Session I 1 4.109 , 0.0001

Percentage of management

duties

1 0.007 0.35

Years of works 1 0.951 0.43

Error 59

Total 63b

Panel B. Adjusted mean scores

Expert system (ES group) 3.67

No decision aid (NDA group) 1.18

a One-tailed directional test. No signi®cant interaction effects beyond

level 1 were found.b One participant had an emergency call and left while performing case

II. His data was, therefore, taken out.

Table 6

Participants satisfaction with accuracy

Source DF F-value P-valuea

Panel A. Analysis of covariance

Experimental group 1 6.205 0.10

Initial score in Session I 1 4.109 0.002

Percentage of management

duties

1 0.007 0.12

Years of works 1 0.951 0.16

Error 59

Total 63b

Panel B. Adjusted mean scores

Expert system (ES group) 3.94

No decision aid (NDA group) 3.00

a One-tailed directional test. No signi®cant interaction effects beyond

level 1 were found.b One participant had an emergency call and left while performing case

II. His data was, therefore, taken out.

Table 4

Improvement in accuracy score

Source DF F-value P-valuea

Panel A. Analysis of covariance

Experimental group 1 6.205 0.008

Initial score in Session I 1 4.109 0.02

Percentage of management

duties

1 0.007 0.47

Years of works 1 0.951 0.17

Error 59

Total 63b

Panel B. Adjusted mean scoresb

Expert system (ES group) 11.50

No decision aid (NDA group) 3.67

a One-tailed directional test. No signi®cant interaction effects beyond

level 1 were found.b One participant had an emergency call and left while performing case

II. His data was, therefore, taken out.

Sessions II and III and losing some con®dence when

performing the task without an expert system. The results

in Table 9 indicate that, on average, participants in the ES

group perceived that the task as more interesting than parti-

cipants in the NDA group (4.18 vs 2.87, respectively,

p , 0.05).

5. Conclusions and direction for future research

In the future, it is likely that operations managers will

become more involved in performing reviews of internal

control systems as operating processes are re-engineered

and control systems for those processes are re-evaluated.

Regarding operating processes, such operations managers

are likely to have more expertise than student-novices and

external auditors; however, because of their educational

background, they are likely to have less fundamental knowl-

edge of accounting. This research is the initial investigation

of the use of expert systems to assist managers who are not

auditing experts in detecting potential internal control weak-

nesses.

The major ®nding of this study is that after being trained

with the expert system, participants in the ES group signi®-

cantly outperformed participants in the NDA group on a

variety of dimensions. This indicates that it is feasible for

operations managers to acquire internal control evaluation

knowledge via the use of expert systems. Such systems can

help managers detect the weaknesses in their organizations'

internal control systems more effectively and more ef®-

ciently. In addition, the results show that the system can

help improve the users' satisfaction with their performances

and perceptions of internal control evaluation tasks. On

average, the users in the ES group reveal that they are

more satis®ed with their accuracy than participants in the

NDA group. The users in the ES group also perceive that the

task was easier and more interesting than those in the NDA

group.

These ®ndings suggest that installing such a system

may help managers maintain an effective internal

control system, thus providing more reliable ®nancial

data and better safeguarding assets. Such systems

could enable organizations to save time and money by

allowing weaknesses in internal control systems to be

detected and resolved more quickly. The systems could

also bene®t auditing ®rms by facilitating more reliable

internal control in client ®rms, thereby reducing planned

detection risk (i.e., less work and time).

The scope of this research study may limit general-

izability of the results in several respects. First, this

research concentrates only on the evaluation of controls

commonly found in the sales and collection cycle.

Second, it investigates internal control systems

commonly found in the merchandising industry. It is

not expected to handle novel (uncommonly different)

accounting systems. Third, the knowledge of the expert

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291282

Table 7

Participants satisfaction with speed

Source DF F-value P-valuea

Panel A. Analysis of covariance

Experimental group 1 6.205 0.05

Initial score in Session I 1 4.109 0.02

Percentage of management

duties

1 0.007 0.08

Years of works 1 0.951 0.10

Error 59

Total 63b

Panel B. Adjusted mean scores

Expert system (ES group) 4.04

No decision aid (NDA group) 2.87

a One-tailed directional test. No signi®cant interaction effects beyond

level 1 were found.b One participant had an emergency call and left while performing case

II. His data was, therefore, taken out.

Table 8

Participants perception of task dif®culty

Source DF F-value P-valuea

Panel A. Analysis of covariance

Experimental group 1 6.205 0.17

Initial score in Session I 1 4.109 0.17

Percentage of management

duties

1 0.007 0.10

Years of works 1 0.951 0.23

Error 59

Total 63b

Panel B. Adjusted mean scores

Expert system (ES group) 3.65

No decision aid (NDA Group) 2.93

a One-tailed directional test. No signi®cant interaction effects beyond

level 1 were found.b One participant had an emergency call and left while performing case

II. His data was, therefore, taken out.

Table 9

Participants perception of task attractiveness

Source DF F-value P-valuea

Panel A. Analysis of covariance

Experimental group 1 6.205 0.05

Initial score in Session I 1 4.109 0.005

Percentage of management

duties

1 0.007 0.10

Years of works 1 0.951 0.06

Error 59

Total 63b

Panel B. Adjusted mean scores

Expert system (ES group) 4.18

No decision aid (NDA group) 2.87

a One-tailed directional test. No signi®cant interaction effects beyond

level 1 were found.b One participant had an emergency call and left while performing case

II. His data was, therefore, taken out.

system developed for this study is based primarily on

one auditor who is a partner of an international account-

ing ®rm. The resulting system closely represents his

reasoning about internal control evaluation. Thus, the

system's knowledge may be ®rm-speci®c or expert-

speci®c. These limitations point to directions in which

the research presented here can be extended by future

investigations.

Future research might investigate the results of using

expert systems over a long period of time (e.g.,

conducting additional sessions one week or month

after the ®rst four sessions). Researchers might incorpo-

rate additional transaction cycles, industries, or other

auditing functions into the expert system. They might

develop additional expert systems by acquiring expertise

from an internal auditor instead of an external auditor.

Finally, researchers might investigate the feasibility of

integrating this kind of expert system with the compa-

ny's databases.

Appendix A

A.1. Basic internal control activities

Internal control activities are those policies and proce-

dures that management has established to meet its objec-

tives for ®nancial reporting. There are potentially many

such control activities in any entity. The extent of separation

of duties depends heavily on the size of the organization.

However, in general, the control activities may be classi®ed

into ®ve categories as follows:

A.1.1. Adequate segregation of duties

When one person performs both functions, there is an

excessive risk that person's disposing of the asset for perso-

nal gain and adjusting the records to relieve himself or

herself of responsibility.

For example, if the cashier receives cash and is respon-

sible for data entry for cash receipts and sales, it is possible

for the cashier to take the cash received from a customer and

adjust the customer's account by failing to record a sale or

by recording a ®ctitious credit to the account.

A.1.2. Proper authorization of transactions and accounts

To achieve effectiveness of internal control, appropriate

individuals must authorize transactions. If there is any

person in an organization who could acquire or expend

assets at will, the complete chaos would result.

For example, a merchandising business would require

speci®c authorization to be obtained before incurring a

long-term debt.

A.1.3. Adequate documents and records

Documents and records are the physical objects upon

which transactions are entered and summarized. In order

to prevent fraud and irregularity, certain relevant principles

should be adopted to dictate the proper design and use of

documents and records.

A.1.4. Physical control over assets and records

It is essential to adequate internal control to protect assets

and records. If assets and records are not adequately

protected, they can be stolen, damaged, or lost. In the

event of such an occurrence, the accounting process as

well as normal operations could be seriously disrupted.

The most important type of protective measure for safe-

guarding assets and records is the use of physical precau-

tions. For example, a storeroom for inventory can be used to

guard against pilferage. When the storeroom is under the

control of a competent employee, there is also further assur-

ance that obsolescence is minimized.

A.1.5. Independent check on performance

The need for independent checks arises because an inter-

nal control structures tends to change over time unless there

is a mechanism for frequent review. Personnel are likely to

forget or intentionally fail to follow procedures, or become

careless unless someone observes and evaluates their perfor-

mances. In addition, both fraudulent and unintentional

misstatements are possible, regardless of the quality of the

controls.

Appendix B

B.1. List of internal control weaknesses for sales and

collection cycle of medium-size merchandising

organizations grouped by basic control activities

B.1.1. Adequate segregation of duties

1. Credit department is not independent from sales

department.

2. Credit department is not independent from shipping

department.

3. Credit department is not independent from accounting

department.

4. Finance department is not independent from accounting

department.

5. Collection department is not independent from shipping

department.

6. The person who records sale invoices is the same or

related to the persons who perform the following

functions:

6.1. prepares sales invoices

6.2. prepares credit memo

6.3. records shipment

6.4. records goods returns

6.5. records customers' claims

6.6. receives cash/cheque

6.7. authorizes the voiding of invoices

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 283

7. The person who records credit memos is the same or

related to the persons who perform the following

functions:

7.1. prepares sales invoices

7.2. prepares credit memo

7.3. records shipment

7.4. records goods returns

7.5. records customers' claims

7.6. receives cash/cheque

7.7. authorizes the voiding of invoices

8. The person records adjustments to customers' accounts

is the same or related to the persons who perform the

following functions:

8.1. prepares sales invoices

8.2. prepares credit memo

8.3. records shipment

8.4. records goods returns

8.5. records customers' claims

8.6. receives cash/cheque

8.7. authorizes the voiding of invoices

9. The person reconciles customers' account balances with

control balances is the same or related to the persons

who perform the following functions:

9.1. records accounts receivable

9.2. prepares customers' control accounts

9.3. receives cash/cheque

10. The person records general ledgers is the same or

related to the persons who perform the following

functions:

10.1. records accounts receivable

10.2. receives cash/cheque

11. The person who receives cash/cheque is the same or

related to the persons who records cash/cheque.

12. The person who opens mails is the same or related to the

persons who perform the following functions:

12.1. receives cash/cheque

12.2. records cash/cheque

12.3. records accounts receivable

12.4. records general ledger

B.1.2. Proper authorization of transactions and accounts

1. Sales orders are not approved by a sale manager.

2. Sales invoices are not approved by both a sales manager

and a credit manager.

3. Credit memos are not approved by both a sales manager

and a credit manager.

4. Changes in payment conditions memos are not

approved by both a sales manager and a credit manager.

5. Debit notes are not approved by both a credit manager

and an accounting manager/controller.

6. Adjustments to customers' account memos are not

approved by both a credit manager and an accounting

manager/controller.

7. Sales invoices with special terms and discounts are not

approved by both a credit manager and a managing

director.

8. Speci®c sales invoices (e.g., with substantial amount)

are not approved by both a credit manager and a mana-

ging director.

9. Uncollectable account memos are not approved by both

a credit manager and a managing director.

10. Write-off account memos are not approved by both a

credit manager and a managing director.

11. Sales orders are not approved in writing.

12. Sales invoices are not approved in writing.

13. Credit memos are not approved in writing.

14. Changes in payment conditions memos are not

approved in writing.

15. Debit notes are not approved in writing.

16. Adjustments to customers' accounts memos are not

approved in writing.

17. Uncollectable accounts memos are not approved by in

writing.

18. Write-off accounts memos are not approved in writing.

B.1.3. Adequate documents and records

1. There is no space for an authorized signature in a sales

order.

2. There are no spaces for authorized signatures in a sales

invoice.

3. There are no spaces for authorized signatures in a credit

memo.

4. There are no spaces for authorized signatures in a

changes in payment conditions memo.

5. There are no spaces for authorized signatures in a debit

note.

6. There are no spaces for authorized signatures in a

adjustments to customers' accounts memo.

7. There is no space for authorized signatures in an

uncollectable accounts memo.

8. There is no space for authorized signatures in a write-off

accounts memo.

9. There is no space for a customer's signature in a

delivery order to acknowledge receipt of goods.

10. Sales orders are not prenumbered consecutively.

11. Sales invoices are not prenumbered consecutively.

12. Credit memos are not prenumbered consecutively.

13. Changes in payment conditions memos are not

prenumbered consecutively.

14. Debit notes are not prenumbered consecutively.

15. Adjustments to customers' accounts memos are not

prenumbered consecutively.

16. Uncollectable accounts memos are not prenumbered

consecutively.

17. Write-off accounts memos are not prenumbered

consecutively.

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291284

18. Delivery orders are not prenumbered consecutively.

19. Cash receipts are not prenumbered consecutively.

20. There is no registration book for controlling customers'

purchase orders.

21. Customer purchase order registration book does not

contain receiving purchase order date, purchase order

number, customer name, and contact person name.

22. There is no stock card for controlling inventory in and

out.

23. A stock card does not contain date, reference number of

instruction document (e.g., delivery order), item

description, and quantity.

24. There is no policy that all customer purchase orders

(both written and verbal) must be transcribed into a

uniform internal sales order form.

25. There is no aging report showing the proportion of

accounts receivable due in 30, 60, and 90 days.

26. A list of collections, which provides details of cash,

cheque, and remittance advice received by mail, is not

prepared when opening mails.

27. All cash/cheque received are not deposited on a daily

basis.

28. All cash/cheque received are not posted to accounts

receivable on a daily basis.

29. Account balance statements are not sent to all customers

informing their current balances at least once a month.

30. The bank reconciliation is not made at least once a

month.

31. Insuf®cient copies (seven copies should be prepared) of

sales invoices to advise related departments of its action

and retain in the sales department's ®le as the evidence

of performance.

32. Original copies of sales invoices are not sent to

customers for their references.

33. Copies of sales invoices are not distributed to appropri-

ate departments as follows:

² the original to a customer

² one copy retained in sales department

² one copy to credit department

² one copy to accounting department

² one copy to storeroom department

² two copies to shipping department

34. Insuf®cient copies of credit memos for cash discount

(four copies) to advise related departments of its action

and retain in the ®nance department ®le as the evidence

of performance.

35. Insuf®cient copies of credit memos for goods returns

(six copies) to advise related departments of its action

and retain in receiving department's ®le as the evidence

of performance.

36. Original copies of credit memos are not sent to

customers for their references.

37. Copies of credit memos for cash discount are not

distributed to appropriate departments as follows:

² the original to a customer

² one copy retained in ®nance department

² one copy to credit department

² one copy to accounting department

38. Copies of credit memos for goods returns are not

distributed to appropriate departments as follows:

² the original to a customer

² one copy retained in receiving department

² one copy to credit department

² one copy to accounting department

² one copy to ®nance department

² one copy to storeroom department

39. Insuf®cient copies (four copies should be prepared) of

changes in payment conditions memos to advise related

departments of its action and retain in the credit depart-

ment's ®le as the evidence of performance.

40. Original copies of changes in payment conditions

memos are not sent to customers for their references.

41. Copies of changes in payment conditions memos are not

distributed to appropriate departments as follows:

² the original to a customer

² one copy retained in credit department

² one copy to ®nance department

² one copy to sales department

42. Insuf®cient copies (®ve copies should be prepared) of

debit notes to advise related departments of its action

and retain in the credit department's ®le as the evidence

of performance.

43. Original copies of debit notes are not sent to customers

for their references.

44. Copies of debit notes are not distributed to appropriate

departments as follows:

² the original to a customer

² one copy retained in credit department

² one copy to accounting department

² one copy to ®nance department

² one last copy to sales department

45. Insuf®cient copies (®ve copies should be prepared) of

adjustments to customers' accounts memos to advise

related departments of its action and retain in the credit

department's ®le as the evidence of performance.

46. Original copies of adjustments to customers' accounts

memos are not sent to customers for their references.

47. Copies of adjustments to customers' accounts memos

are not distributed to appropriate departments as

follows:

² the original to a customer

² one copy retained in credit department

² one copy to accounting department

² one copy to ®nance department

² one copy to sales department

48. Insuf®cient copies (four copies should be prepared) of

uncollectable accounts memos to advise related depart-

ments of its action and retain in the credit department's

®le as the evidence of performance.

49. Copies of uncollectable accounts memos are not

distributed to appropriate departments as follows:

² the original retained in credit department

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 285

² one copy to accounting department

² one copy to ®nance department

² one copy to sales department

50. Insuf®cient copies (four copies should be prepared) of

write-off accounts memos to advise related departments

of its action and retain in the credit department's ®le as

the evidence of performance.

51. Copies of write-off accounts memos are not distributed

to appropriate departments as follows:

² the original retained in credit department

² one copy to accounting department

² one copy to ®nance department

² one last copy to sales department

B.1.4. Physical control over assets and records

1. The of®ce used to keep sales documents (e.g., sales

orders, sales invoices, credit memos, debit memos,

etc.) is unlocked after hours.

2. The cabinets used to keep sales documents (e.g., sales

orders, sales invoices, credit memos, debit memos, etc.)

are unlocked after hours.

3. The of®ce used to keep books of accounts (e.g., sales

journal books, accounts receivable books, control

account books, etc.) is unlocked after hours.

4. The cabinets used to keep books of accounts (e.g., sales

journal books, accounts receivable books, control

account books, etc.) are unlocked after hours.

5. Persons other than accounting manager/controller and

accounts receivable clerks have key to open cabinets

used to keep sales journal books and accounts

receivable books.

6. Persons other than accounting manager/controller and

general ledger clerks have key to open cabinets used to

keep control account books.

7. Storeroom is not under supervision all time during the

storeroom hours.

8. Storeroom is unlocked after hours.

9. Persons other than storeroom staff have key to open the

storerooms.

10. Cashier's drawers are not locked all time.

11. Persons other than cashiers have key to open cashier's

drawers.

B.1.5. Independent check on performances

1. Credit-worthy customers' purchase orders are not

reviewed at least once a month.

2. Overdue accounts are not reviewed at least once a

month.

3. A list of collections by mail is not reviewed against a

cash book by a person other than the person who open

mails, the person who receives cash, and the person who

records cash at least once a month.

4. Cash/cheque on hand is not counted against a cash book

by a person other than the person who receives cash/

cheque on a surprise basis.

5. Inventories in stock are not counted against the record

in a book of accounts by a person other than the store-

room staff at least once a year.

6. Sales invoices are not checked by a person other than

the person who prepares them or salesperson that prices

and trade discounts are based on approved sales orders

or price lists.

7. Sales invoices are not checked by a person other than

the person who prepares them or the person who records

shipments that quantities are based on actual records of

goods shipped.

8. Customers' names in sales invoices are not checked by a

persons other than the person who prepares them

against the customers' master ®le or customers'

purchase orders.

9. The extensions and additions in sales invoices are not

recomputed by a person other than the person who

prepares them.

10. Credit memos are not checked by a person other than

the persons who prepares them that prices and cash

discount conditions are agreed with original sales

invoices.

11. Credit memos issued for goods returned are not checked

by a person other than the person who prepares them or

the person who records goods returns that quantities are

based on actual records of goods returned.

12. Customers' names in credit memos are not checked by a

persons other than the person who prepares them

against the original sales invoices.

13. The extensions and additions in credit memos are not

recomputed by a person other than the person who

prepares them.

Appendix C

C.1. Case a: PC mart company

PC Mart Company (PCM) is a Kentucky merchandiser

of personal computers and related equipment such as

monitors, printers, modems, hard disk drives, diskettes,

and business software package. The company is estab-

lished in 1960. During its several years of existence,

PCM has done very well. Not only have sales grown

fairly rapidly, but the product lines have been expanded

and the employees have increased in number. Currently,

the company employs ®fty persons, ranging from the

managing director to a janitor.

C.1.1. Organization and functions

PCM has nine departments, which are independent from

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291286

each other. The company's managing director is Mr.

Richard Gere. Reporting to him are nine managers as

follows:

1. Ms. Kim BarsingerÐThe Accounting Manager

2. Mr. Jackie ChanÐThe Administrative Manager

3. Mr. Tom CruiseÐThe Credit Manager

4. Ms. Julia RobertÐThe Treasurer

5. Mr. Kerk RusselÐThe Purchasing Manager

6. Mr. Robert RedfordÐThe Receiving Manager

7. Mr. Charlie SheanÐThe Sales Manager

8. Mr. Harrison FordÐThe Shipping Manager

9. Mr. Clint EastwoodÐThe Storeroom Manager

C.1.2. Accounting department

Ms. Kim Barsinger oversees clerks and bookkeepers who

process transactions and maintain the accounting records.

Reporting to her are all accounting clerk and bookkeeper as

follows:

² Ms. Barbara Bailey: Reconciles Account Balances and

Control Accounts; Prepares Bank Reconciliation;

Records Customers' Claims

² Mr. Richard Cosner: Prepares Sales Journal; Records

Sales Invoices

² Mr. Darick Lykins: Prepares Purchase Journal; Records

Purchase Orders

² Mr. Andrew Mabson: Records Credit Memos and

Adjustments to Customers' Accounts

² Ms. Karen Stainley: Prepares Cash Receipts Journal,

Cash Disbursement Journal, Accounts Receivable

Subsidiary Ledger, and Accounts Payable Subsidiary

Ledger; Records Cash, Cheque and Accounts Receivable

² Ms. Victoria Parks: Prepares General ledger, Trial

Balances, and Financial Statements; Records General

Ledgers

² Ms. Martha Kastner: Prepares General Journal and

Payroll Register; Records General Expenses, Payrolls

The accounting of®ce is always locked after hours to

prevent unauthorized access to the books of accounts. In

addition, there are two cabinets in the of®ce. The ®rst one

is used to keep sales journal and accounts receivable books.

The second is used to keep general ledger, trial balances,

and ®nancial statements books. These two cabinets are kept

locked after hours. Only Ms. Kim Barsinger (the accounting

manager) and Ms. Karen Stainley (the accounts receivable

clerk) have the keys to open these two cabinets. Ms. Victoria

Parks (the general ledger clerk) only has the key to open the

second cabinet.

C.1.3. Administrative department

Mr. Jackie Chan serves as an administrative manager,

with responsibilities for personnel, insurance, budgeting,

cost analysis, and systems and procedures.

C.1.4. Credit department

Mr. Tom Cruise is in charge of the credit department. The

department handles the preparations of debit notes, changes

in payment conditions memos, adjustments to customers'

account memos, uncollectable account memos, and write-

off accounts memos. Mr. Tom Cruise also reviews all over-

due account at least once a month in order to keep track on

the performances of collections and to ensure that all follow-

up procedures are being done properly and ef®ciently.

At the end of each month, the credit department's staff

will send the account balance statements to all active custo-

mer informing their current balances with the company.

They also prepare an aging report showing the proportion

of accounts receivable due in 30, 60, and 90 days. Each

month, this report is reviewed for setting up the uncollect-

able accounts. The uncollectable accounts memos must be

approved by Mr. Richard Gere (the managing director) and

Mr. Tom Cruise (the credit manager) in writing in order to

ensure that only bad accounts with high uncollectable

potential are transferred to the provision for uncollectable

accounts. The memos also have spaces provided for

authorized signatures and are prenumbered consecutively.

Every two months, the uncollectable accounts are

reviewed to check the performances of collections. If all

follow-up procedure is performed but the accounts are

still uncollectable, the write-off accounts memos will be

issued. These memos must be approved by Mr. Richard

Gere (the managing director) and Mr. Tom Cruise (the

credit manager) in writing. The memos have spaces

provided for authorized signatures and are prenumbered

consecutively.

Four copies of both uncollectable accounts memos and

write-off accounts memos are prepared and distributed as

follows:

² The original copy is retained in credit department's ®le

for their own record,

² One copy is sent to accounting department,

² One copy is sent to ®nance department, and

² The last copy is sent to sales department

C.1.5. Finance department

Ms. Julia Robert performs the duties of treasurer. Report-

ing to her are cashiers and ®nance clerks. This department

handles all collection from customers.

C.1.6. Purchasing department

Mr. Kerk Russel is in charge of purchasing activities and

supervises several purchasing staffs who specialize in the

various product lines. The department receives the purchase

requisitions from other departments and further processes

the acquisition of goods or any other assets.

C.1.7. Receiving department

Mr. Robert Redford handles all receiving function which

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 287

includes receiving of goods from suppliers and also receiving

the goods returned from customers. Ms. Julie Garwood, a

receiving clerk, is in charge for the record of all goods received

from suppliers and all goods returned from customers.

C.1.8. Sales department

Mr. Charlie Shean directs the operations of the sales func-

tions. There are ten salespersons and two sales clerks under

his supervision. Mr. Thomas Newston and Mr. Jimmy Carl-

ton, the sales clerks, are responsible for preparing sales

invoices. The of®ce is always locked after hours to prevent

unauthorized access to sales documents. In addition, in order

to reduce the opportunity of missing documents, the cabinets

used to keep sales documents are also locked after hours.

C.1.9. Shipping department

Mr. Harrison Ford is in charge of the shipping depart-

ment. This department maintains all shipping operation.

Mr. Stephen Bendon, the shipping clerk, is in charge for

the record of each shipment.

C.1.10. Storeroom department

Mr. Clint Eastwood maintains the storeroom operations.

The storeroom is under the supervision of at least one store-

room staff all time during the store hours and is always

locked after hours. Only storeroom staffs have the key to

open the storeroom. At the end of each year, the inventory in

stock will be counted against the record in the books of

accounts by the person other than the storeroom staff in

order to verify the actual amount of inventory in stock as

well as to alert the storeroom staff to take a good care of

inventory in and out.

C.2. Sales and collections procedures

The company sells its products for cash or on credit. It has

about 200 credit customers. The company acquires products

from 20 suppliers. The perpetual inventory method for

recording purchases is employed by PCM. Most products

are sold at established prices, but trade discounts are

allowed on occasion to small businesses and professional

®rms.

PCM maintains its cash in two bank accounts. There is

one account for general funds and one for payrolls. The

company also has petty cash and cashier drawer funds.

Among the resources that it owns are the of®ce building,

storeroom, the land on which these buildings are located, the

furniture and ®xtures in each of the building, cash registers,

other of®ce equipment, and two trucks for delivering

ordered merchandise.

Sales and collections procedures are divided into seven

groups of functions. The following describes all the proce-

dure performed by the company's employees pertaining to

the sales and collections cycle.

C.2.1. Receiving customers' purchasing orders

Customers' purchase orders are received by salespersons

either in person, by phone, or by mail. Normally, ®fteen to

twenty orders are received each day. If a customer's

purchase order is received by mail, a salesperson will

forward such order to a sales clerk for preparing a sales

invoice. However, if an order is received in person or by

phone, he/she must transcribe it into a uniform sales order

before further processing.

If cash in full is paid by the customer at the time of the

sale, the sales order copy is marked paid, and the sale is rung

up on a cash register. The customer also receives the cash

receipt tape from a cashier. If no cash or if only a partial

payment (i.e., a deposit) is received at the time of the sale,

the sale is treated as a credit sale. The cash/cheque received

each day via the cash registers is checked at the end of the

day, then combined with the cash/cheque received by mail

and deposited into the bank on a daily basis. The cashiers'

drawers are locked all time. Only Ms. Kathy Mcdonald and

Ms. Deborah Adams, the company's cashiers, have the key

to open the drawers.

The company also has a customers' purchase order regis-

tration book which contains purchase order receiving date,

purchase order number, customer name, and the name of

contact person. In addition, the credit-worthy customers'

purchase orders are reviewed monthly by Mr. Charlie

Shean (the sales manager) in order to ensure that such orders

are handled properly and ef®ciently.

C.2.2. Preparing sales order forms

All sales order must be approved by Mr. Charlie Shean

(the sales manager) in writing prior to further processing.

The sales orders have a space provided for the sales

manager's authorized signature and are also prenumbered

consecutively.

C.2.3. Authorizing credit terms and discounts and preparing

sales invoices

After approval is granted for a sale, seven copies of a

sales invoice are prepared by the sales clerks, Mr. Thomas

Newston and Mr. Jimmy Carlton. These sales invoices must

also be approved by Mr. Charlie Shean (the sales manager)

in writing in order to ensure that the credit sales are not

approved to a poor-credit customer. Then, these copies are

distributed as follows:

² The original copy is sent to a customer as a billing and

also for his/her record.

² One copy is retained in sales department's ®le for their

own record,

² One copy is sent to credit department,

² One copy is sent to ®nance department,

² One copy is sent to store department, and

² The last two copies are sent to shipping department

Any voiding of sales invoices must be authorized by Mr.

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291288

Richard Cosner. Occasionally, a special term and discount

may be given to speci®c customers. This type of sales

invoices including some speci®c invoices (e.g., an invoice

with substantial amount) must be approved by Mr. Tom

Cruise (the credit manager) and Mr. Richard Gere (the

managing director) in order to ensure that such sales are

in conform with the company's sales and credit policies.

Sales invoices have spaces provided for authorized signa-

tures and are prenumbered consecutively.

Typically, Ms. Barbara Bailey (the chief accountant) will

review sales invoices against sales orders and records of

shipments in order to verify if the prices and trade discounts

are based on approved sales orders and the quantities are

agreed with the actual records of goods shipped. She also

checks the customers' names against the customer master

®le and recomputes the extensions and additions in sales

invoices.

C.2.4. Preparing credit memos for cash discounts or goods

returns, changes in payment conditions, debit notes, and

adjustments to customers' accounts

The company allows returns and allowances within a

speci®ed period from the dates of sales. It also allows

cash discounts on sales if paid in full within ten days, other-

wise, it expects full payment within thirty days. In both

cases, a credit memo will be prepared by Mr. Kevin Cosner

or Mr. Dennis Justice. All credit memos must be approved

by Mr. Charlie Shean (the sales manager) and Mr. Tom

Cruise (the credit manager) in writing before further proces-

sing. Credit memos have spaces provided for authorized

signatures and are prenumbered consecutively.

In case of cash discounts, four copies of credit memo are

prepared and distributed as follows:

² The original copy is sent to a customer for his/her record.

² One copy is retained in ®nance department's ®le for their

own record,

² One copy is sent to credit department, and

² The last copy is sent to accounting department

In case of goods returns, six copies of credit memo are

prepared and distributed as follows:

² The original copy is sent to a customer for his/her record.

² One copy is retained in receiving department's ®le for

their own record,

² One copy is sent to credit department,

² One copy is sent to accounting department,

² One copy is sent to ®nance department, and

² The last copy is sent to storeroom department

Typically, Ms. Barbara Bailey (the chief accountant) will

review credit memos against sales invoices and records of

goods returns in order to verify if the prices and cash

discount conditions are agreed with the original sales

invoices and the quantities are based on the actual records

of goods returned. She also checks the customers' names

against the original sales invoices and recomputes the exten-

sions and additions in credit memos.

A change in payment conditions, which requires the issu-

ance of debit notes, is sometimes allowed to a customer.

Such change must be approved by Mr. Charlie Shean (the

sales manager) and Mr. Tom Cruise (the credit manager) in

writing. The changes in payment conditions memos are

prenumbered consecutively and also have spaces provided

for authorized signatures. Four copies of the memos are

prepared and distributed as follows:

² The original copy is sent to a customer for his/her record.

² One copy is retained in credit department's ®le for their

own record,

² One copy is sent to ®nance department, and

² The last copy is sent to sales department

An increase in sales amounts as well as other adjustments

to customers' accounts must be approved by Mr. Tom

Cruise (the credit manager) and Ms. Kim Barsinger (the

accounting manager) in writing prior to further processing

in order to prevent unauthorized changes to customers'

accounts. Both debit notes and adjustments to customers'

accounts memos have spaces provided for authorized signa-

tures and are prenumbered consecutively. Five copies of

them are prepared and distributed as follows:

² The original copy is sent to a customer for his/her record.

² One copy is retained in credit department's ®le for their

own record,

² One copy is sent to accounting department,

² One copy is sent to ®nance department, and

² The last copy is sent to sales department

C.2.5. Preparing execution instruction, withdrawing goods

from stock, and shipping goods

After receiving a copy of sales invoices from sales depart-

ment, a storeroom staff prepares goods for shipment. If an

order cannot be completely ®lled because there is an insuf-

®ciently quantity of an item on hand, the storeroom staff

uses the copy of the sales invoice to prepare a back order

and sends it to the purchasing manager for further proces-

sing.

After receiving two copies of sales invoices from sales

department, a shipping staff will contact the storeroom

department for the goods to be shipped and, then ships the

goods to customers. One copy of sales invoices is retained in

shipping department for their own record. Another copy,

which may be called a delivery order, is accompanied

with the goods shipped. The delivery order has a space for

the customer's signature and is prenumbered consecutively.

The shipping staff will have the customer signed the deliv-

ery order as the acknowledgment of goods received. Then,

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 289

such delivery order will be forwarded to ®nance department

for further collection processes.

C.2.6. Receiving cash, cheque, and remittance advice by

mail

Ms. Kathy Mcdonald and Ms. Deborah Adams perform

the duty of cashiers. Besides the main duty of receiving

cash, Ms. Kathy Mcdonald is also assigned to open the

mail received each day. Each letter containing a cash remit-

tance is set aside. While opening all mail, they will prepare a

list of collections by mail, which provides the details of all

cash, cheque, and remittance advice received by mail. At the

end of each month, this list of collections will be reviewed

against a cash book by Ms. Julia Robert (the treasurer) in

order to ensure the correctness of cash/cheque recorded as

well as to prevent the lapping on cash/cheque received.

Ms. Kathy Mcdonald and Ms Deborah Adams (the cash-

iers) also prepare a deposit slip, then combine the cash/

cheque received in the mail with the cash/cheque from the

cashiers' drawers and deposit into the bank on a daily basis.

At the end of each day, Ms. Julia Robert (the treasurer) will

count the cash/cheque on hand against the record in cash

book in order to reduce the opportunity of lapping on cash/

cheque.

Ms. Kathy Mcdonald and Ms. Deborah Adams will issue

cash receipts to customers to acknowledge the receipts of

their payments. They also send to pile of remittance noti®-

cations and advises (together with a copy of the deposit slip)

to the accounting clerks for entry into the cash receipts

journal, for posting to the accounts receivable and general

ledgers, and for ®ling.

C.2.7. Recording accounts receivable, sales and general

ledgers

The accounting records are entered and posted manually.

Ms. Karen Stainley (the accounts receivable clerk) will post

all cash/cheque received to an accounts receivable book on a

daily basis.

Ms. Victoria Parks (the general ledger clerk) supervises

the posting to the general ledgers. Then, she prepares trial

balances and ®nancial statements with the aid of an electro-

nic spreadsheet package to compete key ®nancial ratios and

to print hard copy ®nancial reports for the other managers.

Ms. Barbara Bailey (the chief accountant) is responsible

for recording all customers' claim and reconciling custo-

mer's accounts with the control accounts. She also prepares

a bank reconciliation at the end of each month in order to

cross check the ªCash in Bankº account with the transac-

tions provided in the bank statement.

C.3. Relationships of the company's employees

² Ms. Julia Robert is Ms. Julie Garwood's sister

² Mr. Thomas Newston is Mr. Dennis Justice's son

² Mr. Kevin Cosner is Mr. Richard Cosner's brother

² Mr. Jackie Chan and Ms. Karen Stainley are husband and

wife

Appendix D

D.1. Post-experiment questionnaires

Please answer the following questions. Please keep in

mind that the following questions refer ONLY to the last

case study you have done (Table 10).

References

American Institute of Certi®ed Public Accountants (AICPA) (1996). Codi-

®cation of statements on auditing standards, New York: AICPA.

Bailey Jr, A. D., Duke, G. L., Gerlach, J., Ko, C., Meservy, R., & Whinston,

A. B. (1985). TICOM and the analysis of internal controls. Accounting

Review, 60 (2), 186±211.

Biggs, S. F., Messier Jr, W. F., & Hansen, J. V. (1987). A descriptive

analysis of computer audit specialists' decision-making behavior in

advanced computer environments. Auditing: A Journal of Practice &

Theory, Spring, 1±21.

Boritz, J. E. (1985). The effect of information presentation structures on

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291290

Table 10

1. On a scale of 1 (very dif®cult) to 7 (very easy), how dif®cult was it to do the case study?

1 2 3 4 5 6 7

very dif®cult dif®cult slightly dif®cult neutral slightly easy easy very easy

2. On a scale of 1 (very unsatisfactory) to 7 (very satisfactory), how satis®ed were you with your accuracy in answering the case study?

1 2 3 4 5 6 7

very unsatisfactory unsatisfactory slightly unsatisfactory neutral slightly satisfactory satisfactory very satisfactory

3. On a scale of 1 (very unsatisfactory) to 7 (very satisfactory), how satis®ed were you with your speed in answering the case study?

1 2 3 4 5 6 7

very unsatisfactory unsatisfactory slightly unsatisfactory neutral slightly satisfactory satisfactory very satisfactory

4. On a scale of 1 (very boring) to 7 (very interesting), how interesting was the task you performed?

1 2 3 4 5 6 7

very boring boring slightly boring neutral slightly interesting interesting very interesting

audit planning and review judgments. Contemporary Accounting

Research, 8 (1), 193±218.

Borthick, A. F., & West, O. D. (1987). Expert systems-A new tool for the

professional. Accounting Horizons, March, 9±16.

Committee of Sponsoring Organizations of Treadway Commission

(COSO) (1992). Internal control-integrated framework, New York:

AICPA.

Cormier, S., & Hagman, J. (1987). Transfer of learning, San Diego, CA:

Academic Press.

Cummings, B. K., & Apostolou, N. G. (1987). Expert systems in auditing:

an emerging technology. Internal Auditing, 3 (2), 3±10.

Cummings, B. K., Lauer, J., & Baker, R. (1988). Expert systems in internal

auditing. Internal Auditing, 4 (1), 49±64.

Curtis, M. B., & Viator, R. E. (2000). An investigation of multidimensional

knowledge structure and computer auditor performance. Auditing: A

Journal of Practice and Theory, 19 (2), 83±103.

Davis, F. D., Bagozzi, R. D., & Warshaw, P. R. (1989). User acceptance of

computer technology: a comparison of two theoretical models.

Management Science, 35 (8), 982±1003.

Eining, M. M., & Dorr, P. B. (1991). The impact of expert system usage on

experiential learning in an auditing setting. Journal of Information

Systems, 5 (1), 1±16.

Fedorowicz, J., Oz, E., & Berger, P. D. (1992). A learning curve analysis of

expert system use. Decision Sciences, July/August, 797±818.

Gal, G. F. (1985). Using auditor knowledge to formulate data model

constraints: an expert system for internal control evaluation. Ph.D.

Dissertation, Michigan State University.

Gal, G. F., & Steinbart, P. J. (1992). Interface style and training task

dif®culty as determinants of effective computer-assisted knowledge

transfer. Decision Sciences, 23, 128±143.

Graham, L. E., Damens, J., & Ness, G. V. (1991). Developing risk advi-

sorSM: An expert system for risk identi®cation. Auditing: A Journal of

Practice & Theory, 10 (1), 69±96.

Holsapple, C. W., & Whinston, A. B. (1986). Managers' guide to expert

systems, Homewood, IL: Dow Jones-Irwin.

Libby, R. (1995). The role of knowledge and memory in audit judgment. In

R. Ashton & A. Ashton, Judgment and decision making research in

accounting and auditing. Cambridge, MA: Cambridge University

Press.

Lieb, E. B., & Gillease, J. K. (1996). Du Pont uses a decision support

system to select its audit portfolio. Interfaces, 26 (3), 10±21.

Miller, G. A. (1956). The magical number seven, plus or minus two: some

limits on our capacity for processing information. Psychological

Review, 63, 81±97.

Murphy, D. S. (1990). Expert system use and the development of expertise

in auditing: a preliminary investigation. Journal of Information

Systems, Fall, 18±35.

Nelson, M. W., Libby, R., & Bonner, S. E. (1995). Knowledge structure and

the estimation of conditional probabilities in audit planning. Accounting

Review, January, 27±47.

Odem, M. D., & Dorr, P. B. (1995). The impact of elaboration-based expert

system interfaces on de-skilling: An epistemological issue. Journal of

Information Systems, 9 (1), 1±17.

O'Leary, D. E., & Watkins, P. R. (1989). Review of expert systems in

auditing. Expert Systems Review for Business & Accounting, 9 (2),

3±22.

Oz, E. (1989). A study of improvement in decision-making skills through

the use of expert systems. Unpublished Ph.D. Dissertation. Boston

University.

Sharda, R., Barr, S. H., & McDonnell, J. C. (1988). Decision support system

effectiveness: a review and an empirical test. Management Science, 34

(2), 139.

Smith, V. L. (1982). Microeconomic systems as experimental science.

American Economic Review, 72 (5), 923±955.

Viator, R. E., & Curtis, M. B. (1998). Computer auditor reliance on auto-

mated and non-automated controls as a function of training and experi-

ence. Journal of Information Systems, Spring, 19±30.

Vinze, A. S., Karan, V., & Murthy, U. S. (1991). A generalizable knowl-

edge-based framework for audit planning expert systems. Journal of

Information Systems, Fall, 78±91.

Weber, R. (1999). Information systems control and audit, Upper Saddle

River, NJ: Prentice-Hall.

C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 291