transferring auditors’ internal control evaluation knowledge to management
TRANSCRIPT
Transferring auditors' internal control evaluation knowledge tomanagementq
Chuleeporn Changchita, Clyde W. Holsappleb,*, Ralph E. Viatorc
aDepartment of Management Sciences, Henry B. Tippie College of Business, W284 John Pappajohn Business Building, University of Iowa, Iowa City,
IA 52242-1000, USAbSchool of Management, Carol M. Gatton College of Business and Economics, University of Kentucky, Lexington, KY 40506-0034, USA
cArea of Accountancy, College of Business Administration, Texas Tech University, Lubbock, TX 79409, USA
Abstract
Due to both regulatory and competitive forces, attention to an organization's internal controls has increased signi®cantly in the 1990s.
Although management is ultimately responsible for ensuring internal controls are adequate, managers often lack knowledge of internal
control concepts. This study reports on an experiment testing an expert system developed to facilitate the transfer of internal control
knowledge to management. Experimental results indicate that expert systems are viable aids for transferring internal control knowledge
to managers whose work experience is outside of accounting and control systems. q 2001 Elsevier Science Ltd. All rights reserved.
Keywords: Intelligent system; Expert system; Management; Internal controls; Knowledge transfer; Learning theory
1. Introduction
Internal control is de®ned as a process designed to
provide reasonable assurance regarding the achievement
of objectives in the following categories: reliability of ®nan-
cial reporting, effectiveness and ef®ciency of operations,
and compliance with applicable laws and regulation
(AICPA, 1996). Due to both regulatory and competitive
forces, attention to an organization's internal controls has
increased signi®cantly in the 1990s (Weber, 1999). Both
accounting ®rms and researchers have devoted signi®cant
effort to the development of decision aids, especially expert
systems, to assist auditors in internal control evaluations
(Bailey, Duke, Gerlach, Ko, Meservy & Whinston, 1985;
Boritz, 1985; Gal, 1985; Cummings & Apostolou, 1987;
Cummings, Lauer & Baker, 1988; O'Leary & Watkins,
1989; Graham, Damens & Ness, 1991; Vinze, Karan &
Murthy, 1991).
Researchers have long contended that expert systems
could be used not only as decision aids but also to train
non-expert users (Holsapple & Whinston, 1986; Biggs,
Messier & Hansen, 1987; Borthick & West, 1987; Gal &
Steinbart, 1992; Odem & Dorr, 1995). Prior studies found
that subjects who practiced making decisions with the aid of
an expert system were better and quicker at reaching deci-
sions than subjects who practiced without the support of the
expert system (Oz, 1989; Murphy, 1990; Eining & Dorr,
1991; Fedorowicz, Oz & Berger, 1992). However, there
are no previous reported studies of an expert system to
facilitate the transfer of internal control evaluation knowl-
edge to management. There are several reasons why
managers are likely to perform internal control evaluations.
First, establishment and supervision of internal control
systems are the responsibility of management, not external
auditors (COSO, 1992). Second, external auditors do not
have the immediate and detailed insight into the operation
of the internal control systems that management does. And
third, the evaluation of internal control systems should be
treated as a continuous rather than a discrete process, so that
any weaknesses can be prevented or detected as soon as
possible.
Even though the literature has reported that an expert
system can facilitate the transfer of internal control evalua-
tion knowledge to novice subjects (Libby, 1995; Lieb &
Gillease, 1996; Weber, 1999), all such studies used either
auditors or accounting students as subjects. Without
conducting the research, we are reluctant to assume that
having managers as subjects will yield the same positive
result. There are signi®cant reasons why managers who
lack training in accounting systems may be less likely to
Expert Systems with Applications 20 (2001) 275±291PERGAMON
Expert Systemswith Applications
0957-4174/01/$ - see front matter q 2001 Elsevier Science Ltd. All rights reserved.
PII: S0957-4174(00)00066-X
www.elsevier.com/locate/eswa
q Data Availability: Data is available from the authors; however, requests
for data must be speci®c regarding intended use.
* Corresponding author. Tel.: 11-606-257-5236; fax: 11-606-257-8031.
E-mail addresses: [email protected] (C. Changchit),
[email protected] (C.W. Holsapple), [email protected] (R.E. Viator).
acquire internal control evaluation knowledge. First,
managers, including corporate internal auditors, come
from diverse backgrounds, such as operations and informa-
tion systems training, and do not necessarily have an
accounting educational background (Viator & Curtis,
1998). Second, differences in educational background
have been directly linked to differences in knowledge struc-
ture (Curtis & Viator, forthcoming). Studies have shown
that a mismatch between knowledge structure and task
structure can have a detrimental effect on performance
(Nelson, Libby & Bonner, 1995), and differences in knowl-
edge structure are systematically associated with the quality
of performance in reviewing of internal control systems
(Curtis & Viator, forthcoming). Given variations in educa-
tional background, work experience, and knowledge struc-
tures, the transfer of internal control evaluation knowledge
appears to be less likely for managers, as compared to exter-
nal auditors or accounting student novices. Third, managers
may not be willing to learn the concept of the internal
controls, or may not feel comfortable using expert systems.
For example, they may provide incorrect inputs, not under-
stand system outputs, ®nd it dif®cult to work with expert
systems, or resist considering the advice it gives. This
research examines whether managers' use of expert systems
in the evaluation of internal controls can overcome the
barriers identi®ed above, resulting in the transfer of internal
control knowledge to those managers.
2. Background and hypotheses
One of the most fundamental learning concepts is acqui-
sition of knowledge (Cormier & Hagman, 1987). Informa-
tion processing theory uses the computer as a model for
human learning (Miller, 1956). Like the computer, the
human mind takes in information, performs operations to
change its form and content, stores and locates it, and gener-
ates responses to it. Thus, processing involves gathering and
representing information (encoding), holding information
(retention), and getting at the information when needed
(retrieval). Several decision aids were developed to facili-
tate the transfer of the knowledge of one source to another.
In this study, we are concerned with the impact of using
expert systems to facilitate such learning. The following six
hypotheses were designed to examine the value of the
system as follows:
H1: The improvement in accuracy scores of participants
trained with an internal control evaluation expert system
is higher that the improvement in accuracy scores of
participants with no decision aid.
Hypothesis H1 examines whether a participant's accu-
racy in detecting internal control weaknesses improves
after being trained with an expert system. Accuracy of deci-
sion-making is examined as a measure of the system's effec-
tiveness (Sharda, Barr & McDonnell, 1988; Eining & Dorr,
1991). In order to examine the effectiveness of the system,
we compared participants' improvement (Session I vs
Session IV) in the accuracy scores between the Expert
System (ES) group and No Decision Aid (NDA) group.
We hypothesized that, on average, the ES group's improve-
ment in accuracy scores will be higher than the NDA
group's improvement.
H2: The improvement in decision time of participants
trained with an internal control evaluation expert system
is higher than the improvement in decision time of parti-
cipants with no decision aid.
Hypothesis H2 examines whether a participant's decision
time in accurately detecting internal control weaknesses is
reduced after being trained with an expert system. The time
used to make correct decisions (i.e., time per accuracy score)
was examined as a measure of the system's ef®ciency. In
order to measure the improvement in decision time, we
compare the times used by participants in both groups in
accurately detected internal control weaknesses. We hypothe-
sized that, on average, the ES group's improvement in deci-
sion time will be higher than the NDA group's improvement.
Hypotheses H3a, H3b, H4a, and H4b are based on the
Technology Acceptance Model (TAM). The TAM provides
a basis for explaining the determinants of computer accep-
tance and user behavior (Davis, Bagozzi & Warshaw, 1989).
The model suggests that computer usage is determined by
two criteria: perceived usefulness and perceived ease of use.
Perceived usefulness is de®ned as the prospective users'
subjective probability that using a speci®c application
system will increase his or her job performance within an
organizational context. Perceived ease of use is de®ned as
the degree to which the prospective user expects the target
system to be free of effort (Davis et al., 1989).
H3a: Participants trained with an internal control evalua-
tion expert system are more satis®ed with their accuracy
than participants with no decision aid.
H3b: Participants trained with an internal control evalua-
tion expert system are more satis®ed with their speed than
participants with no decision aid.
Hypotheses H3a and H3b examine participants'
perceived usefulness of the expert system in improving
their performances in detecting internal control weaknesses.
The hypotheses test whether there were differences in satis-
faction with performances between participants practicing
with an expert system (ES) and those with no decision aid
(NDA). We hypothesized that, on average, participants in
the ES group would be more satis®ed with their accuracy
and speed than participants in the NDA group.
H4a: Participants trained with an internal control evalua-
tion expert system perceive that performing tasks requires
C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291276
less effort, compared to participants with no decision aid.
H4b: Participants trained with an internal control evalua-
tion expert system perceive that performing tasks is more
interesting, compared to participants with no decision aid.
Hypotheses H4a and H4b examine participants'
perceived usefulness of the expert system in improving
their attitudes toward the internal control evaluation tasks.
The hypotheses test whether there were differences in parti-
cipants' perceptions of the task between the ES group and
the NDA group. We hypothesized that, on average, partici-
pants in the ES group would perceive that the evaluation of
internal controls requires less effort and is more interesting
than participants in the NDA group.
3. Research methodology
To examine the impact of using an expert system to facil-
itate the transfer of the auditor's internal control evaluation
knowledge to management, an experiment was conducted.
The research design consisted of two treatment groups,
three covariates, and six dependent variables. The two treat-
ment groups were subjects with an expert system (ES) and
those with no decision aid (NDA). Three covariates were
measured, including the initial accuracy score from Session
I, percentage of management duties, and years of work. The
six dependent variables included subjects' accuracy, deci-
sion time, satisfaction with accuracy, satisfaction with
speed, perception of task dif®culty, and perception of task
attractiveness.
3.1. Subjects
Because the main purpose of this study is to investigate
whether an expert system can be devised to help managers
more ef®ciently and/or more effectively detect potential
weaknesses of internal control systems in their organiza-
tions, the subjects were persons with managerial responsi-
bilities. These managers were drawn from client ®rms of
three international accounting ®rms. Lists of potential
subjects were obtained from these accounting ®rms. One
hundred and forty subjects were randomly selected from
those lists (one hundred for the experimental group and
forty for a control group). To gain subjects' cooperation,
the invitation letter explained clearly the importance of
the study and assured con®dentiality. All subjects were
told that they would receive a summary of this study.
The experiment was conducted in an isolated, controlled
environment provided by college laboratory and conference
rooms. There were 50 participants in the experimental group
(50% participation rate) and 15 in the control group (37.5%
participation rate). Each participant served as an internal
control decision maker. Even though some participants
hold positions in accounting and ®nancial areas, telephone
interviews were conducted with these participants to ensure
that they had neither a background in internal control
concepts nor prior hands-on experience with an expert
system. Tables 1±3 show the participants' industries, educa-
tion, average years of work, average percentage of manage-
ment duties, and diversity of management roles.
3.2. Internal control case studies
Four case studies (A, B, C, and D) were generated from
the manipulation of several cues for detecting the potential
C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 277
Table 1
Distribution of subjects across industry
Industries No. of subjects
Consultant 12
Legal 8
Manufacturing 8
Government 11
Insurance 3
Banking 2
Construction 2
Distribution 2
Mining 2
Transportation 2
Education 3
Finance 3
Others 7
Total 65
Table 3
Participants' titles
Title No. of subjects
Finance and Accounting
Manager
9
General Manager 7
Supervisor 10
Director 5
Senior Analyst 4
Assistant Manager 6
Vice President 2
Tax Manager 2
Administrative Manager 6
Legal Manager 2
Chief Accountant 2
Engineer 2
System Analyst 2
Lawyer 2
Marketing Manager 2
Operating Manager 1
Joint Venture Manager 1
Total 65
Table 2
Participants' demographics
Education 55 Bachelor, 10 Master
Average years of work 5.46
Average percentage of management duties 53.7
weaknesses in internal control systems. These cues were
obtained from a review of auditing texts, accounting texts,
and input from accounting professors and experienced audi-
tors. (See Appendix B) The scenario in each case dealt with
the adequacy of internal control over a company's sales and
collection cycle. Also, each case included background infor-
mation about the ®ctitious company and a partial organiza-
tion chart. Each case contained ten potential weaknesses in
the internal control system. Three experienced auditors and
three managers were asked to pilot test these cases to ensure
their similarity with respect to the degree of dif®culty in
detecting the potential internal control weaknesses. Revi-
sions to the cases were made based on feedback provided
(see Appendix C).
3.3. Experimental task
The experiment consisted of four sessions. Prior to the
experiment, the four cases were randomly assigned to the
sessions, yielding the result of Case A being assigned to
Session IV, Case B to Session I, Case C to Session III,
and Case D to Session II. In each session, participants
were asked to play the role of a manager trying to detect
the potential weaknesses of the internal control system
described in the case. The maximum time allowed for
each session was two hours.
In Session I, participants in both groups performed the
case without any decision aid. Because no participant had a
background in internal control concepts, a booklet contain-
ing an overview of basic internal control activities (Appen-
dix A) and a list of possible internal control weaknesses
(Appendix B) were provided in this session. In Sessions II
and III, participants in the NDA group performed the second
and third case studies without the expert system, while parti-
cipants in the ES group were allowed to use the expert
system as a decision aid in detecting internal control weak-
nesses in the assigned case. In Session IV, the expert system
was removed, participants in both groups performed the
fourth case study without any decision aid. A booklet
containing an overview of basic internal control activities
and a list of possible internal control weaknesses were
provided again in this session.
For each session, the time required for decision-making
was recorded and an accuracy score was calculated. In addi-
tion, questionnaires were given at the end of each session to
measure participants' satisfaction with their performances
as well as their perceptions of effort and interest in the task.
This questionnaire was also used to gather data about their
demographics.
A performance-based reward system was used here to
induce optimal participant behavior and reduce problems
due to guessing (Smith, 1982). Participants were told that
their performances across the series of cases would be used
to calculate two scores: an accuracy score and a time per
accuracy score. They were informed that prizes would be
given to those participants attaining the highest accuracy
scores and to those attaining the best time per accuracy
scores. Participants appeared to be genuinely interested in
garnering prizes. In addition, as practicing managers who
agreed to devote the many hours required to this study, their
own senses of commitment and professionalism were
factors in securing their efforts within the experiment.
In arriving at an accuracy score, all points related to
correctly identi®ed weaknesses were summed. In addition,
to prevent the subjects from trying to detect weaknesses by
guessing, they were informed at the beginning of the experi-
ment that one-third of all points for inaccurately identi®ed
weaknesses will be subtracted as the penalty for guessing.
Non-response for a potential weakness results in neither
addition nor subtraction.
3.4. Dependent variables
Two experienced auditors and two accounting professors
evaluated the list of internal control weaknesses previously
described. Each evaluator assigned a score to each weakness
using a scale of 0±10, based on the degree of importance of
each weakness (0Ðthe least important, 10Ðvery impor-
tant). The average of the scores for each weakness (i.e.,
divided by four) was assigned as its importance. In order
to examine the impact of using the expert system to facil-
itate the transfer of knowledge, the improvement in accu-
racy was compared across the two treatment groups:
participants in the ES group versus participants in the
NDA group.
The time used to make decisions accurately (i.e., Time
per Accuracy Score) was examined as a measure of ef®-
ciency. The time per accuracy score was computed as
follows:
Time Used to Perform Task
Accuracy Score
For example, Mr. A correctly detects 5 weaknesses and
incorrectly identi®es 5 weaknesses. The time used to do the
case is 40 min. Assuming that each weakness has 3 points
associated with it. Mr. A's accuracy score is
10� ((5 £ 3) 2 (5 £ 3)/3) and his time per accuracy score
is 40/10, or 4 min per accuracy score.
Four questions (detailed in Appendix D) were asked to
investigate participant satisfaction and perception of the
task. These questionnaires were pretested with ten MBA
students to ensure their clarity and construct validity.
1. On a scale of 1 (very unsatisfactory) to 7 (very satisfac-
tory), how satis®ed were you with your accuracy in
answering the case study?
2. On a scale of 1 (very unsatisfactory) to 7 (very satisfac-
tory), how satis®ed were you with your speed in answer-
ing the case study?
3. On a scale of 1 (very dif®cult) to 7 (very easy), how
dif®cult was it to do the case study?
4. On a scale of 1 (very boring) to 7 (very interesting), how
C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291278
interesting was the task you performed?
3.5. Decision aid
A prototype expert system to train management in inter-
nal control evaluation was constructed. The knowledge of
the system was acquired from several sources including an
experienced auditor, who served as the expert for the
system, auditing texts, accounting texts, and the COSO
framework. The expert was asked to describe, in detail
and step-by-step, the techniques and processes he uses in
evaluating an internal control system. The reasons for each
decision were also acquired in an attempt to develop an
expert system that would be able to emulate both the
expert's knowledge and his reasoning behavior.
Because the system was aimed at training managers in
internal control evaluation, additional features, other than
those included in previously reported expert systems, were
integrated into the system for easy understanding and use by
persons who are not familiar with internal control concepts.
For instance, the system incorporated several diagrams to
direct users to the next steps of detecting internal control
weaknesses. Also, the system prompted users for entering
data relevant to detecting the internal control weaknesses. A
menu was also used to reduce the possibility of typing
errors. The system reports each weakness identi®ed and
why the weakness found is considered important, not just
the adequacy of the overall internal control system within
the organization. Figs. 1±3 present examples of diagrams,
input screens with menus, and system output.
Once a prototype expert system was developed, the
expert was asked to review the expertise that had been
captured in the rule sets. Test cases were used during the
validity review.
4. Results and discussion
In order to assess the homogeneity of subjects' internal
control knowledge between the experimental group and the
control group, an ANOVA was performed on the ®rst
session results to test for differences in Accuracy Score
and in Time per Accuracy Score between the groups. At
an alpha level of 0.05, no signi®cant difference was found
between the groups' initial levels of internal control knowl-
edge.
The experimental data were analyzed as a completely
randomized design with a one-way treatment structure
using the Analysis of Covariance (ANCOVA) technique.
The StatVieww system was used to perform the test and
each participant was an experimental unit. Hypotheses H1
and H2 were tested by calculating performance differences
between Session IV and Session I, for each participant.
Tests of hypotheses H3a, H3b, H4a, and H4b were based
on participants' responses to the questionnaire items at the
end of Session IV. The ANCOVA factors included experi-
mental group and three covariates: initial score, percentage
of management duties, and years of work experience.
C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 279
Fig. 1. Diagram for checking the existence of proper authorization of sales orders, sales invoices, credit memo, and changes in payment conditions memos.
Table 4 presents the ANCOVA results testing H1, regard-
ing participants' accuracy in detecting potential weaknesses
in internal control systems after being trained with the
expert system. The results show that the improvement in
accuracy for participants in the ES group was signi®cantly
higher than those in the NDA group (11.50 vs 3.67, respec-
tively, with p , 0.008). Regarding the covariates, only the
initial score was found to have an effect to the improvement
C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291280
Fig. 2. A menu for selecting the person required to approve sales invoices.
Fig. 3. Report of weakness found for improper authorization of sales invoices of transactions and accounts.
of participants' accuracy (p , 0.02). Analysis of the beta
coef®cients indicated that participants with lower initial
scores had a greater improvement than participants with
higher initial scores.
In order to ensure the homogeneity of participant's accu-
racy score at the beginning of the experiment, an F-test was
used to compare initial accuracy scores between the two
groups. No signi®cant difference was found (p . 0.1).
Hypothesis H2 examined whether there was an impact on
participants' speed in accurately detecting potential weak-
nesses in internal control systems after being trained with
the expert system. Because some participants obtained a 0
accuracy score, an ANOVA could not be run directly to test
if there was a signi®cant difference in the mean of time per
accuracy score between the sessions (i.e., the time cannot be
divided by zero). In order to solve this problem, a positive
theoretical minimum score (20) was added to each partici-
pant's accuracy score. The result is referred to as an adjusted
time per accuracy score. The results are summarized in
Table 5 and indicate that participants in the ES group had
signi®cantly greater speed (i.e., the improvement in their
performances in Session IV vs Session I) in accurately
detecting internal control weaknesses than participants in
the NDA group (3.67 vs 1.18, respectively, with
p , 0.0001). This result suggests that using the expert
system as a training tool did affect participants' ef®ciency
in a positive direction.
For the covariates, as with the accuracy score, only the
initial decision time was found to affect the improvement
of participants' decision time (p , 0.0001). Analysis of
the beta coef®cients indicated that participants with
lower initial decision time had a greater improvement
than participants with higher initial decision time. In
order to ensure the homogeneity of participant's decision
time at the beginning of the experiment, an F-test was
used to compare their initial speeds. No signi®cant differ-
ence was found (p . 0.2).
Hypotheses H3a and H3b examined the difference
between the ES group and the NDA group regarding parti-
cipants' satisfaction with performance. The results are
summarized in Tables 6 and 7. On average, participants in
the ES group were more satis®ed with their accuracy, but
the difference between groups was not statistically signi®-
cant (3.94 vs 3.00, respectively, p . 0.1). However, the
result shows that participants in the ES group were signi®-
cantly more satis®ed with their speed than participants in the
NDA group (4.04 vs 2.87, respectively, p , 0.05).
Hypotheses H4a and H4b examined the difference
between the ES and the NDA group regarding participants'
perceptions of the task. The results are summarized in
Tables 8 and 9 and indicate that there was no signi®cant
difference between the ES and the NDA group regarding
attitude on the dif®culty of the task (3.65 vs 2.93, respec-
tively, p . 0.17). These ®ndings might be attributed to
participants in the ES group using the expert system in
C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 281
Table 5
Improvement in decision time
Source DF F-value P-valuea
Panel A. Analysis of covariance
Experimental group 1 6.205 , 0.0001
Initial score in Session I 1 4.109 , 0.0001
Percentage of management
duties
1 0.007 0.35
Years of works 1 0.951 0.43
Error 59
Total 63b
Panel B. Adjusted mean scores
Expert system (ES group) 3.67
No decision aid (NDA group) 1.18
a One-tailed directional test. No signi®cant interaction effects beyond
level 1 were found.b One participant had an emergency call and left while performing case
II. His data was, therefore, taken out.
Table 6
Participants satisfaction with accuracy
Source DF F-value P-valuea
Panel A. Analysis of covariance
Experimental group 1 6.205 0.10
Initial score in Session I 1 4.109 0.002
Percentage of management
duties
1 0.007 0.12
Years of works 1 0.951 0.16
Error 59
Total 63b
Panel B. Adjusted mean scores
Expert system (ES group) 3.94
No decision aid (NDA group) 3.00
a One-tailed directional test. No signi®cant interaction effects beyond
level 1 were found.b One participant had an emergency call and left while performing case
II. His data was, therefore, taken out.
Table 4
Improvement in accuracy score
Source DF F-value P-valuea
Panel A. Analysis of covariance
Experimental group 1 6.205 0.008
Initial score in Session I 1 4.109 0.02
Percentage of management
duties
1 0.007 0.47
Years of works 1 0.951 0.17
Error 59
Total 63b
Panel B. Adjusted mean scoresb
Expert system (ES group) 11.50
No decision aid (NDA group) 3.67
a One-tailed directional test. No signi®cant interaction effects beyond
level 1 were found.b One participant had an emergency call and left while performing case
II. His data was, therefore, taken out.
Sessions II and III and losing some con®dence when
performing the task without an expert system. The results
in Table 9 indicate that, on average, participants in the ES
group perceived that the task as more interesting than parti-
cipants in the NDA group (4.18 vs 2.87, respectively,
p , 0.05).
5. Conclusions and direction for future research
In the future, it is likely that operations managers will
become more involved in performing reviews of internal
control systems as operating processes are re-engineered
and control systems for those processes are re-evaluated.
Regarding operating processes, such operations managers
are likely to have more expertise than student-novices and
external auditors; however, because of their educational
background, they are likely to have less fundamental knowl-
edge of accounting. This research is the initial investigation
of the use of expert systems to assist managers who are not
auditing experts in detecting potential internal control weak-
nesses.
The major ®nding of this study is that after being trained
with the expert system, participants in the ES group signi®-
cantly outperformed participants in the NDA group on a
variety of dimensions. This indicates that it is feasible for
operations managers to acquire internal control evaluation
knowledge via the use of expert systems. Such systems can
help managers detect the weaknesses in their organizations'
internal control systems more effectively and more ef®-
ciently. In addition, the results show that the system can
help improve the users' satisfaction with their performances
and perceptions of internal control evaluation tasks. On
average, the users in the ES group reveal that they are
more satis®ed with their accuracy than participants in the
NDA group. The users in the ES group also perceive that the
task was easier and more interesting than those in the NDA
group.
These ®ndings suggest that installing such a system
may help managers maintain an effective internal
control system, thus providing more reliable ®nancial
data and better safeguarding assets. Such systems
could enable organizations to save time and money by
allowing weaknesses in internal control systems to be
detected and resolved more quickly. The systems could
also bene®t auditing ®rms by facilitating more reliable
internal control in client ®rms, thereby reducing planned
detection risk (i.e., less work and time).
The scope of this research study may limit general-
izability of the results in several respects. First, this
research concentrates only on the evaluation of controls
commonly found in the sales and collection cycle.
Second, it investigates internal control systems
commonly found in the merchandising industry. It is
not expected to handle novel (uncommonly different)
accounting systems. Third, the knowledge of the expert
C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291282
Table 7
Participants satisfaction with speed
Source DF F-value P-valuea
Panel A. Analysis of covariance
Experimental group 1 6.205 0.05
Initial score in Session I 1 4.109 0.02
Percentage of management
duties
1 0.007 0.08
Years of works 1 0.951 0.10
Error 59
Total 63b
Panel B. Adjusted mean scores
Expert system (ES group) 4.04
No decision aid (NDA group) 2.87
a One-tailed directional test. No signi®cant interaction effects beyond
level 1 were found.b One participant had an emergency call and left while performing case
II. His data was, therefore, taken out.
Table 8
Participants perception of task dif®culty
Source DF F-value P-valuea
Panel A. Analysis of covariance
Experimental group 1 6.205 0.17
Initial score in Session I 1 4.109 0.17
Percentage of management
duties
1 0.007 0.10
Years of works 1 0.951 0.23
Error 59
Total 63b
Panel B. Adjusted mean scores
Expert system (ES group) 3.65
No decision aid (NDA Group) 2.93
a One-tailed directional test. No signi®cant interaction effects beyond
level 1 were found.b One participant had an emergency call and left while performing case
II. His data was, therefore, taken out.
Table 9
Participants perception of task attractiveness
Source DF F-value P-valuea
Panel A. Analysis of covariance
Experimental group 1 6.205 0.05
Initial score in Session I 1 4.109 0.005
Percentage of management
duties
1 0.007 0.10
Years of works 1 0.951 0.06
Error 59
Total 63b
Panel B. Adjusted mean scores
Expert system (ES group) 4.18
No decision aid (NDA group) 2.87
a One-tailed directional test. No signi®cant interaction effects beyond
level 1 were found.b One participant had an emergency call and left while performing case
II. His data was, therefore, taken out.
system developed for this study is based primarily on
one auditor who is a partner of an international account-
ing ®rm. The resulting system closely represents his
reasoning about internal control evaluation. Thus, the
system's knowledge may be ®rm-speci®c or expert-
speci®c. These limitations point to directions in which
the research presented here can be extended by future
investigations.
Future research might investigate the results of using
expert systems over a long period of time (e.g.,
conducting additional sessions one week or month
after the ®rst four sessions). Researchers might incorpo-
rate additional transaction cycles, industries, or other
auditing functions into the expert system. They might
develop additional expert systems by acquiring expertise
from an internal auditor instead of an external auditor.
Finally, researchers might investigate the feasibility of
integrating this kind of expert system with the compa-
ny's databases.
Appendix A
A.1. Basic internal control activities
Internal control activities are those policies and proce-
dures that management has established to meet its objec-
tives for ®nancial reporting. There are potentially many
such control activities in any entity. The extent of separation
of duties depends heavily on the size of the organization.
However, in general, the control activities may be classi®ed
into ®ve categories as follows:
A.1.1. Adequate segregation of duties
When one person performs both functions, there is an
excessive risk that person's disposing of the asset for perso-
nal gain and adjusting the records to relieve himself or
herself of responsibility.
For example, if the cashier receives cash and is respon-
sible for data entry for cash receipts and sales, it is possible
for the cashier to take the cash received from a customer and
adjust the customer's account by failing to record a sale or
by recording a ®ctitious credit to the account.
A.1.2. Proper authorization of transactions and accounts
To achieve effectiveness of internal control, appropriate
individuals must authorize transactions. If there is any
person in an organization who could acquire or expend
assets at will, the complete chaos would result.
For example, a merchandising business would require
speci®c authorization to be obtained before incurring a
long-term debt.
A.1.3. Adequate documents and records
Documents and records are the physical objects upon
which transactions are entered and summarized. In order
to prevent fraud and irregularity, certain relevant principles
should be adopted to dictate the proper design and use of
documents and records.
A.1.4. Physical control over assets and records
It is essential to adequate internal control to protect assets
and records. If assets and records are not adequately
protected, they can be stolen, damaged, or lost. In the
event of such an occurrence, the accounting process as
well as normal operations could be seriously disrupted.
The most important type of protective measure for safe-
guarding assets and records is the use of physical precau-
tions. For example, a storeroom for inventory can be used to
guard against pilferage. When the storeroom is under the
control of a competent employee, there is also further assur-
ance that obsolescence is minimized.
A.1.5. Independent check on performance
The need for independent checks arises because an inter-
nal control structures tends to change over time unless there
is a mechanism for frequent review. Personnel are likely to
forget or intentionally fail to follow procedures, or become
careless unless someone observes and evaluates their perfor-
mances. In addition, both fraudulent and unintentional
misstatements are possible, regardless of the quality of the
controls.
Appendix B
B.1. List of internal control weaknesses for sales and
collection cycle of medium-size merchandising
organizations grouped by basic control activities
B.1.1. Adequate segregation of duties
1. Credit department is not independent from sales
department.
2. Credit department is not independent from shipping
department.
3. Credit department is not independent from accounting
department.
4. Finance department is not independent from accounting
department.
5. Collection department is not independent from shipping
department.
6. The person who records sale invoices is the same or
related to the persons who perform the following
functions:
6.1. prepares sales invoices
6.2. prepares credit memo
6.3. records shipment
6.4. records goods returns
6.5. records customers' claims
6.6. receives cash/cheque
6.7. authorizes the voiding of invoices
C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 283
7. The person who records credit memos is the same or
related to the persons who perform the following
functions:
7.1. prepares sales invoices
7.2. prepares credit memo
7.3. records shipment
7.4. records goods returns
7.5. records customers' claims
7.6. receives cash/cheque
7.7. authorizes the voiding of invoices
8. The person records adjustments to customers' accounts
is the same or related to the persons who perform the
following functions:
8.1. prepares sales invoices
8.2. prepares credit memo
8.3. records shipment
8.4. records goods returns
8.5. records customers' claims
8.6. receives cash/cheque
8.7. authorizes the voiding of invoices
9. The person reconciles customers' account balances with
control balances is the same or related to the persons
who perform the following functions:
9.1. records accounts receivable
9.2. prepares customers' control accounts
9.3. receives cash/cheque
10. The person records general ledgers is the same or
related to the persons who perform the following
functions:
10.1. records accounts receivable
10.2. receives cash/cheque
11. The person who receives cash/cheque is the same or
related to the persons who records cash/cheque.
12. The person who opens mails is the same or related to the
persons who perform the following functions:
12.1. receives cash/cheque
12.2. records cash/cheque
12.3. records accounts receivable
12.4. records general ledger
B.1.2. Proper authorization of transactions and accounts
1. Sales orders are not approved by a sale manager.
2. Sales invoices are not approved by both a sales manager
and a credit manager.
3. Credit memos are not approved by both a sales manager
and a credit manager.
4. Changes in payment conditions memos are not
approved by both a sales manager and a credit manager.
5. Debit notes are not approved by both a credit manager
and an accounting manager/controller.
6. Adjustments to customers' account memos are not
approved by both a credit manager and an accounting
manager/controller.
7. Sales invoices with special terms and discounts are not
approved by both a credit manager and a managing
director.
8. Speci®c sales invoices (e.g., with substantial amount)
are not approved by both a credit manager and a mana-
ging director.
9. Uncollectable account memos are not approved by both
a credit manager and a managing director.
10. Write-off account memos are not approved by both a
credit manager and a managing director.
11. Sales orders are not approved in writing.
12. Sales invoices are not approved in writing.
13. Credit memos are not approved in writing.
14. Changes in payment conditions memos are not
approved in writing.
15. Debit notes are not approved in writing.
16. Adjustments to customers' accounts memos are not
approved in writing.
17. Uncollectable accounts memos are not approved by in
writing.
18. Write-off accounts memos are not approved in writing.
B.1.3. Adequate documents and records
1. There is no space for an authorized signature in a sales
order.
2. There are no spaces for authorized signatures in a sales
invoice.
3. There are no spaces for authorized signatures in a credit
memo.
4. There are no spaces for authorized signatures in a
changes in payment conditions memo.
5. There are no spaces for authorized signatures in a debit
note.
6. There are no spaces for authorized signatures in a
adjustments to customers' accounts memo.
7. There is no space for authorized signatures in an
uncollectable accounts memo.
8. There is no space for authorized signatures in a write-off
accounts memo.
9. There is no space for a customer's signature in a
delivery order to acknowledge receipt of goods.
10. Sales orders are not prenumbered consecutively.
11. Sales invoices are not prenumbered consecutively.
12. Credit memos are not prenumbered consecutively.
13. Changes in payment conditions memos are not
prenumbered consecutively.
14. Debit notes are not prenumbered consecutively.
15. Adjustments to customers' accounts memos are not
prenumbered consecutively.
16. Uncollectable accounts memos are not prenumbered
consecutively.
17. Write-off accounts memos are not prenumbered
consecutively.
C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291284
18. Delivery orders are not prenumbered consecutively.
19. Cash receipts are not prenumbered consecutively.
20. There is no registration book for controlling customers'
purchase orders.
21. Customer purchase order registration book does not
contain receiving purchase order date, purchase order
number, customer name, and contact person name.
22. There is no stock card for controlling inventory in and
out.
23. A stock card does not contain date, reference number of
instruction document (e.g., delivery order), item
description, and quantity.
24. There is no policy that all customer purchase orders
(both written and verbal) must be transcribed into a
uniform internal sales order form.
25. There is no aging report showing the proportion of
accounts receivable due in 30, 60, and 90 days.
26. A list of collections, which provides details of cash,
cheque, and remittance advice received by mail, is not
prepared when opening mails.
27. All cash/cheque received are not deposited on a daily
basis.
28. All cash/cheque received are not posted to accounts
receivable on a daily basis.
29. Account balance statements are not sent to all customers
informing their current balances at least once a month.
30. The bank reconciliation is not made at least once a
month.
31. Insuf®cient copies (seven copies should be prepared) of
sales invoices to advise related departments of its action
and retain in the sales department's ®le as the evidence
of performance.
32. Original copies of sales invoices are not sent to
customers for their references.
33. Copies of sales invoices are not distributed to appropri-
ate departments as follows:
² the original to a customer
² one copy retained in sales department
² one copy to credit department
² one copy to accounting department
² one copy to storeroom department
² two copies to shipping department
34. Insuf®cient copies of credit memos for cash discount
(four copies) to advise related departments of its action
and retain in the ®nance department ®le as the evidence
of performance.
35. Insuf®cient copies of credit memos for goods returns
(six copies) to advise related departments of its action
and retain in receiving department's ®le as the evidence
of performance.
36. Original copies of credit memos are not sent to
customers for their references.
37. Copies of credit memos for cash discount are not
distributed to appropriate departments as follows:
² the original to a customer
² one copy retained in ®nance department
² one copy to credit department
² one copy to accounting department
38. Copies of credit memos for goods returns are not
distributed to appropriate departments as follows:
² the original to a customer
² one copy retained in receiving department
² one copy to credit department
² one copy to accounting department
² one copy to ®nance department
² one copy to storeroom department
39. Insuf®cient copies (four copies should be prepared) of
changes in payment conditions memos to advise related
departments of its action and retain in the credit depart-
ment's ®le as the evidence of performance.
40. Original copies of changes in payment conditions
memos are not sent to customers for their references.
41. Copies of changes in payment conditions memos are not
distributed to appropriate departments as follows:
² the original to a customer
² one copy retained in credit department
² one copy to ®nance department
² one copy to sales department
42. Insuf®cient copies (®ve copies should be prepared) of
debit notes to advise related departments of its action
and retain in the credit department's ®le as the evidence
of performance.
43. Original copies of debit notes are not sent to customers
for their references.
44. Copies of debit notes are not distributed to appropriate
departments as follows:
² the original to a customer
² one copy retained in credit department
² one copy to accounting department
² one copy to ®nance department
² one last copy to sales department
45. Insuf®cient copies (®ve copies should be prepared) of
adjustments to customers' accounts memos to advise
related departments of its action and retain in the credit
department's ®le as the evidence of performance.
46. Original copies of adjustments to customers' accounts
memos are not sent to customers for their references.
47. Copies of adjustments to customers' accounts memos
are not distributed to appropriate departments as
follows:
² the original to a customer
² one copy retained in credit department
² one copy to accounting department
² one copy to ®nance department
² one copy to sales department
48. Insuf®cient copies (four copies should be prepared) of
uncollectable accounts memos to advise related depart-
ments of its action and retain in the credit department's
®le as the evidence of performance.
49. Copies of uncollectable accounts memos are not
distributed to appropriate departments as follows:
² the original retained in credit department
C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 285
² one copy to accounting department
² one copy to ®nance department
² one copy to sales department
50. Insuf®cient copies (four copies should be prepared) of
write-off accounts memos to advise related departments
of its action and retain in the credit department's ®le as
the evidence of performance.
51. Copies of write-off accounts memos are not distributed
to appropriate departments as follows:
² the original retained in credit department
² one copy to accounting department
² one copy to ®nance department
² one last copy to sales department
B.1.4. Physical control over assets and records
1. The of®ce used to keep sales documents (e.g., sales
orders, sales invoices, credit memos, debit memos,
etc.) is unlocked after hours.
2. The cabinets used to keep sales documents (e.g., sales
orders, sales invoices, credit memos, debit memos, etc.)
are unlocked after hours.
3. The of®ce used to keep books of accounts (e.g., sales
journal books, accounts receivable books, control
account books, etc.) is unlocked after hours.
4. The cabinets used to keep books of accounts (e.g., sales
journal books, accounts receivable books, control
account books, etc.) are unlocked after hours.
5. Persons other than accounting manager/controller and
accounts receivable clerks have key to open cabinets
used to keep sales journal books and accounts
receivable books.
6. Persons other than accounting manager/controller and
general ledger clerks have key to open cabinets used to
keep control account books.
7. Storeroom is not under supervision all time during the
storeroom hours.
8. Storeroom is unlocked after hours.
9. Persons other than storeroom staff have key to open the
storerooms.
10. Cashier's drawers are not locked all time.
11. Persons other than cashiers have key to open cashier's
drawers.
B.1.5. Independent check on performances
1. Credit-worthy customers' purchase orders are not
reviewed at least once a month.
2. Overdue accounts are not reviewed at least once a
month.
3. A list of collections by mail is not reviewed against a
cash book by a person other than the person who open
mails, the person who receives cash, and the person who
records cash at least once a month.
4. Cash/cheque on hand is not counted against a cash book
by a person other than the person who receives cash/
cheque on a surprise basis.
5. Inventories in stock are not counted against the record
in a book of accounts by a person other than the store-
room staff at least once a year.
6. Sales invoices are not checked by a person other than
the person who prepares them or salesperson that prices
and trade discounts are based on approved sales orders
or price lists.
7. Sales invoices are not checked by a person other than
the person who prepares them or the person who records
shipments that quantities are based on actual records of
goods shipped.
8. Customers' names in sales invoices are not checked by a
persons other than the person who prepares them
against the customers' master ®le or customers'
purchase orders.
9. The extensions and additions in sales invoices are not
recomputed by a person other than the person who
prepares them.
10. Credit memos are not checked by a person other than
the persons who prepares them that prices and cash
discount conditions are agreed with original sales
invoices.
11. Credit memos issued for goods returned are not checked
by a person other than the person who prepares them or
the person who records goods returns that quantities are
based on actual records of goods returned.
12. Customers' names in credit memos are not checked by a
persons other than the person who prepares them
against the original sales invoices.
13. The extensions and additions in credit memos are not
recomputed by a person other than the person who
prepares them.
Appendix C
C.1. Case a: PC mart company
PC Mart Company (PCM) is a Kentucky merchandiser
of personal computers and related equipment such as
monitors, printers, modems, hard disk drives, diskettes,
and business software package. The company is estab-
lished in 1960. During its several years of existence,
PCM has done very well. Not only have sales grown
fairly rapidly, but the product lines have been expanded
and the employees have increased in number. Currently,
the company employs ®fty persons, ranging from the
managing director to a janitor.
C.1.1. Organization and functions
PCM has nine departments, which are independent from
C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291286
each other. The company's managing director is Mr.
Richard Gere. Reporting to him are nine managers as
follows:
1. Ms. Kim BarsingerÐThe Accounting Manager
2. Mr. Jackie ChanÐThe Administrative Manager
3. Mr. Tom CruiseÐThe Credit Manager
4. Ms. Julia RobertÐThe Treasurer
5. Mr. Kerk RusselÐThe Purchasing Manager
6. Mr. Robert RedfordÐThe Receiving Manager
7. Mr. Charlie SheanÐThe Sales Manager
8. Mr. Harrison FordÐThe Shipping Manager
9. Mr. Clint EastwoodÐThe Storeroom Manager
C.1.2. Accounting department
Ms. Kim Barsinger oversees clerks and bookkeepers who
process transactions and maintain the accounting records.
Reporting to her are all accounting clerk and bookkeeper as
follows:
² Ms. Barbara Bailey: Reconciles Account Balances and
Control Accounts; Prepares Bank Reconciliation;
Records Customers' Claims
² Mr. Richard Cosner: Prepares Sales Journal; Records
Sales Invoices
² Mr. Darick Lykins: Prepares Purchase Journal; Records
Purchase Orders
² Mr. Andrew Mabson: Records Credit Memos and
Adjustments to Customers' Accounts
² Ms. Karen Stainley: Prepares Cash Receipts Journal,
Cash Disbursement Journal, Accounts Receivable
Subsidiary Ledger, and Accounts Payable Subsidiary
Ledger; Records Cash, Cheque and Accounts Receivable
² Ms. Victoria Parks: Prepares General ledger, Trial
Balances, and Financial Statements; Records General
Ledgers
² Ms. Martha Kastner: Prepares General Journal and
Payroll Register; Records General Expenses, Payrolls
The accounting of®ce is always locked after hours to
prevent unauthorized access to the books of accounts. In
addition, there are two cabinets in the of®ce. The ®rst one
is used to keep sales journal and accounts receivable books.
The second is used to keep general ledger, trial balances,
and ®nancial statements books. These two cabinets are kept
locked after hours. Only Ms. Kim Barsinger (the accounting
manager) and Ms. Karen Stainley (the accounts receivable
clerk) have the keys to open these two cabinets. Ms. Victoria
Parks (the general ledger clerk) only has the key to open the
second cabinet.
C.1.3. Administrative department
Mr. Jackie Chan serves as an administrative manager,
with responsibilities for personnel, insurance, budgeting,
cost analysis, and systems and procedures.
C.1.4. Credit department
Mr. Tom Cruise is in charge of the credit department. The
department handles the preparations of debit notes, changes
in payment conditions memos, adjustments to customers'
account memos, uncollectable account memos, and write-
off accounts memos. Mr. Tom Cruise also reviews all over-
due account at least once a month in order to keep track on
the performances of collections and to ensure that all follow-
up procedures are being done properly and ef®ciently.
At the end of each month, the credit department's staff
will send the account balance statements to all active custo-
mer informing their current balances with the company.
They also prepare an aging report showing the proportion
of accounts receivable due in 30, 60, and 90 days. Each
month, this report is reviewed for setting up the uncollect-
able accounts. The uncollectable accounts memos must be
approved by Mr. Richard Gere (the managing director) and
Mr. Tom Cruise (the credit manager) in writing in order to
ensure that only bad accounts with high uncollectable
potential are transferred to the provision for uncollectable
accounts. The memos also have spaces provided for
authorized signatures and are prenumbered consecutively.
Every two months, the uncollectable accounts are
reviewed to check the performances of collections. If all
follow-up procedure is performed but the accounts are
still uncollectable, the write-off accounts memos will be
issued. These memos must be approved by Mr. Richard
Gere (the managing director) and Mr. Tom Cruise (the
credit manager) in writing. The memos have spaces
provided for authorized signatures and are prenumbered
consecutively.
Four copies of both uncollectable accounts memos and
write-off accounts memos are prepared and distributed as
follows:
² The original copy is retained in credit department's ®le
for their own record,
² One copy is sent to accounting department,
² One copy is sent to ®nance department, and
² The last copy is sent to sales department
C.1.5. Finance department
Ms. Julia Robert performs the duties of treasurer. Report-
ing to her are cashiers and ®nance clerks. This department
handles all collection from customers.
C.1.6. Purchasing department
Mr. Kerk Russel is in charge of purchasing activities and
supervises several purchasing staffs who specialize in the
various product lines. The department receives the purchase
requisitions from other departments and further processes
the acquisition of goods or any other assets.
C.1.7. Receiving department
Mr. Robert Redford handles all receiving function which
C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 287
includes receiving of goods from suppliers and also receiving
the goods returned from customers. Ms. Julie Garwood, a
receiving clerk, is in charge for the record of all goods received
from suppliers and all goods returned from customers.
C.1.8. Sales department
Mr. Charlie Shean directs the operations of the sales func-
tions. There are ten salespersons and two sales clerks under
his supervision. Mr. Thomas Newston and Mr. Jimmy Carl-
ton, the sales clerks, are responsible for preparing sales
invoices. The of®ce is always locked after hours to prevent
unauthorized access to sales documents. In addition, in order
to reduce the opportunity of missing documents, the cabinets
used to keep sales documents are also locked after hours.
C.1.9. Shipping department
Mr. Harrison Ford is in charge of the shipping depart-
ment. This department maintains all shipping operation.
Mr. Stephen Bendon, the shipping clerk, is in charge for
the record of each shipment.
C.1.10. Storeroom department
Mr. Clint Eastwood maintains the storeroom operations.
The storeroom is under the supervision of at least one store-
room staff all time during the store hours and is always
locked after hours. Only storeroom staffs have the key to
open the storeroom. At the end of each year, the inventory in
stock will be counted against the record in the books of
accounts by the person other than the storeroom staff in
order to verify the actual amount of inventory in stock as
well as to alert the storeroom staff to take a good care of
inventory in and out.
C.2. Sales and collections procedures
The company sells its products for cash or on credit. It has
about 200 credit customers. The company acquires products
from 20 suppliers. The perpetual inventory method for
recording purchases is employed by PCM. Most products
are sold at established prices, but trade discounts are
allowed on occasion to small businesses and professional
®rms.
PCM maintains its cash in two bank accounts. There is
one account for general funds and one for payrolls. The
company also has petty cash and cashier drawer funds.
Among the resources that it owns are the of®ce building,
storeroom, the land on which these buildings are located, the
furniture and ®xtures in each of the building, cash registers,
other of®ce equipment, and two trucks for delivering
ordered merchandise.
Sales and collections procedures are divided into seven
groups of functions. The following describes all the proce-
dure performed by the company's employees pertaining to
the sales and collections cycle.
C.2.1. Receiving customers' purchasing orders
Customers' purchase orders are received by salespersons
either in person, by phone, or by mail. Normally, ®fteen to
twenty orders are received each day. If a customer's
purchase order is received by mail, a salesperson will
forward such order to a sales clerk for preparing a sales
invoice. However, if an order is received in person or by
phone, he/she must transcribe it into a uniform sales order
before further processing.
If cash in full is paid by the customer at the time of the
sale, the sales order copy is marked paid, and the sale is rung
up on a cash register. The customer also receives the cash
receipt tape from a cashier. If no cash or if only a partial
payment (i.e., a deposit) is received at the time of the sale,
the sale is treated as a credit sale. The cash/cheque received
each day via the cash registers is checked at the end of the
day, then combined with the cash/cheque received by mail
and deposited into the bank on a daily basis. The cashiers'
drawers are locked all time. Only Ms. Kathy Mcdonald and
Ms. Deborah Adams, the company's cashiers, have the key
to open the drawers.
The company also has a customers' purchase order regis-
tration book which contains purchase order receiving date,
purchase order number, customer name, and the name of
contact person. In addition, the credit-worthy customers'
purchase orders are reviewed monthly by Mr. Charlie
Shean (the sales manager) in order to ensure that such orders
are handled properly and ef®ciently.
C.2.2. Preparing sales order forms
All sales order must be approved by Mr. Charlie Shean
(the sales manager) in writing prior to further processing.
The sales orders have a space provided for the sales
manager's authorized signature and are also prenumbered
consecutively.
C.2.3. Authorizing credit terms and discounts and preparing
sales invoices
After approval is granted for a sale, seven copies of a
sales invoice are prepared by the sales clerks, Mr. Thomas
Newston and Mr. Jimmy Carlton. These sales invoices must
also be approved by Mr. Charlie Shean (the sales manager)
in writing in order to ensure that the credit sales are not
approved to a poor-credit customer. Then, these copies are
distributed as follows:
² The original copy is sent to a customer as a billing and
also for his/her record.
² One copy is retained in sales department's ®le for their
own record,
² One copy is sent to credit department,
² One copy is sent to ®nance department,
² One copy is sent to store department, and
² The last two copies are sent to shipping department
Any voiding of sales invoices must be authorized by Mr.
C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291288
Richard Cosner. Occasionally, a special term and discount
may be given to speci®c customers. This type of sales
invoices including some speci®c invoices (e.g., an invoice
with substantial amount) must be approved by Mr. Tom
Cruise (the credit manager) and Mr. Richard Gere (the
managing director) in order to ensure that such sales are
in conform with the company's sales and credit policies.
Sales invoices have spaces provided for authorized signa-
tures and are prenumbered consecutively.
Typically, Ms. Barbara Bailey (the chief accountant) will
review sales invoices against sales orders and records of
shipments in order to verify if the prices and trade discounts
are based on approved sales orders and the quantities are
agreed with the actual records of goods shipped. She also
checks the customers' names against the customer master
®le and recomputes the extensions and additions in sales
invoices.
C.2.4. Preparing credit memos for cash discounts or goods
returns, changes in payment conditions, debit notes, and
adjustments to customers' accounts
The company allows returns and allowances within a
speci®ed period from the dates of sales. It also allows
cash discounts on sales if paid in full within ten days, other-
wise, it expects full payment within thirty days. In both
cases, a credit memo will be prepared by Mr. Kevin Cosner
or Mr. Dennis Justice. All credit memos must be approved
by Mr. Charlie Shean (the sales manager) and Mr. Tom
Cruise (the credit manager) in writing before further proces-
sing. Credit memos have spaces provided for authorized
signatures and are prenumbered consecutively.
In case of cash discounts, four copies of credit memo are
prepared and distributed as follows:
² The original copy is sent to a customer for his/her record.
² One copy is retained in ®nance department's ®le for their
own record,
² One copy is sent to credit department, and
² The last copy is sent to accounting department
In case of goods returns, six copies of credit memo are
prepared and distributed as follows:
² The original copy is sent to a customer for his/her record.
² One copy is retained in receiving department's ®le for
their own record,
² One copy is sent to credit department,
² One copy is sent to accounting department,
² One copy is sent to ®nance department, and
² The last copy is sent to storeroom department
Typically, Ms. Barbara Bailey (the chief accountant) will
review credit memos against sales invoices and records of
goods returns in order to verify if the prices and cash
discount conditions are agreed with the original sales
invoices and the quantities are based on the actual records
of goods returned. She also checks the customers' names
against the original sales invoices and recomputes the exten-
sions and additions in credit memos.
A change in payment conditions, which requires the issu-
ance of debit notes, is sometimes allowed to a customer.
Such change must be approved by Mr. Charlie Shean (the
sales manager) and Mr. Tom Cruise (the credit manager) in
writing. The changes in payment conditions memos are
prenumbered consecutively and also have spaces provided
for authorized signatures. Four copies of the memos are
prepared and distributed as follows:
² The original copy is sent to a customer for his/her record.
² One copy is retained in credit department's ®le for their
own record,
² One copy is sent to ®nance department, and
² The last copy is sent to sales department
An increase in sales amounts as well as other adjustments
to customers' accounts must be approved by Mr. Tom
Cruise (the credit manager) and Ms. Kim Barsinger (the
accounting manager) in writing prior to further processing
in order to prevent unauthorized changes to customers'
accounts. Both debit notes and adjustments to customers'
accounts memos have spaces provided for authorized signa-
tures and are prenumbered consecutively. Five copies of
them are prepared and distributed as follows:
² The original copy is sent to a customer for his/her record.
² One copy is retained in credit department's ®le for their
own record,
² One copy is sent to accounting department,
² One copy is sent to ®nance department, and
² The last copy is sent to sales department
C.2.5. Preparing execution instruction, withdrawing goods
from stock, and shipping goods
After receiving a copy of sales invoices from sales depart-
ment, a storeroom staff prepares goods for shipment. If an
order cannot be completely ®lled because there is an insuf-
®ciently quantity of an item on hand, the storeroom staff
uses the copy of the sales invoice to prepare a back order
and sends it to the purchasing manager for further proces-
sing.
After receiving two copies of sales invoices from sales
department, a shipping staff will contact the storeroom
department for the goods to be shipped and, then ships the
goods to customers. One copy of sales invoices is retained in
shipping department for their own record. Another copy,
which may be called a delivery order, is accompanied
with the goods shipped. The delivery order has a space for
the customer's signature and is prenumbered consecutively.
The shipping staff will have the customer signed the deliv-
ery order as the acknowledgment of goods received. Then,
C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 289
such delivery order will be forwarded to ®nance department
for further collection processes.
C.2.6. Receiving cash, cheque, and remittance advice by
Ms. Kathy Mcdonald and Ms. Deborah Adams perform
the duty of cashiers. Besides the main duty of receiving
cash, Ms. Kathy Mcdonald is also assigned to open the
mail received each day. Each letter containing a cash remit-
tance is set aside. While opening all mail, they will prepare a
list of collections by mail, which provides the details of all
cash, cheque, and remittance advice received by mail. At the
end of each month, this list of collections will be reviewed
against a cash book by Ms. Julia Robert (the treasurer) in
order to ensure the correctness of cash/cheque recorded as
well as to prevent the lapping on cash/cheque received.
Ms. Kathy Mcdonald and Ms Deborah Adams (the cash-
iers) also prepare a deposit slip, then combine the cash/
cheque received in the mail with the cash/cheque from the
cashiers' drawers and deposit into the bank on a daily basis.
At the end of each day, Ms. Julia Robert (the treasurer) will
count the cash/cheque on hand against the record in cash
book in order to reduce the opportunity of lapping on cash/
cheque.
Ms. Kathy Mcdonald and Ms. Deborah Adams will issue
cash receipts to customers to acknowledge the receipts of
their payments. They also send to pile of remittance noti®-
cations and advises (together with a copy of the deposit slip)
to the accounting clerks for entry into the cash receipts
journal, for posting to the accounts receivable and general
ledgers, and for ®ling.
C.2.7. Recording accounts receivable, sales and general
ledgers
The accounting records are entered and posted manually.
Ms. Karen Stainley (the accounts receivable clerk) will post
all cash/cheque received to an accounts receivable book on a
daily basis.
Ms. Victoria Parks (the general ledger clerk) supervises
the posting to the general ledgers. Then, she prepares trial
balances and ®nancial statements with the aid of an electro-
nic spreadsheet package to compete key ®nancial ratios and
to print hard copy ®nancial reports for the other managers.
Ms. Barbara Bailey (the chief accountant) is responsible
for recording all customers' claim and reconciling custo-
mer's accounts with the control accounts. She also prepares
a bank reconciliation at the end of each month in order to
cross check the ªCash in Bankº account with the transac-
tions provided in the bank statement.
C.3. Relationships of the company's employees
² Ms. Julia Robert is Ms. Julie Garwood's sister
² Mr. Thomas Newston is Mr. Dennis Justice's son
² Mr. Kevin Cosner is Mr. Richard Cosner's brother
² Mr. Jackie Chan and Ms. Karen Stainley are husband and
wife
Appendix D
D.1. Post-experiment questionnaires
Please answer the following questions. Please keep in
mind that the following questions refer ONLY to the last
case study you have done (Table 10).
References
American Institute of Certi®ed Public Accountants (AICPA) (1996). Codi-
®cation of statements on auditing standards, New York: AICPA.
Bailey Jr, A. D., Duke, G. L., Gerlach, J., Ko, C., Meservy, R., & Whinston,
A. B. (1985). TICOM and the analysis of internal controls. Accounting
Review, 60 (2), 186±211.
Biggs, S. F., Messier Jr, W. F., & Hansen, J. V. (1987). A descriptive
analysis of computer audit specialists' decision-making behavior in
advanced computer environments. Auditing: A Journal of Practice &
Theory, Spring, 1±21.
Boritz, J. E. (1985). The effect of information presentation structures on
C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291290
Table 10
1. On a scale of 1 (very dif®cult) to 7 (very easy), how dif®cult was it to do the case study?
1 2 3 4 5 6 7
very dif®cult dif®cult slightly dif®cult neutral slightly easy easy very easy
2. On a scale of 1 (very unsatisfactory) to 7 (very satisfactory), how satis®ed were you with your accuracy in answering the case study?
1 2 3 4 5 6 7
very unsatisfactory unsatisfactory slightly unsatisfactory neutral slightly satisfactory satisfactory very satisfactory
3. On a scale of 1 (very unsatisfactory) to 7 (very satisfactory), how satis®ed were you with your speed in answering the case study?
1 2 3 4 5 6 7
very unsatisfactory unsatisfactory slightly unsatisfactory neutral slightly satisfactory satisfactory very satisfactory
4. On a scale of 1 (very boring) to 7 (very interesting), how interesting was the task you performed?
1 2 3 4 5 6 7
very boring boring slightly boring neutral slightly interesting interesting very interesting
audit planning and review judgments. Contemporary Accounting
Research, 8 (1), 193±218.
Borthick, A. F., & West, O. D. (1987). Expert systems-A new tool for the
professional. Accounting Horizons, March, 9±16.
Committee of Sponsoring Organizations of Treadway Commission
(COSO) (1992). Internal control-integrated framework, New York:
AICPA.
Cormier, S., & Hagman, J. (1987). Transfer of learning, San Diego, CA:
Academic Press.
Cummings, B. K., & Apostolou, N. G. (1987). Expert systems in auditing:
an emerging technology. Internal Auditing, 3 (2), 3±10.
Cummings, B. K., Lauer, J., & Baker, R. (1988). Expert systems in internal
auditing. Internal Auditing, 4 (1), 49±64.
Curtis, M. B., & Viator, R. E. (2000). An investigation of multidimensional
knowledge structure and computer auditor performance. Auditing: A
Journal of Practice and Theory, 19 (2), 83±103.
Davis, F. D., Bagozzi, R. D., & Warshaw, P. R. (1989). User acceptance of
computer technology: a comparison of two theoretical models.
Management Science, 35 (8), 982±1003.
Eining, M. M., & Dorr, P. B. (1991). The impact of expert system usage on
experiential learning in an auditing setting. Journal of Information
Systems, 5 (1), 1±16.
Fedorowicz, J., Oz, E., & Berger, P. D. (1992). A learning curve analysis of
expert system use. Decision Sciences, July/August, 797±818.
Gal, G. F. (1985). Using auditor knowledge to formulate data model
constraints: an expert system for internal control evaluation. Ph.D.
Dissertation, Michigan State University.
Gal, G. F., & Steinbart, P. J. (1992). Interface style and training task
dif®culty as determinants of effective computer-assisted knowledge
transfer. Decision Sciences, 23, 128±143.
Graham, L. E., Damens, J., & Ness, G. V. (1991). Developing risk advi-
sorSM: An expert system for risk identi®cation. Auditing: A Journal of
Practice & Theory, 10 (1), 69±96.
Holsapple, C. W., & Whinston, A. B. (1986). Managers' guide to expert
systems, Homewood, IL: Dow Jones-Irwin.
Libby, R. (1995). The role of knowledge and memory in audit judgment. In
R. Ashton & A. Ashton, Judgment and decision making research in
accounting and auditing. Cambridge, MA: Cambridge University
Press.
Lieb, E. B., & Gillease, J. K. (1996). Du Pont uses a decision support
system to select its audit portfolio. Interfaces, 26 (3), 10±21.
Miller, G. A. (1956). The magical number seven, plus or minus two: some
limits on our capacity for processing information. Psychological
Review, 63, 81±97.
Murphy, D. S. (1990). Expert system use and the development of expertise
in auditing: a preliminary investigation. Journal of Information
Systems, Fall, 18±35.
Nelson, M. W., Libby, R., & Bonner, S. E. (1995). Knowledge structure and
the estimation of conditional probabilities in audit planning. Accounting
Review, January, 27±47.
Odem, M. D., & Dorr, P. B. (1995). The impact of elaboration-based expert
system interfaces on de-skilling: An epistemological issue. Journal of
Information Systems, 9 (1), 1±17.
O'Leary, D. E., & Watkins, P. R. (1989). Review of expert systems in
auditing. Expert Systems Review for Business & Accounting, 9 (2),
3±22.
Oz, E. (1989). A study of improvement in decision-making skills through
the use of expert systems. Unpublished Ph.D. Dissertation. Boston
University.
Sharda, R., Barr, S. H., & McDonnell, J. C. (1988). Decision support system
effectiveness: a review and an empirical test. Management Science, 34
(2), 139.
Smith, V. L. (1982). Microeconomic systems as experimental science.
American Economic Review, 72 (5), 923±955.
Viator, R. E., & Curtis, M. B. (1998). Computer auditor reliance on auto-
mated and non-automated controls as a function of training and experi-
ence. Journal of Information Systems, Spring, 19±30.
Vinze, A. S., Karan, V., & Murthy, U. S. (1991). A generalizable knowl-
edge-based framework for audit planning expert systems. Journal of
Information Systems, Fall, 78±91.
Weber, R. (1999). Information systems control and audit, Upper Saddle
River, NJ: Prentice-Hall.
C. Changchit et al. / Expert Systems with Applications 20 (2001) 275±291 291