The Importance of Information Order and Management Explanation when Performing Analytical Procedures in Audit Planning

Leslie Berger University of Waterloo

PhD Student

First Year Summer Paper

September 9, 2005

I am grateful to Bill Wright, my summer paper advisor, for his insight, guidance, and helpful comments. I am also thankful to Guoping Liu for her helpful comments.

2

ABSTRACT

This paper examines the impact that the order of client information and the presence of a

fraudulent management explanation have on auditors’ ability to accurately perform

analytical procedures during the planning phase of the audit. Experimental evidence,

using a 2 x 2 experimental design and a sample of 42 junior and senior auditors, is used

to examine auditors’ performance in the completion of analytical procedures. The results

of this study indicate that information order impacts expectation development, the

presence of client explanation impacts the inherent risk assessment, and both factors

affect auditors’ assessment of the correct hypothesis.

3

INTRODUCTION The effectiveness of an audit is impacted by an auditor’s ability to accurately perform

analytical procedures in the planning phase of the engagement. In this study, the

accuracy of auditor performance when presented with client information in a causal

framework is compared to auditor performance when information is presented in an

audit-planning framework. In addition, auditors’ ability to effectively react to a

fraudulent management explanation is examined.

This study presents evidence that the order of client information presentation in the audit

planning stages impacts the auditor’s ability to accurately perform analytical procedures.

Although prior research indicates that information order impacts auditor performance

(Ricchiute (1992), O’Donnell and Shultz (2003)), this experiment considers the impact of

client information order on the accuracy of auditor assessments during the completion of

preliminary analytical procedures in a fraudulent environment. As client explanation is

the most common source of evidence gathered during the performance of planning

analytical procedures (Hirst and Koonce (1996)), this study also considers the impact that

a fraudulent management explanation has on the accuracy of auditor decisions in this

crucial stage of the audit.

The results of this study suggest that both the order in which information is presented and

the existence of an incorrect management explanation impact auditor performance in

analytical procedures. When information is presented in a causal manner, auditors

perform analytical procedures more accurately. Specifically, auditors prepared a more

4

accurate sales estimate than when the same information was presented in an audit-

planning framework. In addition, auditors more accurately identified the correct,

fraudulent explanation of the variance in the causal framework scenario with the

fraudulent, management explanation available.

THEORY AND HYPOTHESES

The Planning Phase of an Audit

In the planning phase, auditors develop a general strategy and a detailed approach for the

expected nature, timing and extent of the audit (CICA Handbook, Section 5150).

Generally Accepted Auditing Principles state that as part of the planning phase, an

auditor must perform analytical procedures (CICA Handbook, Section 5301). In practice,

37% of the planning time is spent performing analytical procedures (Ameen and Strawser

(1994)).

In performing analytical procedures, auditors are required to study relationships among

elements of financial and non-financial information to form expectations about what the

recorded amounts should be and then compare their expectations to the actual balances.

The results of analytical procedures in the planning phase are designed to assist the

auditor in assessing the risks of material misstatement in order to determine the nature,

timing and extent of further audit procedures (CICA Handbook, Section 5301).

5

Experimental evidence indicates that the results of analytical procedures affect an

auditor’s decisions about further audit testing. For example, insufficient analytical

procedures may result in reduced audit quality because satisfactory results will lead to a

reduction in substantive testing (Bedard and Biggs (1991)). Auditors are most likely to

extend audit testing if an issue was identified in the planning phase (Cohen and Kida

(1989)). Thus, an auditor’s ability to accurately identify unusual fluctuations and

accurately assess risk using analytical procedures will have a positive impact on the

engagement.

Analytical Procedures Research

The cognitive phases that an auditor experiences during the performance of analytical

procedures include mental representation, hypothesis generation, information search, and

hypothesis evaluation (Koonce (1993)). There have been a number of studies on the

factors that impact auditor performance during each of these phases.

Forming an Expectation

The first stage in the analytical review process, mental representation, is described as the

period in which auditors gather information from a variety of sources, including draft

financial statements and other non-financial measurements, to form expectations about

the client’s current year financial results (Koonce (1993)). Auditors who intentionally

develop expectations of the balances demonstrate an improved rate of identification of

the potential misstatement (McDaniel and Kinney (1995)).

6

When presented with a variety of information about the client’s financial and non-

financial status, auditors identify patterns in the data to assist them in the development of

expectations about the client’s financial results. An auditor’s inability to correctly

identify the pattern can be attributed to one of three potential errors: failure to consider a

cue, misinterpretation of a cue, or failure to use crucial cues in combination (Bedard and

Briggs (1991)).

Auditors may choose to incorporate or disregard information in their development of

expectations. For example, auditors place insufficient consideration on non-financial

information and place more emphasis on financial trends when conducting analytical

procedures (Cohen, Krishnamoorthy and Wright (2000)).

Client provided information influences auditors’ expectations and perceptions. Biggs and

Wild (1985) observed that auditors’ judgments were biased by unaudited information.

Also, in the performance of analytical procedures if their expectation does not vary

significantly from the client’s number, auditors judge the strength of the analytical

procedures higher than when a material variance exists (Glover, Prawitt, and Wilks

(2004)).

Hypothesis Evaluation

When the auditor establishes an expectation of financial results, and compares it to the

client provided results, a variance between the unaudited results and the auditor

expectation may occur. In this situation, the auditor must consider possible explanations

7

to explain the variance. Koonce (1993) notes that three cognitive steps characterize this

portion of analytical procedures: Hypothesis Generation (generation of potential

explanations), Information Search (identification of facts that may support or refute an

generated hypothesis), and Hypothesis Evaluation (evaluation of the hypothesis validity).

Audit efficiency and effectiveness depend on an auditor’s ability to recognize patterns in

the financial data and generate hypotheses to explain variances. Auditor error most

frequently occurs in the hypothesis generation phase (Bedard and Biggs (1991)). Failure

to generate the correct hypothesis has been shown to negatively impact the outcome of

the analytical procedures (Green and Trotman (2003), Asare and Wright (2003)).

Previous research has shown that various factors impact an auditor’s ability to evaluate

hypotheses. For example, an auditor’s memory of previous auditing engagements has

been shown to impact the nature of hypotheses evaluated. Auditors are more likely to

identify patterns of common or recently encountered financial statement patterns (Libby

(1985)).

A fair amount of research has been done to examine auditors’ ability to deviate from a

concluded hypothesis once additional information becomes available. Once auditors

conclude on a hypothesis, they are unwilling to switch to a different hypothesis even after

additional information indicates another hypothesis may be more valid than the original

(Hienman-Hoffman, Moser and Joseph (1995)). When provided with additional

information, auditors adjust assessments of a given hypothesis one at a time and do not

8

simultaneously adjust the likelihood for any of the other competing hypotheses (Asare

and Wright (1997)).

The Presentation of Information to the Auditor

Previous research indicates that in situations where the decision maker must evaluate a

number of interdependent items, the manner in which information is presented can

significantly impact the overall outcome. For example, psychology research suggests

that when information is presented in a story format, where causal and intentional

relationships are clearly presented, the decision maker’s comprehension of the situation

and subsequent decisions are improved (Pennington and Hastie (1986, 1998)).

The benefits of causal information ordering have been tested in an auditing context.

Ricchiute (1992) examined the impact of information order on audit partners’ ability to

prepare a going concern decision. The evidence indicates that audit partners presented

with information in a causal order more accurately concluded that the client’s going

concern was in doubt than those presented with information in traditional working paper

order.

O’Donnell and Schultz (2003) observed that audit support software organized around

business processes can influence decision performance. Auditors that used business

process focused software documented more risk factors and assessed misstatement risk at

higher levels than those using the transaction cycle focused system.

9

Building on the prior research, this study examines whether the way in which client

information is presented and analyzed in the initial audit planning stages impacts the

auditor’s ability to accurately perform analytical procedures. More specifically, this

study considers the effect that the presentation of information, either in a causal format or

an audit-planning format can have on auditor performance in the planning phase of the

audit. Based on prior research, it is expected that auditors presented with information in

a causal format will provide a more accurate analytical procedures analysis.

H1: The order in which client information is reviewed will positively impact auditors’

performance of analytical procedures.

Auditor Use of Management Explanations

In the performance of analytical procedures, auditors identify and attempt to determine

the cause of unexplained variances in account balances. During audit planning, the client

is the most common source of fluctuation explanations (Hirst and Koonce (1996)).

Therefore, an auditor’s accurate evaluation of the quality of a management explanation is

essential in the completion of analytical procedures in the planning phase.

Prior research suggests that auditors’ perception of a management explanation can impact

auditor performance1. For example, auditors are more likely to increase their planned

testing when there is minimal corroboration of management’s explanation and there is an

explicit incentive for management to misrepresent the financial statements (Glover,

Jiambalvo, and Kenney (2000)). When evaluating management explanations, auditors

are sensitive to both the competence and objectivity of the evidence source (Hirst

1 For a more detailed summary of prior research, please refer to Table 2.

10

(1994)). Auditors who receive an explanation from a client judge it to be more reliable

when the client possesses high competence (Anderson, Koonce and Marchant (1994)).

Koonce (1992) notes failure to perform written explanations and counter explanations

contribute to an auditor’s incorrect acceptance of a non-error cause when the correct

cause involves a financial statement error. Most research about management

explanations examines factors that affect an auditor’s perception of non-error

explanations (Anderson et al. (2004), Glover et al. (2000), Anderson et al (1994)). Many

of the existing studies do not consider the impact of an erroneous or fraudulent

management explanation.

Unlike previous research, this study considers the impact of a fraudulent management

explanation on the accuracy of analytical procedures performed in audit planning. It is

expected that when provided with a non-error explanation by management (intended to

conceal a fraudulent entry) auditor performance will be impacted, however it is not clear

whether the result will be more or less accurate when given a fraudulent management

explanation. If an auditor does not identify the explanation as fraudulent it follows that

their performance would be negatively impacted. Conversely, if an auditor concludes

that the explanation is fraudulent; auditor performance may be positively impacted. The

following non-directional hypothesis has been developed:

H2: The presence of a management explanation will impact the auditor’s performance

in the planning phase of the engagement.

11

METHODOLOGY

A laboratory experiment was designed to test H1 and H2. The between subjects, 2x2

experiment design examined information order (audit planning or causal) and the impact

of the presence of a management explanation (available or unavailable) on auditors

performance in a series of analytical procedures tasks.

Subjects

A group of 44 Masters of Accounting students from a Canadian University participated in

this experiment. As the majority of participants were graduates of a co-operative

education program, the average participant had 11.3 months (standard deviation of 6.2

months) of auditing work experience. Upon graduation, 64% of the participants (n=27)

will return to their auditing firms as audit senior auditors while 33% (n=14) of the

participants will be junior auditors. 47.6% of the participants had prior auditing

experience with manufacturing clients.

Materials

In this experiment, auditors were presented with a scenario in which the participant

assumed the role of the senior associate planning an upcoming audit engagement.

Information provided to the participants included background information about the

company, current year information, the previous two years’ audited financial statements,

the current year’s unaudited financial statements, a graphical representation of 4 years of

financial and non-financial indicators, and a junior auditor’s completion of preliminary

analytical procedures in the sales balance awaiting the participant’s review.

12

The company used in the materials was a fictional Canadian toy manufacturer2. A series

of financial and non-financial cues in the case were designed to illustrate that the

company faced many operational challenges during the year. For example, the company

experienced increased competition in the marketplace, and a disruption in the distribution

of finished goods due to an external strike. Participants were also presented with

information that described the corporate governance environment, including the CEO’s

dominant personality and aggressive promises made to the public shareholders. The

existence of significant operational challenges combined with the corporate governance

environment was intended to create an environment in which the presence of

management fraud was plausible.

A fraudulent entry, a premature recognition of the subsequent year’s sales, was included

in the financial statements received by all participants. All participants were provided

with two cues to identify the fraudulent entry: information detailing a material decrease

in outstanding orders at year-end of $14.6 million dollars, and a graphical presentation of

the decrease in outstanding orders (as a percentage of sales). Please refer to Table 1 for a

detailed analysis of the cues presented to the participants in this experiment.

2 An audit partner and two audit managers reviewed the materials and participant questions for realism.

13

Design

The information order condition was manipulated by providing the participants with the

information in a traditional, audit planning order (background knowledge of the client,

previous year’s audit files, current year information, and corporate leadership) or causal

order (background information, corporate strategy and objectives, the company’s past and

present performance results and corporate leadership information.) In each manipulation,

participants were presented with identical facts about the company.

The existence of a fraudulent management explanation to explain a material variance in

the sales account was manipulated in this experiment. In the scenario, the junior auditor

provides an explanation to explain the variance between his sales expectation and the

client provided balance. The junior auditor either notes that the controller explained that

the sales variance was caused by an increase in orders at year-end (available) or that the

controller has been away from the office and unavailable to comment on the fluctuation

(unavailable). The explanation provided by management is an erroneous explanation

intended to cover up the fraudulent entry to increase sales.

Response Variables

After reviewing the materials, participants were asked to perform three tasks designed to

measure the accuracy of their performance of analytical procedures during the planning

phase of the audit. First, participants calculated an expectation of the client’s current year

sales balance and the variance between the expectation and client provided balance.

Next, participants were provided a list of 7 possible hypotheses that could explain a

14

fluctuation in the sales account. They were asked to allocate 100 points between the

potential hypotheses to indicate the probability that each hypothesis was the correct

explanation for the sales fluctuation calculated in the first task. Finally, using a 100-point

scale, participants were asked to assess inherent risk, fraud risk, risk of material

misstatement, and control risk for the audit. At the end of the experiment, participants

were also asked to provide general demographic information.

RESULTS

To test H1 and H2, three ANOVA were performed. In all cases the independent

variables were information order (audit planning and causal) and management

explanation (available and unavailable). To examine the impact that these factors had on

the auditor’s performance three dependent variables were measured: sales estimate error,

hypothesis probability assignment, and risk assessments.

The ANOVA results using the absolute value of the sales estimate error as a dependent

variable are presented in Table 3 and Figure 1. The absolute value of sales estimate error

was calculated as the absolute value of the difference between the participant’s sales

estimate and the correct response. The correct response was calculated by adjusting the

client provided sales balance for the early recognition of the following year’s sales. The

absolute value of this error was used in this calculation because the magnitude of the

error, not the direction, is of interest. ANOVA showed that the sales estimate error was

influenced by the order of information (one tailed p value = 0.0475), but was not

influenced by the presence of management explanation (p = 0.482) or the interaction (p =

15

0.210). As expected, the average absolute value of the error was higher in the planning

order ($12,629) than in the causal order ($7,744). Thus, auditors presented with the

causal order format were more accurate in estimating the sales balance than those

presented with the audit planning order.

Analysis of the participants’ assessment of inherent risk3 is presented in Table 4 and

Figure 2. ANOVA indicates that client explanation had a significant impact on the

inherent risk assessment (p = 0.074); however, the inherent risk assessment was not

significantly influenced by the information order (0.651) or the interaction (0.185).

Participants that were not given a client explanation assessed the inherent risk of the

engagement at 73.20%. When provided the fraudulent management explanation, the

auditors’ assessment of inherent risk increased to 82.86%. Thus, the presence of a

fraudulent management explanation caused auditors to increase their assessment of

inherent risk.

The auditor’s assessment of the probability that premature recognition of next year’s

sales was tested as a dependent variable. The results of this analysis are presented in

Table 5 and Figure 3. ANOVA indicates that the interaction of the client explanation and

the causal information had a significant effect of the probability assessment (p = 0.052).

The individual effects of client explanation (0.233) and information order (0.404) were

not significant. Of the four cells, participants that were presented with causal information

and the fraudulent client explanation were most accurate in their assessment of the

3 The results (not presented) relating to control risk, risk of material misstatement, and fraud risk assessments were not significant.

16

correct hypothesis. In other words, the participants in this cell assigned the highest

probability (36.82%) to the correct hypothesis. Auditors that used the planning order

appeared to be less accurate in their assessment of the correct hypothesis when

management explanation was provided (20.45%) than when the explanation was not

provided (25%). Conversely, auditors that were presented with the causal format were

more accurate when management explanation was available (36.82%) than when the

explanation was not available (18.33%).

In summary, the order of information had a significant impact on the auditors’

expectation development, the presence of incorrect management explanation impacted

the auditors’ risk assessment, and both factors impacted the auditor’s assessment of the

correct hypothesis. As predicted, the auditor’s performance was impacted by the order in

which they received information (H1) and the presence of a management explanation

(H2).

17

CONCLUSIONS AND DISCUSSION

The results of this study indicate that the order in which client information is presented

impacts the accuracy of auditors’ analytical procedures in the planning of an audit.

Auditors presented with the causal scenario were able to more accurately use the client

information in developing an estimate of sales than those presented with client

information in the traditional audit-planning format.

Further, the results indicate that the presence of a fraudulent management explanation

impacts auditor behavior. Participants provided with the management explanation

increased their assessment of the inherent risk of the audit. This result suggests that

auditors identified the explanation as being potentially erroneous or fraudulent thereby

leading to an increase in the perceived inherent risk of the engagement.

Finally, the effects of the causal framework and the presence of a management

explanation interact to produce a more accurate assessment of the correct hypothesis.

The participants in the causal framework, management explanation condition showed the

most accurate assessment of the correct hypothesis. This is an interesting result because

these participants seemingly identified the management explanation as fraudulent and

were not influenced by the client’s explanation but instead used it as further evidence that

a fraudulent entry caused the variance.

Auditor performance in analytical procedures can have a significant impact on the

effectiveness of the audit engagement. It follows that an understanding of factors and

18

approaches that can improve the accuracy of preliminary analytical procedures can

therefore help to improve the overall audit approach. This paper contributes to the

management explanation research by analyzing the impact of a fraudulent management

explanation on the auditor performance in analytical procedures. This study also

contributes to prior research about the importance of information ordering in decision-

making by examining the impact of the causal information order on the accuracy of audit

performance in planning phase analytical procedures in a fraudulent environment.

Based on the results of this study there are many opportunities for further research. Prior

research indicates that experienced auditors may process evidence and react to various

conditions differently than less experienced, junior auditors (Bedard and Chi (1993)).

Further studies may consider the impact of expertise on these findings. In addition, some

researchers conclude that a risk based auditing approach positively impacts the

performance of the auditors (Bell, Peecher and Solomon (2002)). Further research may

examine if experts in the risk based approach respond differently to information order

and erroneous management explanations when performing analytical procedures.

19

TABLE 1 Cues Available to the Subjects

Financial Performance Cues

Decline in sales in both 2002 and 2003 Sudden improvement in sales in 2004 Increase in Accounts Receivable balance of $10,166,000 from 2003 Increase in Gross Margin Ratio from 0.43 in 2003 to 0.47 in 2004

Non-Financial Performance Cues

Two new major competitors entered the Canadian toy market in 2003 In 2003 and 2004 the Company’s new products were not as well received in the

marketplace as they were previously In 2004 a dockworker’s strike prevented inventory from reaching retail locations

Fraud Environment Cues

The company’s CEO and founder is extremely aggressive, with a dominant personality

CEO publicly stated that company will present much better results in 2004, prior to realizing the challenges that the company would face throughout 2004

Fraudulent Entry Cues

2004 year-end backlog sales decreased $14.6 million from 2003. Graphical representation depicting year end back-log as a percentage of sales

20

TABLE 2

Prior Research Examining the Presence of Management Explanations in Analytical Procedures

Authors Audit

Phase Type of

Explanation Conclusions/Findings

Anderson, Kadous, and Koonce (2004)

Planning Non error The likelihood of a client to distort information affects an auditor’s evaluation of the persuasiveness of a management explanation

Glover, Jiambalvo, and Kenney (2000)

Planning Non error Auditors are more likely to increase testing when there is minimal corroboration of management’s explanation and an explicit incentive for management to misrepresent the financial statements.

Anderson, Koonce and Marchant (1994)

Evidence Gathering

Non error Auditors judge an explanation to be more reliable when the client possesses high competence.

Hirst (1994) Planning Not specified In evaluating management explanations, auditors are sensitive to both the competence and objectivity of the evidence source.

Koonce (1992)

Evidence Gathering

Non error Auditors who provided a written explanation in support of a hypothesis claimed that the explanation was more likely.

Auditors revised the probability of a hypothesis downward when asked to provide a counter explanation.

21

TABLE 3

Analysis of the Absolute Value of the Participant Error for Information Order and Client Explanation Conditions

Absolute Value of Participant

Sales Estimate Error = Participant’s Sales

Estimate - Client Provided

Sales Figure adjusted for Fraudulent Error

Panel A: Analysis of Variance with Absolute Value of Participant Sales Estimate Error as the Dependent Variable Effect d.f. F- Statistic Significance

Two tailed (one tailed)

Client Explanation 1 0.503 .482 Information Order 1 2.930 .095

(.0475) Client Explanation x Information Order 1 1.626 .210 Error 38 - - Panel B: Means (Standard Deviations) of Absolute Value of Participant Sales Estimate Error Information Order Client Explanation Causal Planning Overall Available 5 334 13 392 9 363 (4 477) (9 603) (8 394) n = 11 n =11 n= 22 Unavailable 10 688 11 867 11 336 (10 693) (9 190) (9 641) n = 9 n = 11 n= 20 Overall 7 744 12 629 10 303 (8 134) (9 205) (8 953) n= 20 n = 22 n = 42

22

TABLE 4

Analysis of the Inherent Risk Assessment for Information Order and Client Explanation Conditions

Panel A: Analysis of Variance with Inherent Risk Assessment as the Dependent Variable Effect d.f. F- Statistic Significance Client Explanation 1 3.380 .074 Information Order 1 0.208 .651 Client Explanation x Information Order 1 1.820 .185 Error 38 Panel B: Means (Standard Deviations) of Inherent Risk Assessment Information Order Client Explanation Causal Planning Overall Available 87.73 78.00 82.86 (5.641) (18.65) (14.33) n = 11 n =11 n= 22 Unavailable 70.56 75.36 73.20 (21.23) (20.19) (20.28) n = 9 n = 11 n= 20 Overall 80.00 76.68 78.26 (16.86) (19.01) (17.88) n= 20 n = 22 n = 42

23

TABLE 5

Analysis of the Correct Hypothesis Probability for Information Order and Client Explanation Conditions

Panel A: Analysis of Variance with Correct Hypothesis Probability as the Dependent Variable Effect d.f. F- Statistic Significance Client Explanation 1 1.470 .233 Information Order 1 0.711 .404 Client Explanation x Information Order 1 4.013 .052 Error 38 Panel B: Means (Standard Deviations) of Absolute Value of Correct Hypothesis Probability Information Order Client Explanation Causal Planning Overall Available 36.82 20.45 28.64 21.48 (18.90) (21.45) n = 11 n =11 n= 22 Unavailable 18.33 25.00 22.00 (11.99) (19.37) (16.41) n = 9 n = 11 n= 20 Overall 28.50 22.73 25.48 (19.08) (18.81) (19.28) n= 20 n = 22 n = 42

24

FIGURE 1

Absolute Value of the Participant Error for Information Order and Client Explanation Conditions

$0

$2,500

$5,000

$7,500

$10,000

$12,500

$15,000

Planning Causal

ExplanationAvailable

ExplanationnotAvailable

25

FIGURE 2 Inherent Risk Assessment for Information Order and Client Explanation Conditions

0%

20%

40%

60%

80%

100%

Planning Causal

ExplanationAvailable

Explanationnot Available

26

FIGURE 3 Correct Hypothesis Probability for Information Order and Client Explanation Conditions

0%

10%

20%

30%

40%

50%

Planning Causal

Explanation

NoExplanation

27

REFERENCES Ameen, E. C., J. R. Strawser (1994) “Investigating the Use of Analytical Procedures: An Update and Extension.” Auditing: A Journal of Practice and Theory v.13 n.2: 69. Anderson, U. L., K. Kadous, L. Koonce (2004). "The Role of Incentives to Manage Earnings and Quantification in Auditors’ Evaluations of Management-Provided Information." Auditing 23(1): 11. Anderson, U. and L. Koonce (1998). "Evaluating the sufficiency of causes in audit analytical procedures." Auditing 17(1): 1. Anderson, U. L., L. Koonce and G. Marchant (1994). "The effects of source-competence information and its timing on auditors' performance of analytical procedures." Auditing 13(1): 137. Asare, S. K. and A. Wright (1997). "Hypothesis revision strategies in conducting analytical procedures." Accounting, Organizations and Society 22(8): 737. Asare, S. K. and A. M. Wright (2003). "A note on the interdependence between hypothesis generation and information search in conducting analytical procedures*." Contemporary Accounting Research 20(2): 235. Bedard, J. C. and S. F. Biggs (1991). "Pattern Recognition, Hypotheses Generation, and Auditor Performance in an Analytical Task." The Accounting Review 66(3): 622. Bedard, J. C., and M. Chi (1993). "Expertise in Auditing." Auditing 12: 21. Bedard, J. C., S. F. Biggs and D. M. Frederick (1991). "The Effect of Domain-Specific Experience on Evaluation of Management Representations in Analytical Procedures." Auditing 10: 77. Bell, T., F. Marrs, I. Solomon and H. Thomas. 1997. Auditing Organizations Through a Startegic-Systems Lens. The KPMG Business Measurement Process. KPMG Peat Marwick LLP Bierstaker, J. L., J. C. Bedard and S. F. Biggs (1999). "The Role of Problem Representation Shifts in Auditor Decision Processes in Analytical Procedures." Auditing: A Journal of Practice & Theory 18(1): 18. Biggs, S. F. and J. J. Wild (1985). "An Investigation of Auditor Judgment in Analytical Review." The Accounting Review 60(4): 607. Cohen, J., R, G. Krishnamoorthy and A. Wright, M. (2000). "Evidence on the effect of financial and nonfinancial trends on analytical review." Auditing 19(1): 27.

28

Cohen, J. R. and T. Kida (1989). "The Impact of Analytical Review Results, Internal Control Reliability and Experience on Auditors' Use of Analytical Review." Journal of Accounting Research 27(2): 263-276. Glover, S., M, J. Jiambalvo and J. Kennedy (2000). "Analytical procedures and audit-planning decisions." Auditing 19(2): 27. Glover, S., M, D. F. Prawitt and T. J. Wilks (2004). "Why Do Auditors Over-Rely on Weak Analytical Procedures? The Role of Outcome and Insensitivity to Precision." working paper. Glover, S. M., D. F. Prawitt, J. Joseph J. Schultz and M. F. Zimbelman (2003). "A Test of Changes in Auditors' Fraud-Related Planning Judgments since the Issuance of SAS No. 82." Auditing 22(2): 237. Green, W. J. and K. T. Trotman (2003). "An Examination of Different Performance Outcomes in an Analytical Procedures Task." Auditing: A Journal of Practice & Theory 22(2): 219. Heiman-Hoffman, V. B., D. V. Moser and J. A. Joseph (1995). "The impact of an auditor's initial hypothesis on subsequent performance at identifying actual errors." Contemporary Accounting Research 11(2): 763. Heiman, V. B. (1990). "Auditors' Assessments of the Likelihood of Error Explanations in Analytical Review." The Accounting Review 65(4): 875-890. Hirst, D. E. (1994). "Auditors' Sensitivity to Source Reliability." Journal of Accounting Research 32(1): 113. Hirst, D. E. and L. Koonce (1996). "Audit analytical procedures: A field investigation." Contemporary Accounting Research 13(2): 457. Johnson, E. N. (1995). "Effects of information order, group assistance, and experience on auditors' sequential belief revision." Journal of Economic Psychology 16(1): 137. Koonce, L. (1992). "Explanation and Counter-explanation During Audit Analytical Review." The Accounting Review 67(1): 59. Koonce, L. (1993). "A cognitive characterization of audit analytical review" Auditing 12: 57. Koonce, L. and F. Phillips (1996). "Auditors' comprehension and evaluation of client-suggested causes in analytical procedures." Behavioral Research in Accounting 8: 32. Libby, R. (1985). "Availability and the Generation of Hypotheses in Analytical Review." Journal of Accounting Research 23(2): 648.

29

McDaniel, L. S. and W. R. Kinney, Jr. (1995). "Expectation-formation guidance in the auditor's review of interim financial information." Journal of Accounting Research 33(1): 59. O'Donnell, E. (2002). "Evidence of an association between error-specific experience and auditor performance during analytical procedures." Behavioral Research in Accounting 14: 179. O'Donnell, E. and J. Joseph J. Schultz (2003). "The Influence of Business-Process-Focused Audit Support Software on Analytical Procedures Judgments." Auditing: A Journal of Practice & Theory 22(2): 265. Pennington, N. and R. Hastie (1986). “Evidence Evaluation in Complex Decision Making.” Journal of Personality and Social Psychology 51(2): 242. Pennington, N. and R. Hastie (1988). “Explanation-Based Decision Making: Effects of Memory Structure on Judgment.” Journal of Experiment Psychology: Learning, Memory and Cognition 14(3):521. Ricchiute, D.N. (1992). “Working-Paper Order Effects and Auditors’ Going-Concern Decisions.” The Accounting Review 67(1): 46. “Section 5150, General Assurance and Auditing – Planning and Supervision” CICA Handbook, Canadian Institute of Chartered Accountants, Toronto, Ontario “Section 5301, General Assurance and Auditing – Analysis” CICA Handbook, Canadian Institute of Chartered Accountants, Toronto, Ontario Trotman, K., T. and A. Wright (2000). "Order effects and recency: Where do we go from here?" Accounting and Finance 40(2): 169. Wilks, T. J. and M. F. Zimbelman (2004). "Decomposition of Fraud Risk Assessments and Auditors' Sensitivity to Fraud Cues." Contemporary Accounting Research 21(3): 719. Wright, A. M. and J. C. Bedard (2000). "Decision Processes in Audit Evidential Planning: A Multistage Investigation." Auditing: A Journal of Practice & Theory 19(1): 123.