A C C O U N T I N G & A U D I T I N G

a u d i t i n g

How Do Financial Statement Auditorsand IT Auditors Work Together?

By Joseph F. Brazel

In complex information technologyenvironments, audit quality is partiíü-ly detemiined by financial statement

auditors" accounting information system(AIS) expertise and their evaluation of anIT auditor's assessment. Because ofchanges in the auditing standards for pri-vate and publicly iraded companies. ITauditors have become common fixtures onaudit engagements. Little is understoodabout how these two professions interact.

Through an experiment witii practicingfinancial statement auditors, a recent studytbund that auditors will nsly heavily on thetesting of a competent IT auditor whenassessing control risk and planning sub-stantive testing. IT auditors can improveboth the effectiveness and efficiency of thefinancial statement audit. When IT audi-tor competence is low. it appears thatonly auditors with high levels of AISexpertise are able to effectively compen-sate for this deficiency.

The Role of IT AuditorsStatement on Auditing Standards (SAS)

108 Suggests that in complex IT settingsauditors should consider assigning oneor more computer assurance specialists(i.e., IT auditors) to the engagement inorder to determine the effect of IT onthe audit, gain an understanding of con-trols, and design and pertbrm tests of ITcontrols, fn addition. SAS 109 notes theimportance of IT with respect to auditors"assessments of control risk. For publiclytraded corporations. Public CompanyAccounting Oversight Board (PCAOB)Auditing Standard 5. An Audit oflntenmlControl over Financial Reporting Thath Integrated with an Audit of FinancialStatements, requires auditors to gain anunderstanding and test IT system controlsin order to provide an opinion on the

effectiveness of intemal controls overfmancial reporting. These auditing stan-dards, as well as companies" adoptions ofcomplex IT systems, have suhstantiallyenhanced the role of IT auditors on auditengagements.

tors on fmancial audits will continue togrow, and auditors will need to expamitheir AIS knowledge and skills in order toperform effective and efficieni audits.

The enhanced role of IT auditors onfinancial audits hrings up three qtiestions.

A 20()0 study estimated that the numberof IT auditors employed by one Big Fivefirm would grow from 100 to 5.000between 1990 and 2005 (N.A. Bagranoffand V.P. Vendrz.yk. "The Changing Roleof IS Audit Among the Big Five US-BasedAccounting Firms," hifomiation Sv.stemsand Control Journal, vol. 5). and IT audi-tor testing can now represent a suhstantialportion of the financial statement auditwork. IT auditors have become a chiefsource of audit evidence. For example. ITauditors" tests of system access controls merelied upon by auditors when makingcontrol risk assessments. As technologicaldevelopments continue, the use of !T audi-

First. what do auditors think of IT amiiioisas a source of audit evidence? Second, howdo these two pn)fessions interact on auditengagements? And ihird. under what con-ditions can this relationship be most pro-ductive?

IT Auditois as a Sourceof Audit Evidence

Past research has indicated that audi-tors have substantial concerns about ITauditor competence in practice, andsometimes question the value IT auditorsbring to the audit engagement (Bagranoffand Vendrzyk 2000; James E. Hunton.Amold M. Wright, and Sally Wright. "Are

38 NOVEMBER 2008 / THE CPA JOURNAL

Financial Auditors Overconfident in TheirAbility to Assess Risks Associated withEnterprise Resource Planning Systems?".tounuú of Infonncition Systems. Fall 2(X)4).Currently, increased demand for IT audi-tors due to the aforementioned standardshas resulted in IT auditors' resources beingstretched over more audit engagements,as well as audit firms losing highly com-petent IT auditors to corporations lookinglo improve the effectiveness of their ownintemal controls (C. Annesley. "ManualProcesses Must Be Automated to Cut Costof Sar banes-Ox ley Audits. Says Basda."Computer Weekly, October 25. 2005;Nonii£Ji Marks, "Maintaining Control: Willa Boom in Intemal Auditing Result in aBust in Audit Quality?," Internal Auditor.February 2005).

These findings were confirmed by theauthor s own study. Participants were askedto respond, on a scale from 1 (disagree)to 10 (agree), whether they had experiencedvariation in IT auditor competence. Thenieiui response was 7.23. The author doesnot conclude that there i.s a competencyproblem with IT auditors, but, rather, thattheir competency levels vary in practice.On the other hand, another study hasshown that IT auditors are better atassessing risks in enterprise resourceplann ng (ERP) environments and thatauditors appear to be overconfident in theirabiiititrs in such settings (Hunton, Wright.and W right 2(K)4). This overconfidence. aswell as high IT auditor billing rates, mayhelp explain why auditors are sometimeshesitant to employ IT auditors, beyondthe minimum firm-established require-ments, on their engagements.

Interacting on Audit EngagementsWhile audit managers are typically sen-

sitive to competence deficiencies in theiraudit ;talf and can compensate hy employ-ing additional procedures themselves,auditors" ability to effectively respond toIT autlitor competence deficiencies may bedetem lined by their own AIS expertise level.As the AIS expertise of an auditor incnsas-es, the auditor's km>wledge of system designand controls should be greater and, thus,provide the auditor with a clearer under-standing of what system controls the ITauditor has (or has not) tested, as well asIhe ability to compensate for the FT audi-tor's competence deficiencies. The pairing

of a less competent IT auditor with an audi-tor who maintains a low level of AIS exper-tise may lead to an ineffective audit.Auditors with low AIS expertise mayover-rely on weak IT auditor tests becausethey lack the AiS expertise to indepen-dently identify sy.stem risks and perform IT-related tests themselves. Conversely, whenthe IT auditor assigned to the engagementis highly competent, all auditors should ben-efit from their inclusion on the engage-ment team, as they can rely more on theIT auditor's testing and concentrate more onmatters related to the financial statementaudit. In summary, investigating the audi-tor-IT auditor relationship requires analyz-ing both the competence of the IT auditoras well as the AIS expertise the auditorbrings to the engagement

A Productive RelationshipTo examine how auditors are interacting

with IT auditors, one study had 74 practic-ing auditors complete an experimental auditcase study that asked them to supply riskassessments and plajined testing decisionsin an ERP setting (see Joseph F. Brazel andChristopher P. Agoglia. "An Examinationof Auditor Planning Judgments in aComplex Accounting Infonnation SystemEnvironment," Contemporary AccountingResearch. Winter 2007). Thirty-five of theauditors r^eived intemal ccmtrol testing doc-umentation concluding that system con-trols were reliable from a highly compe-tent IT auditor, while the odier 39 auditorsnxeived the exact same evidence from anIT auditor with low competence. To manipi-ulate IT auditor competence between thetwo groups, auditors were given informa-tion about the extent of the FT auditor's priortraining, experience, and performance (eitherall high or all low). As a check, partici-pants later noted that both the high andlow IT auditor competence descriptions

were equally realistic. The study measuredand assessed each auditor's AIS expertiselevel as either high or low via multiple scalesmeasuring the auditor's experience ;ind train-ing with complex AIS. Thus, the experi-mental study consisted of four groups (seeExhibit I).

The study provided all participantswith a ca.se that contained backgroundinformation for a hypothetical client, rele-vant authoritative audit guidance, andseveral prior-year workpapers. Thefie work-papers included prior-year risk assessmentsand substantive testing for the sales andcollection cycle. Participants also receiveda current-year workpaper documenting theclient's implementation of an ERP systemmodule for the cycle and infomiation thatan IT auditor would be assigned to theengagement to test system controls. Thecurrent-year workpaper noted multiplepotential implementation problems, includ-ing the migration of legacy-system datato the ERP system due to a mid-year con-version and the integration of a bolt-oninternal control package with the system.Next, participants received informationabout the IT auditor's competence level(either high or low) and the IT auditor'scontrol tests, which concluded that "sys-tem-related controls appear reliable."Particip^mts then assessed and dix'unient-ed a control risk assessment and plannedthe nature, staffing, timing, and extent ofsubstantive procedures for the cycle. Lastly.the authors had auditors at the senior man-ager and partner levels evaluate the effec-tiveness of the participants' judgments.These evaluators had extensive experienceauditing companies with complex AIS.

Control Risk AssessmentsAfter reviewing all of the case study

materials, participants assessed control riskfor the cycle on a scale ranging from 0

EXHIBIT 1 ]Participants by Group

High Auditor AIS Expertise

Low Auditor AIS Expertise

Low IT Auditor

Competence

n = 18

n^21

High IT Auditor

Competence

n = 18

n = 17

n - Number of practicing audit senior participants in each experimental group.

NOVEMBER 2œ8 / THE CPA JOURNAL 3 9

EXHIBIT 2Graphical Presentations ofthe Study's Results

Panel A: Participants' Mean Control Risk Assessments

70

SO

120

tío

100

es.s9

52.57

AIS ExpertiseO High

* Low

54.47

* 3a.B7

Low HighIT Auditor Competence

Panel C: Participants' Mean Staffing Decisions

AIS ExpertiseO High^ Low

4.12

23aZ53

127

Low High

iT Auditor Competence

Panel E: Participants' Mean Extent Decisions

AIS ExpertiseO HJQh

* Low

* 104.20

38.70

95.68

Low High

IT Auditor Competence

6.50

6.00

5.50

5.00

4.00

Panel B: Participants' Mean Nature Decisions

13.53

n.n

AIS ExpertiseO Hiflh

* Low

Lovy High

IT Auditor Competence

Panel 0: Participants' Mean Timing Decisions

AIS ExpertiseO High

^ Low

50.47

34.20

3167

"* 34.40

Low HighIT Auditor Competence

Panel F: Participants' Mean Testing Effectiveness

S.24

AIS ExpertiseO High* low

Low HighIT Auditor Competence

Controi risk was assessed by participants on a scale ranging from 0 ("low risk") to 100 {"high risk") percent. Nature refers to the total num-

ber of procédures planned. Staffing was computed as the total number of procedures assigned to a more senior-level auditor than to a staff

assistant Timing was measured as the total number of testing hours budgeted at fiscal year-end (versus interim). Extent refers to the total

number of budgeted audit hours. Effectiveness was determined by experts and computed as the experts' mean effectiveness ratings of par-

ticipants' audit programs on 10-point scales (1 = "very low"; 10 - "very high").

40 NOVEMBER 2008 / THE CPA JOURNAL

(very low) to 100 (very high). Exhibit 2.Panel A. presents the resuhs relating tomean control risk assessments for thestudy's four groups. What the studyfound was that the competence of the ITauditor had a substantial effect on auditors'control risk assessments. Essentially, as thecompetence of the IT auditor increased,auditoi*s tended to rely on their positivecontrol testing results and assessed con-trol risk as lower. Thus, in Panel A, all linesslope downward. This pattern emerged forauditors with both high and low AIS exper-tise. Given their superior knowledge of sys-tems and the potential system risks posedin the case study, however, auditors withhigher AIS expertise tended to assesscontrol risk as higher, regardless of the ITauditoi's level of competence.

Testing DecisionsUpc n completion of their control risk

assessments, participants planned the nature.staiTin; . timing, and extent of substantivetesting for the cycle. The study measuredthe "n;ilure" and "staffing" of participants'testing decisions as the total number of pro-cedures planned and the number of proce-dures assigned to a more senior-level audi-tor than staff assistant, respectively. The"timing" and "extent" of participants' test-ing decisions were computed as the totiilnumber of testing hours budgeted at fiscalyear-end {versus interim) and the total num-ber of budgeted audit hours, respectively.Panels B-F in Exhibit 2 graphically illus-traie the testing decisions of the study's fourgroup^, as well as the effectiveness of theirtesting decisions [evaluated by experiencedauditors on a scale ranging from I {verylow) to 10 {very high)].

For the most part, a typical pattern canbe seen in Panels B-F. When IT auditorcompetence is high {right-hand side ofthe graphs), the testing decisions of audi-tors with low and high AIS expertise aregenerally the same, mtxlerate in scope, andreasonably effective. Competent IT audi-tors appear to let all auditors rely on theirsystem testing and concentrate on the non-system testing in which they are adept {e.g.,accounts receivable confimiation, analyti-cal procedures related to sales). On theother hand, when IT auditor competenceis low (left-hand), there appears to be asubstaiitial difference between the testingdecisions of auditors with low and high

AIS expertise. These results suggest thatthe superior knowledge base of auditorswith high AIS expertise allows them toeffectively expand the scope of substantivetests, to include their own tests of the sys-tem, when there are IT auditor competencedeficiencies. Unfortunately, when auditorAIS expertise is low, it appears that under-auditing may result. Indeed, the pairing oflow IT auditor competence and low audi-tor AIS expertise had the lowest meaneffectiveness rating in Panel F (4.87).

If the above results show that auditorAIS expertise can play an impotiaiit rolein ERP settings, what role does the gen-eral audit expérience of the auditor play?The answer appears to be very little; atthe very least, AIS expertise seems totrump general audit experience in an ERPsetting. The authors examined the effectsof general audit experience on both thejudgments of the participants, as well asthe effectiveness of those judgments.Results showed no relationship betweengeneral experience and these factors.Thus, when assigning staff to an auditengagement, it may be prudent to con-sider the staff members' levels of AISexpertise (with respect to the elient's AIS),in addition to their general audit experi-ence levels. In other words, a fourth-yearsenior with high AIS expertise may bemore valuable than a fifth-year senior witha lower level of AIS expertise.

ConclusionTo answer the three questions posed

above: Auditors perceive that IT auditorcompetence varies in practice, both audi-tor AIS expertise and IT auditor compe-tence affect how these two professionsinteract on an audit engagement, and thisrelationship can be most productive whenat least one {preferably both) of the twoprofessions exhibits expertise or compe-tence related to a company's IT system.The findings of the study have implicationsfor practice and education. For example.given the potential for deleterious effectsin complex IT settings, PCAOB inspectionteams should consider evaluating whetherpolicies {e.g., training, scheduling) are inplace to ensure Kith the competence of theIT auditor and the AIS expertise of audi-tors assigned to the engagement. Theresults of this study clearly point to theadvantages of sufficiently training both

auditors and IT auditors so that they areequipped with the requisite expertise, giventhe complexity of their clients' IT.

In light of recently increased auditorresponsibilities with respect to intemal con-trol assessment, auditors should considerthe implications for audit efficiency andeffectiveness if they either alienate addi-tional intemal control testing to IT auditorsor provide auditors with greater trainingin evaluating IT risks and performingtests of IT. When IT is used to maintainthe general ledger, it's worthwhile notingthat SAS 109, Understanding the Entityand Its Environment and Assessing theRisks of Material Misstatement, discusseshow nonstandard joumal entries "may existonly in electronic fonn and may be moreeasily identified thn)ugh the use of com-puter-assisted audit techniques." Shouldauditors be perfomiing these procedures?Will they be effective? O would it be monseffective and efficient io rely on an IT audi-tor to perform this task? Firms could alsoexplore ways in whieh to improve the ITauditor-auditor relationship (e.g.. eombinedtraining and increased communicationthniughout the audil).

From an eduaiticinal standpoint, the studypoints to an increasing need to improve thesystem-related educational experiences ofaccounting students who will be the IT andfinancial statement auditors of the future.Undergraduate and master's degree pmgramsin accounting might want to partner withmanagement information systems depart-ments, or develop faculty strengths in thefield of AIS, in OTder to incorporate an ITconcentration into their programs. After hav-ing completed three years of Sarbanes-Oxleysection 4(>4 audits, auditors now know thespecific skill sets needed to eflectively per-form these audits in a complex AIS setting.Accounting academics should maintain anopen dialogue with these pn>fe,ssionals whendeveloping and updating their system-retat-ed acaiunting classes. Such advances in edu-cation should help provide the professionwith accounting graduates who have therequired skill set to llourish in the complexIT settings of the future. •

Joseph F. Brazel, PhD, is an assistantprofessor of accountitig at the college ofmanagement at North Carolina StateUniversity, Raleigh, N.C.

NOVEMBER 2008 / THE CPA JOURNAL 41