1 auditing section doctoral consortium 2006 auditing section midyear conference january 2006 linda...
TRANSCRIPT
![Page 1: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/1.jpg)
1
Auditing Section Doctoral ConsortiumAuditing Section Doctoral Consortium2006 Auditing Section Midyear Conference2006 Auditing Section Midyear Conference
January 2006January 2006
Linda McDanielLinda McDanielUniversity of KentuckyUniversity of Kentucky
Experimental Research in Assurance & Auditing
![Page 2: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/2.jpg)
2
Acknowledgements
Many colleagues, but particularly Jane Kennedy Bill Kinney Laureen Maines Mark Peecher
![Page 3: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/3.jpg)
3
Experimental Research Session: Objectives
■ Review strengths and weaknesses of experimental research
■ Discuss how to capitalize on these strengths (i.e., summarize elements of good experimental design)
■ Illustrate with an example arising from SOX reguations
![Page 4: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/4.jpg)
4
Experimental Research is … Systematic, controlled, empiricalcontrolled, empirical, critical
investigation of phenomena guided by guided by theorytheory and hypotheses about the presumed relations among such phenomena (Kerlinger)
characterized by, active manipulation of variables of
interest to generate new data the random assignment of
participants to specified conditions
controllcontrolled,ed,
presumedpresumedrelationsrelationstheotheoryry
guided guided byby
![Page 5: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/5.jpg)
5
Comparative Advantages
Ability to test causal relations, not just associations Manipulate variables of interest
Internal validity Control/randomize effects of other
variables Disentangle variables confounded in
natural setting
![Page 6: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/6.jpg)
6
Timeliness: no need to wait on the real world to create data
See McDaniel and Hand (CAR, 1996)
Ex ante research is possible Conditions that do not exist in natural
settings can be created in the lab Gaynor, McDaniel, and Neal (TAR,
forthcoming) Hirst and Hopkins (JAR, 1998)
Comparative Advantages
![Page 7: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/7.jpg)
7
Comparative Advantages
Examination of sub-judgments (determinants of decisions) and processes
Kadous, Kennedy, and Peecher (TAR, 2003) Hoffman, Joe, and Moser (AOS, 2003) Maines and McDaniel (TAR, 2000)
Thus, experiments can answer Thus, experiments can answer how, how, when, and whywhen, and why important features of important features of the accounting process and the accounting process and environment influence environment influence behaviorbehavior as as well as decisionswell as decisions
![Page 8: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/8.jpg)
8
Relative Disadvantages
External validity Task is abstraction from real world Variables manipulated at discrete levels Participants may not be representative
Small sample size Limited access to participants for real-world,
complex auditing/accounting issues
Reduced ability to replicate No second chances (without significant costs)
![Page 9: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/9.jpg)
9
Designing an Experiment
After the researcher identifies an interesting, relevant question that calls for an experiment … (i.e., post Kinney 3 paragraphs)
… he/she must develop an effective (good) research design, i.e., to draw causal inferences Use theory to guide predictions
““It is the theory that decides It is the theory that decides what can be observed.”what can be observed.” Albert Einstein
Minimize threats to construct, internal, and statistical validity
![Page 10: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/10.jpg)
10
Vs and Zs Prior-influence & contemporaneous factors (alternative explanations)
4. Statisti
cal Validity
Operational Definition
X
Operational Definition Y
Operational
Libby et al. (2002) Predictive Validity Framework
Concept X
Concept Y
Conceptual
Independent (X) Dependent (Y)
1. Theory
2. Construct Validity 3.
5. Internal Validity
![Page 11: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/11.jpg)
11
Designing an Experiment
What variables will you manipulate?
Number & levels of independent variables
Interactions?Libby et al. (2002) Predictive Validity Framework
![Page 12: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/12.jpg)
12
Designing an Experiment
What variables will you control? (internal validity) Account for Vs and Zs (see Kinney (TAR, 1986))
by Random assignment of participants
Hold variables constant by design/ selection (within-participant design; match on Vs)
Measure covariates / statistically remove effects (e.g., covariate analysis, regression)
![Page 13: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/13.jpg)
13
Other Necessary Design Choices
Professional participants?
Incentives?
Within- versus between- participants design?
See Libby, Bloomfield, and Nelson (AOS, 2002) for a discussion of each
![Page 14: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/14.jpg)
14
Professional Participants?
Theory should dictate this choice Libby and Kinney (TAR, 2000) Maines and McDaniel (TAR, 2000) See also Libby and Luft (AOS, 1993)
Professionals are a limited resource
Libby, Bloomfield & Nelson (AOS, 2002)
Professionals exhibit stronger selection bias relative to non-professional groups
Peecher & Solomon (IJA, 2001)
![Page 15: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/15.jpg)
15
Incentives?
Why? “…no skin in the game…”
When? Camerer & Hogart (JRU, 1999)
![Page 16: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/16.jpg)
16
Within- versus Between- Participants?
Enhanced statistical power as participants serve as their own control
See Schepanski, Tubbs, and Grimlund (JAL, 1992)
Increased salience of treatment effects
Vulnerability to carry-over effects
Requires proper statistical analysis
![Page 17: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/17.jpg)
17
Turning Observations into a Researchable Question
Do the new SOX regulations related to NAS result in improved audit quality?
Why is this interesting or important?
How can we examine?
![Page 18: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/18.jpg)
18
Real World Problem and Regulatory Actions After corporate abuses, SEC seeks to ban
all auditor-provided NAS Concerns about auditor independence, audit
quality, and investor confidence
Conceding certain NAS improve audit quality, SEC limits NAS auditors can provide to clients but requires ACs to pre-approve services after considering
auditor independence and audit quality Registrants to disclose AC pre-approvals and
fees paid to auditor (by category)
![Page 19: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/19.jpg)
19
An Example
The Effects of Joint Provision and Disclosures of Non-audit
Services (NAS) on Audit Committee (AC) Decisions and
Investor Preferences
![Page 20: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/20.jpg)
20
Theory
Pre-approval process makes ACs directly accountable to 3rd parties for auditor independence and audit quality
Disclosures (of pre-approvals and audit fees) makes ACs publicly accountable to investors for perceived independence
anecdotal reports suggest ACs are avoiding allowable NAS
![Page 21: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/21.jpg)
21
Predictions / Research Hypotheses
ACs will be more likely to recommend joint
provision when the NAS improves audit quality (AQ)
less likely to recommend joint provision when public disclosures are required
The disclosure effect holds even when ACs believe joint provision improves AQ
![Page 22: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/22.jpg)
22
NAS/AQ relation & required public
disclosures
Predictive Validity Framework
Conceptual
Independent (X) Dependent (Y)
Account-ability
ACs pre-approval decisions
Type of NAS and type of company
Joint provision recommendatio
n
Vs and Zs Experience with NAS approval; beliefs about effects of NAS on auditor
independence and synergies with audit; audit experience, etc.
Operational
![Page 23: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/23.jpg)
23
Operational Independent Variables
Type of Service: Effect of NAS on Audit Type of Service: Effect of NAS on Audit QualityQuality
Risk Management ServicesRisk Management Services
Joint provision improves audit quality
Human Resource ServicesHuman Resource Services
Joint provision has no effect on audit quality
![Page 24: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/24.jpg)
24
Measured Independent Variable
Measured Independent Measured Independent Variable:Variable:
Belief about NAS and audit quality relation
See Libby et al. (AOS, 2002) for when this approach is justified and implications for interpretation
![Page 25: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/25.jpg)
25
Operational Independent Variables
Type of Company: Disclosure Type of Company: Disclosure RequirementRequirement
Publicly-traded companyPublicly-traded company
Company is required to make public disclosures
Privately-held companyPrivately-held company
Company is not required to make public disclosures
![Page 26: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/26.jpg)
26
Operational Dependent Variableand Controls
Measured Dependent Measured Dependent Variable: Variable:
Joint provision recommendation Reasons for and against firm selection
Manipulation Checks / Control Manipulation Checks / Control Variables: Variables:
Audit quality manipulation checkNAS quality by provider Effect of joint provision on auditor objectivityDemographic information
![Page 27: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/27.jpg)
27
Participants & Other Choices
Participants were Corporate Directors attending a KPMG Audit Committee Institute Roundtable
No monetary incentives
Between-participants design
![Page 28: 1 Auditing Section Doctoral Consortium 2006 Auditing Section Midyear Conference January 2006 Linda McDaniel University of Kentucky Experimental Research](https://reader035.vdocuments.mx/reader035/viewer/2022062417/5514c29f550346935c8b471c/html5/thumbnails/28.jpg)
28
Some Lessons Learned
1. Work on projects that really interest you and for which you have a comparative advantage!
2. Good experimental papers require a lot of up-front time and effort. This pays off!
3. Always:Write Kinney 3 paragraphs / have others
review!Prepare Libby boxesPilot test (as many times as necessary)Share with colleagues often throughout the
process