Closing the Gap between Auditor Performance and Regulators’ Expectations when Auditing Fair Value Measurements: Evidence from Practicing Audit Partners

Steven M. Glover Brigham Young University

Mark H. Taylor Case Western Reserve University

Yi-Jing Wu Case Western Reserve University

April 2014

2014 Deloitte Foundation/University of Kansas Auditing Symposium

Acknowledgements: We thank the Center for Audit Quality for funding this project as well as the Center’s Research Advisory Board for its feedback to help refine our survey. We also appreciate those practicing partners who responded to our survey. This paper has benefited from helpful comments from the workshop participants at Case Western Reserve University. Steve Glover is grateful for the funding provided by the K. Fred Skousen Endowed Professorship. Mark Taylor appreciates the funding provided by the Andrew D. Braden Endowed Professorship. Finally, although funding for the research project described in this paper was provided by the Center for Audit Quality, the views expressed in this paper are those of the authors and not those of the Center for Audit Quality or the participating accounting firms.

Closing the Gap between Auditor Performance and Regulators’ Expectations when Auditing Fair Value Measurements: Evidence from Practicing Audit Partners

ABSTRACT: Auditing fair value measurements (FVMs) is a challenging area for auditors, as evidenced by the numerous PCAOB inspection findings in recent years. Ambiguity in auditing standards coupled with the complexity associated with auditing of FVMs results in disagreements between audit firms and the PCAOB regarding what constitutes sufficient appropriate audit evidence. Using a survey, we obtain audit partners’ views regarding the PCAOB inspection process related to FVMs. In doing so, this study aims to narrow the gap between auditor performance and the PCAOB’s expectations by identifying areas for potential improvement. This study provides insights regarding audit partners’ views on the auditability of and reporting on FVMs with significant estimation uncertainty. Our results also provide insights regarding significant challenges auditors face in auditing of FVMs with a focus on two key areas: evaluating the reasonableness of management’s assumptions and valuation methods, and use of valuation experts. We find that the lack of verifiable and corroborative evidence, and auditors’ reliance on valuation experts due to their limited knowledge and expertise regarding complex valuation inputs, analyses, and models contribute to the gap between auditor performance and regulators’ expectations. Our results suggest that current auditing standards may not provide sufficient guidance to auditors or inspectors on what constitutes sufficient, appropriate audit evidence around FVMs. Finally, regarding estimates with extreme estimation uncertainty, our results indicate that extreme estimation uncertainty exists in audited financial statements; and, perhaps no amount of additional audit procedures can limit the irreducible inherent estimation uncertainty to levels that regulators would deem to be reasonable.

1

Closing the Gap between Auditor Performance and Regulators’ Expectations when Auditing Fair Value Measurements: Evidence from Practicing Audit Partners

INTRODUCTION

Auditing fair value measurements (FVMs) is a challenging and complex area given the

subjectivity inherent in estimating future events and the potential for a high degree of

measurement uncertainty (Bratten, Gaynor, McDaniel, Montague, and Sierra 2013; Christensen,

Glover, and Wood 2012; PCAOB 2007). Auditing complex FVMs requires considerable

professional judgment. The inherent uncertainty of FVMs and the subjective nature of the audit

evidence provide regulators and financial statement users more than typical leeway to second-

guess the quality of the auditors’ work (Peecher, Solomon, and Trotman 2013).

PCAOB inspections have repeatedly identified audit deficiencies in auditing FVMs (e.g.,

PCAOB 2008a; PCAOB 2012a). PCAOB inspection reports from 2007-2011 reveal that the

most common deficiency is failure to evaluate the reasonableness of management’s underlying

assumptions and methods. The difficulties of auditing FVMs have increased auditors’ use of

pricing services and valuation specialists, which is an area that has induced the PCAOB to

criticize the firms for over-relying on the work of specialists and pricing services (PCAOB

2010a; PCAOB 2012a; Bratten et al. 2013). However, audit firms have frequently disagreed with

these PCAOB conclusions (Church and Shefchik 2012) highlighting an apparent expectation gap

that exists between auditors and regulators regarding the nature and extent of audit evidence

necessary to support audits of FVMS (hereafter, the “FVM expectation gap”).

While several concurrent studies provide insights regarding the process and challenges of

auditing complex estimates including FVM, and auditors’ use of in-house valuation specialists

(Cannon and Bedard 2013; Griffith, Hammersley, and Kadous 2014; Griffith 2013), research

regarding the audit of FVM remains sparse (Bratten et al. 2013). The current study extends the

literature on auditing of FVM by gathering survey data from audit partners to identify possible

2

causes of the FVM expectation gap as well as potential steps that could be taken to reduce the

gap. We then provide additional insights regarding key challenges auditors face in auditing

FVMs to identify areas for potential improvement in auditor performance, auditing standards,

and the PCAOB inspection process. Specifically, we focus on challenges auditors encounter in

the following two key FVMs areas most frequently mentioned in PCAOB inspection reports to

contain deficiencies: (1) evaluating management’s assumptions and valuation methods, and (2)

use of valuation experts. Finally, we analyze audit partners’ views regarding the auditability of

and reporting on estimates with extreme measurement uncertainty, especially in light of recent

commentary that questions the auditability of these complex estimates (Christensen et al. 2012).

To address our research objectives, we distributed the survey primarily to field practice

audit partners with significant experience in the planning, supervising, and executing audits of

FVMs.1 The survey uses both objective and open-ended questions and is divided into sections

aimed gathering insights regarding auditors’ views regarding PCAOB inspection process,

general challenges in auditing FVMs, specific challenges with use of valuation experts, and high

estimation uncertainty and presentation.

We find that a majority of participants (over 60 percent) believe that the PCAOB

inspectors expect more evidence to be gathered than auditing standards technically require. For

audit partners serving on engagements where inspectors identified a FVMs-related audit

deficiency, 100 percent disagreed with the inspector’s assessment. Primary reasons for

disagreement are (1) the PCAOB did not appropriately consider misstatement risk and totality of

1The survey results presented in this study are based on our primary survey of practicing audit partners. We also designed a separate survey for national office audit partners that is more focused on PCAOB inspections and high estimation uncertainty and presentation of FVMs. We received responses from 5 national office partners, one from each of the participant firms, and we use their responses to open-ended questions to provide further insights regarding auditing FVMs. Thus, in the presentation of results, when appropriate, we report some of the national audit partner responses to corroborate responses from practicing audit partners and to highlight unique insights on the topic experienced by partners at the national level.

3

the evidence gathered, (2) the PCAOB expects a level of precision in auditing FVMs that is

simply not achievable, and (3) PCAOB inspectors have unreasonable expectations regarding

management’s and the auditor’s use of and reliance on third-party specialists. The top two

suggestions audit partners provided for improving the PCAOB inspection process and audits of

FVMs are implementation of a within-engagement risk-based inspection approach and revising

auditing standards to provide better guidance.

With respect to significant challenges in auditing FVMs, a number of key findings

emerged that may be contributing to the FVMs expectation gap. The top three challenges our

participants cite are (1) lack of verifiable and corroborative evidence, (2) degree of difficulty in

assessing reasonableness of assumptions, and (3) high degree of subjectivity involved in

estimates. Moreover, management’s lack of knowledge regarding the valuation process creates

additional challenges for auditors. Over 50 percent of participants report concerns with

management’s knowledge regarding valuation processes. Further, over 60 percent believe that

the audit challenges differ between financial vs. nonfinancial assets/liabilities primarily with

respect to the availability of market data, as auditors must rely more on management’s

assumptions for the latter due to the lack of observable market information. In terms of issues

encountered when using third-party pricing services, partners frequently note that their inability

to obtain sufficient information about the pricing services’ proprietary models is problematic

when trying to understand how the pricing services derive the FV of investment securities. These

challenges may explain why PCAOB inspection reports repeatedly note deficiencies in the area

of evaluating the reasonableness of management’s assumptions and valuation models and use of

valuation experts.

Further adding to the complexity of auditing FVMs, as well as calling into question the

fundamental auditability of FVMs, is “extreme measurement uncertainty” that can exist in

4

financial statements in which the reasonable range of estimation uncertainty on the measurement

date can be many times greater than materiality (Christensen et al. 2012). We find that one of the

primary reasons that auditors disagree with PCAOB inspection findings relates to the estimation

uncertainty inherent in FVMs. A majority of our participants (60 percent) suggest that the current

requirement to provide a high level of assurance that a point estimate is fairly stated within

auditor materiality be reconsidered because providing such assurance on point estimates is

unrealistic and potentially misleading given the level of subjectivity, complexity and uncertainty.

Nearly all participants (93 percent) support the idea of revising auditing standards to provide

additional clarity and guidance around estimates characterized by extreme measurement

uncertainty. The top three areas in which auditors would welcome more guidance from standard-

setters include (1) clarification about an acceptable reasonable range of estimation uncertainty,

(2) clarification regarding audit requirements and procedures, and (3) the standard-setting

process should influence audit procedures rather the PCAOB inspection process.

Our results highlight a number of factors that may be contributing to the FVM

expectation gap. Findings suggest that existing auditing standards may not provide sufficient

guidance to inform auditors or inspectors on what constitutes sufficient, appropriate audit

evidence, and that PCAOB inspectors expect auditors to gather more evidence than auditing

standards technically require. As a result, in some cases auditors may be reacting to inspectors’

preferences rather than carefully applying existing auditing standards. A potential consequence

of an inspection-driven approach to interpreting/applying auditing standards is that audit efforts

may be linked to “inspection risk” rather than audit risk (Glover and Prawitt 2013), which may

lead auditors to devote less attention to areas that present greater audit risk. Our results highlight

two additional issues related to the FVM expectation gap. One concerns the availability of

reliable data to support key assumptions (either management’s or pricing services’). Another

5

relates to auditors’ overreliance on pricing services and valuation specialists, caused in part by

management’s lack of knowledge/expertise to fully evaluate complex valuation inputs, analyses,

and models (Bratten et al. 2013; Griffith et al. 2014; Martin, Rich, and Wilks 2006). These are

additional areas in which regulatory efforts may be able to help close the FVM expectation gap.

To our knowledge, this is the first study that addresses antecedents of and possible

solutions for narrowing the FVMs expectation gap. We extend the literature by providing

insights regarding ways to improve audit practice and the inspection process around FVMs.

Second, our study is also the first to provide important insights from practicing audit partners on

the auditability of and reporting on FVMs with high estimation uncertainty, an area in which

audit partners believe standard-setters can provide additional guidance and clarity. Finally, this

study contributes to the extant research regarding key challenges in auditing FVMs. In particular,

we extend the literature on the use of valuation experts (e.g., Griffith 2013; Griffith et al. 2014)

by providing evidence on the use of and challenges related to different types of experts,

including third-party pricing services and valuation specialists.

The remainder of this paper is organized into the following sections. The next section

discusses relevant background literature followed by descriptions of the survey design, data

collection and analysis. The fourth section presents our results. Finally, we discuss conclusions

and implications of this study.

BACKGROUND & RELATED LITERAURE

Previous research and our own analysis of PCAOB inspection reports identify numerous

deficiencies in the area of auditing FVMs (Church and Shefchik 2012; Griffith et al. 2014;

PCAOB 2010a). Valuation-related audit issues remain one of the top deficiencies noted and the

frequency of reported deficiencies has increased in recent years (Griffith et al. 2014; VRC 2013).

6

However, the firms’ have frequently disagreed with PCAOB conclusions (Church and Shefchik

2012) highlighting an apparent FVM expectation gap between auditors and regulators.

Previous research suggests that one source of the gap may be that auditors lack the

knowledge and expertise to fully understand and evaluate FVMs (Bratten et al. 2013; Griffith et

al. 2014; Martin et al. 2006). To address these limitations, auditors frequently engage valuation

specialists and pricing services to assist with valuation analyses (Bratten et al. 2013; Griffith et

al. 2014). However, extensive reliance on valuation specialists to assess the reasonableness of

assumptions and valuation methods has lead to additional audit deficiencies frequently cited in

PCAOB inspection reports, such as overreliance on the work of specialists (e.g., PCAOB 2010b;

PCAOB 2012a; Bratten et al. 2013).

Research regarding audits of FVMs is limited and in its early stages (Bratten et al. 2013;

Martin et al. 2006).2 To ensure that the direction of future research addressing auditing FVMs is

meaningful to practice and standard setters, Bratten et al. (2013) call for researchers to continue

to gain a better understanding of the process of auditing FVMs and the critical challenges

auditors are encountering in the field. To do so, several concurrent studies, using interview and

field-based survey methods, examine the process of auditing FVM, challenges auditors

encounter when auditing FVM, and auditors’ use of in-house valuation specialists (Bratten et al.

2013; Cannon and Bedard 2013; Griffith et al. 2014; Griffith 2013).

Griffith et al. (2013) interviewed audit partners and senior managers from six large audit

firms to better understand the process of auditing of complex estimates, including FVMs.

Consistent with the PCAOB inspection findings, Griffith et al. (2014) find that auditors tend to

over rely on management’s assertions, fail to notice and/or reconcile conflicting evidence

2We do note that recent experimental research investigates how various environmental and task factors affect auditors’ judgments and decisions in this complex audit area (e.g., Backof, Thayer, and Carpenter 2014; Earley, Hoffman, and Joe 2013; Griffin 2011; Griffith, Hammersley, Kadous, and Young 2013; Joe, Vandervelde, and Wu 2014; Maksymov, Nelson, and Kinney 2013).

7

between management’s assumptions and external evidence, and encounter coordination

difficulties when dealing with the firm’s in-house specialists. The paper identifies two potential

causes for the difficulties encountered by auditors when auditing of complex estimates. One is

that audit firm policies and auditing standards tend to over emphasize verification of

management’s model instead of “critically evaluating the reasonableness of the estimate”

(Griffith et al. 2014, 6). The other cause is that the division of knowledge and work between

auditors and the audit firm’s in-house valuation specialists results in auditors’ lack of knowledge

and expertise to evaluate assumptions and valuation models related to complex estimates.

Conducting interviews with audit partners and managers, Griffith (2013) provides more

detailed description of auditors’ use of the firm’s in-house specialist when auditing FVM and

problems encountered with the involvement of in-house valuation specialists. Griffith finds that

given the lack of clear auditing standards on the use of internal firm specialists, auditors adapt

existing guidance for use of external specialists to their use of an internal specialist. Her study

identifies three main areas that are challenging to auditors when using an in-house specialist.

They include (1) differences in views of auditors and valuation specialists, (2) coordination

difficulties between the two parties, and (3) ambiguity in the responsibilities of each party.

Another related study by Cannon and Bedard (2013), using a survey, provide additional

evidence regarding significant challenges audit managers and senior managers encounter when

auditing FVM. Consistent with the PCAOB inspection findings, they find that evaluating the

reasonableness of management’s assumptions and valuation methods is one of the most

frequently encountered challenge auditors face when auditing FVM. Cannon and Bedard (2013)

also find that a majority of participants report encountering extreme measurement uncertainty

such that the reasonable range is greater than materiality. Based on their results, they conclude

8

that auditors encounter difficulties with providing positive assurance on FVM and proposing

audit adjustments given that model assumptions and valuation methods are highly subjective.

Accessing practicing partners in the field, the current study extends prior literature in the

key following ways. Our study is the first to gather and report on audit partners’ views regarding

the PCOAB inspection process and primary causes for disagreement between auditors and

inspectors. Hence, we contribute to the literature by identifying ways in which the FVM

expectation gap can be closed. Second, while prior literature notes problems with current

auditing standards and guidance that are contributing to challenges auditors encounter with

auditing complex estimates (Bratten et al. 2013; Griffith 2013; Griffith 2014), this study extends

the extant literature by providing insight from audit partners regarding areas in which regulators

can provide clarification and additional guidance to help improve audit performance. Third,

concurrent studies focus primarily on auditors’ use and challenges related to use of the firm’s in-

house valuation specialists (Griffith 2013; Griffith 2014). The current study contributes to the

existing literature by exploring auditors’ use of third-party pricing services and valuation

specialists. Finally, this study contributes to prior literature by reporting on auditors’ views

regarding the auditability of and reporting on estimates with high estimation uncertainty along

with auditors’ views on areas in which they welcome improved clarity in the auditing standards

and guidance from standard-setters.

In light of PCAOB inspection reports and research in the area, we first present results

regarding audit partners’ views about the PCAOB inspection process and why in some cases they

disagree with the PCAOB inspectors. Partners provide their suggestions for how the PCAOB can

help improve auditing of FVMs as well as the inspection process. We next report results

regarding key challenges auditors encounter with respect to two key areas: (1) the

evaluating/testing management’s assumptions and valuation methods, and (2) the use of

9

valuation specialists and pricing specialists. Finally, we present results regarding the auditability

of and reporting on estimates with high estimation uncertainty. Results of this study, along with

insights from prior and other concurrent studies (e.g., Cannon and Bedard 2013; Griffith et al.

2014; Griffith 2013), will allow scholars, practitioners, and regulators to better understand how

to reduce the expectation gap noted above, and in turn improve auditor performance.

SURVEY DESIGN

We designed a survey that included both objective and open-ended questions.

Beforehand, we examined PCAOB inspection findings, prior studies, auditing standards, and we

also conducted interviews with two national-level partners. Two additional audit partners and

one retired audit partner who did not participate in the study provided initial feedback regarding

our survey. Further, partners from participating firms and the CAQ’s Research Advisory Board

provided additional feedback on the survey, which we incorporated to ensure our questions

captured the areas of primary importance and interest to auditors regarding auditing of FVMs.

The survey included several sections.3 The first section includes demographic questions

including position, level of involvement with FVMs, primary industry, specialization, etc. Other

sections were designed to gather information regarding auditors’ views in the following key

areas: key challenges when auditing FVMs and use of valuation experts, the PCAOB inspection

processes, and the auditability and presentation of estimates with high estimation uncertainty.

DATA COLLECTION AND ANALYSIS

Data Collection

We distributed our survey with assistance from the CAQ’s Director of Research, who

coordinated with five audit firms annually inspected by the PCAOB to identify audit partners

with experience in audits of FVMs. The majority of surveys were completed online; the 3Results of this study were collected using a survey that is also part of another study. We only present results of the survey that address the key objectives of the current study.

10

remaining surveys were completed via hard-copy. The survey was provided directly to potential

participants, and follow-ups were initiated to encourage participants to complete the survey.

Coding of Open-Ended Questions

Our survey included numerous open-ended questions to improve the potential richness of

the data gathered. Two of the researchers independently read responses associated with each

open-ended question to derive coding themes to categorize responses to each question and later

met to finalize the themes. Two doctoral students then independently categorized responses

based on the pre-established themes. Responses range in length from a few words to a brief

paragraph. Thus, individual responses could be coded to several themes or to as few as one. The

two Ph.D. students met to resolve coding differences. The Cohen’s Kappa measure is 0.61,

which suggests an acceptable inter-rater agreement level.4

Participants

Thirty-two audit partners responded to the survey. Table 1 presents participants’

demographic information. Mean response regarding the extent to which participants are involved

in the planning, supervising, or executing auditing significant estimates, including FVMs, is

10.28 on an 11-point scale anchored “very little involvement” (1) and “a great deal of

involvement” (11), indicating our participants have had significant involvement in auditing

FVMs. On average, participants have 22.3 years of audit experience. Participants practice

primarily in three industries: Financial Services/ Depository/Financial Institutions/Asset

Management (43.8 percent), Industrial Products and Manufacturing (25.0 percent), and

Consumer Products and Goods (15.6 percent). Participants rated their valuation expertise as 8.53

on an 11-point scale anchored with “very little expertise” (1) and “a great deal of expertise” (11).

4A rule of thumb is that Kappa measures between 0.40 and 0.70 are acceptable (Neuendorf 2002). Further, Landis and Koch (1977) provide benchmarks for interpreting Kappa. Kappa of 0.61 to 0.80 represents “substantial” inter-rater agreement strength.

11

Nearly 19 percent of participants indicate they specialized in a particular type of FVM. Top

specializations include investment securities, asset impairment, real estate and goodwill.

[INSERT TABLE 1 ABOUT HERE]

RESULTS

Auditors’ views regarding PCAOB inspection process

As noted, PCAOB inspection reports identify numerous deficiencies in the area of

auditing FVMs (Church and Shefchik 2012; Griffith et al. 2014; PCAOB 2010a). Consistent

with Griffith et al.’s (2014) analyses of inspection reports for 2008 and 2009, our analysis of

PCAOB inspection reports from 2007 - 2011 indicates that one of the most frequently noted

deficiencies PCAOB inspectors identify is failure to evaluate the reasonableness of

management’s underlying assumptions and valuation methods. Evaluating the reasonableness of

model assumptions is an especially challenging task given the degree of judgment and

measurement uncertainty involved (Bratten et al. 2013; Christensen et al. 2012; Griffith et al.

2014; PCAOB 2012a). To overcome these challenges, auditors have often turned to valuation

specialists and pricing services, which has resulted in PCAOB criticism over relying too much

on these specialists (e.g., PCAOB 2010b; PCAOB 2012a; Bratten et al. 2013).

Examples of both types of deficiencies include:

1. For hard-to-value financial instruments “the Firm failed to obtain an understanding of the specific methods and/or assumptions underlying certain fair value measurements that were obtained from pricing services or other third parties and used in the Firm’s testing of the fair value of the hard-to-value financial instruments…. for certain securities, the Firm tested the issuer’s process of obtaining prices from third parties and corroborating these prices through the use of models. The Firm, however, failed to evaluate the appropriateness of the models and the reasonableness of the underlying assumptions” (PCAOB 2011c, 17).

2. “The Firm failed to obtain an understanding of the specific methods and/or assumptions underlying fair value measurements that were obtained from pricing services or other third parties and used in the Firm's testing of the fair value measurements of certain financial instruments without readily determinable fair values (hard-to-value financial instruments)” (PCAOB 2012d, 16).

12

3. For goodwill impairment testing “the Firm selected the market approach to support its

audit conclusions, one of the three valuation methods used by the specialist…The Firm failed to sufficiently evaluate the reasonableness of the assumptions used by the specialist in that it failed to evaluate the comparability of the transactions used to develop the control premium, despite indications that the transactions included companies that differed significantly from the issuer” (PCAOB 2010b, 7).

The high degree of subjectivity inherent in FVMs along with ambiguity in auditing

standards require auditors to exercise significant professional judgment, which is susceptible to

second-guessing by the PCAOB in hindsight (Peecher et al. 2013). Consequently, the firms’

responses to the PCAOB inspection findings often indicate firm’s disagreement with the PCAOB

inspectors’ conclusions (Church and Shefchik 2012). For instance, firms often express in their

letters included in the PCAOB inspection reports that differences between auditor performance

and the PCAOB inspectors’ expectations should be expected given differences in professional

judgment (e.g., PCAOB 2011c; PCAOB 2008b).

Although a number of factors undoubtedly contribute to FVMs expectation gap,

including situations in which the audit actually was deficient, we first consider partners’ views of

the inspection process and address auditors’ reported challenges in subsequent sections. To learn

more, our survey addresses the following questions related to the PCAOB inspections.

1. Do the PCAOB inspectors expect the audit team to collect more evidence than required by existing auditing standards?

2. What are primary reasons audit firms disagree with the PCAOB’s assessment when inspectors identified an audit deficiency related to the auditing of FVMs?

3. How can the PCAOB help improve the auditing of FVMs as well as the inspection

process? Over 75 percent of our participants (N=23) served as the engagement partner on an

engagement that has been selected for PCAOB inspections. Participants whose engagement had

been selected for PCAOB inspection were instructed as follows: “In answering the following

13

questions in this section, if you have had more than one audit engagement inspected, please

consider an engagement that contained the most complex FVMs.”

To address our first key question, we asked engagement partners: “In your opinion, did

the PCAOB inspectors expect the audit team to collect more evidence than what the firm

believes represents the requirements of auditing standards (e.g., PCAOB AU 328) when auditing

FVMs?” As shown in Panel A of Table 2, approximately 60 percent of participants believe that

the PCAOB inspectors expected more than the auditing standard requires. Consistent with these

engagement partners’ views, one national partner provided an example:

“The engagement team documented risks associated with various investments and thus bi-furcated the population into various risk buckets. The engagement team took each bucket and designed testing strategies. [The] PCAOB did not have any issues with the higher risk buckets. However, in the low risk bucket (only had 49 securities - level 1), the engagement team tested 38 of the 49 but did not test an item over [tolerable misstatement]. Overall, 80 percent of fair values had been tested through various strategies involving risk ratings and we disagree that [auditing standards require that] the engagement team MUST test the item over [tolerable misstatement] in the low risk bucket. The PCAOB inspector disagreed. When they write observations, you must address them. Most of the time, it is less costly to agree and perform some minor testing and put the matter behind us, than to fight.”

[INSERT TABLE 2 ABOUT HERE]

This raises a potentially interesting question for PCAOB leadership to consider and

clarify for low risk securities that represent low risk of material misstatement: do the auditing

standards require substantive testing of each security over tolerable misstatement? Audit theory

and practicing experts would say no. However, as the respondent points out, theory and practice

are of little comfort when inspector’s criticisms must be addressed.

Another national partner also elaborated on the challenge of gathering sufficient evidence

in subjective and judgmental areas:

“By experience, no matter how much evidence you gather and you believe you have gathered sufficient appropriate audit evidence on judgmental assumptions, someone can always second guess you. What is reasonable? Reasonable is extremely difficult concept and one that causes massive issues in the audit industry. Your concept of reasonable and

14

my concept of reasonable will always likely be different. When you add pages of documentation to support why you believe something is reasonable, it provides additional information to be challenged as well. This is a never-ending cycle. Auditors must make decisions based on the evidence gathered and sometimes looking at something [with the benefit of hindsight] a year later - even the auditor may think - hmmm, maybe I should have done something different. But, really - at the time the judgment had to be made - it was reasonable based on [the circumstances and] all the evidenced gathered.” This dilemma provides another fertile area for the PCAOB to consider both the

profession’s frustration with the “never-ending cycle” and the practical realities of the

tremendous subjectivity inherent in the concept of “reasonable.” The PCAOB should also

recognize the fact that assessments of what is “reasonable” in hindsight versus foresight are

likely to differ. Perhaps auditing standards and inspection policies could be revised to

contemplate these issues and provide guidance in ways auditors can appropriately exercise

professional judgment to gather sufficient appropriate evidence without fear of PCAOB

inspector reprisals based on hindsight and differences of interpretation rather than legitimate

shortcomings in the auditor’s performance at the time of the judgment.

To identify reasons for auditor disagreements with PCAOB inspection results, we asked

audit partners that had their engagements inspected by the PCAOB whether PCAOB inspectors

identified any audit deficiency related to the auditing of FVMs. Roughly half (N=11) of our

participants indicated “yes.” For those that answered “yes” we asked whether they agreed with

the PCAOB’s assessment and to explain why or why not. Even with the benefit of hindsight and

knowing the concerns articulated by the PCAOB inspectors, 100 percent of the engagement

partners disagreed with the PCAOB’s conclusions. We asked why they disagreed. Table 3

reports the three themes that emerged from the open-ended responses. The most frequently

mentioned reason for disagreement is that the partners believed that the PCAOB did not give

appropriate consideration to the risk of misstatement, the nature of the security, and the totality

of the audit evidence gathered. For example, one partner noted:

15

“The PCAOB comments related to roll-forward procedures on a low risk securities portfolio (the roll-forward analytical was inside tolerable range but the overall balance was greater than materiality) and the PCAOB did not give appropriate consideration to the risk of misstatement, the nature of the securities, and the totality of audit procedures performed in the roll-forward period. The PCAOB is attempting to legislate auditing standards and improvements to management’s internal controls through their [inspection] process instead of issuing appropriate guidance or examples.”

Another partner indicated:

“We believe we had tested a significant portion of the investment portfolio and had covered the key risks. We also evaluated the key controls over valuation and placed reliance on those controls, yet the PCAOB gave us no “credit” for the controls testing.”

This is another area for the profession as well as PCAOB leadership and inspectors to consider—

What evidential “credit” or assurance is provided to the audit of internal control over financial

reporting? Auditing theory and practice suggests risk assessment and controls testing are

important considerations in evaluating the nature, timing, and extent of substantive testing.

Consistent with practice partners’ responses, one national office partner noted that differences

exist regarding what PCAOB inspectors and auditors believe constitutes an audit failure:

“Evaluation to the audit as a whole. [The] PCAOB believes if a [potential] difference is greater than performance materiality, an audit fails. No credit is given to other evidence. They measure to [a hypothetical standard of] what might happen, regardless of [the] remote [likelihood].”

[INSERT TABLE 3 ABOUT HERE]

This may be yet another area for the profession and the PCAOB to consider—What

constitutes an overall audit failure and how should auditors properly factor in the totality of audit

evidence and the risk of misstatement in the evaluation of the magnitude of a perceived audit

deficiency when considering whether it constitutes an audit failure.

The second most frequently mentioned reason for disagreement is that the PCAOB

expects a level of precision that is not be achievable, as one partner notes:

“In my opinion, the PCAOB standard of auditing regarding estimates, assumptions and inputs to valuation models are wholly unreasonable. By definition, the fair value models

16

are estimates [that] cannot be audited to the level of precision that the PCAOB is expecting.” Our findings combined with similar findings in other studies (e.g., Christensen et al.

2012, Cannon and Bedard 2013) suggests that the profession and the PCAOB may find it fruitful

to acknowledge the potential for inherent and irreducible uncertainty in complex Level 2 and

Level 3 FVMs and to consider the appropriate ways to report the uncertainty and what

constitutes sufficient appropriate audit evidence to support an FVM with a reasonable range of

estimation uncertainty that exceeds auditor materiality. 5 Related to this issue, we provide further

discussions later in the paper regarding the reporting of and auditability of estimates with

extreme measurement uncertainty.

The third most frequently mentioned reason for the disagreement with the PCAOB is

participants’ belief that the PCAOB’s expectation regarding management and auditors’ reliance

on third-party specialist is not reasonable. As one partner describes:

“The PCAOB’s expectation as to the level of management’s review of [third] party pricing is at a level that is likely not reasonable. The regulator seemed to assert that the issuer is required to actually calculate a specific fair value for a given security as its control, or to specifically validate each assumption in a specific security’s value in order to monitor its pricing vendor. However many company’s have knowledge of broad asset classes and movements within market indices that would detect significant differences in security values, without having to actually employ another specialist to validate their chosen specialist (pricing vendor).”

Later we discuss challenges auditors encounter when using third-party valuation and pricing

services and ways in which regulators can help improve auditor performance.

Obviously, reasonable disagreements between experienced audit partners and PCAOB

inspectors will occur. The fact that all of our partners disagreed with the PCAOB inspector’s

conclusions along with the reasons for disagreement noted above provide the PCAOB an

opportunity to consider ways to (1) improve auditing standards and guidance to clarify what 5We discuss further below that partners report that it is common for complex estimates to have significant and irreducible estimation uncertainty many times larger than performance materiality.

17

constitutes sufficient, appropriate evidence when auditing complex FVMs, (2) better train

inspectors on audit risk, evidence, accounting for FVMs, and what constitutes an audit deficiency

in the area of auditing FVMs, and (3) create an appeal or arbitration process whereby important

but reasonable differences of opinion between engagement teams and inspectors can be elevated

and resolved, perhaps by independent experts. Some form of appeal or arbitration process would

undoubtedly go a long way toward improving audit quality as it would help to clarify areas

where auditors need to improve. Also, such a process would identify areas where inspectors may

be imposing unnecessary costs and inefficiencies in terms of the firms’ responses to specific

inspection findings (i.e., additional audit procedures performed) as well as firms’ changes to

audit methodologies vis-à-vis areas of “inspection risk” (Glover and Prawitt 2013).

Finally, we asked partners to provide suggestions about how the PCAOB could help

improve auditing FVMs. Results presented in Table 3 show that the most frequently mentioned

suggestion is the PCAOB to use a risk-based approach consistent with auditing standards and to

consider the totality of audit evidence when auditing fair values. One partner stated:

“While risk-based audit approaches are routinely discussed in SEC/PCAOB literature, [inspection] teams have struggled to understand or be satisfied with various sampling approaches rather than achieving near or actual 100 percent coverage of the population. Fair value measurements routinely involve the use of multiple assumptions. PCAOB inspectors struggle to differentiate the audit work performed on certain, but not all of those assumptions, when certain of those assumptions have the greatest potential for impact or change in that value.”

Another partner also indicated:

“Focus on where the risk of misstatement is highest. This would be in the area of whether a security is Level 2 or Level 3. Then focus on how we look for management bias or incompetence in their estimation process. The PCAOB seems to preach about adequate risk assessments, then looks at every security like it should be treated the same way. And if they really want to improve audit quality, then they should treat all firms in the inspection process the same way. My understanding is that they do not look at the ICFR work of the firms that are not inspected annually (that was told to me by the PCAOB during this year’s inspection). That creates an uneven playing field in the market place that requires larger firms to do more audit work than smaller firms (that may lack

18

expertise and not even know it). If you want to know how that is working out, just ask Madoff’s investors.”

The second most frequently mentioned recommendation is for the PCAOB to provide

clearer guidance of audit requirements regarding, for example, evidence, audit procedures and

documentation. One partner explained:

“We have moved away from auditors being able to apply audit judgment in the area of fair value measurement to where it is now a rules-based approach. That is all fine and good as long as the rules are clear and effectively communicated. However, the problem is that the PCAOB continues to change their expectations and the primary mechanism for communicating these changes seems to be through inspection comments versus proactively educating the profession on their expectations. This “gotcha” approach is counterproductive to improving the quality of audits. Rather, more energy should be spent by the PCAOB on auditor outreach.”

A national office partner’s response further emphasizes the practicing partner’s opinion:

“The audit standards need to address both the accounting literature as well as the ability to operationalize the requirements. We have no responsibility to manufacture evidence, but what [audit evidence] is available to all market participants and evaluated as to consistency and method (such as used by pricing services) is not accepted as adequate. We evaluate the audit evidence available. If we conclude it is relevant and reliable it may be used. The PCAOB would assume we need to evaluate all other choices. Is the auditor required to hire a specialist for every security for which there is no market? Or is the auditor to assess the risk of error to the financial statements as a whole?”

The third most frequently mentioned recommendation is for the PCAOB to acknowledge

that evidence gathered may not be able to get to the level of precision desired (i.e., within

materiality, inspectors need more realistic expectations). One partner explains:

“Evaluate the procedures in light of a reasonable range around the fair value estimate rather than in light of materiality, which can often be too small a range to use for purposes of evaluating an estimate for which there is no correct answer.”

Another partner adds:

“[For] allowance for loan losses or impaired loans, the level of precision between cause and effect is not as real as they would like it to be. Therefore their demands on conclusions to be based on accurate estimates using judgment cannot be supported as precisely as they would like. So a lot of time is spent in drafting memos and other support for estimates that end up being [imprecise] estimates in the end.”

19

In summary, our results suggest that differences exist between auditors and the PCAOB

inspectors regarding interpretations/judgments about the amount of evidence required when

auditing FVMs. In addition, participants overwhelming report disagreeing with the PCAOB

inspectors’ identified audit deficiencies related to FVMs. One primary reason for the

disagreement is that participants felt that the PCAOB inspectors did not give appropriate

consideration to the level of audit risk, nature of the security, and the totality of the audit

evidence gathered. Consequently, the top suggestion participants provide regarding how the

PCAOB could improve the audit of FVMs is for inspectors to use a risk-based approach

consistent with auditing standards, and to consider the totality of audit evidence gathered.

Results also reveal that current auditing standards may not provide sufficient guidance to

inform auditors on how complex FVMs should be audited. Instead, changes in audit practice are

being driven by the inspection process rather than through changing existing auditing standards

(cf. Griffith et al. 2014). This approach is problematic in that audit effort is driven the likelihood

of PCAOB scrutiny or “inspection risk” rather than audit risk (Glover and Prawitt 2013). This

concern is further emphasized by one national office partner’s response:

“We are less likely to do what is reasonable and required by standards and more likely to appease the inspection team.”

Another practice partner also notes audit strategies that are being driven by the inspection

process rather than audit risk:

“The deficiency related to the precision of the review by management of the pricing assessments made by the third party engaged by management. The issuer has a non-complex securities portfolio, engaged a national broker/dealer to value securities, obtained a SAS 70 that included testing of securities pricing as a key control, but it wasn't precise enough for the PCAOB. The PCAOB seems to be focused in the right areas in their exams, but they don’t pay particular attention to real risk of misstatement. The focus should be solely on identifying and valuing hard to price securities, not every security. In this case, we engaged a third party every year to value 100 percent of the securities and we never had a security price fall outside of our acceptable range of prices, but this was a substantive test the PCAOB didn’t consider a dual test when assessing our ICFR work.”

20

Audit efforts driven by inspection risk rather than audit risk could potentially lead to audit

efficiency and effectiveness issues. For lower audit areas, auditors may be over-auditing which

potentially shifts auditors’ focus and resources away from higher-risk areas (e.g., level 3 FVMs).

Moreover, professional judgment must be exercised when deciding which of several

applicable auditing standards (e.g., AU 328, Auditing Fair Value Measurements and Disclosures

and AU 342, Auditing Accounting Estimates) and practice alerts to apply when auditing FVMs

(AICPA 1989; AICPA 2003). The presence of multiple standards, practice alerts, and additional

clarifications through informal channels further compounds auditors’ task difficulty when

auditing FVMs (Bratten et al. 2013). Our findings emphasize the need for improvement and

clarification to existing auditing standards with respect to auditing FVMs.

Challenges in auditing of fair value measurements

In the previous section we report results regarding auditors’ views of the PCAOB

inspection process and areas in which audit partners suggest that regulators can improve auditing

of FVMs. In this and the next section, we explore key challenges auditors encounter when

auditing FVMs. Specifically, we focus on two key areas that PCAOB inspection reports

frequently identify as containing audit deficiencies: (1) evaluating the reasonableness of

management’s assumptions and valuation methods, and (2) using valuation experts, including

pricing services, in-house and third-party valuation specialists. In doing so, we identify areas that

may be contributing to the expectation gap. This section focuses on general challenges related to

auditing of FVMs; the next section focuses on challenges related to use of valuation experts.

When we asked participants what are the most significant challenges encountered when

auditing inputs for significant level 2 or level 3 FVMs where small changes to the inputs have

potentially material changes to the reported value, the three top challenges reported in Table 4

are (1) lack of verifiable and corroborative evidence including lack of 3rd party evidence (42.9

21

percent), (2) degree of difficulty in assessing reasonableness of assumptions and benchmarks

(14.3 percent) and (3) high degree of subjectivity involved in estimates (14.3 percent). To

elaborate on the challenges around obtaining verifiable and corroborative evidence, one partner

points out the following:

“The most difficult challenge is determining whether the assumption represents "market". Usually you can look at historical experience to determine if assumption is reasonable for the specific asset but it is hard to tell is such is what the market might use. For example prepayment assumption, you can determine based one experience of the asset, but that does not mean the marketplace would use similar assumption as they would build in some expectation of what interest rates are going to do in the future.”

Another partner echoes the above point:

“[A key challenge is] obtaining enough evidence to support the overall assertion, which is generally going to be based on numerous judgments that can't be proven but have to be considered for reasonableness.”

Consistent with PCAOB inspection findings, audit partners note significant challenges with

assessing the reasonableness of assumptions. For example one practicing partner notes:

“Where there is a lack of liquidity, fair value measurements are incredibly challenging. With the benefit of hindsight, what appeared to be reasonable assumptions at the time they were developed can appear to be entirely unreasonable. Often times, I do not think management is doing enough documentation and memorialize the thinking and rational at the time assumptions are developed.”

Finally, in terms of challenges with high degree of subjectivity involved in estimates, one

practicing partner points out the following:

“[the] biggest challenge is that inputs within a reasonable range could result in indicated values that differ greater than materiality. This precludes the ability to "stress-test" management's analysis. Thus, supporting one number for an input over another is a matter of significant judgment.”

[INSERT TABLE 4 ABOUT HERE]

Results and audit partners’ comments provided above highlight key challenges auditors face

when auditing complex FVM. These very challenges are likely causes of why the PCAOB

inspection findings repeatedly report deficiencies in the area of evaluating the reasonableness of

22

management’s assumptions and valuation models. Further, these challenges reported by our

participants raise questions regarding the auditability of estimates with extreme measurement

uncertainty (discussed later).

We also asked participants whether they believe audit challenges they previously

described are different between financial vs. nonfinancial asset/liabilities. Approximately 63

percent report that they believe challenges differ between these two types of FVMs. As shown in

Table 4, the top two most frequently mentioned differences are (1) difference in availability of

market data (56.5 percent), and (2) degree of auditor familiarity (13 percent). In terms of

differences between financial and non-financial assets/liabilities, one partner notes,

“It has been my experience, that when dealing with nonfinancial assets/liabilities you are typically having to rely on management prepared data (e.g. forecasts), where for many financial assets you are able to source some other independent data.”

Similarly, another partner mentions,

“Financial instruments are more likely to have some observable market comparisons than non-financial instruments.”

Finally, another partner expresses the following:

“There seem to be more third party valuation experts involved in valuing financial assets/liabilities. The valuation for nonfinancial assets/liabilities relies more heavily on management’s assumptions.”

Our study extends prior research by providing evidence that the challenges encountered in

auditing of FVMs differ between financial and nonfinancial assets/liabilities. The above findings

and audit partner comments indicate that auditors encounter ever more difficulty when trying to

obtain verifiable and corroborative evidence from independent sources for non-financial

instruments, thus leading to more reliance on management’s assumptions.

Management’s lack of knowledge can create additional challenges for audit teams when

auditing FVMs. As presented in Panel B of Table 2, approximately 58 percent of participants

report frequently encountering situations in which management’s knowledge of valuation

23

processes and methodologies is relatively shallow. We asked participants to describe their team’s

audit responses when management’s knowledge of valuation processes and methodologies is

relatively shallow, and how such situations impact audit procedures around FVMs. Table 4

reports the top four audit responses are (1) auditor expects that management will place greater

reliance on outside specialists (20.7 percent), (2) consider impact on internal control

effectiveness and inherent risk (19 percent), (3) auditors place greater reliance on valuation

specialists (17.2 percent), and (4) perform additional audit procedures (17.2 percent). Results

suggest that when management’s knowledge regarding the valuation process and methods is

limited, auditors anticipate greater reliance on valuation specialists both by management and the

audit team.

Taken together our results suggest that auditors consider the impact of management’s

lack of knowledge of the valuation processes and methodologies on internal controls assessment

and inherent risk. However, the above results indicate that auditor’s react to potentially higher

control and inherent risks by placing greater reliance on valuation specialists. This finding

highlights auditors’ and management’s potential overreliance on specialists when auditing

complex FVMs, consistent with the PCAOB inspection findings.

Auditors’ use of valuation experts

Research regarding the use of pricing services and valuation specialists when auditing

FVMs is limited. Based on interviews with audit partners and managers, Griffith (2013) provides

insight regarding auditors’ use of in-house valuation specialists. The current study extends the

literature by distinguishing between auditors’ use of third-party pricing services and valuation

specialists (both in-house and 3rd party). In this section, we highlight challenges auditors

encounter when using valuation experts that may be contributing to the FVMs expectation gap.

24

PCAOB inspecting findings often note that auditors over rely on pricing services and

valuation specialists (both third-party and in-house) by failing to obtain sufficient understanding

of the specific assumptions and methods used by the pricing service to derive the FV estimate

(e.g., PCAOB 2008b; PCAOB 2011a; PCAOB 2011b; PCAOB 2012b; PCAOB 2012c).

Therefore, we ask questions directed at better understanding how auditors assess the

reasonableness of assumptions and methods used by pricing services and valuation specialists.

First, with use of pricing services, results in Table 5 indicate that for audit areas considered to be

“higher” risk, participants most frequently mention using the following approaches to assess the

reasonableness of specific assumptions, methods, and models used by third-party pricing

services: (1) involve the firm’s in-house specialists (26.7 percent), (2) gain an understanding of

the pricing methodology—e.g., method, model, and assumptions (24.4 percent), and (3)

comparison with auditor’s independent estimate, another pricing service, or similar instrument

(22.2 percent). Second, with use of valuation specialists, we find that the three most frequently

mentioned approaches, as presented in Table 5, are (1) obtain support/audit evidence for data

used in model from both internal and external sources (30.2 percent), (2) reliance on audit firm’s

in-house valuation specialist to evaluate the appropriateness of assumptions (24.5 percent), and

(3) auditor to gain understanding of key assumptions via the firm’s valuation specialists (15.1

percent). These results further emphasize that auditors may lack the training and expertise to

perform the most subjective aspects related to auditing FVMs. Consequently, they extensively

use in-house valuation specialists to assess reasonableness of assumptions underlying the FVMs.

[INSERT TABLE 5 ABOUT HERE]

The PCAOB has set up a Pricing Source Task Force to address challenges auditors

encounter when auditing FVMs of financial instruments. To inform regulators, we asked our

participants what challenges they encounter when using pricing services. One of the primary

25

challenges reported is obtaining sufficient evidence to support the service’s proprietary model

and assumptions used to derive the FV of investment securities. As presented in Panel B of Table

2, results show that one third of our participants report that the audit team is frequently unable to

obtain sufficient information on the third-parties’ proprietary models or assumptions. This issue

further exacerbate difficulties auditors are encountering when trying to understand and assess the

reasonableness of management’s assumptions and valuation methodologies. The national office

partners we surveyed indicated that some pricing services continue to be reluctant to share

proprietary information. A national office partner stated:

“Pricing services will not share with you their assumptions and models on how they priced investments. We have worked with one in which we can get that information and thus, much of our practice in this area relies on that pricing service. For Level 3 investments we tend not to have the pricing service provide a price since we believe it is more effective and efficient to test management’s assumptions used to price the instrument.”

Another national office partner notes:

“Inconsistency between pricing services and a lack of full transparency of the computation. I recognize that the latter is a function of safe guarding corporate processes and is unlikely to change. The former is extremely challenging - if the client uses one pricing service and the team uses a different pricing service - at times the approaches are often different. We see this with CMOs as an example.”

Further emphasizing the issue, a national office partner points out:

“Although substantial progress has been made in the last couple years, brokers and pricing services often treat their valuation methods and models as proprietary. So, "getting behind the model" is easier said than done. It is expensive and often a matter of triangulation rather than transparent insight.”

Finally, a national office partner expresses the need for clearer guidance in this area:

“We need or would like more immediate transparency but currently it is largely only available for actively traded securities when transparency is not needed. There are no guidelines for providing pricing, minimum requirements or standards to assure greater consistency.”

One national office partner questioned whether it would make sense for PCAOB standards to

cease requiring the auditor to obtain such information. Alternatively, perhaps the PCAOB can

26

work with the pricing industry to foster cooperation in sharing information with auditors, under

confidentiality agreements, to comply with PCAOB auditing standards.

In summary, our findings provide insights that should be helpful for regulators as they

consider how to improve performance in this area and narrow the FVM expectation gap. Our

results suggest that auditors frequently rely on both pricing services and valuation specialists to

assist with the most subjective and challenging aspects of FVMs, evaluating management’s

assumptions and/or valuation models. As previously discussed, overreliance on pricing services

and valuation specialists is a frequently noted deficiency in the PCAOB inspection reports

(PCAOB 2011a; PCAOB 2012b; PCAOB 2012c). Despite auditors’ reliance and inspection

findings, current PCAOB auditing standards do not provide clear guidance to auditors’ regarding

use of in-house specialists vs. third-party pricing services and valuation specialists (e.g., Griffith

2013). Therefore, providing additional guidance to auditors regarding use of pricing services and

valuation specialists is another area in which regulators can help close the FVM expectation gap.

High estimation uncertainty and presentation

The degree of measurement uncertainty and prevalence of FVMs in financial statements

have increased over time (e.g., Bratten et al. 2013; Christensen et al. 2012; PCAOB 2012a).

Consistent with this notion, as presented in Panel C of Table 2, nearly 97 percent of the audit

partners’ surveyed in our study agreed that over the last two decades the number of FVMs

included in financial statements has significantly increased and nearly 94 percent agreed that the

relative proportion of FVMs with high degrees of measurement uncertainty has increased.

Christensen et al. (2012) highlight “extreme measurement uncertainty” that can exist in

the financial statements in which the reasonable range of estimation uncertainty on the fair value

measurement date can be many times larger than materiality. Results of the current study

support Christensen et al. (2012) assertions. Approximately 33 percent of participants (Panel C

27

of Table 2) indicate that they frequently encounter situations in which the degree of measurement

uncertainty exceeds materiality. Cannon and Bedard (2013) report that over 70 percent of their

participants indicate encountering measurement uncertainty that is equal to or greater than

materiality when considering the most complex FVMs.

Given the increase in measurement uncertainty, Christensen et al. (2012) question

whether auditors can provide assurance on accounts that contain extreme measurement

uncertainty. They assert that “no amount of auditing can remove the extreme uncertainty inherent

in reported values derived from management’s valuation models based on unobservable inputs

subject to estimation uncertainty” (Christensen et al. 2012, p. 143). Bell and Griffin (2012) also

question the auditability of estimates with extreme measurement uncertainty, including FVMs.

Further, Christensen et al. (2013) report that auditors and investors believe that high estimation

uncertainty significantly affects auditors’ ability to provide the requisite level of assurance.

Highlighting estimation uncertainty could be the subject of proposed critical audit matter

paragraphs as described in the PCAOB proposal to revise audit reports (PCAOB 2013). The

proposal retains the existing pass/fail reporting model, but would require auditors to

communicate critical audit matters in a separate paragraph. Christensen, Glover, and Wolfe

(2014) find that investors who receive a critical audit matter paragraph highlighting uncertainty

in a FVM are more likely to change their investment decision than are investors who receive the

standard report or investors who receive the same information as in management’s footnotes.

The views expressed by Christensen et al. (2012) and Bell and Griffin (2012) appear

contrary to expectations of inspectors and regulators. Previously, we reported that PCAOB

inspectors and some in the PCAOB leadership may believe that more auditing can reduce the

apparently irreducible inherent uncertainty associated with complex FVMs. While the prior

literature calls into question the auditability of estimates with such extreme measurement

28

uncertainty, ours is the first study to gather and report on audit partners’ views regarding this

issue. Our findings provide important insights and recommendations for how practice can be

improved. Specifically, in this section we address the following key questions:

1. How can the level of measurement uncertainty in complex estimates be best communicated to users of financial statements?

2. Given the increase in estimates with extreme measurement uncertainty, do auditors believe the current requirement to provide a high level of positive assurance that a point estimate is fairly stated be reconsidered? Why?

3. How can existing auditing standards provide further clarification and guidance?

We asked audit partners’ views regarding the current communication of measurement

uncertainty to users of financial statements and how best to convey the level of uncertainty.

Responses in Panel C of Table 2 indicate that 50 percent of the participants disagreed that current

footnote disclosures around the level of uncertainty at the measurement date are consistent

among preparers and only 50 percent agree that current footnote disclosures about measurement

uncertainty are adequate. Our findings raise potential concerns regarding reporting consistency

and adequacy. We also asked: “For estimates with a reasonable range of estimation uncertainty

that is greater than overall materiality, which of the following methods do you believe would be

most useful to users in conveying the level of uncertainty.” Table 6 provides a list of suggested

methods and the frequency that audit partners selected each method. The top three methods

selected are: (1) narrative description of estimation methods and the source and level of

uncertainty (37 percent), (2) quantitative disclosure of the range of reasonable estimates at the

measurement date (30 percent), (3) tables in footnotes listing measurement assumptions and the

values used (18 percent).

[INSERT TABLE 6 ABOUT HERE]

Recent commentaries suggest that the current audit approach for reported amounts with

extremely high estimation uncertainty is more consistent with negative assurance rather than

29

positive assurance. PCAOB and SEC rules require that auditors provide reasonable positive

assurance (interpreted in the auditing standards to be a high level of assurance) that all account

balances are fairly stated in all material respects. To address the issue of auditability of estimates

with extreme measurement uncertainty (the second key question in this section), we asked

participants whether they believe the current requirement to provide a high level of positive

assurance that a point estimate is fairly stated be reconsidered when the estimate’s reasonable

range of estimation uncertainty is many times larger than overall materiality (e.g., consider

instead providing positive assurance on the estimation process). Approximately 60 percent of the

audit partners responded “yes” to this question, which is telling for a couple of reasons. First, it

raises the very real possibility that the current requirement for auditors to provide a high level of

positive assurance that all point estimates reported in the financial statements are materially

correct is not realistic and that current auditing standards and regulator expectations may be

demanding a service that even the best and brightest simply cannot provide. Second, our

conversations with some national office partners suggests that the profession takes enormous

pride in providing positive assurance on the most difficult and complex estimates, including

FVMs, and that no one can do a better job than the outstanding professionals at the largest

firms,6 and (3) auditors, like most people, get comfortable with the status quo and are hesitant to

consider or recommend what some might consider radical change.

We asked the 60 percent of participants that responded “yes” to explain why they think

auditing standards and the level of assurance associated with highly uncertain estimates should

be reconsidered. Table 7 indicates that the top two reasons are (1) it is unrealistic to provide

assurance on point estimate given the level of subjectivity, complexity and uncertainty of

6Christensen et al. (2013) report one partner in their survey commented, “in regard to impacting the provision of assurance, there’s always a way to get there. If there isn’t, you shouldn’t be issuing an opinion.” The authors note that the profession recognizes “the practical realities they face. Until and unless the auditing standards change, auditors are prepared to continue doing their best to provide the desired level of assurance.”

30

estimates (28.6 percent), and (2) no amount of evidence could achieve high level of assurance

given high estimation uncertainty (23.8 percent). For those who responded “no,” standards and

the level of assurance should not be reconsider, their top two reasons are (1) the problem is

inherent in the audit process, thus no change is warranted (31.6 percent), and (2) auditing and

financial reporting has always involved estimation and uncertainty (21.1 percent).

[INSERT TABLE 7 ABOUT HERE]

To address our third key question in this section, we asked participants whether their firm

would support additional clarity and guidance from standard setters and/or regulators regarding

auditing estimates including FVMs. Approximately 93 percent of participants indicated “yes”

they would support additional clarity and guidance. For participants that answered “yes,” we

asked them to provide specific recommendations to add to the existing literature. For the 57

percent of participants that did provide a recommendation, Table 7 indicates that the top three

recommendations are (1) more guidance regarding an acceptable reasonable range of estimation

uncertainty (29.4 percent), (2) more clarification in standards regarding audit requirements and

procedures (29.4 percent), (3) the standard-setting process should influence audit procedures and

requirements rather than using PCAOB inspection process to drive behavior (17.6 percent). A

specific recommendation mentioned by one practice partner is:

“I think [the issue] needs more clarity just so there is a common understanding of what is required. Clients don’t understand the extent of procedures that are required because they have been forced on the auditor by the PCAOB via the inspection process, but are not well spelled out in their published standards.”

Another practice partner recommends adding:

“…clarity around range of tolerable differences when there is high estimation uncertainty.” A national office partner also specifically noted an area for improvement:

31

“It needs to [come] from standard setters - frankly there needs to be an emphasis from the SEC on the need for the registrants to tighten their estimates within an appropriate range. This is an area of audit confusion - more guidance of the application will be very useful.”

We concluded the survey by asking participants, in the context of high estimation

uncertainty, whether they believe a potentially viable approach in the future (assuming standards

change accordingly) is to require the auditor to provide assurance on the rigor and quality of the

estimation process and negative assurance on the reported point estimate (i.e., the entity followed

an appropriate and rigorous process to form the estimate and nothing came to our attention that

would suggest the reported point estimate is not fairly stated). Approximately, 36 percent of

participants responded “yes.” We follow up this question by asking these participants to explain

their response. Table 7 shows that the top two explanations are (1) suggestion is consistent with

other private equity fund opinions, represents current environment, and is reasonable (40

percent), and (2) suggestion offsets the extremely high expectations of regulators regarding level

of assurance possible for estimates with high estimation uncertainty (20 percent).7

Collectively, we find that the majority of audit partners surveyed (60 percent) believe that

the current requirement to provide a high level of positive assurance on a point estimate should

be reconsidered. Results in this section further highlight the FVM expectation gap that exists

between auditors and regulators. As noted earlier, PCAOB inspectors and some in the PCAOB

leadership may believe that more auditing can reduce the irreducible inherent uncertainty as

evidenced by PCAOB inspection findings. Our findings provide insight for regulators and

standard-setters regarding areas in which they can help improve the auditing of FVMs, and

perhaps reduce the FVM expectation gap. These areas include the following: regulators and

standard setters consider the feasibility of the requirement to provide a high level of positive

7For those who responded “no” to positive assurance on the estimation process and negative assurance on the point estimate, the top explanation provided is that such a change is inconsistent with other standards as the auditor is responsible to provide positive assurance the financial statements taken as a whole, so that the positive opinion cannot exclude FVMs (27.8 percent).

32

assurance on estimates with extreme estimation uncertainty; standard-setters and regulators

should reconsider how estimation uncertainty is conveyed to financial statement users; and

provide more guidance and clarification about what is sufficient, appropriate audit evidence to

support FVMs with extreme estimation uncertainty.

CONCLUSIONS

The current study aims to begin to close the gap between auditor performance and

regulators’ expectations around audits of FVMs. Participants largely believe that the PCAOB

inspectors expect more evidence than auditing standards require, thus suggesting that existing

guidance requires further clarification. This discrepancy often leads to auditor disagreement with

the PCAOB inspector’s assessments of audit deficiencies.

We find that one frequently reported reason for auditor disagreement with the PCAOB is

that inspectors did not appropriately consider risk of misstatement and totality of the audit

evidence gathered. This is also the area audit partners surveyed most frequently mentioned as an

area in which the PCAOB could help auditors improve performance and improve the inspection

process. This issue if unaddressed could result in audit efforts that are not driven by audit risk as

prescribed by auditing standards, but instead incorrectly driven by “inspection risk” (Glover and

Prawitt 2013). Consequently, auditors may end up focusing more attention on managing

inspection risk while devoting less effort to areas with the greatest audit risk.

Another frequently reported reason for auditor disagreement with the PCAOB inspectors’

assessment is that inspectors expect a level of precision that may not be achievable given the

nature of FVM. This finding is consistent with questions raised in prior literature regarding the

auditability of estimates with extreme measurement uncertainty (Bell and Griffin 2012;

Christensen et al. 2012). Further, our results show that a majority (60 percent) of participants

believe that the current requirement to provide a high level of positive assurance that a point

33

estimate is fairly stated should be reconsidered particularly when the estimate’s reasonable range

of estimation uncertainty is many times larger than overall materiality. Participants frequently

expressed that providing assurance on point estimate given subjectivity, complexity and

uncertainty of estimates is unrealistic to and that no amount of evidence could achieve high level

of assurance given high estimation uncertainty as a key reasons why the current requirement

should be reconsidered. Moreover, an overwhelming majority of participants support additional

clarity and guidance from regulators regarding auditing of estimates with high estimation

uncertainty. The top two recommendations mentioned by participants to improve existing

auditing standards are (1) provide more guidance regarding an acceptable reasonable range of

estimation uncertainty, and (2) provide more clarification in standards regarding audit

requirements and procedures.

Finally, the third most frequently mentioned reason for auditor disagreement with the

PCAOB’s assessment is that inspectors have unreasonable expectations regarding management

and auditor’s reliance on third-party specialists. Consistent with this finding, our results

regarding auditors’ use of pricing services and valuation specialists suggest that auditors

frequently rely on pricing services and valuation specialists to evaluate assumptions and/or

valuation models. However, the PCAOB auditing standards currently do not provide clear

guidance for auditors’ use of in-house specialists vs. third-party pricing services and valuation

specialists (Griffith 2013). Results also indicate that auditors encounter problems with obtaining

sufficient information from third-party pricing services regarding their proprietary model and

assumptions used to determine the FV of investment securities.

In summary, results of our study identify areas in which auditor performance, regulation,

and auditing standards need improvement. In doing so, this study highlights areas for future

research regarding auditability of complex accounting estimates as well as suggestions for how to

34

reduce the apparent gap between current auditor performance and regulator expectations.

Importantly, our results provide important insights for regulators regarding unintended

consequences of the PCAOB inspection process. Results reveal that existing auditing standards

do not provide sufficient guidance to auditors regarding how to audit complex FVMs. Instead,

changes in audit performance are driven by the inspection process. This consequence could be

significant from both the audit effectiveness and efficiency standpoints. That is, from an

effectiveness standpoint, auditors may be shifting focus away from higher-risk areas (e.g., level 2

and level 3 FVM), which has a greater risk of misstatement, to attend to lower risk areas (e.g.,

level 1 FVM). From an efficiency standpoint, auditors may be over-auditing low-risk audit areas.

35

REFERENCES

American Institute of Certified Public Accountants (AICPA). 2000. Auditing Derivative Instruments, Hedging Activities, and Investments in Securities. AU Section 332. New York, NY: AICPA.

American Institute of Certified Public Accountants (AICPA). 2003. Auditing Fair Value

Measurements and Disclosures. AU Section 328. New York, NY: AICPA. Bell, T. B., and J. B. Griffin. 2012. Commentary on auditing high-uncertainty fair value

estimates. Auditing: A Journal of Practice & Theory 31: 147 – 155. Backof, A. G., J. Thayer, and T. Carpenter. 2014. Auditing Complex Estimates: The Presentation

Format of Management’s Supporting Evidence and Auditors’ Professional Skepticism. Working paper, University of Virginia and University of Georgia.

Bratten, B., L. M. Gaynor, L. McDaniel, N. R. Montague, and G. E. Sierra. 2013. The audit of

fair values and other estimates: The effects of underlying environmental, task, and auditor-specific factors. Auditing: A Journal of Practice & Theory 31: 127 – 146.

Cannon, N., and J. C. Bedard. 2013 Auditing challenging fair value measurements: Evidence

from the field. Working Paper, Bentley University. Christensen, B. E., S. M. Glover, and D. A. Wood. 2012. Extreme estimation uncertainty in fair

value estimates: Implications for audit assurance. Auditing: A Journal of Practice & Theory 31 (1): 127 – 146.

Christensen, B. E., S. M. Glover, T. C. Omer, and M. K. Shelly. 2013. Understanding audit quality: Insights from audit partners and investors, working paper, Texas A&M University.

Christensen, B.E., S. M. Glover, and C. J. Wolfe. 2014. Do critical audit matter paragraphs in the

audit report change nonprofessional investors’ decision to invest? Working paper, Texas A&M University.

Church, B. K., and L. B. Shefchik. 2012. PCAOB inspections and large auditing firms. Auditing:

A Journal of Practice & Theory 26 (1): 43 – 63. Earley, C. E., V. B. Hoffman, and J. R. Joe. 2013. Are auditors skeptical of management’s Level

2 versus level 3 fair value classification judgments? Working paper, Providence College, University of Pittsburgh, and University of Delaware.

Griffin, J. B. 2011. The effects of uncertainty and disclosure on auditors’ fair value materiality

decisions. Working paper, University of Notre Dame. Griffith, E. E. 2013. How Do Auditors Use Valuation Specialists when Auditing Fair Values?

Working Paper, The University of Georgia.

36

Griffith, E. E., J. S. Hammersley, K. Kadous, and D. Young. 2013. Auditor Mindsets and Audits of Complex Estimates. Working Paper, The University of Georgia, Emory University, and Georgia Tech University.

Griffith, E. E., J. S. Hammersley, and K. Kadous. 2014. Audits of Complex Estimates as

Verification of Management Numbers: How Institutional Pressures Shape Practice. Forthcoming Contemporary Accounting Research.

Glover, S. M., and D. F. Prawitt. 2013. Enhancing Auditor Professional Skepticism. Monograph

produced by the Standards Working Group of the Global Public Policy Committee, www.thecaq.org/docs/research/skepticismreport.pdf.

Joe, J. R., S. D. Vandervelde, and Y. Wu. 2014. Use of Third Party Specialists’ Reports When

Auditing Fair Value Measurement: Do Auditors Stay in their Comfort Zone? Working Paper, The University of Delaware, University of South Carolina, and Case Western Reserve University.

Landis, J.R., and G. G. Koch. 1977. The measurement of observer agreement for categorical

data. Biometrics 33: 159- 174. Maksymov, E., M. W. Nelson, and W. R. Kinney, Jr. 2012. Effects of procedure frame,

procedure verifiability, and audit efficiency pressure on planning audits of fair values. Working paper, Cornell University and The University of Texas at Austin.

Martin, R. D., J. S. Rich, and T. J. Wilks. 2006. Auditing fair value measurement: A synthesis of

relevant research. Accounting Horizons 20 (3): 287 – 303. Neuendorf, K. A. 2002. The content analysis guidebook. Sage publishing. Peecher, M. E., I. Solomon, and K. T. Trotman. 2013. An accountability framework for financial

statement auditors and related research questions. Accounting, Organizations and Society (Forthcoming).

Public Company Accounting Oversight Board (PCAOB). 2007a. Auditing Accounting Estimates

and Fair Value Measurements. Standing Advisory Group Meeting. June 21. Washington, DC: PCAOB.

Public Company Accounting Oversight Board (PCAOB). 2008a. Report on 2007 Inspection of

Deloitte & Touche LLP. May 19. Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2008b. Report on 2007 Inspection of

KPMG LLP. August 12. Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2010a. Report on Observations of

PCAOB Inspectors Related to the Audit Risk Areas Affected by the Economic Crisis. Release No. 2010-006. September 29. Washington, DC: PCAOB.

37

Public Company Accounting Oversight Board (PCAOB). 2010b. Report on 2009 Inspection of McGladrey & Pullen, LLP. June 24. Washington, DC: PCAOB.

Public Company Accounting Oversight Board (PCAOB). 2011a. Report on 2010 Inspection of

Ernst and Young, LLP. November 30. Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2011b. Report on 2010 Inspection of

Deloitte & Touche LLP. December 7. Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2011c. Report on 2010 Inspection of

PricewaterhouseCoopers LLP. November 8. Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2012a. Auditing the Future. June 7.

Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2012b. Report on 2011 Inspection of

KPMG LLP. August 15. Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2012c. Report on 2012 Inspection of

KPMG LLP. July 30. Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2012d. Report on 2011 Inspection of

Ernst and Young, LLP. December 6. Washington, DC: PCAOB. Public Company Accounting Oversight Board (PCAOB). 2013. Proposed Auditing Standards:

The Auditor’s Report on an Audit of Financial Statements when the Auditor Expresses an Unqualified Opinion; The Auditor’s Responsibilities Regarding Other Information in Certain Documents Containing Audited Financial Statements and The Related Auditor’s Report; and Related Amendments to PCAOB Standards. Release 2013-005. Washington, D.C.: PCAOB.

Valuation Research Corporation (VRC). 2013. Alert: PCAOB comments increase auditor

scrutiny of fair value measurements. January. Boston, MA: VRC.

38

* For certain questions participants are able to select more than one option. Thus, for those questions, the total percentage is greater than 100%. Percentage calculations are based on participants who finished the entire survey.

TABLE 1 Participant Demographics*

Extent of involvement in the planning, supervision, or execution of auditing significant estimates, including fair value measurements. Mean (1 - 'very little involvement' to 11 - 'a great deal of involvement') 10.28 Total Number of Participants 32 Percent of Participants who are Audit Partners 100% Mean Years of audit experience 22.3 Industry

Financial Services/ Depository/ Financial Institutions/ Asset management 43.8% Industrial Products and Manufacturing 25.0% Consumer Products and Goods 15.6% Energy and Public Utilities - Regulated and Non-Regulated 12.5% Healthcare and Managed Care 9.4% Technology 9.4% Commercial 6.3% Life Sciences 6.3% Other 6.3%

Valuation and audit expertise in fair value measurement Mean (1 - 'very little expertise’ to 11 - ' great deal of expertise') 8.53 Do you specialize in a particular type of estimate or fair value measurement? Yes 18.8% No 81.2% If “yes,” areas of specialization

Investment Securities 66.7% Other Financial Instruments 50.0% Liabilities at Fair Value (e.g., asset retirement and debt obligations 16.7% Asset Impairment 50.0% Real Estate Portfolio 50.0% Business Valuation 16.7% Goodwill 50.0% Other (Please describe below) 33.3%

39

TABLE 2 Scale - Rated Responses

Panel A: Questions Regarding PCAOB Inspections

Panel B: Challenges in Auditing of FVM and Use of Valuation Experts

Scale Rating Difference from Scale's

Midpoint?*

Question

1-5 Expect what the standard

requires

6 Expect more

than the standard requires

7-11 Expect

significantly more than

the standard requires Mean t p - value

In your opinion, did the PCAOB inspectors expect the audit team to collect more evidence than what the firm believes represents the requirements of auditing standards (e.g., PCAOB AU 328) when auditing fair value measurements? 30.43% 8.70% 60.87% 7.13 1.65 0.11

Scale Rating Difference from Scale's

Midpoint?*

Question 1-5

Infrequently

6 Neither

infrequently nor frequently

7-11 Frequently Mean t p - value

1. Based on your experience, how frequently do auditors encounter situations in which management’s knowledge of valuation processes and methodologies is relatively shallow? 29.03% 12.90% 58.06% 6.74 1.94 0.06

1-5

Infrequently

6 Neither

infrequently nor frequently

7-11 Frequently

2. Based on your experience, how often is the audit team unable to obtain sufficient proprietary information from the third party pricing service to support the model and assumptions management used to determine the fair value of these investments? 50.00% 16.67% 33.33% 5.57 -0.99 0.33

40

TABLE 2 (Cont.)

Panel C: Questions on High Estimation Uncertainty and Presentation

* We performed t-tests to determine if the mean rating for each question differs from the midpoint of the scale (p = two-tailed).

Scale Rating Difference from Scale's

Midpoint?*

Question

1-5 More strongly

disagree

6 Neither agree or

disagree

7-11 More strongly

agree Mean t p - value 1. For the following two questions, please rate how strongly you

agree or disagree with each of the following statements: Over the last two decades the number of fair value measurements included in financial statements has significantly increased (e.g., estimates that were not previously on the balance sheet that now are). 3.1% 0.0% 96.9% 9.53 11.60 <0.01 Over the last two decades, the relative proportion of fair value measurements with high degrees of measurement uncertainty (i.e., lack of measurement precision resulting in reasonable range of uncertainty around the estimate that is equal to or greater than overall materiality applied by your firm) has increased. 6.3% 0.0% 93.8% 8.88 8.82 <0.01

2. Based on your experience, for significant level 2 or level 3 fair value estimates WHERE INPUTS ARE NOT OBSERVABLE, how frequently does the degree of measurement uncertainty or reasonable range around the reported point estimate exceed overall materiality? 52.9% 14.7% 32.4% 5.69 -0.80 0.43

3. Current footnote disclosures around the level of uncertainty at the measurement date are consistent among preparers. 50.0% 18.8% 31.3% 5.16 -2.03 0.05

4. Current footnote disclosures about measurement uncertainty are adequate. 37.5% 12.5% 50.0% 6.28 0.65 0.52

41

TABLE 3 Questions Regarding PCAOB Inspections – Open-ended Responses

Question1: On your engagement, did the PCAOB inspectors identify any audit deficiency related to the auditing of fair value measurements? If yes, did you agree with the PCAOB’s assessment? Why or why not?

Responses by Themes Frequency (%) 1. PCAOB did not give appropriate consideration to the risk of misstatement, nature of the security, and the totality of the audit evidence gathered. 54.5% 2. PCAOB expects a level of precision that may not be achievable given nature of FVM 27.3% 3. PCAOB’s expectation regarding management and auditor’s reliance on 3rd party specialist is not reasonable 18.2% Question 2: Given your experience with PCAOB inspections specifically in the area of fair value measurements, please provide one or two suggestions for how the PCAOB could help improve the auditing of fair value measurements. 1. Inspectors could use a more risk-based approach which is consistent with auditing standards and consider the totality of audit evidence gathered by auditors 25.9% 2. PCAOB to provide clearer guidance of audit requirements (e.g., evidence, procedures, documentation) 18.5% 3. Acknowledgement that there will be a range of FV estimates and evidence gathered may not be able to get to the level of precision desired (i.e., within materiality, inspectors need more realistic expectations) 18.5% 4. Need for more knowledgeable, qualified inspectors 11.1% 5. Acknowledgement that audit of FVMs requires judgment on the part of the auditor 11.1% 6. More clearly define and understand the role of the auditor vs. third-party specialists 7.4% 7. Consistent inspection requirements for both annually and non-annually inspected firms 3.7% 8. Bigger picture…work with accounting standard setters to reduce the number of items accounted for using FV 3.7%

42

TABLE 4 General Questions Regarding the Auditing of FVMs – Open-ended responses

Question 1: In your opinion, what are the most significant challenges in auditing inputs for a significant level 2 or 3 fair value measurement that are not observable and where small changes to the inputs have potentially material changes to the reported value?

Responses by Themes Frequency (%) 1. Lack of verifiable/corroborative evidence (including lack of 3rd party evidence) 42.9% 2. High degree of subjectivity involved in estimates 14.3% 3. Degree of difficulty in assessing reasonableness of assumptions and benchmarks 14.3% 4. Developing a range and placing point estimate within the range 9.5% 5. Inability to audit to the level of precision desired 7.1% 6. Lack of disclosure about imprecision 4.8% 7. Removing management bias 4.8% 8.Ensuring the competency and expertise of the valuation specialists 2.4% Previous Question: Do you believe the audit challenges are different between financial vs. nonfinancial assets/liabilities? Question 2: If “yes,” please briefly describe the differing challenges.

Responses by Themes Frequency (%) 1. Difference in availability and relevance of market data 56.5% 2. Degree of auditor familiarity 13.0% 3. Difference in perceptions of precision by regulators and users 8.7% 4. Degree of consistency possible when applying processes, methods, models, and/or assumptions) 8.7% 5. Difference in dollar amounts (i.e., size) 4.3% 6. Degree of subjectivity 4.3% 7. Complexity of models 4.3% Question 3: Based on your experience, what is your audit response when management’s knowledge of valuation processes and methodologies is relatively shallow, and how does this impact your audit procedures when testing the issuer’s fair value measurements?

Responses by Themes Frequency (%) 1. Auditor expectation that management will place greater reliance on third-party specialists 20.7% 2. Consider impact on internal control effectiveness and inherent risk 19.0% 3. Auditors place greater reliance on valuation specialists 17.2% 4. Perform additional audit procedures 17.2% 5. Reconsider auditor’s risk assessment 10.3% 6. Auditor to develop independent estimate 8.6% 7. Increase skepticism and scrutiny 5.2% 8. Communication with board or audit committee 1.7%

43

TABLE 5 Questions Regarding Auditors’ Use of Valuation Experts – Open-ended responses

Question 1: When estimates are obtained from a third-party pricing service in an audit area considered to be “higher” risk, what does the audit team usually do to assess the reasonableness of specific assumptions, methods, and models used by the pricing service to value the specific financial instruments in order to establish a basis for reliance on the information provided by the pricing service?

Responses by Themes Frequency (%) 1. Involve firm’s in-house specialists 26.7% 2. Gain understanding of the pricing methodology (e.g., method, model, and assumptions) 24.4% 3. Comparison with auditor’s independent estimate, another pricing service, or similar instrument 22.2% 4. Assess qualifications and reliability of information supplied by pricing service 20.0% 5. Assess the pricing service’s policies and procedures 6.7% Question 2: If the valuation specialist’s work used to support the audit involves use of significant assumptions and methods, how does the audit team test the data and assess the reasonableness of those assumptions and methods?

Responses by Themes Frequency (%) 1. Obtain support/audit evidence for data used in model (internal and external sources) 30.2% 2. Reliance on audit firm’s valuation specialist to evaluate appropriateness of assumptions 24.5% 3. Auditor gains understanding of key assumptions (via the firm’s valuation specialists) 15.1% 4. Comparison to company’s historical information 13.2% 5. Test completeness and accuracy of data used in management’s valuation 9.4% 6. Follow firm and/or PCAOB recommended practices 5.7% 7. Search for contrary evidence 1.9%

44

TABLE 6 Questions on High Estimation Uncertainty and Presentation

Question: For estimates with a reasonable range of estimation uncertainty that is greater than overall materiality, which of the following methods do you believe would be most useful to users in conveying the level of uncertainty (please ignore the potential costs associated with the method). (check all that apply)

Responses by Themes Frequency (%) 1. Narrative description of estimation methods and the source and level of uncertainty 37.1% 2. Quantitative disclosure of the range of reasonable estimates at the measurement date 30.0% 3. Tables in footnotes listing measurement assumptions and the values used 18.6% 4. Statistical distributions providing the upper and lower bounds at say 90 or 95%

confidence 5.7% 5. Rounding reported estimates to a higher level than other reported values in the financial

statements to remove the illusion of exactitude 2.9% 6. Separate columns on the face of the financial statements for each year that add up to the

current columnar totals and presentations. The separate columns would include a column that presents reported vales with relatively low estimation uncertainty (a reasonable within materiality) and a column for reported values with a reasonable range of estimation uncertainty greater than materiality 1.4%

7. Other 4.3%

45

TABLE 7 Questions on High Estimation Uncertainty and Presentation – Open-ended Responses

Previous question: In your opinion, should the current requirement to provide a high level of positive assurance that a point estimate is fairly stated be reconsidered when the estimate’s reasonable range of estimation uncertainty is many times larger than overall materiality (e.g., consider instead providing positive assurance on the estimation process)? Question1: Please explain why or why not.

Responses by Themes (‘Yes’ response to the previous question) Frequency (%) 1. Unrealistic to provide assurance on point estimate given level of subjectivity, complexity and

uncertainty of estimates 28.6%

2. No amount of evidence could achieve high level of assurance given high estimation uncertainty 23.8%

3. Current presentation may mislead readers of the financial statements – investors may not understand the level of uncertainty in estimates 14.3%

4. However, not sure if negative assurance on the point estimate and positive assurance on the process is the way to go 14.3%

5. Disclose range and confidence level so investors can evaluate 14.3% 6. Auditors’ need to rely on specialist, thus makes more sense to focus on the estimation process 4.8%

Responses by Themes (‘No’ response to the previous question) Frequency (%) 1. Problem inherent in the audit process, thus no change is warranted. 31.6% 2. Auditing and financial reporting has always involved estimation and uncertainty 21.1% 3. Responsibility of auditor to provide opinion on the whole financial statement not just fair value measurements 15.8% 4. More or less auditing will not solve problem. More disclosure is necessary 15.8% 5. Excessive disclosures may cause confusion and information overload for users 10.5% 6. Suggestion inconsistent with other audit standard 5.3% Previous Question: Regarding the existing guidance on how to audit estimates with high estimation uncertainty, would your firm support additional clarity and guidance from standard setters and/or regulators regarding auditing estimates or do you believe the current guidance is sufficient? Question 2: Are there specific recommendations you have on what could be added to the literature?

Responses by Themes Frequency (%) 1. More guidance regarding an acceptable reasonable range of estimation uncertainty 29.4% 2. More clarification in standards regarding audit requirements and procedures 29.4% 3. The standard-setting process should influence audit procedures and requirements rather than using PCAOB inspection process to drive behavior 17.6% 4. Realize that in some cases PCAOB guidance and expectations assume greater accuracy and precision than is achievable in reality 11.8% 5. Provide more concrete examples 5.9% 6. More guidance regarding reliance on specialists for hard-to-value FVM 5.9%

46

TABLE 7 (Cont.) Questions on High Estimation Uncertainty and Presentation – Open-ended Responses

Previous question: For estimates with high estimation uncertainty, in your opinion, would a potentially viable approach in the future, assuming standards change accordingly, be to require the auditor to provide positive assurance on the rigor and quality of the estimation process and negative assurance on the reported point estimate (i.e., the entity followed an appropriate and rigorous process to form the estimate and nothing came to our attention that would suggest the reported point estimate is not fairly stated)? Question 3: Please explain why or why not?

Responses by Themes (‘Yes’ response to the previous question) Frequency (%) 1. Suggestion is consistent with other private equity fund opinions, represents current environment, and is reasonable 40.0%

2. Unrealistic to provide assurance on point estimate given level of subjectivity, complexity and uncertainty of estimates 20.0%

3. Different assurance levels may cause confusion 10.0% 4. Additional clarification regarding range of tolerable differences 10.0% 5. Suggestion offsets the extremely high expectations of regulators regarding level of assurance

possible for estimates with high estimation uncertainty 10.0%

6. Less judgment involved with auditing the process than outcome 10.0%

Responses by Themes (‘No’ response to the previous question) Frequency (%) 1. Suggestion is inconsistent with other standards as the auditor is responsible to provide positive assurance the financial statements taken as a whole, so can’t exclude fair value measurements 27.8% 2. Too difficult to separate process from outcome, both are needed 16.7% 3. Additional change can’t address the inherent imprecision in FVMs so suggestion may still not reduce risk of material misstatement or identify inappropriate estimates 16.7% 4. Suggestion is similar to the ICFR audit 16.7% 5. Results in potential differences due to differences in auditors’ judgments 11.1% 6. Believe current audit process and reporting is sufficient 5.6% 7. Create confusion for users and the disclosures necessary would be difficult to manage 5.6%