experimenting with multi-attribute utility survey methods in a multi-dimensional valuation problem

Ecological Economics 36 (2001) 87–108

ANALYSIS

Experimenting with multi-attribute utility survey methods ina multi-dimensional valuation problem�

Clifford Russell a,*, Virginia Dale b, Junsoo Lee c, Molly Hadley Jensen a,Michael Kane d, Robin Gregory e

a Vanderbilt Institute for Public Policy Studies, 1207 18th A6enue South, Nash6ille, TN 37212, USAb En6ironmental Sciences Di6ision, Oak Ridge National Laboratory, P.O. Box 2008, Oak Ridge, TN 37831-6036, USA

c Department of Economics, P.O. Box 161400, Uni6ersity of Central Florida, Orlando, FL 32816-1400, USAd Department of Ecology and E6olutionary Biology, Uni6ersity of Tennessee, Knox6ille, TN 37996-1410, USA

e Decision Research, 1124 West 19th Street, No. Vancou6er, B.C., Canada

Received 24 May 1999; received in revised form 22 May 2000; accepted 1 June 2000

Abstract

The use of willingness-to-pay (WTP) survey techniques based on multi-attribute utility (MAU) approaches has beenrecommended by some authors as a way to deal simultaneously with two difficulties that increasingly plagueenvironmental valuation. The first of these is that, as valuation exercises come to involve less familiar and more subtleenvironmental effects, such as ecosystem management, lay respondents are less likely to have any idea, in advance,of the value they would attach to a described result. The second is that valuation questions may increasingly be aboutmulti-dimensional effects (e.g. changes in ecosystem function) as opposed for example to changes in visibility from agiven point. MAU has been asserted to allow the asking of simpler questions, even in the context of difficult subjects.And it is, as the name suggests, inherently multi-dimensional. This paper asks whether MAU techniques can be shownto ‘make a difference’ in the context of questions about preferences over, and valuation of differences between,alternative descriptions of a forest ecosystem. Making a difference is defined in terms of internal consistency ofanswers to preference and WTP questions involving three 5-attribute forest descriptions. The method involves firstasking MAU-structured questions attribute-by-attribute. The responses to these questions allow researchers to infereach respondent’s preferences and WTP. Second, the same respondents are asked directly about their preferences andWTPs. The answer to the making-a-difference question, based largely on comparing the inferred and stated results,

www.elsevier.com/locate/ecolecon

� Based on research supported by EPA Grant cR824699. Neither the results nor their interpretations as reported in this paperhave been reviewed or endorsed by USEPA. The senior author would like to thank participants in seminars at the University of EastAnglia, the Norwegian Agricultural University, the Swedish Agricultural Universities at Uppsala and Umea, AKF in Copenhagen,and the Beijer Institute of the Royal Swedish Academy of Sciences for helpful comments and suggestions. We are also grateful forcomments on an earlier draft by Nick Hanley and Eirik Romstad.

* Corresponding author. Fax: +1-615-3228081.E-mail address: [email protected] (C. Russell).

0921-8009/01/$ - see front matter © 2001 Elsevier Science B.V. All rights reserved.

PII: S 0921 -8009 (00 )00207 -X

C. Russell et al. / Ecological Economics 36 (2001) 87–10888

is not straightforward. Overall, the inferred results are good ‘predictors’ of what is stated. But the agreement is byno means perfect. And the individual differences are not explainable by the socio-economic characteristics of theindividuals. Since the technique involves creating a long, somewhat tedious, and even apparently confusing series oftasks (though each task may itself be simple), it is by no means clear that the prescription, ‘use MAU techniques’,holds the same level of practical as of theoretical promise. © 2001 Elsevier Science B.V. All rights reserved.

Keywords: Ecosystem valuation; Multi-attribute utility; Willingness to pay surveys; Forest attributes

1. Introduction

The literature dealing with direct methods ofenvironmental damage or benefit estimation islarge, complex, fascinating, and growing at aprodigious rate. The central concern for perhaps90% of that literature is how seriously to take theanswers that respondents give to varieties of will-ingness-to-pay (or to accept) (WTP/WTA) ques-tions. One version of that concern is thetraditional economics worry — that people willfigure out how their possible answers might affecttheir future welfare and conceal their true prefer-ences (reveal false ones), either free-riding or over-bidding as the situation seems to make desirable(e.g. Mitchell and Carson, 1989, ch. 6 and 7). Asecond version, broadly stated, is that, so far fromunderstanding how to conceal true preferences toreap greater potential rewards, lay respondents toenvironmental valuation questions do not knowwhat their preferences are and cannot possiblypredict what they would be in a real as opposedto the hypothetical survey choice situation. Wemay think of this as the psychologists’ concern(e.g. Fischhoff, 1991; Schkade and Payne, 1994).Bohm (1994), takes a position that might be seenas a blend of the two simplified ones set outabove. He stresses the hypothetical structure ofthe questions and the resulting unreliability of theanswers. But his evidence points to a tendency tooverstate WTP.

Sharpening this latter concern is the trend inthe field of direct valuation toward taking onmore subtle, complex, and long-term problems,such as those dealing with ecological systems,their condition, management, and futureprospects. These extensions mean that the situa-tions being sketched and the preference and valu-ation decisions being sought are becoming more

distant from lay experience. At the same time, theinformation that must be transferred to the re-spondent is becoming more extensive and morecomplicated. In particular, questions often involvemore than one environmental dimension, as con-trasted with, for example, visibility changes forwhich the policy effect is naturally captured by ascalar.

The work reported on here was motivated bythe cognitive concern. It picks up the suggestionmade by Gregory et al. (1993) (also Gregory andSlovic, 1997) that multi-attribute-utility theory(MAUT or just MAU) can provide the founda-tion for an alternative approach to valuation.1

They make the case that MAU, in principle,addresses both the multi-dimensionality and theunfamiliarity of the new valuation challenges byproviding a set of cognitively simpler tasks. Ourgoal was to test the proposition that the use ofMAU Survey techniques will make an identifiableand useful difference in results obtained fromrespondents to a survey instrument dealing with amulti-dimensional environmental ‘good’. Otherresearchers have made a similar point in the con-text of using multi-criteria decision-making meth-ods (MCDM) as a means to quantify stakeholdervalues in complex economic or environmental riskproblems where prior experience is limited (e.g.Hobbs and Horn, 1997).

1 MAU is not the only option here. Another that is set up todeal directly with multiple dimensions asks people to statepreferences between alternative bundles of attributes, one ofwhich is (or may be) cost. This approach is called ‘statedpreference’ by transportation researchers and ‘choice experi-ment’ by environmental economists. For a review, see Hanleyet al. (1997). The obvious contrasts with MAU are that thelatter involves attribute-by-attribute questions and requires, inits simplest form, the imposition of simple functional forms.Two other alternatives are ‘contingent ranking’ and ‘conjointanalysis’ (e.g. Bergland, 1994).

C. Russell et al. / Ecological Economics 36 (2001) 87–108 89

The remainder of the paper is divided into fivesections. Section 2 describes the setting, the choiceof attributes, the survey instrument and methodof its application, and summarizes the characteris-tics of the respondent samples. Section 3 containsthe results from the full MAU survey, including:most- and least-preferred levels of the attributesand their ranks and weights; stated WTP for theavailability of the most- rather than least-pre-ferred level of the respondent’s most importantattribute; and the implications of these responsesfor the respondent’s preferences over and WTPfor differences between the blended forests.

In Section 4 the tests and their results aredescribed: evidence of confusion when respon-dents directly confront the multi-dimensionalcomparisons of the blended forests; and matchesbetween implied and stated preferences over andWTP for differences between those forests. Effortsto explain the respondent-by-respondent resultsfor these matches are also reported. Section 5takes up the possibility that the results reflectedthe educational effect of completing the full MAUquestionnaire. Results are reported in terms ofextent of apparent confusion (intransitive prefer-ence statements) and comparisons of stated pref-erences and WTP amounts with thecorresponding responses of the ‘educated’ sample.Section 6 includes our interpretation of the resultsand suggestions for further work.

2. Setting, attributes, and sample

2.1. The setting and the attributes

The setting we chose for this test is a SouthernAppalachian forest ecosystem.2 We have else-where (Russell et al., 1997) discussed the develop-

ment of the attributes, how they were described inwords and given visual form, and how the settingwas simplified. Briefly, however, we required theattributes to satisfy (as closely as possible) fiveconditions:1. Because the questions asked involve having

the respondent imagine changing each at-tribute independently, they should be orthogo-nal, or as close to that condition as feasible,given the other requirements.

2. The number of attributes should be small —certainly smaller than the 18 ecological indica-tors used in the Environmental Monitoringand Assessment Program (EMAP) as descrip-tors of forests (Lewis and Conkling, 1994).Our goal was to stay below eight as suggestedby some of the literature on the difficultiespeople have with multidimensional judgments(e.g. Miller, 1956; Phelps and Shanteau, 1978).(A key part of our test described later involvessuch a judgment.)

3. The attributes should be describable by combi-nations of simple words and straightforwardvisual images (photographs, schematics,cartoons).

4. The attributes should be ecologically meaning-ful (i.e. interpretable by ecologists as providinga summary description of the forest). Severalindividual measurements might be summarizedin one or more index-like measures.3

5. The attributes should relate to people’s rea-sons for valuing forests as well as to scientificconcerns.

The last two of these requirements for at-tributes interact in an interesting way. One mightimagine trying to satisfy the last (c5) by includ-ing a ‘forest quality ladder’, akin to the ‘waterquality ladder’ developed at Resources for theFuture for a contingent valuation study of thebenefits of water pollution control. This laddershowed supportable recreation uses of water bod-

2 This choice was originally made because we were, at the timeof writing the proposal, working on an EPA/EMAP projectinvolving linking Environmental Monitoring and AssessmentProgram (EMAP) ecological indicators for a forest system tosocietal values. We had anticipated that the EMAP projectwould give us a substantial head start in devising the attribute setfor this study. In the event, however, the project was canceled byEPA before we had obtained the anticipated results, though wehad gathered focus group input on how lay people think aboutforests and what they value in and about them.

3 For our purposes this mapping could be one way. That is,ecological measurements could be used to determine levels of theattributes. But given levels of the attributes would not in generalimply unique levels of the underlying measurements.


ies changing as water quality — proxied by dis-solved oxygen — increased (Mitchell and Carson,1984). The analog would have been to attach thesources of human values for forests to an index offorest ‘quality’ for those uses, thus making theforest valuation problem unidimensional. The keyto the modest success of this approach in thewater quality arena, however, was the plausibilityof choosing dissolved oxygen as the single under-lying measurement.

But in forests, there is no simple array of‘functions’ (sources of values) related to any singleunderlying attribute or measurement determiningthe suitability of a particular piece of forest forproviding all or even many of those services. And,given a multidimensional description of a particu-lar forest, it is possible, even likely, that differentindividuals will judge that forest differently in thematter of how well it would serve a particularfunction, such as recreation service provision.Said the other way round: describing a forest as‘good for activity A’ would, in general, call upecologically different forests in the imaginationsof different respondents. Thus, it seemed (andseems) to us dangerous to substitute functions forecological descriptors, while still claiming to befaithful to ecological consistency. That is, thestraightforward approach to satisfying c5 (and,in the process, reducing the problem to a singledimension) promised to violate c4.

The six attributes we used, along with theirscales and special notes on visual presentation, aresummarized in Table 1. All descriptions consistedof combinations of words and pictures. Schemat-ics were used to help people picture ‘patchiness’.

The reader will note that nothing is said aboutwater features of the landscape/ecosystem. Wetold respondents to imagine any water featuresthey wanted, but to hold them constant across thequestion situations. We did not want to risk hav-ing our choice of water features distort the per-ceptions of respondents, for we knew from theliterature on landscape perception that water fea-tures tend to be dominant (e.g. Coss and Moore,1990; also Hanley and Ruffell, 1993). Similarly fortopography, though we suggested they imaginethe steep, rolling hills typical of much of theregion (Kaplan and Kaplan, 1989). We also asked

respondents to ‘think summer’. Finally, and fur-ther to simplify, we asked respondents to assumethat the conditions were to be maintained con-stant over many years. We realize that this is notecologically realistic, but we feared making theproblem more complex by trying to describe thedifferent paths of forest change that might arisefrom a given current condition.

2.2. The instrument

The MAU technique involves finding attribute-by-attribute functions that relate WTP to the levelof the attribute ‘provided’. For economy of pre-sentation we refer to these as ‘sub-WTP’ func-tions. Added together, they imply a total WTP forany combination of the attributes. For each re-spondent and each sub-WTP function the baselineis that respondent’s least preferred level of theattribute. This, in turn, implies that differencesbetween alternative forests can be valued, but nota forest as opposed to no forest.

Gregory et al. (1993) give some guidance aboutthe choice of attributes, but do not provide eventhe beginning of a cookbook for structuring thequestions necessary to capture these functions.And while there is a huge literature on applyingMAU in decisions (e.g. Keeney and Raiffa, 1976;Edwards, 1977; Merkhofer and Keeney, 1987; vonWinterfeldt, 1987; Keeney, 1992), much of thework on elicitation of MAU-related judgmentshas involved sophisticated decision makers oper-ating in their areas of expertise (e.g. Jenni et al.,1994). Our setting is different in two very impor-tant ways: we were interested in the views of lay,not expert, respondents; and, we could expect thatonly a handful of respondents would know muchabout forests when we first encountered them.This would be true even though a substantialfraction might actually be users of forests forsightseeing, hiking, and even camping.

We developed a simple MAU question struc-ture that seems to make very small demands, atall but one stage, on the cognitive capabilities ofrespondents. In the course of the construction andrevision of the questionnaire, we made consider-able use of focus groups and of one-on-one, thinkaloud interviews. These were held at the Vander-


Tab

le1

Att

ribu

tes:

thei

rde

finit

ion

and

mea

sure

men

tt

Spec

ial

note

sN

ame

Defi

niti

onSc

ale

unit

sE

xten

t

Thi

sis

apr

oxy

for

tim

esi

nce

last

exte

nsiv

edi

stur

banc

eT

ree

size

Dia

met

erof

larg

est

tree

sin

the

fore

stIn

ches

1–72

such

asse

riou

sfir

eor

clea

rcu

ttin

g.D

escr

ipti

onin

clud

esso

me

info

rmat

ion

abou

tbi

rds

and

0–10

0%

For

est

type

Per

cent

age

offo

rest

cove

rm

ade

upof

need

le-b

eari

ngtr

ees

othe

rm

amm

als

likel

yto

befo

und

inth

efo

rest

.N

oca

use

issp

ecifi

ed.

Cou

ldbe

dise

ase,

inse

cts,

Ext

ent

oflo

ssof

plan

tm

ater

ial

(fol

iage

)%

0–10

0V

isib

lepl

ant

dam

age

pollu

tion

ora

com

bina

tion

.T

opi

ndo

wn

anu

mer

ical

scal

eit

isne

cess

ary

toP

atch

ines

s1–

60c

Num

ber

offo

rest

patc

hes

ofm

inim

umsi

ze8

acre

sin

spec

ify

tota

lac

res,

min

imum

patc

hsi

zean

dto

tal

%a

2000

acre

fore

stth

atis

over

all

70%

cove

red

byfo

rest

fore

stco

ver.

1–5

Not

ast

atem

ent

ofsu

itab

ility

but

ofac

tual

cond

itio

ns.

Cat

egor

ies

Inte

nsit

yof

faci

litie

sde

velo

pmen

tan

dof

recr

eati

onal

Rec

reat

ion

inte

nsit

yus

eF

rom

esse

ntia

lw

ilder

ness

tohi

ghly

deve

lope

dan

d(1

=lo

wes

tle

vel)

inte

nsiv

ely

used

.A

lso

ast

atem

ent

ofac

tual

cond

itio

ns.

Fro

mes

sent

ially

Inte

nsit

yof

extr

acti

veus

eE

xtra

ctiv

eC

ateg

orie

s1–

5ze

roex

trac

tion

tom

inin

gor

exte

nsiv

elo

ggin

g.in

tens

ity

(1=

low

est

leve

l)


bilt Institute for Public Policy Studies (VIPPS),almost all under the direction of Molly HadleyJensen. In all we held nine focus groups over themonths from May to October 1996. A total of 33people participated in these. We also conductedseven think-aloud interviews using the entire in-strument as it then existed. These were held in themonths of August through October as the instru-ment neared what we guessed would be its finalform. We tried hard to bring in a wide range ofages, education levels, and (presumed) back-grounds with respect to forest-oriented activities.The earliest groups helped us settle on usefulvisual images and meaningful scales for the inten-sity variables. Later groups focused on structureand wording. The message of these latter groupswas consistent: simplify, simplify, simplify. Wetried very hard to respond, though we realize thateven after about a dozen redraftings, we stillprobably had residual problems with jargon andtechnical language, too many words, too longsentences, and too complex instructions.

The tasks asked of respondents were asfollows:4

1. Each respondent identified her/his most- andleast-preferred levels for each of the six forestattributes. This was done while the attributeswere being explained and the relevant visualsdisplayed.

2. Each respondent put the attributes in decreas-ing order of subjective importance. This rank-ing was triggered by the question: ‘If you werevisiting a forest in which every attribute was atyour least preferred level, which one of theattributes would you change first to your mostpreferred level, if you could?’ This same ques-tion form was used to find the second ranked,third ranked, and so on.

3. Each respondent supplied weights for the or-dered attributes, beginning with an arbitrary100 for her/his most important. It was stressedthat these did not have to add up to anyparticular number but could equally well be100, 98, 96, . . ., 90 and 100, 10, 9, . . ., 6 or

any other decreasing but non-negativepattern.5

4. We asked people to complete two exercisesthat bring in the notion of WTP for changes inthe levels of individual attributes. The firstexercise asked about annual WTP to help in-sure that the respondent’s most important at-tribute would be maintained at his/her mostpreferred (rather than least preferred) level in aforest of about 20 000 acres (if a square, about5.5 miles on a side) within 1.5 h of their city,Nashville. The second asked the same questionabout the second most important attribute foreach respondent.

The specificity about the park did not extend toname or location. The idea was just to create acontext far enough from the city that it would nothave to be crowded and intensively used, but closeenough that it could be visited for day trips. Thesize is arbitrary but is intended to create a sense ofscale well larger than familiar local forested parks.(It is 10 times the size of one such park usedelsewhere in the instrument to remind people ofthe region’s topography.)

Finally, it is worth noting that arriving at thisformat for the question connecting attributes toWTP was a painful process. (It is described inRussell et al., 1997.) Suffice it to say here, MAUpractitioners are likely to find it less than satisfac-tory. They would prefer to see a more ‘natural’connection via another attribute linked itself tomoney.

Examples discussed and rejected in creating thequestionnaire used here were forest industrywages and profits in the region. In our judgmentthe available alternatives were either just as ‘artifi-cial’ as the approach used here or would haveviolated the attribute independence requirement.The latter, for example, is true of the examplescited above, for the respondent could hardly be

5 An extensive literature exists discussing alternative meth-ods for eliciting weights and warning how difficult it can be toachieve consistency. The methods include pairwise compari-sons and subsequent hierarchical re-composition as part of anAnalytic Hierarchy Process (e.g. Saaty, 1983) or variations onthe swing-weighting, lottery, or pricing-out procedures typi-cally used by decision analysts (e.g. von Winterfeldt andEdwards, 1986).

4 The survey instrument, including black and white versionsof the visuals, is available from the senior author.


Table 2The ‘blended’ forest descriptionss

Attribute Second forestFirst forest Third forest

25%% diameter(1) Tree size 25%% diameter25%% diameter(2) Forest type 0% needle-bearing 0% needle-bearing 50% needle bearing

10% loss vegetation 30% loss vegetation(3) Visible plant damage 30% loss vegetation5 patches1 patch 5 patches(4) Patchiness3 4(5) Recreation intensity 23 34(6) Extraction intensity

expected to believe that forest industry profits orwages could rise in the absence of an increase inextractive activity. The former would be true ofan ‘attribute’ that was the cost of an admissionticket or other version of a fee for use.

2.3. Testing for ‘a difference’

In brief, our approach to testing whether MAU‘makes an identifiable difference’ involved creat-ing what we call ‘blended’ forests, using varyingcombinations of five of the six attributes. Theseare described in Table 2. Respondents were firstasked the MAU questions listed above that al-lowed us to infer the value they would put on anysuch forest. Then they were asked directly abouttheir ordinal preferences over and WTP for differ-ences between three ‘blended’ forests. (The forestswere labeled ‘first’, ‘second’, and ‘third’.) At thesimplest level, we asked three preference ques-tions, one for each of the possible pairings of theblends. This approach gave respondents enoughscope to give answers implying cyclic (intransitive)preferences. A straightforward result would thenbe to find a substantial number of intransitiveresponses, implying that people had trouble withthe direct multi-dimensional comparisons. TheWTP question asked, specifically, for first andsecond and for second and third, ‘How muchwould you be willing to pay annually to be able tovisit regularly your preferred forest [of the pair] asopposed to the other forest’.

More complex, and more difficult to interpret,were the comparisons of:� stated and implied preference patterns� stated and implied WTPs for the differences

between the blended forests.

Again, the most straightforward outcomeswould be to find very little agreement. Then, whilewe would have no way of saying which is ‘cor-rect’, we would at least know that there was adifference between the MAU and the more di-rectly obtained results. Because in later results,second forest turns out to be preferred to theother two by a majority of respondents, it isworth pointing out that when we ‘designed’ theblended forests, we had only informal focus groupinformation about which attributes were likely tobe most important and which levels of the at-tributes most and least preferred. We strove toproduce three descriptions that would be differentenough to be distinguishable, but not so differentas to make the ranking questions asked aboutthem trivially easy. The last two pages of theinstrument asked basic socio-economic questionsand also sought information about experiencewith forests, such as camping, gathering herbs,biking, and picnicking. The experience informa-tion was used in dummy variable (did/did not)form in subsequent regression analysis.

3. Data gathering and the respondent sample

By the time we had finished constructing thebasic MAU survey instrument, we sensed that wehad discovered an analogy to the rule of thumbone hears expressed for PC software creation. Inour study that rule seemed to become: the simplereach question is made for the respondent — givena particular overarching goal for the survey —the more questions there have to be and thelonger and, possibly, more boring the instrument.We felt that our instrument, even without the


blended forest questions, was sufficiently dauntingthat trying to use it as a mail survey would likelybring serious non-response problems. But neitherwas there money in the budget to allow us tocontract for more than a handful of individualinterviews. And, of course, the heavy need forvisuals made telephone surveys impossible. Facedwith this dilemma, we adopted a data-gatheringapproach based on ‘deliberative polling’ (Fishkin,1995). That is, we convened large groups (roughly75 people) in a conference room at VIPPS andworked through the survey with them.6 The visu-als were made available to the entire group simul-taneously and at low cost by using acomputerized bank of TV monitors available inthe room.7 These sessions took a bit over twohours, a fact that made it difficult to schedulethem in the evenings, which in turn interferedwith sampling from the working population.

The groups assembled did cover a wide rangeof ages, income and educational levels. The fourbasic socio-economic characteristics requestedfrom respondents were age (years), education,gender (1= female), and income. Characteristicsof the original 175 and of those successfully com-pleting the questionnaire are described in Table 3.We refer to an exercise involving only an intro-duction to the attributes followed directly by theblended forest questions as the ‘truncated survey’and show the characteristics of those starting andcompleting this exercise separately.

The message of this table is clear. Though wegathered diverse groups for our deliberativepolling exercises, those groups could not be calledrepresentative of the community. In particular,they were older, better educated, better off eco-nomically, and contained more women. There is

hardly any difference between the full groups thatsat down to take either survey and the corre-sponding sub-groups that successfully finished.There is, on the other hand, a somewhat bettermatch between the smaller group that saw thetruncated questionnaire and the Nashville popula-tion than between the full MAU sample and thatpopulation. But the differences between the twogroups who successfully finished the surveys (131and 43) are not statistically significant (except thatthe gender composition is significant at the 10%level).

There is nothing particularly striking in thedata from the self-reports of forest use, and we donot show a summary here. Driving, hiking andpicnicking are commonly engaged in. At the otherextreme, ATV use, herb gathering, hunting, andcutting wood (as an occupation) are quite rare.The two sample groups have similar patterns ofreported experience, and these experiences havebeen concentrated in the Southeastern US.

Note that 75% of the people who began the fullsurvey exercise finished. Most of the ‘failures’ (26of 44) occurred in the questions involving at-tribute preferences. Ten more people could not ordid not answer the WTP question about theirmost important attribute. Six could not or did notanswer the blended forest comparisons. One didnot complete the personal characteristics section.And, finally, one person who completed all partsof the questionnaire was an outlier in WTP state-ments by more than an order of magnitude. Onlyone person failed to successfully complete thetruncated survey.

4. Results from the surveys

4.1. Most- and least-preferred le6els; ranks andweights

The results from the questions about most- andleast-preferred levels of the attributes and theranking/weighting exercises are summarized inTable 4. The most important observations hereare the following:� All the differences between most- and least-pre-

ferred attribute levels are highly significant,

6 The analogy to Fishkin’s method and goals cannot becarried too far. We were not comparing people’s judgments onissues before and after an educational experience. But we didoffer information to, and answer questions from, our samplesin the context of a large group gathering.

7 The monitor wall, the associated scanning and computerequipment, and the person required to put them all together ina useful way, were generously provided by the First Amend-ment Center at Vanderbilt, an operating arm of The FreedomForum (ex-Gannett) Foundation.


Table 3Sample characteristics: age, education, gender and income

Age (years) Education Gender Income (categorical)(categorical) (1= female)

Full set of original respondents to full MAU instrument52.2(N=175) 3.3Mean 0.66 3.923.1 1.4 0.47S.D. 2.314 1Min 0 1

Max 86 5 1 8

Successfully completing full instrument49.2 3.3Mean 0.66(N=131) 3.7

S.D. 22.8 1.4 0.47 2.3Min 14 0 0 1

83 5 1 8Max

Full set of original respondents to truncated instrument46.8 3.5 0.59 3.1(N=44) Mean19.6 1.2S.D. 0.49 1.7

Min 20 2 0 1Max 84 5 1 8

Successfully completing truncated instrument46.5(N=43) 3.6Mean 0.59 3.119.7 1.3S.D. 0.49 1.720 2Min 0 1

Max 84 5 1 8

35.2 2.2Metro Nashville 0.52 3.1means

Education categories

Category cDescriptionassigned

Less than high school 1High school diploma 2

34-year college degreeSome graduate school 4

5Graduate degree

Income categories

Category cDescriptionassigned

Less than $10 000 1$10 001–20 000 2$20 001–40 000 3

4$40 001–65 000$65 001–90 000 5

6$90 001–115 0007$115 001–175 0008More than $175 000


Table 4Summary of responses to questions about attributes: most- and least-preferred levels, ranks and weightsa

Most- and least-preferred levels Ranks and weights of attributesof attributes

S.D. Mean rank Mean weight Rank S.D.Mean Weight S.D.

Tree size 50.8%% 17.0%% 4.6 55.3 1.4 28.1 most20.3 least10.6%%19.2 4.2 57.739.8% 1.6Forest type 28.8 most

68.6% 42.5 least14.1 3.1 74.8Visible plant damage 1.514.2% 25.2 most24.190.0% least

12.6Patchiness 13.8 4.0 62.1 1.3 25.5 most22.144.4 least

2.4 0.9 2.8 79.0 1.4Recreational 22.7 mostintensity

4.3 1.4 least0.8 2.2 84.9 1.7 23.9Extraction intensity most2.11.63.6 least

a n=131.

which suggests respondents on average knewwhat they were choosing.

� Similarly, the differences between the weightsand the ranks for adjacently ranked attributesare almost all significant. (Only the differencein rank between forest type and patchiness andthe weight differences between tree size andforest type, and between visible plant damageand recreational intensity fail a 10% signifi-cance test.)It is hardly surprising that the average most-

and least-preferred levels in Table 4 are in theinterior of the extent scales for each attribute(Table 1). Any other result would imply perfectagreement on these questions. But we believe it isimportant to note that for the most part, individ-ual respondents chose most-preferred levels thatwere themselves in the interiors of the extentscales (Table 5). Lower percentages chose interiorleast-preferred levels. We interpret this result asevidence that people gave some care to answeringthese questions, for it seems to us it would havebeen easier to have decided which direction was‘better’ and just picked the better endpoint asmost-preferred. (It is possible that some bias to-ward interior choices was introduced via the visu-als, which for the first four attributes did not

show the extremes, though levels close to theextremes were pictured. A micro-level examina-tion of the responses convinced us, however, thatonly for least-preferred tree size was this really alikely explanation. That is, choices did not line upwith pictured levels for the most part.)

Interior most- and least-preferred levels do adda difficulty to the construction of the attribute-by-attribute sub-WTP functions that are central tothe tests. This is because we have no informationabout the function ‘beyond’ an interior most-pre-ferred level. (Similarly, but, we believe, less seri-ously for the region between an interiorleast-preferred on the scale end beyond it.) Oursolutions are described just below, and are opera-tionalized in Appendix A.

4.2. WTP and deri6ing the attribute-by-attributeWTP functions

Responses to the WTP questions displayedlarge variances and were not significantly differentacross attributes. At least it is reassuring to findthat the stated amounts are significantly related toincome and to some of the indicators of forestexperience as summarized in Table 6. Two alter-native models are shown: in model 2 the mostimportant attribute for each respondent is iden-


Tab

le5

Shap

ety

pes

impl

ied

byan

swer

sto

mul

ti-a

ttri

bute

utili

ty(M

AU

)qu

esti

ons

—by

attr

ibut

e(%

of13

1)

Tre

esi

zeF

ores

tty

peV

isib

lepl

tE

xtra

ctio

nin

tens

ity

Pat

chin

ess

Rec

reat

ion

dam

age

inte

nsit

y

Per

cent

wit

hin

teri

orm

ost

pref

erre

d78

.693

.984

.780

.187

.777

.951

.921

.416

.1P

erce

ntw

ith

inte

rior

leas

tpr

efer

red

28.2

14.5

12.2

Per

cent

wit

hbo

thin

teri

orm

ost

and

leas

tpr

efer

red

45.8

19.1

15.3

18.3

12.2

9.2


tified by a dummy variable. In model 1 this is notdone. (The base case for this is the choice of eitherpatchiness or tree size as the most importantattribute. Only six people chose the former andone person the latter.)

From the responses to the questions aboutmost- and least-preferred attribute levels, attributeweights, and WTP for the most important at-tributes, plus a linearization assumption, it is pos-sible to construct a set of attribute-by-attributeWTP functions (‘sub-WTP’ functions, for short).These allow us to infer the value, for any person,

Fig. 1.

of any forest as described by the six attributes.More to the point, they allow us to infer howeach respondent should feel about the blendedforests we asked about — preference direction foreach paired comparison; and his/her WTP for thedifference between the two.

The linearization is shown schematically in Fig.1 for a response in which both an interior most-and least-preferred were chosen. This responseform was the modal one for the intensity at-tributes. Generalizing to the other seven possibleshapes is straightforward. The linearity assump-tion allows us to connect the WTP height of thefunction at the most-preferred attribute level tothe zero WTP for the least-preferred level. (Recallthat the question asked involves WTP for thedifference between these two levels.) We furtherassume: that zero applies to all attribute levelsbeyond the stated least-preferred; and that theportion of the function beyond the most-preferredlevel and to the end of the scale may itself belinearized. The algebra for these calculations is setout in Appendix A.

4.3. Implied preferences and WTP for the blendedforests

Using the sub-WTP functions we inferred pref-erences over, and WTP for differences between,the blended forests described in Table 2. Theseresults are reported in Tables 7a and 7b, parts (a)and (b).

4.3.1. Implied preferencesOur calculations of WTP for the three forests

produce implied preference patterns favoring

Table 6Analyzing stated willingness-to-pay (WTP) for most importantattribute

Coefficient values

Model c1 Model c2

Socio-economic characteristicsAge 1.53* 1.46*

16.22***Income 17.93***Gender 27.60 34.13

10.72 12.98Education

Identity or most important attributeForest type −36.17

−2.76Visible plant damage72.10Recreation intensity

−38.76Extraction intensity

Experience characteristics (yes/no)Camp 66.55*72.72**

−73.47*Herb −45.4210.22Flower −10.64

Wooda −183.73**−157.3*115.5*** 111.50***Firewooda

Bike 65.97* 88.36***47.14Picnic 39.41

Constant −160.4*** −162.6*R2 0.216 0.262

0.143Adj R2 0.166Log likelihood −859.6 −855.5F-stat 2.97 2.73Prob (F-stat) 0.002 0.001

a ‘Wood’ involves cutting wood for commercial purposes.‘Firewood’ involves simply gathering wood for that purpose;not for sale.

* Significant at B15%.** Significant at B10%.*** Significant at B5% are the levels of coefficient signifi-

cance.


Table 7aSummary of implied rank orderings of blended forests

% Preferring 2 to 1 Preferring 2 to 3Forest rank orders 2–3% c Implied

c % c %

1�2�3 9 6.9 9 6.92.331�3�28.4 11 8.42�1�3 1111 8.4

38.2 50 38.250 502�3�1 38.212.23�1�2 1632.1 42 32.1423�2�1

Total 100.1131 103 78.7 70 53.5

Table 7bWillingness-to-pay (WTP) calculated for each of the blended forests and the implied value of the difference between them ($/year;N=131)

Forest 1 Forest 2 Forest 3

Calculated mean WTP (BLENDi) $160.4 $215.4 $226.7391.0356.3272.4S.D.

Forest 2 vs. Forest 3Forest 1 vs. Forest 2Differences

−$11.3−$55.0Mean of signed differences131.0BLENDIi−BLENDIj S.D. 96.1

Mean of absolute differences $60.4 $42.5BLENDIi−BLENDIj S.D. 86.8128.6

forest 2. About 79% of respondents ‘should’prefer forest 2 to forest 1; and 53.5% ‘should’prefer forest 2 to forest 3. The implied patternfound most often is 2�3�1 with 3�2�1 aclose second, so there is broad implied agreementthat forest 1 is the least desirable (70.2% ‘should’agree).

4.4. Implied WTP

Averaging implied WTPs for the differencesamong the forests across the sample gives aslightly different picture than the ‘vote count’based on individual implied orderings. While inthe aggregate forest 2 is valued at $55 per personper year over forest 1, forest 3 is implicitly valuedmore highly than 2 by the group, though thedifference is only $11 per person per year. If thecalculated values of the blended forests are takento be drawings from a normal distribution, themean calculated WTP difference between forests 1

and 2 is statistically significant; that betweenforests 2 and 3 is not.

For completeness, and because we will laterexamine absolute differences, we also report, inTable 7b, means of the unsigned differences inimplied WTP for the forest differences.

4.5. Initial tests of the MAU ‘difference’

4.5.1. Intransiti6e responses to the blended forestpreference questions

There was little evidence of confusion amongrespondents when they faced the three preferencequestions concerning the possible pairs of thethree blended forests. Only 5 of 131 people (about4%) gave responses implying intransitivity. Wehad expected more such responses. One practicaland partial explanation is that in their statedpreference responses, almost 60% of respondentsselected forest 2 over both forest 1 and forest 3.These answers (2�1, 2�3) leave no room for


intransitivity and thus reduce the size of the set ofpeople who might exhibit confusion. That is, thedominance of forest 2 may be in part responsiblefor the results. But only in part; 40% of respondentsdid not in effect ‘lock in’ transitivity.

This result tells us that at least by this measureand in this setting there is not much room for MAUto ‘make a difference’. (The preferences implied bythe MAU responses could never be intransitive.)

4.6. The extent of agreement between implied andstated preferences and WTP amounts

If the preference and WTP amounts implied bythe answers to our MAU questions were in perfectagreement with those stated directly, MAU couldbe said merely to reproduce answers arrived atmore directly. Zero agreement would mean thatMAU makes a huge difference, though it would notbe possible to claim that the MAU versions were‘more correct’ than the direct statements. This is areflection of the general problem of ‘verifying’stated preferences or WTP results. Here, one mightbe inclined to suspect the MAU results, if onlybecause of the importance of lin-earity to themethod of deriving the sub-WTP relations for theattributes.

4.6.1. Agreement of implied and statedpreferences

The implied preference orderings over theblended forests have already been reported. InTable 8 we compare the implied and stated pat-terns and the differences between the two (statedminus implied).

Informally, we see that, in the aggregate, theorderings 1�2�3 and 2�1�3 are under-pre-dicted; while 3�1�2 and 3�2�1 are over-pre-dicted. Or, roughly speaking, the MAUmechanism led us to infer that forest 3 was a gooddeal more popular than was consistent with theactual statements of individuals.

But these comparisons of aggregate countscould reflect massive mis-prediction at the level ofthe individuals, mis-predictions that, in effect,

cancel out.8 We therefore need to look at individ-ual predictions and statements. One fairlystraightforward way to do this is to create a scorevariable according to the following rule:

Stated order Implied order Score

i� j�k 5i� j�k4i�k� j

j� i�k 3j�k� i 2

1k� i� jk� j� i 0

Using this scheme, we find results at the level ofthe individuals as shown in Table 9.

For a bit over a third of the sample, predictionand statement matched perfectly. For a bit overhalf, there was, at worst, a difference in orderingof the 2nd and 3rd ranked alternatives. And foralmost three quarters, either the first and secondor the second and third choices — but not both— were transposed.

Slightly less impressionistically, we can look atthe correlation coefficients between the stated andimplied preference relations for forests 1 and 2and forests 2 and 3. These are both highlysignificant:

Correlation coefficient(N=126)

forest 1 vs. 2 0.3210.401forest 2 vs. 3

8 Thus, assume six people were involved; label them A,B, . . ., F. We would get excellent agreement in the aggregatein the following situation in which e6ery individual predictionwas off:

CountStated Implied Countby for

1 1F1�2�3 A1B A1�3�2 1

2�1�3 1C B 11C12�3�1 D

E 13�1�2 D1F E 113�2�1


Table 8Comparing stated and implied preference orderings

Percentage impliedRank order Differences (stated−implied)Percentage stated

20.61�2�3 6.9 13.72.32.4 0.11�3�28.42�1�3 14.623.0

38.234.9 −3.32�3�112.23�1�2 −9.03.232.115.9 −16.23�2�1

100.0Total 100.1 −0.1n=126 n=131

We can also calculate the Spearman rank ordercorrelation to see whether the ranks (popularity)of the six possible full orderings are ‘close to’being the same. For the comparison betweenstated and implied preference orderings, the calcu-lated correlation coefficient is 0.543 for sixgroups, which means that we can reject the nullhypothesis of no relation between the orderings.

Finally, we can apply the McNemar test tocompare the distributions of two related variables,using the numbers of matches and misses betweenstated and implied preference orders for the pairsconsidered separately. The key to this test is, ineffect, how big a difference there is between thenumbers of erroneous inferences: inferring 1�2when the stated preference is the opposite; andconversely inferring 2�1 when the stated prefer-ence is the opposite. This explains why, eventhough the numbers of correct inferences are veryclose for the two comparisons (95 for 1 vs. 2 and86 for 2 vs. 3) the test statistics are wildlydifferent:

1 vs. 2 X2=1.16 assymp. sig 0.281assymp. sig 0.0002 vs. 3 X2=24.03

So the null of no difference between the statedand implied orderings is not rejected for the 1 vs.2 question, but is rejected for the 2 vs. 3 question,and we cannot say either that the MAU techniqueproduces totally different answers or that itclosely mimics the summary judgments ofindividuals.

4.6.2. Stated and implied WTP for differencesbetween forests

Above, we reported on the mean implied WTPfor the differences between forests 1 and 2 andbetween 2 and 3. We now bring together the meanof stated WTP with the mean implied response inTable 10.

The differences between the stated and im-plied values for the two forest pairs are bothsignificant at a level less than 5%. (A similarconclusion is reached using the Wilcoxon signedranks test.)

4.6.3. Preference ordering analysisWhat can be said about the relationship be-

tween the level of agreement of stated and impliedpreference rankings over the blended forests andthe characteristics of the respondents involved?

Table 9Individual comparisons of stated and implied preferences overthe blended forests

Cumulative %Score c of respondents %

5 34.9444 55.520.626

16.7 72.221382.52 13 10.3

151 11.9 94.47 100.05.60

126a1Total 100.0

a Those stating circular preferences have been dropped.


Table 10Mean stated and implied willingness-to-pay (WTP) for the difference between two blended forests (N=131)

Forest 2 vs. 3Forest 1 vs. 2

S.D.Mean stated WTP S.D. S.D.Mean implied WTP Mean implied WTPS.D. Mean stated WTP

$−11.3$−19.9 133.9 $−55.0 131.0 $15.3 96.0105.6

First, in order to keep the cell sizes up, we havesimplified the score variable as follows:

CellModifiedStated Implied Originalorder scoreorder score sizes

70i� j�k i� j�k 5 22i�k� j 41 493j� i�k1j�k� i 2

12k� i� j 1 00 0k� j� i

To explore our ability to explain the scoredifferences across individuals we tried both or-dered and sequential probit. The latter offered noimprovement over the former, and in Table 11only results of the former estimation exercises arereported. Neither the model using only the socio-economic variables nor the expanded version withall the experience variables is impressive in itsperformance.9 In the first equation, education issignificant at less than 15% and the proportion ofcorrectly predicted scores is 0.58. The expandedmodel has several significant coefficients — butnone of those is attached to a socio-economicvariable. It only does slightly better as a predictorof scores, with 0.60 correct. In neither case is anyof the actual zero scores predicted; and in boththere is substantial under-prediction of scoresequal to one, while the number of scores equal totwo is substantially over-predicted. It appearsthat our available information about the respon-

dents does not allow us to explain our failure toinfer (from the MAU answers) the preferencesacross the blended forests that the respondentsstate.

4.6.4. WTP analysisIn Table 12 we summarize results from at-

tempting to explain variations across respondentsin the absolute size of the deviation betweenstated and implied WTP for the differences be-tween the blended forest. The forest 1, 2 andforest 2, 3 deviations are dealt with separately.(We examined models with only socio-economicexplanatory variables, but these had very lowF-statistics and are not reported.)

For neither set of deviations are the explana-tory models very satisfactory. For the forest 1, vs.2 deviations, none of the socio-economic charac-teristics has a significant coefficient, though threeof the experience dummies have coefficients sig-nificant at between 5 and 10% (camping, fire-wood gathering, and ATV use). All thesecoefficients are positive, suggesting that forest ex-perience somehow makes it more difficult for theMAU technique to infer the correct summaryjudgments across the multi-dimensional forests.Perhaps, for example, the experience leads peopleto formulate rules of thumb for judging forests— rules that are not adequately captured by theattribute-by-attribute, linearized MAU method.

The most interesting results are those for theforest 2 vs 3 deviations. The overall relation ishighly significant by the F-test, two of the socio-economic variables are significant at between 5and 10% (age and education), and two of theexperience dummies are significant at least the10% level. Again, all the significant coefficientsare positive, suggesting that our MAU inferences

9 The base state for the experience variables — the omitteddummy — is the condition of reporting no experience in orwith forests.


Table 11Ordered probit analysis of modified score (implied vs. stated preference) N=126

Model 1 Model 2

Socio-economic 6ariablesAge −0.0009−0.0020

−0.0838−0.0621Income−0.0879Gender −0.0666−0.09380.1307*1Education

Experience 6ariables0.2308Hike

Camp −0.03190.4211Hunt

−0.6829***1Herb0.5511**1Flower

Wood −0.03810.2207Firewood

Drive 0.4084*1

Bike −0.3698−0.7814ATV

Picnic −0.0442

1.55***1Constant 1.46***1

1.606MU(1) 1.474

−106.2Log likelihood −99.915.452.93X2

15Deg. free. 40.420.57Significance

Frequency of correct prediction 73/126=0.58 75/126=0.60

PredictedActual Total actualPredicted

1 2 0 10 2

0 0 1 6 0 5 2 77 42 01 170 32 494 66 00 122 58 70

12 114Total predicted 00 34 92 126

* Significant at B15%.** Significant at B10%.*** Significant at B5%.

‘work’ better for younger, less well-educated re-spondents, and those with less active forest expe-rience. This has the virtue of being consistentwith one of the arguments for using MAU tech-niques — that they are easier for people of mod-est experience and intellectual capacity to dealwith.

5. The educational value of the MAU questions

One of the most striking findings to us of thefirst phase of survey work was the low number ofintransitive responses to the blended forest prefer-ence questions. To explore whether this camefrom an education effect of the MAU survey


instrument or really did demonstrate a greaterability to deal with multiple dimensions than peo-ple are usually given credit for, we administered atruncated survey without the MAU questions to44 respondents. This survey went directly from adescription of the attributes to the blended forestquestions. Therefore, for this group of respon-dents, we do not have MAU answers from whichto infer blended forest preferences and WTP. Weonly know what the respondents stated directlyabout their rankings of the blended forests andtheir WTP for the differences between them. Theonly meaningful comparisons are with those samestatements from the respondents to the full sur-vey. Recall that the demographic characteristicsof these two samples are similar, though the

Table 13Comparison of stated blended forest preferences — full multi-attribute utility (MAU) and truncated survey samples

Truncated surveyRank orders Full MAU survey(% stating) (% stating)

1�2�3 20.6 25.61�3�2 2.4 2.3

23.02�1�3 30.227.934.92�3�1

3.23�1�2 4.69.415.93�2�1

Total 100.0100.0

numbers involved are not large, and neither sam-ple mimics the population from which they weredrawn.

The first important result is that only one ofthe 44 people who responded to the truncatedsurvey gave preference answers implying intransi-tivity. Thus, in the most obvious area for educa-tion to be helpful, no such effect is seen.

We can also compare stated preference rank-ings for the blended forests by the two groups(Table 13). The numbers certainly do not lookvery different. And using either the rank ordercorrelation coefficient (0.943), or the Kendalltau–b (0.867), we find it possible to reject thenull hypothesis of no relation with considerableconfidence (a=0.005 for the rank order test anda=0.015 for the Kendall test).

And finally, we can look at stated (and signed)WTP amounts (Table 14). This comparison ofstated WTP for forest differences conveys amixed message. The means for the forest 1 vs. 2comparison are almost identical. But the differ-ence between the sample means for the forest 2vs. 3 comparison are different at a significantlevel less than 0.1%.

6. Concluding comments

The results of this experiment leave the readerfree to judge the MAU glass either half full orhalf empty, depending on his/her predisposition.Someone positively inclined could stress that:

Table 12Explaining absolute deviations between stated and impliedwillingness-to-pay (WTP) for the blended forest differences

Blended forest Blended forestc2 vs. c3c1 vs. c2

Socio-economic characteristics0.039 0.745**1Age8.24Income 5.56*1

6.59Gender 27.21*1

13.98Edu 11.01**1

Experience characteristics−42.07*1Hike −13.14

Camp 47.08***153.90**1

−1.31Hunt 50.18*1

Herb −10.24−18.83−1.81Flower 17.67

Wood −62.37 −36.14Firewood 55.76***1 36.66**1

16.59Drive 8.5836.10Bike 25.2392.74**1ATV −10.96

Picnic −8.672 −2.54

−31.37Constant −92.66***1

R2 0.202 0.254Adj R2 0.1570.098Log likelihood −747.8−807.9

2.611.94F-StatProb (F-Stat) 0.026 0.002

* Significant at B15%.** Significant at B10%.*** Significant at B5% are the levels of coefficient signifi-

cance.


Table 14Comparison of stated willingness-to-pay (WTP) for differences between blended forests — full multi-attribute utility (MAU) andtruncated survey samples

Forest 1 vs. 2 Forest 2 vs. 3

Full MAU (N=131) Truncated survey (N=43)Full MAU (N=131) Truncated survey (N=43)

Mean WTP S.D. Mean WTPMean WTP S.D.S.D. Mean WTP S.D.

−$20.8 218.5 $15.3 105.6 $56.1−$19.9 230.9133.9

� It is possible to construct an MAU-based sur-vey instrument, embodying multiple indepen-dent dimensions of a complex valuationproblem (in our case, forests). The questionsabout preferences over the scale of each dimen-sion, relative importance of the dimensions(numerically expressed), and WTP to alter oneof the dimensions can be answered, even bypeople with limited education.

� Participants, who ranged in age from highschool students to volunteers from a nursinghome, were generally quite willing to workthrough the tasks given to them and to thinkabout valuation in the context of multiple at-tributes for a forest ecosystem. This positiveresult underlies the appeal of a constructiveapproach to valuation (Payne et al., 1992) andits fundamental assumption that our notions ofvalue are built up, piece by piece, much as abuilding is constructed. Of course, some build-ings are built better than others, and protocolsfor the design of multi-attribute environmentalevaluation efforts are still at an early stage(Gregory and Slovic, 1997). Nevertheless, thewillingness of diverse respondents to undertakethis rather lengthy task, and to stick with itthrough to a monetary valuation, suggests a fitbetween the way the questions were posed andhow many participants naturally think aboutthe types of policy questions that might affectmanagement of a forest ecosystem.

� Their answers, combined with a quite restric-tive linearity assumption, allow the derivationof a ‘sub-WTP function’ for each dimension orattribute. These functions can, in turn, be usedto infer at least relative values for the particu-lar multi-dimensional good at issue (here

forests) described by combinations of the at-tributes. In particular, it is possible to makejudgments among alternative possible goods,either on the basis of ‘votes’ (aggregating ordi-nal preferences) or total WTP.

� The inferred preferences and WTP figures ap-proximate, though they do not perfectly match,the stated preferences and WTP numbers ob-tained directly from respondents.

� The stated WTP answers themselves appear, ingeneral, to be sensibly related to key socio-eco-nomic characteristics of the respondents.But a more skeptical person might question the

importance of these findings by pointing to someawkward facts.� It is not clear that the MAU process makes

much difference in the chosen setting, multi-di-mensional though it is, because: (i) a subsampleasked for blended forest preferences without thebenefit of the MAU educational process exhib-ited even less cyclicity (taken as evidence ofconfusion about the vector comparisons); (ii)the mean WTP answers of this group for thedifferences between blended forests were in onecase identical to the mean from the ‘educated’sample and in one case different; (iii) the statedpreference orderings over the blended forestswere not significantly different for the unedu-cated and educated samples.Thus, it may be that MAU could be useful in

more complex problem settings, for which thevector comparisons would be overwhelming — ifthere were more dimensions, for example. But theskeptic might well say, in addition, somethingalong the following lines:� Even granting that each question is quite sim-

ple, the facts are that: (i) the overall instrument


took a long time to complete (so was almostcertainly not a good candidate for a mail sur-vey, which in turn implies the technique maybe expensive to use; (ii) only 75% of those whosat down to do the survey successfully (com-pletely) finished.

� The several stated WTPs are about the onlyresults that seem to ‘make sense’, if the test is:can we explain the variation across respondentsby their characteristics and self-reported expe-riences (with forests)? In particular, there ap-pears to be no straightforward explanation ofvariation in the matches between implied andpredicted preferences and WTP numbers forthe blended forests pairs.So, it seems clear that the jury is still out on the

promise of MAU as an alternative to the conven-tional contingent valuation technique for prob-lems such as ecosystem valuation. The approachcannot be rejected as without promise. But neithercan it be embraced as the answer to the problemsof cognitive challenges — especially multi-dimen-sionality — identified in the literature and likelyto become more common as the boundaries of thesearch for dollar values in the environment arepushed out by the needs of policymakers.

How might one extend the investigation of thepotential for the MAU survey technique? Ourrecommendations are aimed at avoiding some ofthe difficulties observed in this study, and increas-ing the chance of finding a definitive result.� First, it would seem desirable to concentrate on

attributes that are believably ‘manageable’. Ifan attribute is clearly the result of naturalforces and events, respondents may wonderwhat the point of asking about their prefer-ences is.

� Second, we would suggest a method of surveyadministration, perhaps via laptop computersor using an Internet sampling company suchthat each person could be offered a randomlydesigned set of multidimensional ‘forests’ (orwetlands or streams, or whatever) to answerpreference and WTP questions about. Thiswould avoid the ‘dominance’ problem that isreflected in our intransitivity results.

� Finally, as almost goes without saying, wewould push for enough funding to produce

completed surveys in at least the 8009 rangerather than the 2009 managed here. Thismight be achieved by simplifying the questionseven more than we managed, so that a cheaper‘delivery’ method would be possible.

Appendix A. Deriving the parameters of the‘Sub-WTP’

A.1. Functions from sur6ey responses

Successfully completed surveys contain the fol-lowing information:� most- and least-preferred levels of each

attribute� importance ranks and associated weights for

the attributes� stated WTP of each respondent to change her/

his most important attribute from her/his least-to most-preferred level.The exposition here will be notationally simpler

if we use subscripts to indicate order of attributeimportance rather than order in the list of at-tributes. So let us call the basic data:� most- and least-preferred levels of the attribute

that is ith in descending order of importance:bM

i , bLi

� importance weight for the ith attribute: wi

� WTP for the most important attribute: P1

Two important relations may be inferred fromthe way the WTP and the weight questions areasked.� for WTP it must be true that:10

w1a1bL1 + · · ·+w6a6b6

M+Z

=w1a1bM1 + · · ·+w6a6b6

M+Z−P1

or

P1=w1a1(b1M−bL

1 ),

10 This exposition assumes that the second through sixthmost important attributes are to be held constant at their mostpreferred level, but no level for the other attributes wasspecified in the question that produced the statement of P. Thealgebra works whatever level respondents had in mind so longas they held it constant. But it is possible that, if we hadspecified either least- or most-preferred, we would have seendifferent answers.


where the ai are scaling constants, and Z isincome.

Define bMi −bL

i =Dbi

� For the weights, it must be true that:

wia1Db1=aiDbi

Now: P1=a1Db1, which in effect, defines a1.But, it must be true that there are also in principleP2, . . ., P6 that could have been obtained fromrespondents by questions of the form asked forP1. Thus, Pi=wiaiDbi would be true.

This allows us to construct estimates for the Pi,based on P1 and the other known parameters.Thus,

If P2=w2a2Db2

and,

a2Db2=w2a1Db1

we get

P. 2=w22a1Db1=w2

2P1

where the hat indicates that we are estimating theattribute WTP rather than recovering it from astatement. In terms of what we call the attributeWTP functions, then, we have the height of thefunctions at the attribute values bM. The slope ofthe function between bM

i and bLi is Pi/(bM

i −bLi ),

which is positive when bMi is to the right (higher)

on the attribute scale. The WTP for some level ofbi, call it ba

i , between bL and bM, is given by:

Pi(bai )=Pi [1+ (ba

i −bMi )/Dbi ].

Notice that if bLi is to the left of bM

i , thenumerator of the second term in the brackets isnegative, while the denominator is positive. Viceversa, for the case in which bL

i is to the right ofbM

i .Finally, in calculating the WTP for the blended

forests, it is necessary to allow for cases in whichthe actual ba

i falls outside the bLi to bM

i range. Therules used here are:� If ba

i is ‘on the other side’ of bLi from the most

important level, Pi(bai ) for that attribute is

zero.� If the ba

i is ‘on the other side of’ the most-pre-ferred point from the least-preferred, then welinearize the function so that it’s slope is Pi/

(bMi −b( )Pi/(bM

i −b6 ), depending on whether thebM

i \bLi or bM

i BbLi . (Here, b( denotes the

right-hand end of the scale and b6 the left-handend.)

References

Bergland, O., 1994. Valuing Multidimensional Changes withContingent Ranking, Discussion Paper D-15/1994, Depart-ment of Economics, Agricultural University of Norway,A, s, Norway.

Bohm, P., 1994. CVM spells responses to hypothetical ques-tions. Nat. Resources J. 34 (winter), 37–50.

Coss, R.G., Moore, M., 1990. All that glistens: water connota-tions in surface finishes. Ecol. Psychol. 2, 367–380.

Edwards, W., 1977. Use of multi-attribute utility measurementfor social decision making. In: Bell, D., Keeney, R., Raif-fer, H. (Eds.), Conflicting Objectives in Decisions. JohnWiley, New York.

Fischhoff, B., 1991. Value elicitation: is there anything inthere? Am. Psychol. 46, 835–847.

Fishkin, J., 1995. The Voice of the People. Yale UniversityPress, New Haven.

Gregory, R., Slovic, P., 1997. A constructive approach toenvironmental valuation. Ecol. Econ. 21 (3), 175–182.

Gregory, R., Lichtenstein, S., Slovic, P., 1993. Valuing envi-ronmental resources: a constructive approach. J. Risk Un-certainty 7, 177–197.

Hanley, N., Ruffell, R.J., 1993. The contingent valuation offorest characteristics: two experiments. J. Agric. Econ. 44(2), 218–229.

Hanley, N., R.E. Wright, and V. Adamowicz, 1997. Usingchoice experiments to value the environment: design issues,current experience, and future prospects. Unpublished, re-submitted to a special issue of Environmental and Re-source Economics.

Hobbs, B., Horn, G., 1997. Building public confidence inenergy planning: a multi-method MCDM approach todemand-side planning at BC gas. Energy Policy 25, 357–375.

Jenni, K.E., M.W. Merkhofer, and C. Williams, 1994. TheRise and Fall of a Risk-Based Priority System: Lessonsfrom DOE’s Environmental Restoration Priority System.Discussion paper, unpublished, Applied Decision Analysis,Menlo Park, CA.

Kaplan, R., Kaplan, S., 1989. The Experience of Nature: APsychological Perspective. Cambridge, New York.

Keeney, R.L., 1992. Value-Focused Thinking: A Path to Cre-ative Decision-Making. Harvard University Press, Cam-bridge, MA.

Keeney, R., Raiffa, H., 1976. Decisions with Multiple Objec-tives: Preferences and Value Tradeoffs. Wiley and Sons,New York.

Lewis, T.E., Conkling, B.L., 1994. Forest Health Monitoring:


Southeast Loblolly/Shortleaf Pine Demonstration InterimReport. United States Environmental Protection Agency,Environmental Monitoring and Assessment Program Cen-ter, Research Triangle Park, N.C., Report EPA/620/SR–94/006.

Merkhofer, M.W., Keeney, R.L., 1987. A multi-attribute util-ity analysis of alternative sites for the disposal of nuclearwaste. Risk Anal. 7 (2), 173–194.

Miller, G.A., 1956. The magical number seven, plus or minustwo: some limits on our capacity for processing informa-tion. Psychol. Rev. 63, 81–97.

Mitchell, R., Carson, R.T., 1984. A Contingent ValuationEstimate of National Freshwater Benefits: Technical Re-port to U.S. EPA. Washington, DC: Resources for theFuture.

Mitchell, R., Carson, R.T., 1989. Using Surveys to ValuePublic Goods: The Contingent Valuation Method. Re-sources for the Future, Washington, DC.

Payne, J., Bettman, J., Johnson, E., 1992. Behavioral decisionresearch: a constructive processing perspective. Annu. Rev.Psychol. 43, 87–131.

Phelps, R.H., Shanteau, J., 1978. Livestock judges: how muchinformation can an expert use? Organ. Behav. Hum. Per-form. 21, 209–219.

Russell, C.S., V. Dale, J. Lee, M.H. Jensen, M. Kane, and R.Gregory, 1997. Applying Multi-Attribute Utility Tech-niques to Environmental Valuation: A Forest EcosystemExample. Final Project Report to the Environmental Pro-tection Agency, December.

Saaty, T., 1983. Priority setting in complex problems, IEEETrans. Eng. Manage., EM-30, August.

Schkade, D., Payne, J., 1994. How people respond to contin-gent valuation questions: a verbal protocol analysis ofwillingness to pay for environmental regulation. J. Envi-ron. Econ. Manage. 26 (1), 88–109.

von Winterfeldt, D., 1987. Value tree analysis: an introductionand an application to offshore oil drilling. In: Kleindorfer,P., Kunreuther, H. (Eds.), Insuring and Managing Haz-ardous Risks: From Seveso to Bhopal and Beyond.Springer-Verlag, New York, pp. 349–377.

von Winterfeldt, D., Edwards, W., 1986. Decision Analysisand Behavioral Research. Cambridge University Press,Cambridge.

.

experimenting with multi-attribute utility survey methods in a multi-dimensional valuation problem

Documents