A Comparison of Telephone and Personal In te rv iewing

Tom W. Smith

Apr i l , 1984

National Opinion Research Center Univers i ty of Chicago

This r e p o r t was prepared f o r t he General Soc ia l Survey (GSS) Board of Overseers. The GSS is d i r e c t e d by James A. Davis and Tom W. Smith. The p r o j e c t is funded by the National Science Foundation, Grant No. SOC7.7-03279.

GSS : T-PIntervw

In our review of telephone-personal interviews w e examined

approximately 80 pieces of l i t e r a t u r e ( see bibl iography) and corresponded with

s i x colleagues about t h e i r experience using both modes. Approximately 32

s t u d i e s have compared telephone and personal interviewing. These include 21

complete o r p a r t i a l experimental comparisons of telephone and personal cross-

sec t ions , two experimental comparisons of reinterviews, two laboratory

experiments, three examinations of telephone and non-telephone households on

personal surveys, and four miscellaneous non-experimental s tud ies . Nine were

nat ional s tud ies , 14 communities o r s t a t e s , 7 s p e c i a l populat ions and sub-

communities, and two convenience samples. Health care ( 1 1 ) was the most

f requent ly inves t iga ted topic , followed by crime ( 3 ) , mental s t a t e s (31, c i t y

se rv ices ( 2 ) , p o l i t i c s ( 21 , and miscellaneous ( 1 1 - unions, c a f e t e r i a , r a c i a l

a t t i t u d e s , paper readership, e t c . )

I n the fol lowing sec t ion we w i l l compare the r e l a t i v e s t rengths and

weaknesses of personal and telephone interviewing, focusing on the themes of

q u a l i t y and comparability.

1 , A telephone survey would not cover the sample populat ion covered by

personal household surveys. About 7-9% of households a r e without

telephones. While t h i s number has been dec l in ing s t e a d i l y over the l a s t

seve ra l decades, the re is some specula t ion t h a t the trend may not

continue because of sa tu ra t ion . Also, some concern has been ra ised t h a t

the d i v e s t i t u r e of ATCT w i l l a c t u a l l y lower the telephone penetrat ion

r a t e , because of higher c o s t s and l imi ted service. Telephone coverage

is not uniform across subpopulations and is e s p e c i a l l y low among blacks,

the poor, r u r a l r e s iden t s , Southerners, and the t r a n s i e n t , Because of

the incomplete and d ispropor t ionate coverage by telephone, a switch t o

telephone surveys would mean t h a t one would e i t h e r have t o 1 ) redefine

GSS : T-PIntervw

t h e sample popula t ion and e i t h e r a ) r e s t r i c t a n a l y s i s of previous

personal surveys t o te lephone households only o r b ) a d j u s t the telephone

sample through some type of p o s t s t r a t i f i c a t i o n weight t o at tempt t o

r ep re sen t the genera l household popula t ion o r 2 ) adopt a dua l frame

des ign wi th a personal in te rv iewing component among non-telephone

households t o supplement t h e te lephone survey. (This would however

negate c o s t sav ings gained from telephone interviewing.

2. Response r a t e s a r e lower on the te lephone than i n person. In e i g h t

s t u d i e s t he personal response rate was c l e a r l y higher , i n s i x ca ses

pe r sona l response r a t e s were s l i g h t l y h igher , i n f i v e cases no

d i f f e r e n c e s were repor ted and i n two cases response r a t e s were s l i g h t l y

h ighe r on the telephone. The b e s t e s t ima te based p r imar i ly on t h e

exper ience of the SRC is t h a t a te lephone GSS would have a response r a t e

of f i v e t o t e n pe rcen t p o i n t s lower on the telephone. This is b a s i c a l l y

due t o h igher r e f u s a l and break-off r a t e s on telephone surveys.

Likewise i n a l l t en surveys t h a t compared r e f u s a l l e v e l s on an income

q u e s t i o n te lephone r e f u s a l s w e r e s u b s t a n t i a l l y higher . I t is a l s o

harder t o judge non-response b i a s i n telephone surveys s i n c e a t t r i b u t e s

of the household, neighborhood, e t c . can no t be known f o r

nonrespondents.

3. Telephone RDD surveys have sma l l e r design e f f e c t s than mul t i s tage ,

c l u s t e r e d personal surveys. The des ign e f f e c t s from more cases per

i n t e rv i ewer l eads t o a g r e a t e r i n t e rv i ewer design e f f e c t on telephone

in te rv iewing , bu t the e l imina t ion of a r e a c l u s t e r i n g l eads t o a lower

des ign e f f e c t o v e r a l l on the te lephone and t h e r e f o r e t o g r e a t e r sampling

e f f i c i e n c y .

4. Because it is much e a s i e r t o monitor in te rv iewers on a telephone survey,

the l e v e l of s t a n d a r d i z a t i o n is much higher . On the o t h e r hand, because

in te rv iews are in te rv iewing "b l indly" from usua l ly one l o c a l i t y , it is

d i f f i c u l t o r impossible t o match respondents r a c i a l l y o r reg iona l ly . In

add i t i on , because the telephone in te rv iewing s t a f f is more homogeneous

than a n a t i o n a l f i e l d s t a f f t he re may be more p o t e n t i a l f o r interviewer

b i a s a s opposed t o in te rv iewer variance.

5. Because of t h e more l i m i t e d flow of communication i n t he telephone mode,

respondent / interviewing feedback and c l a r i f i c a $ i o n s a r e more

d i f f i c u l t . Likewise, probing is found t o work more e a s i l y ' a n d flow more

n a t u r a l l y i n pe r sona l s e t t i n g s . This r e s t r i c t e d communication exchange

may c o n t r i b u t e t o c e r t a i n d i f f e r e n c e s i n response as covered below.

6, Ce r t a in t ypes of complex in te rv iewing can be handled b e t t e r on the

te lephone whi le o t h e r s a r e more d i f f i c u l t . Questions s t r i n g s involving

a long s e r i e s on complicated f i l t e r s and s k i p s can be handled v i r t u a l l y

e r r o r f r e e on CAT1 telephone interviews. It is more d i f f i c u l t , however,

f o r respondents t o understand and fo l low complex response t a sks on the

phone such a s rankings, random response ques t ions , magnitude measurement

s ca l ing , o r long ques t ions . There a r e a l s o c e r t a i n problems with

l i s t i n g even t s and behaviors and wi th open ended ques t ions t h a t w i l l be

covered below. Also, des igns t h a t c a l l f o r t h e respondent checking

c e r t a i n r eco rds (e.g. check s tubs , brand of appl iances , e t c . ) a r e

considered more d i f f i c u l t on the telephone. F i n a l l y , in te rv iewer

observa t ions of respondents , t h e i r household, o r t h e i r neighborhood a r e

n o t poss ib le .

7. Telephone in t e rv i ews tend t o be more cursory , less cogn i t i ve ly

s t imu la t ing , and more burdensome t o respondents than pe r sona l

G S S : T-PIntervw -5-

i n t e r v i e w s . A d m i n i s t r a t i o n of the same i n s t r u m e n t on the te lephone

t a k e s less t i m e t h a n i n person. Th is is t r u e even though temporary

i n t e r r u p t i o n s (e.g. by phone c a l l s , peop le i n household) a r e more

f r e q u e n t i n p e r s o n a l i n t e r v i e w s . The b r i s k e r pace o f t e lephone

i n t e r v i e w s seems t o r e s u l t from 1) less exchange between respondent and

i n t e r v i e w e r and 2 ) b r i e f e r o r less complete answers on t h e te lephone.

Open ended responses a r e l o n g e r on p e r s o n a l i n t e r v i e w s and more e v e n t s

o r b e h a v i o r s tended t o be mentioned. Respondents t end t o rate t h e

t e l e p h o n e i n t e r v i e w e x p e r i e n c e more n e g a t i v e l y (less e n j o y a b l e , less

l i k e l y t o d o a g a i n , more s u s p i c i o u s , e t c . ) t h a n p e r s o n a l i n t e r v i e w s .

T h i s d i f f e r e n c e is e s p e c i a l l y l a r g e when i t comes t o e v a l u a t i n g the

l e n g t h o f t h e i n t e r v i e w . While t h e t e lephone i n t e r v i e w a c t u a l l y r a n 20

minu tes s h 6 r t e r than the p e r s o n a l i n t e r v i e w i n t h e Groves t Kahn s t u d y ,

36 p e r c e n t r a t e d t h e t e l e p h o n e i n t e r v i e w as t o o l o n g w h i l e o n l y 10

p e r c e n t r a t e d the p e r s o n a l i n t e r v i e w as t o o long. T h i s g r e a t e r

r e s p o n d e n t burden i n a major f a c t o r c o n t r i b u t i n g t o much h i g h e r break-

o f f rates on t e l e p h o n e t h a n i n person.

8. The g r e a t e r anomin i ty o f t h e t e l e p h o n e i n t e r v i e w is o f t e n assumed t o

r e d u c e s o c i a l d e s i r a b i l i t y e f f e c t s . Exper imental r e s u l t s are e q u i v o c a l

a t best. S t u d i e s o f race o f i n t e r v i e w / r e s p o n d e n t i n t e r a c t i o n s i n d i c a t e

t h a t t h e e f f e c t is a s l a r g e i n t e l e p h o n e s u r v e y s as i n person. ( T h i s

w a s t r u e b o t h i n s t u d i e s t h a t t r i e d t o make r a c i a l i d e n t i f i c a t i o n c l e a r

t o r e s p o n d e n t s and i n n a t u r a l s i t u a t i o n s . ) S t u d i e s that compared l e v e l s

o f s o c i a l l y d e s i r a b l e r e s p o n s e s a c r o s s mode showed no c l e a r

d i f f e r e n c e s . When l e a n i n g s are c o n s i d e r e d , s i x s t u d i e s r e p o r t e d more

s o c i a l l y d e s i r a b l e r e s p o n s e s i n p e r s o n a l i n t e r v i e w s , two s t u d i e s w i t h no

d i f f e r e n c e s , and f o u r s t u d i e s w i t h more s o c i a l l y d e s i r a b l e r e s p o n s e s t o

t e l e p h o n e surveys . However, when w e look a t o n l y clear and s i g n i f i c a n t

GSS : T-PIntervw - 6-

d i f f e r e n c e s , i n only one survey d id personal in te rv iews ob ta in more

s o c i a l l y d e s i r a b l e responses, i n seven cases t he re were no d i f f e r ences ,

and i n four cases telephone surveys showed more s o c i a l l y d e s i r a b l e

responses. Of some s p e c i a l i n t e r e s t is the f ind ing of Bradburn, Sudman,

e t a l . t h a t a l l modes and methods of asking s e n s i t i v e ques t ions produced - l a r g e b iases .

9. Comparisons ac ros s modes have usua l ly been very l imi t ed , f r equen t ly

r e s t r i c t e d t o ana lyses of a few u n i v a r i a t e d i s t r i b u t i o n s . Comparisons

invo lv ing m u l t i v a r i a t e e f f e c t s , v a l i d i t y , and r e l i a b i l i t y have been very

scarce .

A. When comparisons a r e made between telephone households interviewed

by telephone and i n person, d i s t r i b u t i o n s a r e f r equen t ly

comparable. In eleven s t u d i e s no d i f f e r e n c e s were found, four

s t u d i e s found small and l i m i t e d d i f f e r e n c e s , and s i x s t u d i e s found

l a r g e o r widespread d i f f e rences . There appears t o be a tendency f o r

d i f f e r e n c e s t o be l e s s common on f a c t u a l i tems and demographics and

more f r equen t on a t t i t u d e s .and some types of behavior.

B. I n only s i x s t u d i e s were any more e l a b o r a t e o r complex comparisons

c a r r i e d out . Three s t u d i e s looked a t s c a l e r e l i a b i l i t y o r i n t e r -

i t e m c o r r e l a t i o n s . One s tudy found a c l e a r s u p e r i o r i t y f o r personal

i n t e rv i ews ( l e s s agreement b i a s , evasiveness , and extremeness) while

t he o t h e r found no d i f f e r e n c e i n t he r e l i a b i l i t y c o e f f i c i e n t o r

extremeness. In a t h i r d s tudy the in t e r - i t em c o r r e l a t i o n s on a

number of s a t i s f a c t i o n i tems were judged comparable ac ros s modes.

Two s t u d i e s compared t e s t / r e t e s t c o e f f i c i e n t s . I n one very small

s tudy , educat ion responses were more c o n s i s t e n t on te lephone

r e in t e rv i ews than on pe r sona l r e in t e rv i ews ( i n a l l c a s e s the i n i t i a l

i n t e rv i ew w a s i n person) . The d i f f e r e n c e s do no t appear

s i g n i f i c a n t , however. In the second s tudy r e l i a b i l i t y coe f f i c i en t s

were h i g h e s t i n t h e personal-personal mode, (.83, .88) intermediate

on the telephone-telephone mode (.82, .86) and lowest on switched

modes ( -71 , .75, .75, ,811. The d i f f e r e n c e s between the two

c o n s i s t e n t modes were not s i g n i f i c a n t . F i n a l l y , one s tudy found

t h a t te lephone in t e rv i ews underrepresented the less educated and

t h a t t h i s b i a s i nc reased s i g n i f i c a n t l y among the e l d e r l y . These s i x

s t u d i e s u s ing t h r e e s c a l e s and f i v e o t h e r v a r i a b l e s a r e a very

sparce b a s i s f o r g e n e r a l i z i n g about r e l i a b i l i t y and mul t iva r i a t e

d i f f e r e n c e s ac ros s modes. Four show no s i g n i f i c a n t d i f f e r ences , bu t

two f i n d b e t t e r performance from personal in te rv iews .

C. V a l i d i t y checks were c a r r i e d o u t i n nine in s t ances . Personal

surveys had more a c c u r a t e responses i n t h r e e cases , i n f i v e cases no b

d i f f e r e n c e s were de t ec t ed , and i n one case te lephone d id b e t t e r . On

four s t u d i e s t h a t assumed h i g h e r . v a l i d i t y based on t h e o r e t i c a l

a n t i c i p a t i o n of under o r over repor t ing , more complete r epo r t s were

found on te lephone i n two cases and i n two c a s e s f o r personal .

In very genera l terms the i n v e s t i g a t o r s ' summary conclus ions on da t a

q u a l i t y can be c l a s s i f i e d a s fol lows:

personal s u p e r i o r t o te lephone 8

Personal equal t o o r s u p e r i o r t o te lephone 3

No important d i f f e r e n c e s 12

Telephone equa l t o o r s u p e r i o r t o personal 2

Telephone s u p e r i o r t o personal 2

Telephone and pe r sona l should be combined (mixed mode)

Both. should be improved

The tilt of t he i n d i v i d u a l r e sea rch f ind ings towards b e t t e r q u a l i t y da ta

from personal i n t e rv i ews is shared by our summary of d i f f e r ences . on

the 25 s tandards l i s t e d i n Table 1 , we f i nd personal supe r io r i n

t h i r t e e n cases , te lephone i n s i x cases , i n s u f f i c i e n t d a t a i n four cases ,

and no d i f f e r e n c e i n two ins tances . Among the n ine teen a reas on which

d i f f e r e n c e s were de tec t ed , twelve are considered of importance t o the

GSS. On t he se twelve, personal in te rv iews had the advantage i n nine

c a s e s while te lephone in te rv iews lead i n t h ree a r eas .

10. Telephone surveys (and e s p e c i a l l y CAT1 surveys) can be done more quick ly

than personal in te rv iews . While personal i n t e rv i ews have been designed,

f i e l d e d , and completed wi th in a week, t h i s r e p r e s e n t s a ex t r ao rd ina ry

and usua l ly e s p e c i a l l y c o s t l y e f f o r t . Given a n t i c i p a t e d ' l e v e l s of

phys i ca l equipment and personnel , it would probably be poss ib l e t o

reduce the design t o d a t a process ing s t a g e from 6 months t o four - f ive

months. Advantages to the GSS of t h i s reduct ion , o t h e r than those t h a t

a r e p a r t of the c o s t sav ings l i s t e d below, a r e minimal however.

A l l s t u d i e s have shown telephone surveys t o be s i g n i f i c a n t l y l e s s

expensive than comparable personal interviews. Cost advantages

d iminish and/or d i sappea r , however, when dual frames a r e adopted

and i n des igns t h a t use some type of personal fo l low up (e.g.

matching h o s p i t a l r eco rds ) on s c a t t e r e d te lephone samples (e.g.

n a t i o n a l RDD). NORC e s t i m a t e s t h a t the t o t a l r educ t ion i n da t a

c o l l e c t i o n and process ing c o s t ( i . e . from ques t ionna i r e

cons t ruc t ion t o product ion of a c lean , raw d a t a t a p e ) would be 25

percent .

G S S : T-PIntervw -9-

What should be the course of a c t i o n f o r 19851 On one hand we would expect

t h a t wi th adjustments f o r non-telephone households most i tems would not

s i g n i f i c a n t l y m r y by mode. On the o t h e r hand, i n genera l the q u a l i t y of

personal in te rv iews seems h igher than te lephone in te rv iews and comparative

ana lyses of m u l t i v a r i a t e d i f f e r e n c e s and comparative r e l i a b i l i t y and v a l i d i t y

a r e too sparce f o r f i rm conclusions. Therefore, an experimental comparison

should focus on t h e s e underexamined a reas .

To adequate ly a s s e s s the comparative r e l i a b i l i t y of personal and

telephone in t e rv i ewing a t e s t / r e t e s t des ign should be adopted. Following our

exper ience wi th e a r l i e r t e s t / r e t e s t des igns , w e sugges t a s i n g l e r e in t e rv i ew

about four t o s i x weeks a f t e r the i n i t i a l interview. This r e t e s t should

involve both same mode and c r o s s mode admin i s t r a t i ons . The fol lowing t a b l e

i l l u s t r a t e s t h i s design:

Reinterview

None Telephone Personal

I n i t i a l

In te rv iew

Telephone

Personal

500 500 500 1500

This des ign would maximize our a b i l i t y t o s tudy the almost t o t a l l y

ignored a r e a of comparative r e l i a b i l i t i e s , b u t would be c o s t l y and n e c e s s i t a t e

a d d i t i o n a l resources. On the r e t e s t on ly a subsample of ques t ions would be

readministered. These would be chosen t o r ep re sen t s i x c l a s s e s of items:

1 ) unchanging demographics, c u r r e n t s t a t u s , 2 ) unchanging demographics, p a s t

s t a t u s , 3 ) changeable demographics, 4 ) s t eady a t t i t u d i n a l items, 5) one o r two

a t t i t u d i n a l s c a l e s , and 6) ep i sod ic and mood items. In a d d i t i o n , t o check the

q u a l i t y and r e l i a b i l i t y of the data, items should be added including 1 ) s e l f -

va l ida t ing f a c t u a l items (e. g., Who is the Secre tary of S ta te? ), 2 ) i terns with

a high degree of both p o s i t i v e and negative s o c i a l d e s i r a b i l i t y (e.g.,

r eg i s t e red t o vote/have l i b r a r y card vs. d r iv ing a f t e r dr inking/ la te paying

b i l l s ) . Serious cons idera t ion should be given t o including record checks t o

va l ida te r epor t s on voting, e t c . In addi t ion , w e should r e p l i c a t e the

f indings t h a t telephone interviews produce shor te r and less complete answers

t o open-ended ques t ions . Since both the personal reinterview of i n i t i a l telephone respondents

and any record checks necess i t a t e having an interviewer i n the v i c i n i t y and

would be e s p e c i a l l y expensive i f sca t t e red , it would probably be bes t t o

c l u s t e r telephone interviews i n the same PSUs a s the personal interviews.

Also, both &e r o t a t i o n of GSS items (which means t h a t roughly a t h i r d

of our a t t i t u d e i tems w i l l not appear on the 1985 survey) and our experience

with our f u l l probabi l i ty /b lock quota experiments i n 1975 and 1976 argue t h a t

the mode comparisons would have t o be rep l i ca ted i n 1986.

There is, however, a se r ious quest ion whether t h e GSS could be

administered on the telephone. Due t o respondent burden, success can not be

predic ted f o r a telephone survey the length of the GSS (approximately 75

1 minutes ) . ....................

'Examples of the adminis t ra t ion of general top ic surveys t o a na t ional o r heterogeneous populat ion l a s t i n g over an hour a r e extremely rare. Most telephone surveys a r e very shor t . Observe the following repor t s on length of telephone surveys :

Source Surveys Studied Length

1. Wiseman & McDonald, 1979 Market research Less than 10 minutes-40.3% g r e a t e r than 1 0 mins. 49.7%

2. Dillman, 1978 Mostly various popu- l a t i o n s , Washington

GSS: T-PIntervw

Our b e s t e s t ima te is t h a t a reasonable expec ta t ion of success could be

expected only i f t h e survey schedule was reduced by 50-60 pe rcen t (and a few

ques t ions dropped o r s u b s t a n t i a l l y reformatted t o make them workable on the

te lephone 1.

There a r e c e r t a i n r a d i c a l changes i n the GSS des ign t h a t could

accommodate such a d r a s t i c reduct ion i n the survey instrument . These would

inc lude 1 ) continuous c ros s - sec t ions ( a l a CNS), 2 ) multi-wave panels , 3) and

t h e sampling of i tems ac ros s t i m e . While these designs each have c e r t a i n

a t t r a c t i o n s , it is d i s q u i e t i n g t o s e e them considered p r i m a r i l y t o c a t e r t o

l i m i t a t i o n s of te lephone interviewing. In' add i t i on , it is doub t fu l t h a t any

of t hese des igns would b e t t e r monitor h i s t o r i c a l developments and he lp expla in

generat ion-to-generat ion s o c i a l change than the annual c ros s - sec t iona l des ign

of t he c u r r e n t GSS.

Thus, i n a t tempt ing t o implement an experimental comparison between

t h e te lephone and personal i n t e rv i ewing on the 1985 GSS w e are l e f t with t h ree

choices 1 ) a t t empt a GSS in t e rv i ew on the telephone d e s p i t e concerns about its

p r a c t i c a l i t y , 2) r a d i c a l l y r edes ign t h e GSS t o make it f u n c t i o n a l wi th in

te lephone l i m i t a t i o n s , and 3) contaminate t h e te lephone/personal comparison on

the experiment by admin i s t e r ing a s h o r t form on the te lephone ( a s i n the 1982

S t a t e 0-5 mins -1 4 5-1 0 -1 2 11-1 5 - 1 20-30 - 4

3. Survey Research, 1973 Academic research Longest eve r done: 5-60, average of longest : 26 mins.

S i m i l a r l y , i n the twelve exper imenta l s t u d i e s , which mentioned l eng th of interview, covered he re i n only one case d i d t h e t i m e exceed 50 minutes on t h e telephone. I n a d d i t i o n , most p r a c t i o n e r s a r e wary about long telephone in te rv iews . Survey Research, i n its p o l l of 14 academic survey organiza t ion , found only one prac t ioner who thought t h a t a survey over 45 minutes was poss ib le . ( H e a c t u a l l y s a i d the re was n o t l i m i t i f t he survey was i n t e r e s t i n g ) . Many au tho r s i n t h e experimental s t u d i e s a l s o expressed concern about long telephone interviews.

GSS : T-PIntervw -1 2-

ANES experiment). (This l a s t choice presumes t h a t a f u l l GSS is not p r a c t i c a l

on the telephone and i n d i r e c t l y argues f o r u l t imate adoption of the second

option. 1

why ca r ry out a telephone/personal experiment? Cer ta in ly there a re

many methodological quest ions about personal/telephone interviewing t h a t the

design w e proposed would g r e a t l y c l a r i f y . Y e t it is doubtful t h a t such an

expensive and extens ive methodological experiment should be c a r r i e d out

without the se r ious in ten t ion of moving the GSS t o the telephone mode. Since

the re a r e no conceptual o r t h e o r e t i c a l reasons f o r t h i s s h i f t , it a l l comes

down t o money. Our best es t imate is t h a t a f t e r our two experimental years

(which would n e c e s s i t a t e an increase i n our budget of some unknown amount),

the telephone survey would save us 25 percent i n c o l l e c t i o n and processing

cos ts . his would mean t h a t the t o l d p r o j e c t savings ( inc luding codebook

prepara t ion , ana lys i s , user support, e t c . ) would be on the order of 18

percent. I f the GSS survey could be c a r r i e d out i n t a c t with a s high a degree.

of q u a l i t y i n the telephone mode a s i n the personal mode, we would be foo l i sh

t o ignore these savings. Unfortunately the f e a s i b i l i t y of doing the necessary

experimental comparison t o determine the r e l a t i v e r e l i a b i l i t y , v a l i d i t y , and

. r e p l i c a b i l i t y of the two modes is quest ionable. .

GSS: T-PIntervw -1 3-

TABLE 1

Comparison of Telephone and Personal In te rv iews

Magnitude of Important

Topic Advantage Advantage t o GSS? Comments

.I. SAMPLING

A. Complete Coverage P Strong Yes 7-9% of households without phones undercoverage much high i n some subpopulation

B. S e l e c t i n g Respondent N -- D i f f e r e n t techniques a r e used, b u t R can be s e l e c t e d

C. Nonresponse P S t rong Y e s

D. Design E f f e c t

11 INTERV IEWING

S t rong Y e s

A. In te rv iewer S t anda rd iza t ion T S t rong Yes

B. In te rv iewer x Respondent I n t e r - a c t i o n s U - - Race of in te rv iewer /

respondent i n t e r a c t i o n s happen on both T and P , b u t race matching not a t tempted on T r e g i o n a l / l i n g u i s t i c matches d i f f i c u l t on T

C. Respondent Feedback/ c l a r i f i c a t i o n s P

D. Probing P

E. Complex Ques t i o n s P

1. V i s u a l a i d s P

Yes

Y e s

Yes

Strong

S l i g h t

S t rong

St rong Yes I n some, bu t not a l l i n s t ances , d i f f e r e n t T techniques can handle wi thout v i sua l a id s

2. Complicated f i l t e r s Strong N o T advantage i f CAT1 only;

given c u r r e n t GSS con ten t manual f i l t e r i n g s works w e l l

3. Open ended P S t rong

4. L i s t s of events , exper iences P

5. Poss ib l e con- s u l t i n g of r eco rds , family P

F. Respondent Pr ivacy T

S t rong S l i g h t

No

S l i g h t

S t rong

S l i g h t

G S S : T - P I n t e r v w -1 4-

TABLE 1

(Continued) - -

Magnitude of Important

Topic Advantage Advantage t o GSS? Comments

111 . RESPONSES

A. General (non th rea t ing ) N - - Dis t r ibu t ions usua l ly not s i g n i f i c a n t l y d i f f e r e n t

B. Soc ia l D e s i r a b i l i t y U/N -- - - C. Va l ida t ion P S l i g h t S l i g h t

D. R e l i a b i l i t y U -- -- V i r t u a l l y unexamined

E. I t e m Nonrespbnse

1. Income Refusa ls

IV. OTHER DESIGN ISSUES

A. Quest ionnaire Length

B. Respondent ~ u r h e n

V. SPEED

V I . COST

U/N - - Unsure

P Strong Yes

P Strong Yes

P Strong Yes

T Strong No

T . Strong Yes

T g e t s s l i g h t l y higher DKs, d i f f i c u l t t o judge a s p o s i t i v e o r nega t ive development

T in te rv iew runs f a s t e r bu t lower acceptance of long in te rv iews

R p r e f e r s P t o T and r a t e s T a s too long and more of burden'

Dual frame designs o f ten negate T savings

- - - - -- -- - - -

P = Personal N = No d i f f e r e n c e

T = Telephone U = Unknown

TELEPHONE BIBLIOGRAPHY

Anderson, C.-Leroy, and Halford, Larry J., "A Four-State Comparisun of V a r i a b l e Sampling and Data C o l l e c t i o n Procedures," P a c i f i c Soc io logica l Review, 13 (Summer, 19701, 149-1 55.

Aneshensel, Carol S.; F re r i chs , Ralph R e , Clark, V i r g i n i a A * , and Yokopenic, P a t r i c i a A * , "Measuring Depression i n t he Community: A Comparison of Telephone and Personal In te rv iews , ' POQ, 46 (Spring, 19821, 110-121. -

Assael, Henry, "Comparison of Brand Share Data by.Three ~ e p o r t i n g Systems," J o u r n a l of Marketing Research, 4 (November, 19671, 400-401.

B i e m e r , Paul P:, "Optimal Dual Frame Design: Resu l t s of a Simulat ion Study," Paper presen ted t o t he American S t a t i s t i c a l Assoc ia t ion , Toronto, August, 1983.

Blankenship, A. B., P ro fe s s iona l Telephone Surveys. New York: McGraw-Hill, 1977.

Bradburn, Norman M.; Sudman, Seymour; and Assoc ia tes , Improving In te rv iew Method and Quest ionnaire Design: Response E f f ec t s to Threa ten ing Questions i n Survey Research. San Francisco: Jossey-Bass Pub l i she r s , 1979.

Bushery, John M.; Cowan, Char les D.; and Murphy, Linda R., "Experiments i n Telephone-Personal V i s i t Surveys," Paper presen ted t o the American S t a t i s t i c a l Assoc ia t ion , San Diego, August, 1978.

~ a h a l a n , Don, "Measuring Newspaper Readership by Telephone: Two Comparisons wi th Face-to-Face Interview's, " Journa l of Adver t i s ing Research, 1 - (19601, 1-6.

Cannel l , Char les F.; Groves, Robert M.; Magilavy, Lou J.; Mathiowetz, Nancy; and Miller, P e t e r V., "An Experimental Comparison of Telephone and Persona l Health Surveys,' Unpublished paper, Survey Research Center , Un ive r s i t y of Michigan, Apr i l , 1983.

Cannel l , Char les F., and Fowler, Floyd J., " In te rv iewers and In te rv iewing Techniques," i n Advances i n Heal th Survey Research Methods: Proceedings of a Nat iona l I n v i t a t i o n a l Conference, NCHSR Research Proceedings S e r i e s , ' 1975.

C a t l i n , Gary and Murray, Susan, "Canadian Vic t imiza t ion Surveys: A Report on P r e t e s t s i n Edmonton and Hamilton, " Survey Methodology, 5 (December, 1979).

Colombotos, John, "The E f f e c t s of Persona l vs. Telephone In te rv iews on S o c i a l l y Acceptable Responses," POQ, 19 ( F a l l , 19651, 457-458.

Colombotos, John, "Personal versus Telephone Interviews: E f f e c t on Responses," Pub l i c Health Reports , 84 (September, 19691, 773-782.

Coombs, Lolagene and Freedman, Ronald, "Use of Telephone Interviews i n a Lonqitudinal F e r t i l i t y Study," 28 (Sprinq, 1964), 112-117.

Co t t e r , Pa t r i ck R.; Cohen, Je f f rey ; and Coulter , P h i l i p B., "Race-of- Interviewer Effec ts i n Telephone Interviews , " POQ, 46 (Summer, 1982), - 278-2040

Czaja, Ronald; Bla i r , Johnny; and Sebest ik, Ju t t a P., "Respondent Select ion i n a Telephone Survey: A Comparison of Three Techniques," Journal of Marketinq Research, 19 (August, 19821, 381 -385.

Dillman, Don A., Mail and Telephone Surveys: The Tota l Desiqn Method. New York: John Wiley & Sons, 1978.

Frankel, Martin R., and Frankel, Les ter R., "Some Recent Developments i n Sample Survey Desiqn," Journal of Marketinq Research, 14 (Auqust, 19771, 280-293

Frey, James H., Survey Research by Telephone. Saqe Library of Socia l Research, Vol. 1 50. Beverly H i l l s : Sage ~ u b l i c a t i o n s , 1983.

Friedman, Linda, and Friedman, Hershey H., "Does the Preceived Race of a Telephone Interviewer Affect the Responses of W h i t e Subjec ts t o an At t i tudes toward Neqroes Scale?," Paper presented t o the American

. S t a t i s t i c a l Association, San Dieqo, August, 1978.

Groves, Robert M., "Actors and Questions i n Telephone and personal ~ n t e r v i e w Surveys," POQ, 43 (Summer, 19791, 190-205. -

Groves, Robert M., "Comparinq Telephone and Personal Interview Surveys," Economic Outlook USA, Summer, 1978, 49-51. -

Groves, Robert M., "An Empirical Comparison of Two Telephone Sample Designs," Journal of Marketing Research, 15 (November, 19781, 622-631.

Groves, Robert M., "An Experimental Comparison of National Telephone and Personal Interview Surveys," Paper presented t o the American S t a t i s t i c a l Association, 1977.

Groves, Robert M., "Implications of CATI: Costs, Errors , and Orqanization of Telephone Survey Research," Socioloqica l Methods and Research, 1 2 (November, 19831, 199-21 5.

Groves, Robert M., "A Researcher's V i e w of the SRC Computer-Based Interviewinq System: Measurement of Some Sources of Error i n Telephone Survey Data," ???

Groves, Robert M.; Berry, Marianne; and Mathiowetz, Nancy, "Some Impacts of Computer Assisted Telephone Interviewing on Survey Methods," Paper presented t o the American S t a t i s t i c a l Association, Auqust, 1980.

Groves, Robert M., and Kahn, Robert L., Surveys by Telephone: A National Com~arison with Personal Interviews. New York: Academic Press, 1979.

Groves, Robert Y., and Lepkowski, James M., "Dual Frame, Mixed Mode Survey Desiqns," Unpublished paper, Survey Research C e n t e r , University of Michigan.

Groves, Robert M., and Magilavy, Lou J., "Estimates of Interviewer Variance in Telephone Surveys," Paper presented t o the American S t a t i s t i c a l Association, 1980.

Groves, Robert M., and Magilavy, Lou J., "Increasing Response Rates to Telephone Surveys: A Door i n the Face f o r Foot-in-the-Door?, " POQ, 45 - ( F a l l , 19811, 346-358.

Groves, Robert M., and Magilavy, Lou J., "Interviewer Ef fec t s i n Telephone Survey Data," Unpublished paper, Survey Research Center, University of Michiqan.

Groves, Robert M., Yaqilavy, Lou J., and Bia lecki , Anthony, "HIS/SRC Telephone Experiment: Report on Samplinq Ac t iv i t i e s , " unpublished paper, Survey Research Center, Universi ty of Michigan, 1980.

Groves, Robert Y., Magilavy, Lou J., and Mathiowetz, Nancy, "Comparison of Results between Cornouter Assisted Telephone Interviewing (CAT11 and Paper and Penci l Quest ionnaires," Report t o the National Center f o r Health S t a t i s t i c s , September, 1981.

Groves, Robert Y., Maqilavy, Lou J., and Mathiowetz, Nancy A., "The Process of Interviewer Var iab i l i ty : Evidence from Telephone Surveys," Paper presented to the American S t a t i s t i c a l Association, 1981 .

Groves, Robert M., and Mathiowetz, Nancy A., "Computer ~ s s i s t e d Telephone Interviewinq: Ef fec t s on Interviewers and Respondents," Unpublished paper, Survey Research Center, University of Michigan.

Groves, Robert M.; Miller, Pe te r V.; and Cannell, Charles F., "A Methodolo- q i c a l Study of Telephone and Face-to Face Interviewinq," Paper presented to the American Associat ion f o r Public Opinion Research, Buck H i l l Fa l l s , Pa, 1981.

Groves, Robert M.; Miller, Pe te r V.; and Handlin, V e l m a J., "Telephone Survey Methodoloqy: A Review, " Unpublished paper, Survey Research C e n t e r , Universi ty of Michiqan, 1982.

Hauck, Mathew and Goldberq, J e f f r e y , "Telephone Interviewinq on the NLRB Elect ion Study," Survey Research, 5 (Jan., 19731, 11-16.

Henson, Ramon; Qoth, Aleda; and Cannell, Charles F., "Personal versus Telephone Interviews and the Ef fec t s of Telephone Reinterviews on Reporting of Psych ia t r i c Symptomatolog~," Survey Research Center, Universi ty of Michiqan, October, 1 974.

Herman, Jean Bre t t , "Mixed-Mode Data Collect ion: Telephone and Personal Interviewinq," Journal of Applied Psycholoqy, 6 2 (19771, 399-404.

n e r d ~ q , 3 . K e q u ~ d ; fi.oaonger3, n l ~ L a r 2 L* ; dnU kCLIICa, K l C f l a r n A., i f iZ?. ivle iJ lnq Older ~ d u l t s : A Comparison of Telephone and Face-to-Face Modalities," POQ, 47 ( F a l l , 19831, 405-418. -

Hochstim, Joseph R., "A Cri t ical Comparison of Three S t r a t e q i e s of Collect ing Data from ~ouseho lds , " Journal of the American S t a t i s t i c a l Association, LXII (Sept., 19671, 976-989.

Ibsen, Charles A., and Ballweg, John A., "Telephone ~ n t e r v i e w s i n Social R$search: Some Methodoloqical Considerations," Quality and Quantity, 8 (dune, 19741, 181-192.

"Interviewing i n Telephone Surveys," Survey Research, 5 (Jan., 1973), 9-13.

Janofsky, A. I rene, "Affective Self-Disclosure i n Telephone versus Face-to- Face Interviews, " Journal of Humanistic Psycholoqy, 11 (Spring, 1971 ) , 93-1 03.

Jordan, Lawrence A*; Yarcus, Alfred C.; and Reeder, Leo G., "Response Styles i n Telephone and Household Interviewing: A F ie ld Experiment," POQ, 44 - (Summer, 19801, 210-222.

Yegeles, S. Stephen; Fink, Clinton F.; and Kirscht , John P., "Interviewing a National Sample by Long-Distance Telephone, POQ, 33 ( F a l l , 19691, 41 2- - 41 9.

Klecka, William R., and n tchfarber , Alfred J., "Random D i q i t Dialing: A Comparison to Personal Surveys," POQ, 42 (Spring, 19781, 105-114. -

Larsen, Otto N., "The Comparative Va l id i ty of Telephone and Face-to-Face Interviews i n the Measurement of Messaqe Diffusion from Leaf le ts ," American Socioloqica l Review, 17 (Auqust, 1952), 471-476.

Locander, William B., and Burton, John P., "The Ef fec t of Question Form on Gatherinq Income Data by Telephone," Journal of Marketing Research, 13 (May, 19761, 189-1 92.

Locander, W i l l i a m ; Sudman, Seymourt and Bradburn, Norman, "An Inves t iga t ion of Interview Method, Threat and Response Dis to r t ion , " Journal of the American S t a t i s t i c a l Association, 71 (June, 19761, 269-275.

Lucas, William I., and Adams, W i l l i a m C., An Assessment of Telephone Survey Methods. Santa Monica. Cal.: Rand C o r ~ o r a t i o n , 1977.

Mathiowetz, Nanw A., and Cannell, Charles F., "Codinq Interviewer Behavior as a Method of Evaluating Performance," Paper presented t o the American S t a t i s t i c a l Association, 1980.

Monteiro, Lois A., Moni to r inq Health S ta tus and Medical Care. Cambridge, Mass.: Bal l inqer , 1976.

Mulry-Liggan, Mary Ha, "A Comparison of a Random D i g i t Dialing'Survey and the Current Population Survey," Paper presented t o t h e American S t a t i s t i c a l Association, Toronto, Auqust, 1984.

Nealon, Jack, "The Ef fec t s of Male vs . Female Telephone In te rv iewers , S t a t i s t i c a l Reportinq Serv ice of t he U.S. Department of Aqr icu l ture , S t a f f - Report No. AGE83061 7.

Oakes, Ralph H., "Differences i n Responsiveness i n Telephone versus Personal interview^,^^ Journa l of Marketinq, 19 (1954), 169.

O ' N e i l , Michael J., and Groves, Robert M., "Telephone ~ n t e r v i e w In t roduct ions and Refusal Rates: Experiments i n Increas inq Respondent C o o ~ e r a t i o n , ~ ~ Survey Research Center , Un ive r s i t y of Michigan, Auqust, 1979.

OIRourke, Diane and B l a i r , Johnny, "Improving Random Respondent Se l ec t ion i n Telephone Surveys," Jou rna l of Marketinq Research, 20 (November, 1983), 428-4320

Quinn, Robert P.; Gutek, Barbara A. ; and Walsh, J e f f r e y T., "Telephone Interviewing: A Reappra isa l and a F i e l d ~ x p e r i m e n t , " Basic and Applied S o c i a l Psycholoqy, 1 (1980), 127-153.

Rogers, Theresa F., "Interviews by Telephone and i n Person: Qual i ty of Responses and F i e l d Performance,' POQ, 40 (Sprinq, 19761, 51-65. -

Roshwalb, I rv ing ; Spector , Leonard; and Madansky, Alber t , "New Methods of Telephone In te rv iewing A & S/CATI, " ???

Salmon, Cha r l e s T., and Nichols, John Sp ice r , "The ~ e x t - B i r t h d a y Method of Respondent Se lec t ion , ' POQ, 47 (Summer, 1983), 270-276. -

San Auqustine, Andre J., and Friedman, Hershey Y., "The Use of t h e Telephone In te rv iew i n Obtaininq Information of a S e n s i t i v e Nature: A Comparative Study," Paper presented t o t h e American S t a t i s t i c a l Associat ion, San Diego, Auqust, 1978.

Schiedskamp, Jay W., "Reinterviews by Telephone," Jou rna l of Marketinq, 26 (Jan., 19621, 28-34.

Shanks, J. Merrill; Sanchez, Maria; and Morton, Betsey, "Al t e rna t ive Approaches t o Survey Data Co l l ec t ion f o r t h e Nat iona l E lec t ion S tud ie s , " Report t o NES Board of Overseers, March, 1983.

Shee ts , Thomas; Radl insk i , Allen; Kohne, James; and Brunner, Allen, "Deceived Respondents: Once B i t t en , m i c e Shy," POQ,, 38 (Summer, 1974), 261-263. -

Shih, Wen-Fu P., 'Nonresponse to Income Questions i n Telephone Surveys, "Paper presented t o the American S t a t i s t i c a l Associat ion, Toronto, August, 1984.

Siemiatycki , Jack, "A Comparison of Mail, Telephone, and Home In terv iew S t r a t e q i e s f o r Household Heal th Surveys, American Jou rna l of Pub l i c Heal th, 69 (March, 19791, 238-245.

Sykes, Wendy and Hoinvi l l e , Gerald, " ~ e t h o d o l o q i c a l Research on Telephone . In te rv iews ," Paper presented t o t h e Market Research Socie ty , March,

1984.

mornbe r ry , Owen T., "An valuation of Three S t r a t e q i e s f o r t h e Col lec t ion of Data from Households," Unpublished d i s s e r t a t i o n , Brown Universi ty , 1976.

Thornberry,. Owen T., and Massey, James T O , "Correct ing f o r Undercoveraqe Bias i n Random D i q i t Dialed Nat iona l Health Surveys," Paper presented t o the American S t a t i s t i c a l Associat ion, 1978.

mornbe r ry , Owen T., and Massey, James T., "Coverage and Response i n Random D i g i t Dialed National Surveys," Paper presented t o t he American S t a t i s t i c a l ~ s s o c i a t i o n , 1983.

~ u l l , Donald S., and Albaum, Gerald S., " B i a s i n Random ~ i q i t Dialed Surveys," POQ, 41 ( F a l l , 1977), 389-395. -

White, Andrew A., "Response Rate Ca lcu la t ion i n RDb ~ e l e p h o n e Health Surveys: Current P rac t i ce s , " Paper presented t o t he American S t a t i s t i c a l Associat ion, 1983.

W i l l i a m s , Fderyn, "Medium of Messaqe: Communications ~ e d i u m as a Determinant of I n t e r p e r s o n a l Evaluat ion," Sociometry, 38 (1 9751, 11 9-1 30.

Wiseman, Freder ick , "Methodoloqical B i a s i n Publ ic Opinion Surveys," POQ 36 (Spring, 1 972 1. 1 05-1 05.

Wiseman, Freder ick , and McDonald, P h i l i p , "Noncontact and Refusa l Rates i n Consumer Telephone Surveys, " ~ o u r n a l of Marketinq ~ e s e a r c h , 1 6 (November, 19791, 478-484.

Wolf le, Lee M., " C h a r a c t e r i s t i c s of Persons With and Without Home '~e l ephones , Journa l of Marketinq Research, 16 (August, 19791, 421-4250

Yaffe, Qichard; Shapiro, Sam; Fwhsberg, Robert R.; Rohde, Char les A.; and Corpeno, Helen C., "Medical Economics Survey-Methods Study: Cost- E f fec t iveness of A l t e rna t ive Survey s t r a t e g i e s , " Medical Care, 16 (Auqust, 19781, 641 -659.