measurement quality in social networks - pàgines de la .1 universitat de girona measurement quality

Download Measurement Quality in Social Networks - Pàgines de la .1 Universitat de Girona Measurement Quality

Post on 07-Jun-2019

212 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

1

Universitat de Girona

Measurement Quality in Social Networks

LLus Coromina

Quantitative Methods, University of Girona Department of Economics,

Campus Montilivi, 17071 Girona, Spain E-mail: lluis.coromina@udg.edu

2Universitat de Girona

Outline

Studies on Social Network Data Quality

Survey Design o Data collection o Questionnaire Design

Types of hypothesis tested in other Quality studies

Quality Measurement (Validity and Reliability)

Measurement procedure o Multitrait-Multimethod Model (MTMM) o Multilevel MTMM o Interpretation

3Universitat de Girona

Outline

INSOC PROJECT: o Social Network Data collection Design o Multitrait-Multimethod Model results o Meta-analysis Design o Meta-analysis Results

Results and comparison from others studies.

Intro

4

Objective

Universitat de Girona

4

Main purpose of scientific research is the interpretation and/or prediction

of phenomena

Quality measurement instruments (empirical data) is of crucial importance

It is important to know how well we can measure with certain

measurement instruments and how much a researcher can rely on findings

based on data collected.

Intro

5

Objective

Universitat de Girona

5

Egocentered network data are especially important.

Network characteristics (network size, structure and composition)

Characteristics of network members (feeling of closeness or importance;

frequency of advice, collaboration, social support) collected from the

respondent (ego).

Intro

6Universitat de Girona

Quality in Social Network Survey Data

6

Studies on Social Network Data Quality

Killworth & Bernard, 1976, 1980; Bernard et al., 1979,1980, 1982;

Hammer, 1984; Sudman, 1985, 1988; Freeman and Romney, 1987;

Freeman, Romney & Freeman, 1987; Corman and Bradford, 1993;

Hlebec, 1993; Brewer and Webster, 1997,

Intro

7Universitat de Girona

Quality in Social Network Survey Data

7

People are generally very inaccurate in reporting on their past

interactions with other people.

People remember long-term or typical patterns of interaction with other

people rather well.

Accuracy of reporting about interactions is also influenced by the

frequency of interaction (more frequent contact with group members,

more accurate reports) and by the reliability of an individual

respondent on about actual interactions (Romney and Faust, 1982; Romney

and Weller, 1984).

Intro

8Universitat de Girona

Quality in Social Network Survey Data

8

Quality of measurement, specially reliability, can be affected by the type of

social support (Ferligoj & Hlebec, 1999).

+ Emotional and informational support dimensions

Reliability

- Social companionship and material support dimensions

Reasons for higher quality

More intimate types of support more important to the respondents.

Relatively small number of very

important people.

Less effortful for the respondent. Network

members are important and close

Intro

9Universitat de Girona

Data Collection Methods

9

DATA COLLECTION METHODS

In Social Sciences, the most frequent measurement instrument is a SURVEY

Intro

10Universitat de Girona

Data Collection Methods

10

Intro

11Universitat de Girona

Data Collection Methods

11

With interviewer Self-administration

Paper Personal (Face-to-face) telephone

paper self-administered

mail Electronic computer assisted personal

interview (CAPI)

computer assisted telephone interview (CATI)

E-mail Web surveys Computer Assisted Self-

Interview (CASI)

ACASI Mobile phone surveys

NO ONE COLLECTION METHOD IS BEST FOR ALL CIRCUMSTANCES.

THE CHOICE MUST BE MADE WITHIN THE CONTEXT OF THE PARTICULAR

OBJECTIVES OF THE SURVEY AND THE RESOURCES AVAILABLE.

Intro

12Universitat de Girona

Data Collection Methods

12

Characteristics of the Population

+

Characteristics of The Sample

+

Types of Questions

+

Question Topic

+

Response Rate

+

Cost $$ +

Time

Intro

13Universitat de Girona

Data Collection Methods

13

PERSONAL INTERVIEWING

ADVANTAGES: DISADVANTAGES:

Generally: highest cooperation and lowest refusal rates

Most costly mode of administration

Allows for longer, more complex interviews Longer data collection period High response quality Interviewer concerns Takes advantage of interviewer presence

Multi-method data collection

TELEPHONE INTERVIEWING

ADVANTAGES: DISADVANTAGES:

Less expensive than personal interviews Samples of general population Non-response Shorter data collection period than personal Questionnaire constraints Interviewer administration (vs. mail) Difficult to administer questionnaires on

sensitive or complex topics

Better control and supervision of interviewers (vs. personal)

Biased against households without telephones, unlisted numbers

Better response rate than mail for list samples

Intro

14Universitat de Girona

Data Collection Methods

14

MAIL SURVEYS

ADVANTAGES: DISADVANTAGES:

Generally lowest cost Most difficult to obtain cooperation Can be administered by smaller team of

people (no field staff)

No interviewer involved in collection of data

Access to otherwise difficult to locate, busy populations

More likely to need an incentive for respondents

Respondents can look up information or consult with others

Need good sample

Slower data collection period than telephone

Intro

15Universitat de Girona

Data Collection Methods

15

WEB SURVEYS

ADVANTAGES: DISADVANTAGES

Lower cost (no paper, postage, mailing, data entry costs)

% of homes own a computer; % have home e-mail.

Can reach international populations Representative samples difficult - cannot generate random samples of general

population

Time required for implementation reduced Differences in capabilities of people's computers and software for accessing

Web surveys

Complex skip patterns can be programmed Different ISPs/line speeds limits extent of graphics that can be used

Sample size can be greater

Intro

16Universitat de Girona

Data Collection Methods

16

MEASUREMENT QUALITY IMPLICATIONS OF MODE SELECTION

Social Desirability

Under-represent (illegalities) or Over-represent (voting) different population groups.

Sensitive information from respondents

Self-administered methods produce fewer social desirability effects.

Response effects (wording, answer categories order, or questions order)

Response order (Primacy-visually- and recency telephone)

Data completeness (with interviewers more questions answered)

Completion times

Intro

17Universitat de Girona

Data Collection Methods

17

Mean completion times(in seconds)

by mode (web vs Telephone) and age group

Intro

18Universitat de Girona

Data Collection Methods

18

Differences in data quality for different methods????

Data collection method for ego-centered network data.

Most used face-to-face data collection mode.

Face to face vs. Telephone (Kogovek et al., 2002)

Validity and Reliability of web surveys

(Couper, 2000; Dillman, 2000; Couper et al., 2001; Vehovar et al., 2002)

Quality of Egocentered Network Data Collected via Web

Marin (2002) ; Lozar Manfreda et al. (2004).

Intro

19Universitat de Girona

19

Questionnaire Design

QUESTIONNAIRE DESIGN

Problems in the response process Survey ERRORS

Krosnick (1999); Groves (2004),

Tourangeau, Rips & Rasinski (2000), Dillman, (2007)

Intro

20Universitat de Girona

20

Questionnaire Design

1) Failure to encode the information required

Problem of asking people about what topics that they do not pay much

attention OR are not important.

Improvement: Pre-test; Filter questions (or multiple name generator)

2) Misinterpretation of the questions

Hospital study: During the past 12 months, since _______, how many times have you

seen or talked to doctor [NAME i ] about your health? Do not count any time you might

have seen a doctor while you were a patient in a hospital, but count all other times your

actually saw or talked to a medical doctor of any kind.

Intro

21Universitat de Girona

21

Questionnaire Design

3) Forgetting and other memory problems

Example:

Low options High res

Recommended

View more >