Quantitative research – variables, measurement

levels, samples, populations

HEM 4112 – Research methods I

Martina Vukasovic

Variables (1)• Independent and dependent

– Independent – suspect for cause– Dependent – outcome of interest

• Types (measurement levels):– Nominal/categorical

• Dichotomies as special type

– Ordinal– Interval – Ratio

Variables (2)

• Why is the choice important?– If not in line with the concept – then

jeopardizing construct validity– Use of statistical tools depends on types of

variables

• How to choose?– What are the attributes of the particular

concept?

Variables (3)1. Are there more then two categories?

a) NO variable is dichotomous

b) YES go the next question

2. Can the categories be rank ordered?a) NO variable is nominal/categorical

b) YES go to next question

3. Are the distances between categories equal?a) NO Variable is ordinal

b) YES go to next question

4. Does a zero value of the variable make sense?a) NO Variable is interval

b) YES Variable is on the ratio measurement level

Variables (4) - exercise• Determine measurement level of the following

variables:– Age– Gender– Education attainment– Occupation– Duration of studies– Research productivity– Approach to teaching

• Discuss your choices

Sources of data (1)• Sources:

– statistical data bases already collected data– questionnaires/surveys you are collecting the data

• Often from a trustworthy source– e.g. Ministry, national statistical bureau, UIS– But are they always trustworthy?

• Sometimes you have to collect data by yourself

Sources of data (2) - tips

• Make sure you understand the definitions of indicators• Make sure the indicators are comparable (if doing a

comparative study)• Check who is the actual source, esp. for international

data bases• If you are collecting the data on your own, reliability of

data depends on – how good the questionnaire/survey is– how representative is the sample

Operationalisation (1)From concepts to indicators

• Example 1:– Concept – intelligence

– Indicator – IQ

– Tool for measuring intelligence: tests that yield IQ

• Example 2:– Concept – social intelligence

– Indicator – social IQ

– Tool for measuring social intelligence: tests that yield SIQ (?)

• Example 3:– Concept – intelligence

– Indicator – several different IQs

– Tool for measuring intelligence: tests that yield IQ

• Sometimes you can use several indicators for one concept

– BUT you need to have a reason to do so

– AND you need to be clear how you combine these indicators

Operationalisation (2)

• Can be presented as a 2-step proces:– Development of an indicator– Development of a tool to measure the indicator

• Sometimes already defined (if using data bases) – make sure you understand the definition and the tool– be critical about them

• Sometimes you define it – be careful about construct validity!

Operationalisation (3)

• IQ – good indicator of intelligence?

• IQ test – good measuring tool?

• How would you operationalise quality in HE?– Why?

Sampling (1)

• Sampling – A sample is a representative part of the

population you are interested in• Important for generalization!• An assumption for using statistical tools in the first

place

• Different techniques for sampling– Depends on your research topic

Sampling (2)

• Random sampling– In some computer programmes (e.g.

SPSS/PASW) this can be done automatically • Generators of random numbers

– Can be done in various ways, although some techniques may introduce bias if not careful

– Selection from the population is done entirely at random no bias (?)

Sampling (3) - exercise

• Imagine you need a random sample for a study. Discuss what bias, if any, can be introduced by using the following methods:– Stopping people in public and asking questions:

• On the street• In the theatre

– Distributing a questionnaire inside the classroom– Calling people on the phone– Asking people to complete an online survey

Sampling (4)

• But completely random sampling can in some ways introduce bias even if done correctly

• For some topics and populations, stratified sampling is more appropriate– E.g. when you know in advance that the

distribution is not normal

Sampling (5)

• Stratified (random) sampling– Divide the population in several groups, or strata

– Identify how many respondents or “cases” you need in each group on the basis of their proportion in the entire population

– Do random sampling within this strata– Check after data collection if your stratification

worked

Sampling (6) - exercise

• What kind of sampling procedure you need, if interested in the following issues (if you think you need to stratify the sample, also discuss what strata you need):– Female students are more successful than male

students;– Students who pay for their education are more

concerned with quality of education;– Mobile students have difficulties in obtaining jobs after

graduation?

Constructing a questionnaire (1)• Which data do you need?

– back to conceptual framework (+ research questions)

• How do you operationalise the concepts?– Use the literature, see what others did before you – Several questions can serve as indicators of one

concept– Sometimes “control” questions are used to check the

internal consistency of answers

• Be clear what is the purpose of each question– Useful also to see them terms of how they relate to

independent or dependent variables

Constructing a questionnaire (2)• Questions MUST NOT be ambiguous

– Piloting is necessary– Be aware of language issues

• Options for answers need to be clear

• Layout needs to be user friendly– The respondent needs to be able to complete

the questionnaire easily– Otherwise, you risk if incomplete

questionnaire incomplete data bases

Practicalities (1)

• Make sure you do not jeopardize your sample

• Make sure you allow enough time for responses

• The collecting procedures needs to be as simple as possible

• Expect low response rates, not everyone will answer

Practicalities (2)

• online or paper?

• Think if this affects your sample

• If paper questionnaire – putting data into the data base requires time,

discipline and concentration– useful to label each completed questionnaire

with a unique number and introduce that into the data base as well (for later checks)

Practicalities (3)

• Analysis– Make a plan beforehand! Statistical packages can be

seductive– Be systematic in building the data base, especially in

terms of variable labels, types etc.

• Always make notes of all the manipulations to the data base that you make

• Keep your data base as well as files with results of analysis safe and make regular backups

SPSS workshop as part of HEM 4113