1.Research Design :Research Design : Measurement &Measurement & Data Collection FormsData Collection Forms Research Design :Research Design : Measurement &Measurement & Data Collection FormsData Collection Forms MBA2216 BUSINESS RESEARCH PROJECTMBA2216 BUSINESS RESEARCH PROJECT by Stephen Ong Visiting Fellow, Birmingham City University, UK

2. 6-2 Design in the Research ProcessDesign in the Research Process 3. MeasurementMeasurement ConceptsConcepts 133 4. 134 LEARNING OUTCOMESLEARNING OUTCOMESLEARNING OUTCOMESLEARNING OUTCOMES 1. Determine what needs to be measured to address a research question or hypothesis 2. Distinguish levels of scale measurement 3. Know how to form an index or composite measure 4. List the three criteria for good measurement 5. Perform a basic assessment of scale reliability and validity After this lecture, you should be able to 5. 6. Describe how business researchers think of attitudes 7. Identify basic approaches to measuring attitudes 8. Discuss the use of rating scales for measuring attitudes 9. Represent a latent construct by constructing a summated scale 10. Summarize ways to measure attitudes with ranking and sorting techniques 11. Discuss major issues involved in the selection of a measurement scale 135 LEARNING OUTCOMESLEARNING OUTCOMES (contd)(contd) LEARNING OUTCOMESLEARNING OUTCOMES (contd)(contd) 6. 12. Explain the significance of decisions about questionnaire design and wording 13. Define alternatives for wording open-ended and fixed-alternative questions 14. Summarize guidelines for questions that avoid mistakes in questionnaire design 15. Describe how the proper sequence of questions may improve a questionnaire 16. Discuss how to design a questionnaire layout 17. Describe criteria for pretesting and revising a questionnaire and for adapting it to global markets LEARNING OUTCOMES (contd)LEARNING OUTCOMES (contd)LEARNING OUTCOMES (contd)LEARNING OUTCOMES (contd) 7. 11-7 FromFrom InvestigativeInvestigative toto MeasurementMeasurement QuestionsQuestions 8. WHAT DO I MEASURE?WHAT DO I MEASURE? Before the measurement process can be defined, researchers have to decide exactly what it is that needs to be produced. The decision statement, corresponding research questions and research hypotheses can be used to decide what concepts need to be measured. Measurement is the process of describing some property of a phenomenon of interest usually by assigning numbers in a reliable and valid way. When numbers are used, the researcher must have a rule for assigning a number to an observation in a way that provides an accurate description. All measurement, particularly in the social sciences, contains error. 138 9. WHAT DO I MEASURE?WHAT DO I MEASURE? (contd)(contd) Concepts A researcher has to know what to measure before knowing how to measure something. A concept is a generalized idea that represents something of meaning. Concepts such as age, sex, education and number of children are relatively concrete properties and present few problems in either definition or measurement. Concepts such as brand loyalty, corporate culture, and so on are more abstract and are more difficult to both define and measure. 139 10. WHAT DO I MEASURE?WHAT DO I MEASURE? (contd)(contd) Operational Definitions Researchers measure concepts through a process known as operationalization, which is a process that involves identifying scales that correspond to variance in the concept. Scales provide a range of values that correspond to different values in the concept being measured. Scales provide correspondence rules that indicate that a certain value on a scale corresponds to some true value of a concept, hopefully in a truthful way. 1310 11. WHAT DO I MEASURE? (contd)WHAT DO I MEASURE? (contd) Operational Definitions (contd) Variables Researchers use variance in concepts to make diagnoses. Variables capture different concept values. Scales capture variance in concepts and as such, the scales provide the researchers variables. For practical purposes, once a research project is underway, there is little difference between a concept and a variable. 12. WHAT DO I MEASURE?WHAT DO I MEASURE? (contd)(contd) Operational Definitions (contd) Constructs Sometimes a single variable cannot capture a concept alone. Using multiple variables to measure one concept can often provide a more complete account of some concept than could any single variable. A construct is a term used for concepts that are measured with multiple variables. Can be very helpful in operationlizing a concept. 1312 13. EXHIBIT 13.EXHIBIT 13.33 Susceptibility to Interpersonal Influence: An Operational DefinitionSusceptibility to Interpersonal Influence: An Operational Definition 14. 11-14 MeasurementMeasurement SelectSelect measurable phenomenameasurable phenomena Develop a set ofDevelop a set of mapping rulesmapping rules Apply the mapping ruleApply the mapping rule to each phenomenonto each phenomenon 15. 11-15 Characteristics of MeasurementCharacteristics of Measurement 16. 11-16 Types of ScalesTypes of Scales OrdinalOrdinal intervalinterval NominalNominal RatioRatio 17. 11-17 Levels of MeasurementLevels of Measurement OrdinalOrdinal intervalinterval RatioRatio NominalNominalNominalNominal ClassificationClassification 18. 11-18 Nominal ScalesNominal Scales Mutually exclusiveMutually exclusive andand Collectively exhaustiveCollectively exhaustive categoriescategories Exhibits onlyExhibits only classificationclassification 19. 11-19 Levels of MeasurementLevels of Measurement OrdinalOrdinalOrdinalOrdinal intervalinterval RatioRatio NominalNominal ClassificationClassification OrderOrder ClassificationClassification 20. 11-20 Ordinal ScalesOrdinal Scales Characteristics ofCharacteristics of nominal scalenominal scale OrderOrder Implies greater thanImplies greater than or less thanor less than 21. 11-21 Levels of MeasurementLevels of Measurement OrdinalOrdinal IntervalIntervalIntervalInterval RatioRatio NominalNominal ClassificationClassification OrderOrder ClassificationClassification OrderOrder ClassificationClassification DistanceDistance 22. 11-22 Interval ScalesInterval Scales Characteristics ofCharacteristics of nominal and ordinalnominal and ordinal scalesscales Equality of interval.Equality of interval. Equal distanceEqual distance between numbersbetween numbers 23. 11-23 Levels of MeasurementLevels of Measurement OrdinalOrdinal intervalinterval RatioRatioRatioRatio NominalNominal ClassificationClassification OrderOrder ClassificationClassification OrderOrder ClassificationClassification DistanceDistance Natural OriginNatural Origin OrderOrder ClassificationClassification DistanceDistance 24. 11-24 Ratio ScalesRatio Scales Characteristics ofCharacteristics of nominal, ordinal,nominal, ordinal, interval scalesinterval scales Absolute zeroAbsolute zero 25. Levels of Scale MeasurementLevels of Scale Measurement The level of scale measurement is important because it determines the mathematical comparisons that are allowed. The four levels of scale measurement are: 26. 1326 Levels of Scale MeasurementLevels of Scale Measurement (contd)(contd) Nominal Assigns a value to an object for identification or classification purposes. Most elementary level of measurement. Ordinal Ranking scales allowing things to be arranged based on how much of some concept they possible. Have nominal properties. 27. 1327 Levels of Scale MeasurementLevels of Scale Measurement (contd)(contd) Interval Capture information about differences in quantities of a concept. Have both nominal and ordinal properties. Ratio Highest form of measurement. Have all the properties of interval scales with the additional attribute of representing absolute quantities. Absolute zero. 28. EXHIBIT 13.EXHIBIT 13.44 Nominal, Ordinal, Interval, and Ratio Scales Provide DifferentNominal, Ordinal, Interval, and Ratio Scales Provide Different InformationInformation 29. EXHIBIT 13.EXHIBIT 13.55 Facts About the Four Levels of ScalesFacts About the Four Levels of Scales 30. 12-30 Measurements are RelativeMeasurements are Relative Any measurement must take into account the position of the observer. There is no such thing as measurement absolute, there is only measurement relative. Jeanette Winterson journalist and author 31. 12-31 The Scaling ProcessThe Scaling Process 32. 12-32 Nature of AttitudesNature of Attitudes Cognitive I think oatmeal is healthier than corn flakes for breakfast. Affective Behavioural I hate corn flakes. I intend to eat more oatmeal for breakfast. 33. 12-33 Improving PredictabilityImproving Predictability Reference groups Reference groups Multiple measures Multiple measures FactorsFactors StrongStrong Specific Basis DirectDirect 34. 12-34 Measurement ScalesMeasurement Scales All survey questions must be actionable if you want results. Frank Schmidt, senior scientist The Gallup Organization 35. 12-35 Selecting aSelecting a Measurement ScaleMeasurement Scale Research objectives Response types Data properties Number of dimensions Forced or unforced choices Balanced or unbalanced Rater errors Number of scale points 36. 12-36 Response TypesResponse Types Rating scaleRating scale Ranking scaleRanking scale CategorizationCategorization SortingSorting 37. 12-37 Number of DimensionsNumber of Dimensions Unidimensional Multi-dimensional 38. 12-38 Balanced or UnbalancedBalanced or Unbalanced Very badVery bad BadBad Neither good norNeither good nor badbad GoodGood Very goodVery good PoorPoor FairFair GoodGood Very goodVery good ExcellentExcellent How good an actress is Angelina Jolie? 39. 12-39 Forced or Unforced ChoicesForced or Unforced Choices Very badVery bad BadBad Neither good nor badNeither good nor bad GoodGood Very goodVery good Very badVery bad BadBad Neither good nor badNeither good nor bad GoodGood Very goodVery good No opinionNo opinion Dont knowDont know How good an actress is Angelina Jolie? 40. 12-40 Number of Scale PointsNumber of Scale Points Very badVery bad BadBad Neither good norNeither good nor badbad GoodGood Very goodVery good Very badVery bad Somewhat badSomewhat bad A little badA little bad Neither good nor badNeither good nor bad A little goodA little good Somewhat goodSomewhat good Very goodVery good How good an actress is Angelina Jolie? 41. 12-41 Rater ErrorsRater Errors Error of central tendency Error of leniency Adjust strength of descriptive adjectives Space intermediate descriptive phrases farther apart Provide smaller differences in meaning between terms near the ends of the scale Use more scale points 42. 12-42 Rater ErrorsRater Errors Primacy Effect Recency Effect Reverse order of alternatives periodically or randomly 43. 12-43 Rater ErrorsRater Errors Halo Effect Rate one trait at a time Reveal one trait per page Reverse anchors periodically 44. ATTITUDES AS HYPOTHETICALATTITUDES AS HYPOTHETICAL CONSTRUCTSCONSTRUCTS Attitude An enduring disposition to consistently respond in a given manner to various aspects of the world. Components of attitudes: Affective Component The feelings or emotions toward an object Cognitive Component Knowledge and beliefs about an object Behavioural Component Predisposition to action Intentions Behavioural expectations 45. Techniques for MeasuringTechniques for Measuring AttitudesAttitudes Ranking Requiring the respondent to rank order objects in overall performance on the basis of a characteristic or stimulus. Rating Asking the respondent to estimate the magnitude of a characteristic, or quality, that an object possesses by indicating on a scale where he or she would rate an object. 46. 1446 Techniques for MeasuringTechniques for Measuring Attitudes (contd)Attitudes (contd) Sorting Presenting the respondent with several concepts typed on cards and requiring the respondent to arrange the cards into a number of piles or otherwise classify the concepts. Choice Asking a respondent to choose one alternative from among several alternatives; it is assumed that the chosen alternative is preferred over the others. 47. Attitude Rating ScalesAttitude Rating Scales Simple Attitude Scale Requires that an individual agree/disagree with a statement or respond to a single question. This type of self-rating scale classifies respondents into one of two categories (e.g., yes or no). Example: THE PRESIDENT SHOULD RUN FOR RE-ELECTION _______ AGREE ______ DISAGREE 48. 12-48 Simple Category ScaleSimple Category Scale I plan to purchase a MindWriter laptop in the 12 months. Yes No 49. Attitude Rating Scales (contd)Attitude Rating Scales (contd) Category Scale A more sensitive measure than a simple scale in that it can have more than two response categories. Question construction is an extremely important factor in increasing the usefulness of these scales. Example: How important were the following in your decision to visit San Diego? (check one for each item) VERY SOMEWHAT NOT TOO IMPORTANT IMPORTANT IMPORTANT CLIMATE ___________ ___________ ___________ COST OF TRAVEL ___________ ___________ ___________ FAMILY ORIENTED ___________ ___________ ___________ EDUCATIONAL/HISTORICAL ASPECTS ___________ ___________ ___________ FAMILIARITY WITH AREA ___________ ___________ ___________ 50. EXHIBIT 14.EXHIBIT 14.11 Selected Category ScalesSelected Category Scales 51. 12-51 Multiple-Choice,Multiple-Choice, Single-Response ScaleSingle-Response Scale What newspaper do you read most often for financial news? East City Gazette West City Tribune Regional newspaper National newspaper Other (specify:_____________) 52. 12-52 Multiple-Choice,Multiple-Choice, Multiple-Response ScaleMultiple-Response Scale What sources did you use when designing your new home? Please check all that apply. Online planning services Magazines Independent contractor/builder Designer Architect Other (specify:_____________) 53. 12-53 Likert ScaleLikert Scale The Internet is superior to traditional libraries for comprehensive searches. Strongly disagree Disagree Neither agree nor disagree Agree Strongly agree 54. Attitude Rating Scales (contd)Attitude Rating Scales (contd) Likert Scale A popular means for measuring attitudes. Respondents indicate their own attitudes by checking how strongly they agree or disagree with statements. Typical response alternatives: strongly agree, agree, uncertain, disagree, and strongly disagree. Example: It is more fun to play a tough, competitive tennis match than to play an easy one. ___Strongly Agree ___Agree ___Not Sure ___Disagree ___Strongly Disagree 55. EXHIBIT 14.EXHIBIT 14.22 Likert Scale Items for Measuring Attitudes toward PatientsLikert Scale Items for Measuring Attitudes toward Patients Interaction with a Physicians Service StaffInteraction with a Physicians Service Staff 56. 12-56 Semantic DifferentialSemantic Differential 57. 1457 Attitude Rating Scales (contd)Attitude Rating Scales (contd) Semantic Differential A series of seven-point rating scales with bipolar adjectives, such as good and bad, anchoring the ends (or poles) of the scale. A weight is assigned to each position on the scale. Traditionally, scores are 7, 6, 5, 4, 3, 2, 1, or +3, +2, +1, 0, -1, -2, -3. Example: ExcitingExciting ___ : ___ : ___ : ___ : ___ : ___ : ___ Calm___ : ___ : ___ : ___ : ___ : ___ : ___ Calm InterestingInteresting ___ : ___ : ___ : ___ : ___ : ___ : ___ Dull___ : ___ : ___ : ___ : ___ : ___ : ___ Dull SimpleSimple ___ : ___ : ___ : ___ : ___ : ___ : ___ Complex___ : ___ : ___ : ___ : ___ : ___ : ___ Complex PassivePassive ___ : ___ : ___ : ___ : ___ : ___ : ___ Active___ : ___ : ___ : ___ : ___ : ___ : ___ Active 58. EXHIBIT 14.EXHIBIT 14.33 Semantic Differential Scales for Measuring Attitudes TowardSemantic Differential Scales for Measuring Attitudes Toward SupermarketsSupermarkets 59. 12-59 Adapting SD ScalesAdapting SD Scales Convenience of Reaching the Store from Your Location Nearby ___: ___: ___: ___: ___: ___: ___: Distant Short time required to reach store ___: ___: ___: ___: ___: ___: ___: Long time required to reach store Difficult drive ___: ___: ___: ___: ___: ___: ___: Easy Drive Difficult to find parking place ___: ___: ___: ___: ___: ___: ___: Easy to find parking place Convenient to other stores I shop ___: ___: ___: ___: ___: ___: ___: Inconvenient to other stores I shop Products offered Wide selection of different kinds of products ___: ___: ___: ___: ___: ___: ___: Limited selection of different kinds of products Fully stocked ___: ___: ___: ___: ___: ___: ___: Understocked Undependable products ___: ___: ___: ___: ___: ___: ___: Dependable products High quality ___: ___: ___: ___: ___: ___: ___: Low quality Numerous brands ___: ___: ___: ___: ___: ___: ___: Few brands Unknown brands ___: ___: ___: ___: ___: ___: ___: Well-known brands 60. 12-60 SD Scale for Analyzing ActorSD Scale for Analyzing Actor CandidatesCandidates 61. 12-61 Graphic of SD AnalysisGraphic of SD Analysis 62. Other Scale Types (contd)Other Scale Types (contd) Image Profile A graphic representation of semantic differential data for competing brands, products, or stores to highlight comparisons. Because the data are assumed to be interval, either the arithmetic mean or the median will be used to compare the profile of one product, brand, or store with that of a competing product, brand, or store. 63. EXHIBIT 14.EXHIBIT 14.44 Image Profiles of Commuter Airlines versus Major AirlinesImage Profiles of Commuter Airlines versus Major Airlines 64. 12-64 Numerical ScaleNumerical Scale 65. Attitude Rating Scales (contd)Attitude Rating Scales (contd) Numerical Scales Scales that have numbers as response options, rather than semantic space or verbal descriptions, to identify categories (response positions). In practice, researchers have found that a scale with numerical labels for intermediate points on the scale is as effective a measure as the true semantic differential. Example: Now that youve had your automobile for about one year, please tell us how satisfied you are with your Ford Taurus. Extremely Dissatisfied 1 2 3 4 5 6 7 Extremely Satisfied 66. 12-66 Multiple Rating List ScalesMultiple Rating List Scales Please indicate how important or unimportant each service characteristic is: IMPORTANT UNIMPORTANT Fast, reliable repair 7 6 5 4 3 2 1 Service at my location 7 6 5 4 3 2 1 Maintenance by manufacturer 7 6 5 4 3 2 1 Knowledgeable technicians 7 6 5 4 3 2 1 Notification of upgrades 7 6 5 4 3 2 1 Service contract after warranty 7 6 5 4 3 2 1 67. 12-67 Stapel ScalesStapel Scales 68. Other Scale Types (contd)Other Scale Types (contd) Stapel Scale Uses a single adjective as a substitute for the semantic differential when it is difficult to create pairs of bipolar adjectives. Tends to be easier to conduct and administer than a semantic differential scale. 69. EXHIBIT 14.EXHIBIT 14.55 A Stapel Scale for Measuring a Stores ImageA Stapel Scale for Measuring a Stores Image 70. 12-70 Constant-Sum ScalesConstant-Sum Scales 71. Other Scale Types (contd)Other Scale Types (contd) Constant-Sum Scale Respondents are asked to divide a constant sum to indicate the relative importance of attributes. Respondents often sort cards, but the task may also be a rating task (e.g., indicating brand preference). Example: Divide 100 points among each of the following brands according to your preference for the brand: Brand A _________ Brand B _________ Brand C _________ 72. 12-72 Graphic Rating ScalesGraphic Rating Scales 73. EXHIBIT 14.EXHIBIT 14.77 A Ladder ScaleA Ladder Scale 74. EXHIBIT 14.EXHIBIT 14.88 Graphic Rating Scale with Picture ResponseGraphic Rating Scale with Picture Response Categories Stressing Visual CommunicationCategories Stressing Visual Communication 75. Other Scale Types (contd)Other Scale Types (contd) Graphic Rating Scale A measure of attitude that allows respondents to rate an object by choosing any point along a graphic continuum. Advantage: Allows the researcher to choose any interval desired for scoring purposes. Disadvantage: There are no standard answers. 76. EXHIBIT 14.EXHIBIT 14.66 Graphic Rating ScaleGraphic Rating Scale 77. EXHIBIT 14.EXHIBIT 14.99 Summary of Advantages and Disadvantages of Rating ScalesSummary of Advantages and Disadvantages of Rating Scales 78. 12-78 Ranking ScalesRanking Scales Paired-comparison scale Forced ranking scale Comparative scale 79. RankingRanking An ordinal scale may be developed by asking respondents to rank order (from most preferred to least preferred) a set of objects or attributes. Paired comparisons Sorting 80. 12-80 Paired-Comparison ScalePaired-Comparison Scale 81. Paired ComparisonPaired Comparison A measurement technique that involves presenting the respondent with two objects and asking the respondent to pick the preferred object; more than two objects may be presented, but comparisons are made in pairs. Number of comparisons = [(n)(n-1)/2] Example: I would like to know your overall opinion of two brands of adhesive bandages. They are MedBand and Super-Aid. Overall, which of these two brands MedBand or Super-Aiddo you think is the better one? Or are both the same? MedBand is better _____ Super-Aid is better _____ They are the same _____ 82. 12-82 Forced Ranking ScaleForced Ranking Scale 83. 12-83 Comparative ScaleComparative Scale 84. 12-84 SortingSorting 85. SortingSorting Require that respondents indicate their attitudes or beliefs by arranging items on the basis of perceived similarity or some other attribute. Example: Here is a sheet that lists several airlines. Next to the name of each airline is a pocket. Here are ten cards. I would like you to put these cards in the pockets next to the airlines you would prefer to fly on your next trip. Assume that all of the airlines fly to wherever you would choose to travel. You can put as many cards as you want next to an airline, or you can put no cards next to an airline. Cards American Airlines _____ Delta Airlines _____ United Airlines _____ Southwest Airlines _____ Northwest Airlines _____ 86. 12-86 Example : MindWriter ScalingExample : MindWriter Scaling Likert Scale The problem that prompted service/repair was resolved Strongly Disagree Disagree Neither Agree Nor Disagree Agree Strongly Agree 1 2 3 4 5 Numerical Scale (MindWriters Favourite) To what extent are you satisfied that the problem that prompted service/repair was resolved? Very Dissatisfied Very Satisfied 1 2 3 4 5 Hybrid Expectation Scale Resolution of the problem that prompted service/repair. Met Few Expectations Met Some Expectations Met Most Expectations Met All Expectations Exceeded Expectations 1 2 3 4 5 87. 12-87 Ideal Scalogram PatternIdeal Scalogram Pattern Item Participant Score 2 4 1 3 X X X X 4 __ X X X 3 __ __ X X 2 __ __ __ X 1 __ __ __ __ 0 88. Measuring Behavioural IntentionMeasuring Behavioural Intention Behavioural Component The behavioural expectations (expected future actions) of an individual toward an attitudinal object. Example: How likely is it that you will purchase a Honda Fit? I definitely will buy I probably will buy I might buy I probably will not buy I definitely will not buy 89. Measuring Behavioural IntentionMeasuring Behavioural Intention (contd)(contd) Behavioural Differential A rating scale instrument similar to a semantic differential, developed to measure the behavioural intentions of subjects toward future actions. A description of the object to be judged is placed on the top of a sheet, and the subjects indicate their behavioural intentions toward this object on a series of scales. Example: A 25 year-old woman sales representative Would ___ : ___ : ___ : ___ : ___ : ___ : ___ : Would Not ask this person for advice. 90. Mathematical and StatisticalMathematical and Statistical Analysis of ScalesAnalysis of Scales Although you can put numbers into formulas and perform calculations with almost any numbers, the researcher has to know the meaning behind the numbers before useful conclusions can be drawn (e.g., averaging the numbers used to identify school busses is meaningless). 91. Mathematical and StatisticalMathematical and Statistical Analysis of Scales (contd)Analysis of Scales (contd) Discrete Measures Discrete measures are those that take on only one of a finite number of values. Most often used to represent a classificatory variable and thus do not represent intensity of measures, only membership. Common discrete scales include any yes-no response, matching, colour choice or practically all scales that involve selecting from a small number of categories. Nominal and ordinal scales are discrete measures. The central tendency of discrete measures is best captured by the mode (i.e., most frequent level). 92. 1392 Mathematical and StatisticalMathematical and Statistical Analysis of Scales (contd)Analysis of Scales (contd) Continuous Measures Continuous measures are those assigning values anywhere along some scale range in a place that corresponds to the intensity of some concept. Ratio measures are continuous measures. Strictly speaking, interval scales are not necessarily continuous. e.g., Likert item ranging from 1=strongly disagree to 5=strongly agree. This is a discrete scale because only the values 1, 2, 3, 4, or 5 can be assigned. 93. Index MeasuresIndex Measures Attributes Single characteristics or fundamental features that pertain to an object, person, or issue. Index Measures Assign a value based on how much of the concept being measured is associated with an observation. Indexes often are formed by putting several variables together. Composite Measures Assign a value to an observation based on a mathematical derivation of multiple variables. 94. Computing Scale ValuesComputing Scale Values Summated Scale A scale created by simply summing (adding together) the response to each item making up the composite measure. Reverse Coding Means that the value assigned for a response is treated oppositely from the other items. 95. EXHIBIT 13.EXHIBIT 13.66 Computing a Composite ScaleComputing a Composite Scale 96. Three Criteria for GoodThree Criteria for Good MeasurementMeasurement SensitivitySensitivitySensitivitySensitivity ReliabilityReliabilityReliabilityReliability ValidityValidityValidityValidity GoodGood MeasurementMeasurement GoodGood MeasurementMeasurement 97. ReliabilityReliability Reliability Reliability is an indicator of a measures internal consistency. A measure is reliable when different attempts at measuring something converge on the same result. When the measuring process provides reproducible results, the measuring instrument is reliable. Internal Consistency Represents a measures homogeneity or the extent to which each indicator of a concept converges on some common meaning. Measured by correlating scores on subsets of items making up a scale. 98. Internal ConsistencyInternal Consistency Split-half Method Assessing internal consistency by checking the results of one-half of a set of scaled items against the results from the other half. The two scale halves should correlate highly. They should also produce similar scores. 99. Internal Consistency (contd)Internal Consistency (contd) Coefficient alpha () The most commonly applied estimate of a multiple item scales reliability. Represents the average of all possible split-half reliabilities for a construct. The coefficient demonstrates whether or not the different items converge. Ranges in value from 0 (no consistency) to 1 (complete consistency). Generally, scales with a coefficient : 100. Test-Retest ReliabilityTest-Retest Reliability Test-retest Method Administering the same scale or measure to the same respondents at two separate points in time to test for stability. Represents a measures repeatability. Problems: The pre-measure, or first measure, may sensitize the respondents and subsequently influence the results of the second measure. Time effects that produce changes in attitude or other maturation of the subjects. 101. ValidityValidity ValidityValidity Good measures should be both precise (i.e., reliable) and accurate (i.e., valid). Validity is the accuracy of a measure or the extent to which a score truthfully represents a concept. Does a scale measure what was intended to be measured? When a measure lacks validity, any conclusions based on that measure are also likely to be faulty. 102. Validity : Face, Content Validity : Face, Content Establishing Validity: The four basic approaches to establishing validity are face validity, content validity, criterion validity, and construct validity. Face validityFace validity refers to the subjective agreement among professionals that a scale logically reflects the concept being measured. Content validityContent validity refers to the degree that a measure covers the domain of interest. 103. Validity : Criterion Validity : Criterion Criterion validityCriterion validity addresses the question: Does my measure correlate with measuresDoes my measure correlate with measures of similar concepts or known quantities?of similar concepts or known quantities? May be classified as either concurrent validity or predictive validity depending on the time sequence in which the new measurement scale and the criterion measure are correlated. If measures taken at the same time concurrent validity. If measures taken at different times predictive validity. 104. Validity : Construct Validity : Construct Construct validityConstruct validity exists when a measure reliably measures and truthfully represents a unique concept and consists of several components: Face and Content validityFace and Content validity Convergent validityConvergent validity another way of expressing internal consistency; highly reliable scales contain convergent validity. Criterion validityCriterion validity Discriminant validityDiscriminant validity represents how unique or distinct is a measure; a scale should not correlate too highly (i.e., above .75) with a measure of a different construct. 105. EXHIBIT 13.EXHIBIT 13.77 Reliability and Validity on TargetReliability and Validity on Target 106. SensitivitySensitivity Sensitivity A measurement instruments ability to accurately measure variability in stimuli or responses. Generally increased by adding more response points or adding scale items. 107. Selecting a Measurement ScaleSelecting a Measurement Scale Some Practical Questions: Is a ranking, sorting, rating, or choice technique best? Should a monadic or a comparative scale be used? What type of category labels, if any, will be used for the rating scale? How many scale categories or response positions are needed to accurately measure an attitude? Should a balanced or unbalanced rating scale be chosen? Should a scale that forces a choice among predetermined options be used? Should a single measure or an index measure be used? 108. Selecting a Measurement ScaleSelecting a Measurement Scale (contd)(contd) Monadic Rating Scale Asks about a single concept in isolation. The respondent is not given a specific frame of reference. Example: Now that youve had your automobile for about 1 year, please tell us how satisfied you are with its engine power and pickup. 109. Please indicate how the amount of authority in your present position compares with the amount of authority that would be ideal for this position. TOO MUCH ABOUT RIGHT TOO LITTLE Selecting a Measurement ScaleSelecting a Measurement Scale (contd)(contd) Comparative Rating Scale Asks respondents to rate a concept in comparison with a benchmark explicitly used as a frame of reference. Example: 110. Selecting a MeasurementSelecting a Measurement Scale (contd)Scale (contd) What Type of Category Labels, If Any? Verbal labels for response categories help respondents better understand the response positions. The maturity and educational levels of the respondents will influence the labeling decision. How Many Scale Categories or Response Positions? Five to eight points are optimal for sensitivity. The researcher must determine the number of positions that is best for the specific project. 111. Selecting a Measurement ScaleSelecting a Measurement Scale (contd)(contd) Balanced Rating Scale A fixed-alternative rating scale with an equal number of positive and negative categories; a neutral point or point of indifference is at the center of the scale. Example: 112. Selecting a Measurement ScaleSelecting a Measurement Scale (contd)(contd) Unbalanced Rating Scale A fixed-alternative rating scale that has more response categories at one end than the other resulting in an unequal number of positive and negative categories. Example: 113. Selecting a MeasurementSelecting a Measurement Scale (contd)Scale (contd) Forced-choice Rating Scale A fixed-alternative rating scale that requires respondents to choose one of the fixed alternatives. Non-forced Choice Scale A fixed-alternative rating scale that provides a no opinion category or that allows respondents to indicate that they cannot say which alternative is their choice. 114. Selecting a MeasurementSelecting a Measurement Scale (contd)Scale (contd) Factors affecting the choice of using a single measure or an index measure: The complexity of the issue to be investigated. The number of dimensions the issue contains. Whether individual attributes of the stimulus are part of a holistic attitude or are seen as separate items. The researchers conceptual (problem) definition will be helpful in making this choice. 115. 13-115 Overall Flowchart for InstrumentOverall Flowchart for Instrument DesignDesign 116. 13-116 Flowchart for InstrumentFlowchart for Instrument Design Phase 1Design Phase 1 117. Strategic Concerns inStrategic Concerns in Instrument DesignInstrument Design What type of scale is needed? What communication approach will be used? Should the questions be structured? Should the questioning be disguised? 118. Technology AffectsTechnology Affects Questionnaire DevelopmentQuestionnaire Development WebSurveyor used to write an instrument. Write questionnairesWrite questionnaires more quicklymore quickly Create visually drivenCreate visually driven instrumentsinstruments Eliminate manualEliminate manual data entrydata entry Save time in dataSave time in data analysisanalysis 119. 13-119 Disguising Study ObjectivesDisguising Study Objectives Situations where disguise is unnecessary Situations where disguise is unnecessary Willingly shared, Conscious-level information Reluctantly shared, Conscious-level information Knowable, Limited-conscious-level information Subconscious-level information 120. Dummy Table forDummy Table for American Eating HabitsAmerican Eating Habits Age Use of Convenience Foods Always Use Use Frequently Use Sometime s Rarely Use Never Use 18-24 25-34 35-44 55-64 65+ 121. 13-121 Flowchart for Instrument DesignFlowchart for Instrument Design Phase 2Phase 2 122. 13-122 Question CategoriesQuestion Categories and Structureand Structure Administrative Target Classification 123. Engagement = ConvenienceEngagement = Convenience Participants are becoming more and more aware of the value of their time. The key to maintaining a quality dialogue with them is to make it really convenient for them to engage, whenever and wherever they want. Tom Anderson managing partner Anderson Analytics 124. 13-124 Question ContentQuestion Content Should this question be asked? Is the question of proper scope and coverage? Can the participant adequately answer this question as asked? Will the participant willingly answer this question as asked? 125. 13-125 Question WordingQuestion Wording CriteriaCriteria Shared vocabulary Single meaning Misleading assumptions Adequate alternatives Personalized Biased 126. 13-126 Response StrategyResponse Strategy FactorsFactors Objectives of the study Participants level of information Degree to which participants have thought through topic Ease and clarity with which participant communicates Participants motivation to share 127. 13-127 Free-Response StrategyFree-Response Strategy What factors influenced your enrollment in Metro U? ____________________________________________ ____________________________________________ 128. 13-128 Dichotomous ResponseDichotomous Response StrategyStrategy Did you attend the A Day at College program at Metro U? Yes No 129. 13-129 Multiple Choice ResponseMultiple Choice Response StrategyStrategy Which one of the following factors was most influential in your decision to attend Metro U? Good academic standing Specific program of study desired Enjoyable campus life Many friends from home High quality of faculty 130. 13-130 Checklist Response StrategyChecklist Response Strategy Which of the following factors influenced your decision to enroll in Metro U? (Check all that apply.) Tuition cost Specific program of study desired Parents preferences Opinion of brother or sister Many friends from home attend High quality of faculty 131. 13-131 Rating Response StrategyRating Response Strategy Strongly influential Somewhat influential Not at all influential Good academic reputation Enjoyable campus life Many friends High quality faculty Semester calendar 132. 13-132 RankingRanking Please rank-order your top three factors from the following list based on their influence in encouraging you to apply to Metro U. Use 1 to indicate the most encouraging factor, 2 the next most encouraging factor, etc. _____ Opportunity to play collegiate sports _____ Closeness to home _____ Enjoyable campus life _____ Good academic reputation _____ High quality of faculty 133. 13-133 Summary of Scale TypesSummary of Scale Types Type Restrictions Scale Items Data Type Rating Scales Simple Category Scale Needs mutually exclusive choices One or more Nominal Multiple Choice Single-Response Scale Needs mutually exclusive choices May use exhaustive list or other Many Nominal Multiple Choice Multiple-Response Scale (checklist) Needs mutually exclusive choices Needs exhaustive list or other Many Nominal Likert Scale Needs definitive positive or negative statements with which to agree/disagree One or more Ordinal Likert-type Scale Needs definitive positive or negative statements with which to agree/disagree One or more Ordinal 134. 13-134 Summary of Scale TypesSummary of Scale Types Type Restrictions Scale Items Data Type Rating Scales Numerical Scale Needs concepts with standardized meanings; Needs number anchors of the scale or end-points Score is a measurement of graphical space One or many Ordinal or Interval Multiple Rating List Scale Needs words that are opposites to anchor the end-points on the verbal scale Up to 10 Ordinal Fixed Sum Scale Participant needs ability to calculate total to some fixed number, often 100. Two or more Interval or Ratio 135. 13-135 Summary of Scale TypesSummary of Scale Types Type Restrictions Scale Items Data Type Rating Scales Stapel Scale Needs verbal labels that are operationally defined or standard. One or more Ordinal or Interval Graphic Rating Scale Needs visual images that can be interpreted as positive or negative anchors Score is a measurement of graphical space from one anchor. One or more Ordinal (Interval, or Ratio) 136. 13-136 Summary of Scale TypesSummary of Scale Types Type Restrictions Scale Items Data Type Ranking Scales Paired Comparison Scale Number is controlled by participants stamina and interest. Up to 10 Ordinal Forced Ranking Scale Needs mutually exclusive choices. Up to 10 Ordinal or Interval Comparative Scale Can use verbal or graphical scale. Up to 10 Ordinal 137. 13-137 Internet SurveyInternet Survey Scale OptionsScale Options 138. 13-138 Internet SurveyInternet Survey Scale OptionsScale Options 139. 13-139 Internet SurveyInternet Survey Scale OptionsScale Options 140. 13-140 Sources of QuestionsSources of Questions Handbook ofHandbook of Marketing ScalesMarketing Scales The Gallup PollThe Gallup Poll Cumulative IndexCumulative Index Measures ofMeasures of Personality andPersonality and Social-Social- PsychologicalPsychological AttitudesAttitudes Measures ofMeasures of Political AttitudesPolitical Attitudes Index to InternationalIndex to International Public OpinionPublic Opinion Sourcebook of HarrisSourcebook of Harris National SurveysNational Surveys Marketing ScalesMarketing Scales HandbookHandbook American SocialAmerican Social Attitudes DataAttitudes Data SourcebookSourcebook 141. 13-141 Flowchart for Instrument Design Phase 3Flowchart for Instrument Design Phase 3 142. 13-142 Guidelines forGuidelines for Question SequencingQuestion Sequencing Interesting topics earlyInteresting topics early Simple topics earlySimple topics early Sensitive questions laterSensitive questions later Classification questions laterClassification questions later Transition between topicsTransition between topics Reference changes limitedReference changes limited 143. Illustrating the Funnel ApproachIllustrating the Funnel Approach How do you think this country is gettingHow do you think this country is getting along in its relations with otheralong in its relations with other countries?countries? How do you think we are doing in ourHow do you think we are doing in our relations with Iran?relations with Iran? Do you think we ought to be dealing withDo you think we ought to be dealing with Iran differently than we are now?Iran differently than we are now? (If yes) What should we be doing(If yes) What should we be doing differently?differently? Some people say we should get tougherSome people say we should get tougher with Iran and others think we are toowith Iran and others think we are too tough as it is; how do you feel about it?tough as it is; how do you feel about it? 144. 13-144 Branching QuestionBranching Question 145. 13-145 Components of QuestionnairesComponents of Questionnaires 146. Example :MindWriter SurveyExample :MindWriter Survey 147. 13-147 Overcoming InstrumentOvercoming Instrument ProblemsProblems Build rapportBuild rapport Redesign question processRedesign question process Explore alternativesExplore alternatives Use other methodsUse other methods PretestPretest 148. Questionnaire Quality and Design:Questionnaire Quality and Design: Basic ConsiderationsBasic Considerations Questionnaire design is one of the most critical stages in the survey research process. A questionnaire (survey) is only as good as the questions it asksask a bad question, get bad results. Composing a good questionnaire appears easy, but it is usually the result of long, painstaking work. The questions must meet the basic criteria of relevance and accuracy. 149. Decisions in QuestionnaireDecisions in Questionnaire DesignDesign 1. What should be asked? 2. How should questions be phrased? 3. In what sequence should the questions be arranged? 4. What questionnaire layout will best serve the research objectives? 5. How should the questionnaire be pretested? Does the questionnaire need to be revised? 150. What Should Be Asked?What Should Be Asked? Questionnaire Relevancy All information collected should address a research question in helping the decision maker in solving the current business problem. Questionnaire Accuracy Increasing the reliability and validity of respondent information requires that: Questionnaires should use simple, understandable, unbiased, unambiguous, and nonirritating words. Questionnaire design should facilitate recall and motivate respondents to cooperate. Proper question wording and sequencing to avoid confusion and biased answers. 151. Wording QuestionsWording Questions Open-ended Response Questions Pose some problem and ask respondents to answer in their own words. Advantages: Are most beneficial in exploratory research, especially when the range of responses is not known. May reveal unanticipated reactions toward the product. Are good first questions because they allow respondents to warm up to the questioning process. Disadvantages: High cost of administering open-ended response questions. The possibility that interviewer bias will influence the answer. Bias introduced by articulate individuals longer answers. 152. Wording Questions (contd)Wording Questions (contd) Fixed-alternative Questions Questions in which respondents are given specific, limited-alternative responses and asked to choose the one closest to their own viewpoint. Advantages: Require less interviewer skill Take less time to answer Are easier for the respondent to answer Provides comparability of answers Disadvantages: Lack of range in the response alternatives Tendency of respondents to choose convenient alternative 153. Types of Fixed-AlternativeTypes of Fixed-Alternative QuestionsQuestions Simple-dichotomy (dichotomous) Question Requires the respondent to choose one of two alternatives (e.g., yes or no). Determinant-choice Question Requires the respondent to choose one response from among multiple alternatives (e.g., A, B, or C). Frequency-determination Question Asks for an answer about general frequency of occurrence (e.g., often, occasionally, or never). Checklist Question Allows the respondent to provide multiple answers to a single question by checking off items. 154. Phrasing Questions for Self-Phrasing Questions for Self- Administered,Telephone, andAdministered,Telephone, and Personal Interview SurveysPersonal Interview Surveys Influences on Question Phrasing: The means of data collectiontelephone interview, personal interview, self- administered questionnairewill influence the question format and question phrasing. Questions for mail, Internet, and telephone surveys must be less complex than those used in personal interviews. Questionnaires for telephone and personal interviews should be written in a conversational style. 155. EXHIBIT 15.EXHIBIT 15.11 Reducing Question Complexity by Providing Fewer Responses forReducing Question Complexity by Providing Fewer Responses for Telephone InterviewsTelephone Interviews 156. Guidelines for ConstructingGuidelines for Constructing QuestionsQuestions Avoid complexity: Simpler language is better. Avoid leading and loaded questions. Avoid ambiguity: Be as specific as possible. Avoid double-barreled items. Avoid making assumptions. Avoid burdensome questions that may tax the respondents memory. Make certain questions generate variance. 157. What Is the Best QuestionWhat Is the Best Question Sequence?Sequence? Order bias Bias caused by the influence of earlier questions in a questionnaire or by an answers position in a set of answers. Funnel technique Asking general questions before specific questions in order to obtain unbiased responses. Filter question A question that screens out respondents who are not qualified to answer a second question. Pivot question A filter question used to determine which version of a second question will be asked. 158. EXHIBIT 15.EXHIBIT 15.22 Flow of QuestionsFlow of Questions to Determine theto Determine the Level of PromptingLevel of Prompting Required toRequired to Stimulate RecallStimulate Recall 159. What Is the Best Layout?What Is the Best Layout? Traditional Questionnaires Multiple-grid question Several similar questions arranged in a grid format. The title of a questionnaire should be phrased carefully: To capture the respondents interest, underline the importance of the research Emphasize the interesting nature of the study Appeal to the respondents ego Emphasize the confidential nature of the study To not bias the respondent in the same way that a leading question might 160. EXHIBIT 15.EXHIBIT 15.33 Layout of a Page from a Telephone QuestionnaireLayout of a Page from a Telephone Questionnaire 161. EXHIBIT 15.EXHIBIT 15.44 Telephone Questionnaire with Skip QuestionsTelephone Questionnaire with Skip Questions 162. EXHIBIT 15.EXHIBIT 15.55 Personal Interview QuestionnairePersonal Interview Questionnaire 163. EXHIBIT 15.EXHIBIT 15.66 Example of a Skip QuestionExample of a Skip Question 164. Internet QuestionnairesInternet Questionnaires Graphical User Interface (GUI) Software The researcher can control the background, colours, fonts, and other features displayed on the screen so as to create an attractive and easy-to-use interface between the user and the Internet survey. Layout Issues Paging layout - going from screen to screen. Scrolling layout entire questionnaire appears on one page and respondent has the ability to scroll down. 165. Internet Questionnaire LayoutInternet Questionnaire Layout Push Button A small outlined area, such as a rectangle or an arrow, that the respondent clicks on to select an option or perform a function, such as submit. Status Bar A visual indicator that tells the respondent what portion of the survey he or she has completed. Radio Button A circular icon, resembling a button, that activates one response choice and deactivates others when a respondent clicks on it. 166. Internet Questionnaire LayoutInternet Questionnaire Layout (contd)(contd) Drop-down Box A space saving device that reveals responses when they are needed but otherwise hides them from view. Check Boxes Small graphic boxes, next to an answers, that a respondent clicks on to choose an answer; typically, a check mark or an X appears in the box when the respondent clicks on it. Open-ended Boxes Boxes where respondents can type in their own answers to open-ended questions. Pop-up Boxes Boxes that appear at selected points and contain information or instructions for respondents. 167. EXHIBIT 15.EXHIBIT 15.77 Question in an Online Screening Survey for Joining a ConsumerQuestion in an Online Screening Survey for Joining a Consumer PanelPanel 168. 15168 EXHIBIT 15.EXHIBIT 15.88 Alternative Ways of Displaying Internet QuestionsAlternative Ways of Displaying Internet Questions 169. Internet Questionnaire LayoutInternet Questionnaire Layout (contd)(contd) Software That Makes Questionnaires Interactive Variable piping software Allows variables to be inserted into an Internet questionnaire as a respondent is completing it. Error trapping software Controls the flow of an Internet questionnaire. Forced answering software Prevents respondents from continuing with an Internet questionnaire if they fail to answer a question. Interactive help desk A live, real-time support feature that solves problems or answers questions respondents may encounter in completing the questionnaire. 170. Pretesting and RevisingPretesting and Revising QuestionnairesQuestionnaires Pretesting Process Seeks to determine whether respondents have any difficulty understanding the questionnaire and whether there are any ambiguous or biased questions. Preliminary Tabulation A tabulation of the results of a pretest to help determine whether the questionnaire will meet the objectives of the research. 171. Designing Questionnaires forDesigning Questionnaires for Global MarketsGlobal Markets Back Translation Taking a questionnaire that has previously been translated into another language and having a second, independent translator translate it back to the original language. A questionnaire developed in one country may be difficult to translate because equivalent language concepts do not exist or because of differences in idiom and vernacular. 172. Further ReadingFurther Reading COOPER, D.R. AND SCHINDLER, P.S. (2011) BUSINESS RESEARCH METHODS, 11TH EDN, MCGRAW HILL ZIKMUND, W.G., BABIN, B.J., CARR, J.C. AND GRIFFIN, M. (2010) BUSINESS RESEARCH METHODS, 8TH EDN, SOUTH-WESTERN SAUNDERS, M., LEWIS, P. AND THORNHILL, A. (2012) RESEARCH METHODS FOR BUSINESS STUDENTS, 6TH EDN, PRENTICE HALL. SAUNDERS, M. AND LEWIS, P. (2012) DOING RESEARCH IN BUSINESS & MANAGEMENT, FT PRENTICE HALL.