chapter thirteen data collection and measurement

Download Chapter Thirteen Data Collection and Measurement

Post on 11-Jan-2016

213 views

Category:

Documents

0 download

Embed Size (px)

TRANSCRIPT

  • Chapter ThirteenData Collection and Measurement

  • MeasurementThe process by which categories or numbers are used to reflect or indicate concepts and constructs

    A concept is a general idea not directly observable in the real world

    A construct is a concept specified in such a way that it is observable in the real world

  • Levels of a Research StudyTheoretical - interconnected propositions or statements of relationship between concepts

    Conceptual - statements of relationships between two or more constructs

    Operational - indicates how each of the constructs will be measured or operationalized. It refers to the indicators used to reflect the constructs as well as to the procedures used to collect & analyze data

  • Theoretical SubstructionThe dynamic thinking process used to move from the theoretical level to the operational or measurement level of a studyIt illustrates the hierarchical order among the major constituents of a studyIt identifies the foundational elements of a study, determines the relationships among the elements, & presents this in a diagram

  • MeasurementMeasurement is the linkage between the conceptual and the operational levels of a research project

    Two key issues in this linkage:validity or the the congruence between a concept and the indicators of the concept, and reliability or the extent to which an instrument yields similar results on repeated measures

  • ValidityFace validity..on the face of it...Content validityreflects the dimension implied by the conceptConcurrent validitycorrelation of one measure with anotherPredictive validity...predict accuratelyConstruct validitydistinguishes participants who differ on the constructInternal validitytreatment produces changes in dependent variable

  • Validity Cont.Internal validitytreatment produces changes in dependent variable

    External validityextrapolation from study to the other groups in general

    In qualitative researchcredibility is the issue

  • Validity in Qualitative ResearchA qualitative study is credible when it presents descriptions of experiences that the people having had that experience immediately recognize as their own the best test of rigor in qualitative work is when the researcher creates true-to-life, and meaningful portraits, stories, & landscapes of human experiences (Sandelowski, 1993)

  • Rigor in Qualitative ResearchKeep careful recordsAvoid the holistic fallacyGuard against elite biasDont be taken over by respondent

  • ReliabilityInstruments ability to produce the same results on repeated measuresTerms such as dependability, consistency, stability & accuracy are often used interchangeablyaccuracy reflects the instruments ability to measure the true value (free from random measurement error) being measured

  • Reliability in Quantitative ResearchReliability is a relative term, expressed as a correlation 1.00 (perfect reliability) to 0.00 (absence of reliability)Reliability coefficients of .70 are acceptable (Nunnally, 1978)Estimates of reliability need to be determined each time the instrument is used

  • Three Attributes of ReliabilityStability

    Internal Consistency

    Equivalence

  • StabilityConcerned with consistency of results with repeated measuresTest-retest procedures - response should be identical on both occasions assuming the variables measured remain the same at the two testing timesGillis (1997) tested the reliability of the ALQ using the test-retest procedure

  • Internal ConsistencyRefers to the homogeneity of the instrument or the ability of the items in the instrument to measure the same variableItems are strongly correlated to each otherThe > intercorrelations, the > internal consistencyMeasures to test internal consistency: KR-20, item-total correlations, split-half method, cronbachs alpha

  • EquivalenceDegree of agreement among 2 or more different observers using the same measurement tool, orDegree of agreement among 2 or more alternate forms of an instrument or toolDetermined by correlating the 2 scores with each otherInterrater reliability may be determined several times in a study

  • Reliability in Qualitative Research

    In qualitative research replication is not possible because the circumstances & individuals can never be the same at some later time

  • Measurement ErrorAny deviation from the true valueTrue value is the underlying exact quantity of a variable at any given timeVariables change over time & any measure will vary slightly from 1 day to the nextMeasure are made up of the following:Measure=TV+ (SE+RE)

  • Measurement Error Cont.Systematic errornon-random error that systematically over- or under-estimates a value (eg., persons not answering a question are given the lowest valueRandom errorrandom fluctuations around the true value. Not a problematicshould average out.

  • Tips for Reducing Measurement ErrorTake average of several measuresUse different indicatorsUse random sampling proceduresUse sensitive measuresAvoid confusion in wordingsError check data carefullyReduce subject/experimenter expectations

  • Levels of Measurement The level of measurement achieved is important because it constrains the type of statistical analysis that can be performed on your data.NominalOrdinalRatio

  • The Effects of Reduced Levels of MeasurementUnderestimating the relative importance of a variable if it is poorly measuredThe greater the reduction in measurement precision, the greater the drop in correlations between variablesPrecisely measured variables will appear to be more important than poorly measured ones

  • Data CollectionProcess of gathering data from identified participants to answer a research questionA variety of quantitative & qualitative methods are available depending upon research questionindexes or scales, biochemical & physiological measures, projective techniques, delphi techniques, unstructured interviews, focus groups, observation sessions, historical documents

  • Item AnalysisGood indexes discriminate well Example of test item developmenttest graded, students divided into upper and lower quartileexamine performance on each questionselect those questions that discriminate best

  • Discrimination of ItemsPercent Correct Each Item Bottom Top Question # 25% 25% 1 40.0 80.0 2 5.0 95.0 3 60.0 55.0 4 80.0 80.0 5 10.0 40.0 6 20.0 60.0

  • Selecting Index ItemsReview conceptual definitionDevelop measures for each dimensionPre-test indexPilot test index

  • Tips for Wording Likert ItemsThe and alert: avoid multiple dimensionsStrongly Agree on right hand side 9-pointsresponse set issueAvoid negatives like not simply use negative wording. Vary strength of wording to produce variation in responseExercise.items for a euthanasia index

  • Other ScalesSemantic Differential: Here a variety of anchors are used and people place themselves or others on a continuum: shy/outgoing; bookworm/social butterfly

  • Other Scales Cont.Magnitude Estimations: subjects use numbers or line lengths to indicate perceptions. Very good for comparisons: yields ratio level measures. Comparing liking of teachers; seriousness of crimes; liking of one community compared to another one, etc.

  • Other Scales (contd)Visual Analogue Scales (VAS) measure the intensity of participants sensations & feelings about the strength of their attitudes, beliefs, & opinions about specific stimuli such as fatigue, pain, health, etc.Usually a 100 mm line is used with anchor words or phrases at each end

  • Delphi techniqueA panel of experts used for multiple data collection, analysis and processingObtains the opinions of experts without the financial cost or inconvenience of bringing expert opinions of a variety of experts are condensed into precise statements people together

  • Physiological MeasuresParticularly appropriate in studies designed to assess the impact of nursing interventions on bodily functionsProvide objective & sensitive measurements that are difficult for the participant to distorte.g. vital signs, % body fat, muscle strength, salivary enzyme levels, serum glucose, etc.

  • Observational MeasurementWell suited to phenomena that are best viewed from a holistic rather than a reductionistic perspective

    Observations maybe structured, unstructured, or semi-structured; occur in natural or controlled settings

    To be scientific they must meet four critieria: consistent with study objectives; systematic & standard plan for recording; checked & controlled; related to scientific concepts or theories

  • InterviewsA face-to-face verbal interaction to illicit information from the respondent usually through direct questioningstructured, semi-structured, nonstructuredAdvantage of probing, in-depth data, used with participants who are not literateLimited by time, cost , sample size