empirical methods – g. grote ethz, hs 07 obectives of the course basic knowledge of social science...

Empirical Methods – G. Grote ETHZ, HS 07

Obectives of the course

• Basic knowledge of social science methods

• Ability to assess quality of existing research

• Ability to carry out studies in an organizational context

• Ability to evaluate/hire external experts for carrying out studies


Examples of social science methods used in organizational contexts

• Interview– Selection interview– Performance appraisal

• Questionnaire– Employee survey– Customer survey– Personality test

• Observation– Job analysis– Assessment center


Special issues in social research

• Control of complexity– Constraints on manipulation of study conditions– Studying "hypothetical constructs"– Limited possibilities for reducing complexity

• Studies with humans– Effects through researcher / researched individual and social

embeddedness– Ethical principles: voluntary participation, informed consent, no

physical/psychological harm, confidentiality

It is all about grey zones, there is no black and white Building competence for navigating in these grey zones.


Special issues for social research in organizational contexts

• Objectives for study by different stakeholders in the organization

• Role/influence of people carrying out the study

• Role/influence of people being studied

• Control over use of results and consequences of study


Empirical methods

Overview Top ic Tutorial 1 Tutorial 2 Tutorial 3 Lecturer

26.9. Overview empirical methods in social research Grote

3.10. Study design Grote

10.10. Defining variables; Approaches to data collection and analysis Grote

17.10. Introduction quantitative methods Günter

24.10. Questionnaires Günter

31.10. Quantitative data analysis - Overview SPSS Günter

7.11. Introduction qualitative methods Raeder

14.11. Qualitative interview Raeder

21.11. Qualitative data analysis Raeder

28.11. Introduction observation Künzle

5.12. Example observation: Assessment Center Künzle

12.12. Guest lecture: Dr. Stefana Broadbent, Swisscom Innovations Grote

19.12. Preparation for exam Grote


Textbook

• Darren Langdridge, Introduction to Research Methods and Data Analysis in Psychology. Pearson Prentice-Hall, 2004. ISBN 0-13-097832-9

• amazon.de: EUR 49.90

• amazon.co.uk: £23.39 (excl. shipping)

• Polybuchhandlung: ca. CHF 89.-

• Orell Füssli Bookshop CHF 80.-


Organization of course

• Lectures

• Three tutorials to be worked on at home

• Written, open book exam during the exam weeks at the end of the semester (2./3. week of January)

• Grade: 50% tutorials, 50% exam


Tutorial 1

• Please read the following article and write a short critique (ca. 2-3 pages) regarding the quality of the article using the criteria presented in the lecture.

• You may work on this tutorial alone or together with one other student.The article: Lipshitz, R. & Strauss, O. (1997). Coping with uncertainty: A naturalistic decision-making analysis. Organizational Behavior and Human Decision Processes, 69, 149-163. (available at www.oat.ethz.ch)


Steps in empirical research

• Define general question of interest• Draw on existing knowledge to develop

specific questions/hypotheses to be studied

• Determine design of study• Data collection• Data analysis• Interpretation/conclusions regarding initial

questions/hypotheses


Criteria for good research

• Internal validity = unambiguous interpretation of results/few alternative explanations for resultsPossible problems: uncontrolled influences by e.g. external events, selection of participants, repeated measuring

• External validity = generalizability of results regarding other samples of people, situations, timePossible problems: special characteristics of participants, reactions to measurement as such, interaction of several "treatments"


Criteria for good research methods

• Reliability: stability of measurement– Internal consistency– Retest– Split half

• Validity: measuring what is intended to be measured– Face validity– Criterion validity: concurrent, predictive– Construct validity


Basic model of empirical studies

Independent variable/sConditions/Causes

Dependent variable/sEffects

Intervening variable/sConfounding variable/s


Quantitative versus qualitative approaches

• Quantitative approaches: measuring and counting phenomena+ precise, controlled, claims on causation/prediction

- oversimplification, no recognition of subjectivity/ individuality

• Qualitative approaches: capturing qualities of phenomena+ recognition of subjectivity/individuality, openness

- difficult to establish internal and external validity, difficult to separate data analysis and interpretation


Developing a study design (1)

• Deciding on overall objective:– Testing hypotheses (incl. evaluation)– Generating hypotheses– Describing a population– Testing methods

• Defining variables of interest– Analytical definition– Operational definition

determining methods for data collection


Developing a study design (2)

• Determine sample – Type: Random, stratified, cluster,

convenience, ...– Size

• Determine methods for data analysis

• Internal and external validity check

• Planning resources


Analytic/operational definition of variables

• Analytic definition:

Content/meaning of variable

• Operational definition:

Measurement of variable


Overview of methods for data collection

• Analysis of documents+ non-reactive, "condensed organizational knowledge"

- not aligned with purpose of the investigation

• Questionnaire + objective, applicable for large samples

- no control over the actual data collection, response biases

• Interview + control over data collection, complex issues possible

- resource-intensive, interviewer influences

• Observation + access to implicit knowledge, natural situation

- subjective meaning of the observed unknown, no control over the occurrence of the events under study


How the questions shape the answers (1)

• "Does it seem possible or does it seem impossible to you that the Nazi extermination of the Jews never happened?" 33% responded with "possible"

versus

"Do you believe the Holocaust happened?" 89% yes, 3.5% no, 7% don´t know

• Taboo behaviors: don´t ask if but when the behavior has been shown in the past


How the questions shape the answers (2) Own behavior

yes/no Percent yes

Own behavior frequency scale 1-5

Percent 4/5

Correct behavior yes/no

Percent yes

Correct behavior scale 1-5

Percent 4/5 Avoid use of spray cans 84 11 72 83 Bio-shopping during the last fortnight

16 9 96 41

Separate glass collection 96 93 100 89 Separate paper collection 94 95 98 85 Separate aluminium collection

78 62 90 69

Use compost 64 38 100 83 Collect batteries 92 98 100 91 Return unused drugs to pharmacy

32 33 91 69

Reduce water when showering/taking a bath

40 31 66 63


How the questions shape the answers (3)

Zur Anzeige wird der QuickTime™ Dekompressor „TIFF (Unkomprimiert)“

benötigt.

From: Schwarz, N. (1999). Self-reports. How the questions shape the answers. American Psychologist, 54, 93-105.


Exercise on defining variables

• Are part-time employees discriminated against in company X?

• Does downsizing reduce loyalty of remaining employees?

• Did a leadership training in company A improve employee motivation?

Which are the independent, dependent, intervening variables? How should the variables be defined analytically and operationally?


Overview of methods for data analysis

• Overall objective of data analysis– Description– Reduction– Preparation for interpretation and

generalization

• Quantitative analysis– Descriptive statistics– Inferential statistics

• Qualitative analysis


Basics on inferential statistics (1)

• Overall objective:– Estimating the likelihood of results found in

a particular sample occurring in the relevant population

– Accuracy of this estimation depends on sample size and variance of variables studied

– Error probability is controlled for both adopting a false hypothesis and rejecting a correct hypothesis



• Steps:– Determining null hypothesis and alternative

hypothesis – Calculate estimate for relevant parameter

(e.g. mean)– Calculate and compare test statistic and

critical value– Decide for/against null hypothesis



• Select appropriate tests– Based on scale level of variables (nominal,

ordinal, interval, ratio)– Based on distribution of variable (e.g. normal

distribution) – Based on research question, e.g.

Comparing means (two groups, e.g. t-test; more groups, e.g. analysis of variance)

Testing associations (e.g. Pearson correlation)


Reporting social science research• Overall objectives

– Consistent, transparent documentation of research process– Building trust in interpretation/conclusions

• Structure of report– Management summary– Introduction/rationale for study– Motivation for specific questions studied based on existing

knowledge– Methods: sample, data collection and analysis– Results– Interpretation/Conclusions– Appendix: Instruments used, raw data etc.


The seminar ... (from ASQ, 1979)

• Qualitative researcher: "Many people these days are bored with their work and are ..."

• Quantitative researcher (interrupting): "What people, how many, when do they feel this way, where do they work, what do they do, why are they bored, how long have they felt this way, what are their needs, when do they feel excited, where did they come from, what parts of their work bother them most, which ..."

• Qualitative researcher: "Never mind."

empirical methods – g. grote ethz, hs 07 obectives of the course basic knowledge of social science...

Documents