research methods lecture 4
TRANSCRIPT
Research of the dayThe survival half-life of a box of chocolates in a hospital ward is 99 minutes.Gajendragadkar, P. R., Moualed, D. J., Nicolson, P. L., Adjei, F. D., Cakebread, H. E., Duehmke, R. M., & Martin, C. A. (2013). The survival time of chocolates on hospital wards: covert observational study. BMJ: British Medical Journal, 347.
Blue is the most attractive eye colour.http://yougov.co.uk/news/2014/09/11/full-results-eye-colour/
Homework ReviewThe Monty Hall problem:
Why is it always better to switch doors?
The Monty Hall ProblemYou are on a game show. The host (Monty) shows you three doors. Behind two doors there are goats. Behind one door there is a car.
A B C
… pick a door...Pick a door. You must tell Monty which door you picked. (Let’s imagine you picked A)
… goat number 1...Monty reveals one goat behind one of the other doors (not the door of your choice).
Change or Stay?Now Monty asks you, do you want to stick to your original door, or do you want to switch to the other one?
ProbabilityYou selected a door with a ⅓ chance of being correct.
Monty then removed one door.
There is now a ½ chance of being correct - this is a better chance than when you made your guess.
Therefore your guess is less likely to be correct, and you should switch.
Imagine the case of you selecting 1 door out of 1000, and Monty gradually removing all the others - your odds of having guessed right in the first place are tiny. You should not stick to the guess you made with such poor odds.
Empirical demonstrationIf you play the Monty Hall Problem game over and over again, you will win more often if you always switch your door.
You will loose more often if you always stay.
Recap●You write for yourself, as well as for an
audience●You must plan and structure your work●Simplicity is better than over-complication●Reading will improve your writing●Always present opposing viewpoints●Avoid logical fallacies
Research4. Experimentation
This work is licensed under the Creative Commons Attribution-ShareAlike 4.0 International License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/4.0/.
by Helena Hollis, 2014
HypothesisRemember Wilson’s quote:
“imagination is of the utmost importance. People differ enormously in their power to construct useful hypotheses, and it is here that true genius shows itself.” (Wilson, 1952, p26)
This applies to experimental approaches.
Scientific MethodExperiments are generated through curiosity, and then formalised through scientific method.
This has developed from philosophy (‘natural philosophy’).
Knowledge from experienceSome philosophers (notably Kant), believe that some knowledge is independent of experience.
This is known as a priori knowledge.
Eg: “All bachelors are unmarried men”
W. V. O. Q.“The lexicographer is an empirical scientist, whose business is the recording of antecedent facts; and if he glosses 'bachelor' as 'unmarried man' it is because of his belief that there is a relation of synonymy between these forms, implicit in general or preferred usage prior to his own work.”
(Quine, 1961)
QuineWillard Van Orman Quineargued that there is no such thing as a priori knowledge, but rather everything we know is known empirically.
This means everything we know is subject to revision.
(Quine, 1961)
...Hume is one of the most important figures in the advancement of what is known as science today.
You may not agree with everything he has to say, but empiricism, an approach he championed, is the key to our understanding of almost everything.
Empiricism“The Empiricism Thesis: We have no source of knowledge in S or for the concepts we use in S other than sense experience.”
(Markie, 2013)
Empirical informationEmpirical data, or information, is that which has been demonstrated and can be ‘experienced’.
Another way to put this, is that results can be observed.
This is what experiments aim to produce.
PurposeExperiments are conducted to support, or oppose hypotheses and theories.
They aim to give concrete, real-world demonstrations in order to do so.
Not every research project requires experimentation. But it does require experiment-literacy.
A warningExperiments can be much harder to do well than they may at first appear.
They are highly time consuming.
Think carefully before committing to conducting experiments in your research.
What is an experiment?“There is no clear-cut distinction between an experiment and a simple observation, but ordinarily in an experiment the observer interferes to some extent with nature and creates conditions or events favourable to his purpose.”
(Wilson, 1952, p. 28)
Scientific Method“Leave nothing to chance. Overlook nothing. Combine contradictory observations. Allow yourself enough time.”(Hippocrates, cited in Sagan, 1996)
VariablesVariables are the essential elements that come into play in your experiment, which can be manipulated to test your hypotheses.
Many variables are beyond our control, unknown, unpredictable, etc. No experiment is perfect. But you must be as aware as possible of as many variables as possible.
Controlling variablesAn ideal experiment will have all variables fixed, aside from the one we are aiming to measure.
In real life, it is more a question of having all other variables as fixed as possible.
Controls and comparisonA measurement taken in isolation does not really mean anything - we need to be able to compare it.
Experiments need control conditions, or control groups, against which they can be compared.
Qualitative / QuantitativeExperiments can aim to gather either kind of data, or both (remember lecture 1).
Qualitative: observing subjective responses, such as opinions and perceptions, which cannot be expressed numerically.
Quantitative: measuring outcomes in a way that can be expressed numerically.
ReplicationA key concept in science is the replication of results.
If one research group in one lab gets a particular set of results, that alone does not entitle us to firmly accept or reject their validity.
Others must run the same experiment, and if the same results are replicated, we can assume they are valid.
Contradictory results can spark interesting debate, and are often found when addressing complex, multi-variable issues.
Experiments with subjectsBoth qualitative, and quantitative research can involve subjects (people).
These experiments typically aim to investigate opinion, attitude, perception, or behaviour.
People complicate thingsPeople bring a huge amount of variables into your experiment based upon their individual traits and backgrounds.
You must be extremely careful when using human subjects.
Sample sizeGenerally speaking, the more people you use as subjects, the better.
Larger sample sizes help to balance out individual variation.
Small samples can still provide interesting information, but you will not be able to generalise from it and make big assumptions about the population in general.
SamplingSampling refers to the selection of your subjects.
It can be random, or directed.
For example, you may want to test a sample that reflects the general population, or you may want to test a specific group (eg: Sound Engineers).
Acquiring subjectsDon’t underestimate how hard it is to find willing guinea pigs for your experiments.
Plan well in advance, and have back-up options.
(Image: HTTP://XKCD.COM/749/)
Be aware of your sampleYou cannot generalise about the population at large from a small sample, or from a specific (ie non-random) sample.
Be very cautious of the kind of conclusions you draw from the sample you have.
Questionnaires / SurveysIn some cases, you may be using subjects without questioning them (ie observing their behaviour, measuring responses physiologically, etc.).
In many cases, you may require your subjects to tell you what they think / experience / feel etc. In this case you may use questionnaires.
StandardisationRemember that you want to keep as many variables stable in experiments as possible.
When conducting questionnaires, you therefore need to standardise as much as possible about how they take place.
Have your subjects in the same room, given precisely the same instructions, by the same person, in the same words, with the same tone, etc.
(May, 2001, pp.91 -92)
CommunicationWhether communicating in speech or writing with your subjects, you need to be very cautious of what you say.
You do not want to convey any bias.
If you are giving instructions, be clear, and objective.
Ideally, you should have a script that you stick to, as this helps with standardisation, and it also helps you to be very careful about your phrasing.
Language and bias“Asking respondents about their attitudes toward welfare will produce lower levels of support than asking about their attitudes toward assistance to the poor.”
(Ruane, 2005, p.72)
Question types●Factual
-How many hours do you typically spend gaming per week?-What is the last film you watched in a cinema?
●Opinion-On a scale of 1-10, where 1 is low and 10 is high, how much do you like the music of Kanye West?-Do you believe that global warming is caused by humans?
(May, 2001, pp.101 -102)
...● Open
Questions in which the participant is free to respond as they wish to (ie write or say an answer without pre-given options).
● ClosedQuestions were the participant is forced to select from pre-determined possible answers.
Closed questions are much, much easier to analyse. Use these rather than open questions as much as possible.
Likert ScalesThese are very useful for making questionnaires quantifiable.
Tips:● Don’t have a middle point. You
may end up with very neutral results.
● Make your scale idiot-proof: if numbered, it must be absolutely obvious what the numbers mean (ie high vs. low)
Test-runWith any experiment, things can go wrong.
You may think your instructions are clear, but your participants may misunderstand you. You may be using phrasing which leads people to give particular answers.
Always do a test-run of any experiment before the real thing.
(Your test-run must not use the same participants as the real experiment.)
Attitude and BehaviourA warning: people’s beliefs and attitudes do not necessarily allow us to predict their behaviour.
Even if your subjects are not motivated to lie on a survey, and we can assume that they believe everything they claim, we cannot make strong assumptions about how this reflects on them outside of the survey.
LaPierre’s journeyIn the 1930’s LaPierre sent a survey to many restaurants and hotels around America asking if they would accept Chinese guests. Almost all replied saying no.
He then travelled with two Chinese friends and visited many of those establishments. They were only refused entry at one hotel. All others accepted them, and behaved in a friendly manner towards them.
LaPierre (1934)
StatisticsStatistics are crucial for assessing the significance of your data.
This is too broad a topic to cover fully here, so if you want to run experiments and collect data you will need to inform yourself about statistical analysis.
Experimental write-upAn experiment needs to be written-up in a set manner, including:
1.Introduction2.Hypothesis3.Methods4.Results5.Discussion6.Conclusion
IntroductionAs usual, this needs to present some background, and orient the reader.
HypothesisState your hypothesis as clearly as possible.
You should also make it clear what would support, or disprove this hypothesis.
Comment on any theory the hypothesis is attached to.
MethodsThis will be an in-depth description of the methods used.
It needs to be systematic, clear, and honest.
Try to be as detailed as possible.
ResultsPresent your results as clearly as you can.
Charts and graphs are advisable.
Note that you do not need to present your raw data here, as this can be an appendix.
Data Visualisation
(McCandless, 2013)
DiscussionThis is where you talk about what you think your results mean.
Do they support or oppose your hypothesis?What conclusions is it possible to draw?Are there alternative interpretations?What might be problematic about your results?
ConclusionAs always, the conclusion sums up what you have found through your research.
Careful to only make the conclusions you are entitled to from your findings.
It may be worth commenting on future directions here.
ReferencesA/V Geeks (2012) Living In A Reversed World. Available at: http://www.youtube.com/watch?v=X5mjU3_vuvM (Accessed : 25th September 2013)
CollegeBinary (2009) Three Minute Philosophy - David Hume Available at: http://www.youtube.com/watch?v=r3QZ2Ko-FOg Accessed : 24th September 2013)
HeroicImaginationTV (2011) Stanley Millgram: Obedience to Authority Available at: https://www.youtube.com/watch?v=y9l_puxcrlM (Accessed: 13th September 2014)
Ichikawa, J. and Steup, M. (2013) "The Analysis of Knowledge", The Stanford Encyclopedia of Philosophy (Fall 2013 Edition), Edward N. Zalta (ed.), Available at:http://plato.stanford.edu/archives/fall2013/entries/knowledge-analysis/ (Accessed: 23rd September 2013).
LaPiere, Richard T. "Attitudes vs. actions." Social forces 13.2 (1934): 230-237.
Markie, Peter, "Rationalism vs. Empiricism", The Stanford Encyclopedia of Philosophy (Summer 2013 Edition), Edward N. Zalta (ed.), Available at: <http://plato.stanford.edu/archives/sum2013/entries/rationalism-empiricism/. (Accessed: 23rd September 2013).
May (2001) Social Research: Issues methods and process. 3rd ed. UK: Open University Press
Ruane, J. M. (2005) Essentials of Research Methods. Malden MA: Blackwell Publishing
Sagan, C. (1996) The Demon Haunted World. London: Headline Book Publishing
TED (2012) Molly Crockett: Beware neuro-bunk. Available at: https://www.youtube.com/watch?v=b64qvG2Jgro (Accessed : 25th September 2013)
Ted-ed (2012) The beauty of data visualization - David McCandless Available at: http://www.youtube.com/watch?v=5Zg-C8AAIGg (Accessed: 27th September 2013)
Wilson, E. B. (1952) An Introduction to Scientific Research. New York: Dover PublicationsW.V.O. Quine, From a Logical Point of View (Harvard University Press, 1953; second, revised, edition 1961)