Qualitative data at your fingertips?

Dr Chris Mann

Qualitative research online needs researchers to be:

• Technically astute

• relationally astute: the human interviewer 'can be a marvelously smart, adaptable, flexible instrument who can respond to situations with skill, tact, and understanding’ (Seidman 1991) ……….. but the challenge is to transfer these qualities online.

Face-to-face cues

• Visual: Age, gender, ethnicity, dress, grooming, conventionality, eccentricity, confidence levels, friendliness, restraint

• Oral: accent - plus pitch and tone suggesting confidence, doubt, humour, irony

We analyse each other and assess:

• If there are signs of mutual understanding• if we can trust the other person’s sincerity and

motivation• if there is sufficient rapport to allow a meaningful

research interaction to take place

Is such rapport possible online?

• SOME SAY NO - ‘the lack of tone or gesture and the length of time between exchanges can lead to something of a formal, structured interview. In contrast to the spontaneous speeding up, slowing down, getting louder, getting quieter, getting excited, laughing together, spontaneous thoughts, irrelevant asides etc. etc. which I have experienced in off-line interviews. [The latter have] FLOW, and DYNAMICS, both of which contribute to greater depth and quality of information in an off-line interview than over e-mail. (Hodkinson 1999: e-mail note)

Some say yes...

• Is rapport online possible? Absolutely!!!! Rapport comes from being very up front with what you are doing and responding as you would with anyone. Laughing, listening and connecting are the key. (Smith-Stoner 1999: e-mail note)

Debate: does online communication have a narrow bandwidth?

• Bandwidth; the ‘volume of information per unit time that a computer, person, or transmission medium can handle’ (Raymond cited in Kollock and Smith 1996).

• ONE VIEW: If investigations seek ‘information that is ambiguous, emphatic, or emotional ... a richer medium should be used.’ (Walther 1992)

Another view..

• The same motives that drive people to communicate in other media drives them to communicate online. Even using a computer they will ‘desire to transact personal, rewarding, complex relationships and ... they will communicate to do so’ (Walther 1992).

Can we increase rapport working online by using linguistic conventions?

• Electronic paralanguage: repetitions, abbreviations and verbal descriptions of feelings and sounds: ‘hehehe’ for laughter - ‘lol’ for ‘lots of laughs’ or ‘LJATD’ for ‘let’s just agree to disagree’.

• 'Emoticons' :-) for smiling/happy; :-o for surprise/shock

But paralanguage might be a blunt instrument in qualitative research

• It may not be transparent e.g. across cultures. In Japan signs of respect are finely graduated so a highly complex system of emoticons attempts to parallel some of the delicacy of FTF interaction. These emoticons are not familiar to most western interviewers - they are read in a traditional horizontal format rather than sideways: [ -o-) I’m sorry; (^o^;) excuse me!; (^o^) happy; (^-^;; ] awkward (Aoki 1994).

Paralanguage may be seen as inappropriate for research

• Seeming friendly but impersonal• lazy and unimaginative• Just not subtle enough: ‘You’ll see people

annotate their mails using smilies, HTML-style tags, capital letters, etc., but even so there is no reliable way of conveying tone’. (undergraduate quote from Mann and Stewart)

So what works?

• A shared research agenda• Participants with a superficial interest in the

research topic may be initially intrigued by interacting online but if they have no particular vested interest in a study, or a low boredom threshold, there may be a tendency for them to drop away:

So what works?

Differing experiences...

• Usually I found that people lost interest before I am able to get to the same degree of detail as a face to face interview. (Hodkinson 1999)

• Respondents often spoke of the value of our dialogues for helping them to make sense of their lives. They remarked on the time they had taken in thinking through their responses (some taking several hours) and messages were usually very long. (Dunne 1999)

Can online interaction sustain personal relationships over time?

• Long term relationships can be part of the research design (with relational bonds strengthened through online disclosure, phone calls, exchange of photographs, FTF meetings) - or can develop spontaneously.

• An intense online relationship can peak or falter• A less intense relationship may be more

sustainable

Developing rapport through interactive skills

• reassurance - frequent and explicit verbal assurances from researcher

• online ‘listening’• verbal expertise• explaining absences

Online listening - words not silence

• the pause can be seen as inattentiveness - indifference - absence

• online listening: ‘responding promptly to questions, overtly expressing interest in particular points made, asking follow-up questions, or perhaps enthusiastically sharing similar experiences to that described by the interviewee’ (Hodkinson 1999).

Online listening - words not silence

listening’ to the written scriptlistening’ to the written script

• being alert to changes in the tone of the conversation, a break in the flow, verbal ‘cues’ when people want to be invited to expound on a topic - “let me know if you want to know more” or bringing the subject up more than once.

Verbal expertise

• Being direct: ‘If there are questions you find odd, or that you do not “appreciate”, please do not hesitate to tell me. That is the only way to make me learn or to avoid repeating the mistake’(Ryen 1999)

• Being tactful: using mild imperatives (for example ‘you may want to check out’ ) and mitigation (‘if you’d like..’)

Verbal expertise

Being self-reflexive

• There were occasions when we were ‘lost for words’, taking some time to decide on what to send as a message, because we felt like our written comments sounded banal or our questions too direct and leading. (O'Connor and Madge 2000).

Explaining absences

• Neglecting to explain absences can seriously jeopardize rapport: ‘when I was ill in bed I still had to check in with my co-investigators; write replies, explain that I was ill and that my conversations would only be short, but thus maintaining the link between us. (Bennett 1998)

Explaining absences

Developing rapport also includes building up trust by acting ethically

• But generating data online presents unprecedented ethical concerns and challenges for the 21st century researcher

Most distinctive features of the virtual setting impact on ethics

• The geographic dispersal of users • The blurred distinction between public and private

domains• The ease of anonymity or pseudonymity• The ability to record and archive communications

without consent• The ability to track participants using the

technology

Generating data face to face we seek to:

• protect subjects from risk - minimizing potential harm resulting from exposure to research methods or results

• assure that deception or fraud do not occur • assure that privacy is protected

Generating data online

• Requires knowledge of legal constraints• Interrelates technology and ethics in an

unprecedented way - and takes place in an technological environment that is constantly evolving

• raises new questions about privacy and consent• Requires protection for the institution and

legitimisation for the researcher

Legal considerations

• Researchers need to apprise themselves of data protection laws, privacy laws and copyright laws relating to the internet.

• Geographical dispersal may mean that the research cross into the legal jurisdiction of more than one country - with different internet laws.

Technical expertise is needed for

• Storage, transmission and destruction of data

• Design of Informed consent procedures that have authenticity and legality (e.g. organising ‘signatures’ for consent forms)

• Briefing and debriefing a ‘virtual’ and often transient research population

• Protecting participants, particularly vulnerable groups

• Establishing the validity of online qualitative and quantitative research methods

New questions about consent

• How do you gain consent from a ‘transient’ population or people using pseudonyms

• How do you find ways to let people withdraw from research in a setting where you can’t banish them?

New questions about privacy

• Is the online world a private or public space

• Who ‘owns’ the words we find on the Net? The person who wrote them? Anyone who reads them?

• Is it OK for researchers to ‘harvest’ words others have written - without their agreement?

One viewpoint...

• A ‘conversation’ includes …– “The right of a set of individuals once engaged in talk

to have their circle protected from entrance and overhearing by others.”

• The offence of ‘overhearing’ …– “Takes from an individual information not intended to

be overheard, it penetrates territory the speakers have claimed as their own, and it defiles that territory by intruding on it.” (Goffman, 1971)

Protecting institutions

• An institution may carry responsibility for Internet research conducted by an individual

• An institution’s name and ‘label’ may be attached to flawed research

• The institution’s computer system may be used inappropriately or dangerously by individuals

Individual researchers depend on:

• The legitimacy offered by a reputable institution to verify their own identity and reliability

• In-house legal advice relating to Internet use• Technical expertise of computer experts to help

design ethical projects that engage with issues such as security of data and confidentiality of participants

"Bogus," "error-ridden," and "just plain wrong" are only a few of the terms being used by journalists and Net experts to describe both the "Cyberporn" cover story in the 3 July issue of Time and the study it used as a story hook. In this special section, HotWired destroys both story and study.

Rimm's Fairy Tales.How Martin Rimm duped Time and the Georgetown Law Journal.

The Shoddy Study.Donna L. Hoffman and Thomas P. Novak take apart the article's foundation.

                                                                                                                                              Real-time Peer Review.Moments after the cyberporn story was released, members of The Well's media conference began deconstructing the Time story and Rimm's study, forcing Time's Philip Elmer-DeWitt to defend them.

What went wrong?

• Generalized claims without supporting data• Invasion of privacy• Seen as deceptive• Dissemination laid subjects open to harm

Immediate fall-out: Rimm's Index

• Number of people thanked who distanced themselves from the study: 9 (of 25)

• Number of weeks before Time distanced itself: 3• Number of critiques of the study: 10• Number of web sites about the study: 9• Number of online newspaper articles questioning the study: 16• Number of times the words "Carnegie Mellon study" appear in the

study: 22• Number of days CMU's Committee of Inquiry took to report: 22

Jonathan Hardwick, Carnegie Mellon University

Long-term fall-out

• It seems indisputable that the study to which Carnegie Mellon University lends its name and its credibility contains disturbing ethical lapses. These lapses seem sufficiently serious that they should be of concern to both the CMU administration and to social scientists and computer professionals elsewhere.

Jim Thomas, Professor of sociology/ criminal justice, North Illinois university

Lesson from the Cyberporn study: researchers need an ethical framework

• Some might argue that the principle investigator bears the responsibility for the ethical lapses. Perhaps. But…the faculty advisor and oversight committees within an institution's administration are ultimately responsible. It is the principle faculty advisor who bears the immediate responsibility for socializing and mentoring the student into the world of empirical research, and this socialization includes imparting ethical precepts.

Jim Thomas

Personal risks for participants

Risks of Internet research

Protection for the most vulnerable

Legal risks

Technical risks

Knowledge about the potential for exposure

Protection from exposure

Debriefing possibilities

Protection from harrassment, abuse and deception

Understanding of legal responsibilities

Privacy

Security