Conducting an empirical investigation

Goals of scientific research

• Description of the data

• Explanation of the data

• Predictions of future data

Criteria of scientific research

• Objectivity• Repeatability• Reliability• Internal validity• External validity• Occam’s Razor

(1) a. When we arrived in Berlin it was dark.

b. It was dark when we arrived in Berlin.

(2) a. Because he didn’t sleep much Peter is tired.

b. Peter is tired because he didn’t sleep much.

(3) a. If it doesn’t rain we'll go to the party.

b. We'll go to the party if it doesn’t rain.

Example: Positiong of ADV-clauses

Literature research

• Search in your library.

• Search in special bibliographies.

• Search for recent publications on your topic.

• Homepage of researchers who have worked on the topic.

• Random search on the internet.

From file <adam43.cha>*CHI: oh # Mommy # I think I have +...*CHI: I think I've [=? I] finished with that .*CHI: I think I've [=? I] finished # Mommy .*CHI: think Robin uses those .*CHI: I think we finished with all dose .*CHI: I think we finished .*CHI: Mom # I think we really [?] finished .*CHI: I think I got it .*CHI: I don't think I have it in de right way .*CHI: I think I do it the other # way .*CHI: I think I finished playing with this .*CHI: I think this bag has a top .*CHI: I think I use this for paper .*CHI: I think we use some different colors # alright ?*CHI: I think it is # but it is on a part .

Impressionistic inspection of the data

If you want to conduct a questionnaire study or an

experiment, you should always start with a pilot study.

Informal pretest

Informal formulation of your theory

Linguistic structures are motivated by underlying

cognitive and functional pressures. Since there are

always multiple pressures (e.g. economy, expressivity,

processing efficiency), there is usually competition

between factors. - The positioning of adverbial clauses

is motivated by competing cognitive and functional

factors:

Informal formulation of your theory

Possible factors influencing the positioning of ADV-clauses:

– Meaning: conditional > temporal > causal– Length: short-initial vs. long-final– Information: old-initial vs. new-final– Subject: same subject-initial vs. different-subject-final– Iconicity: matching order of events and clauses– Genre: scientific > fictional > colloquial discourse

Is your theory falsifiable?

Some theories are not falsifiable:

(1) Language change is progress (or decay).

(2) Cognitive and linguistic development are related / are not related.

Is your theory compatible with other theories/findings?

Many linguists assume that ordering patterns are determined by competing functional and cognitive pressures; but in some languages adverbial clauses almost always precede the main clause (e.g. Japanese). Are the Japanese data compatible with your hypothesis about the positioning of adverbial clauses in English?

Statement of hypothesis

There are two types of empirical studies:

1. Studies testing a given hypothesis

2. Explorative studies

Statistical tests may only be used for hypothesis testing; i.e. do not use statistical tests to test data from an explorative study.

Statement of hypothesis

State your hypothesis informally in terms of a conditional sentence:

If x then yThe more x the more y

X and y are called variables: x is the independent variable, and y is the dependent variable.

Statement of hypothesis: Meaning

i. If the adverbial clause has conditional meaning, it predominantly precedes the main clause.

ii. If the adverbial clause has causal meaning, it predominantly follows the main clause.iii. If the adverbial clause has temporal meaning, it is

about equally frequent before and after the main clause.

Statement of hypothesis: Length

The longer the adverbial clause, the more often it follows the main clause.

Statement of hypothesis: Information

i. If the adverbial clause resumes information from the preceding discourse it tends to precede the main clause.

ii. If the adverbial clause expresses new information it tends to follow the main clause.

Statement of hypothesis: Subject

If the adverbial clauses includes the same subject as the associated main clause it precede the main clause more often than if it includes a different subject.

Statement of hypothesis: Iconicity

If the adverbial clauses expresses an event prior to the event expressed in the main clause it precedes the main clause more often than if the adverbial clauses expresses an event posterior to the event expressed in the main clause.

Statement of hypothesis: Genre

i. In scientific texts, adverbial clauses precede the main clause more often than in fictional texts.

ii. In fictional texts, adverbial clauses precede the main clause more often than in colloquial

discourse.

Experimental hypothesis vs. Null hypothesis

Every hypothesis has an alternative.

Experimental hypothesis:The longer the adverbial clause, the more often it follows the main clause.

Null hypothesis:There is no relationship between the length and positioning of the adverbial clause.

Operationalization of variables

Meaning

• You look up the meaning in a dictionnary.• You analyze each sentence in its discourse

context.• You ask people to judge the meaning (directly or

indirectly).

Operationalization of variables

Length

• Number of words / syllables• Milliseconds

Operationalization of variables

Information status

• Consider each sentence in its context.

Is the information status categorical or scalar?

Operationalization of variables

Subject

• Direct coreference (e.g. Peter … if he …)• Associative link (e.g. The house … if the door …

Operationalization of variables

Iconicity

Is iconicity relevant to conditional and causal clauses?

(1) The accident happened because the driver lost

control of his vehicle.

How do you treat sentences with overlapping events?

(2) When we arrived in Jena, it was raining.

Selection of statistical measure/test

• Type of data (nominal, ordinal, interval)• Type of hypothesis (difference between groups,

correlation)• Experimental design (related vs. independent)• Data distribution (normal vs. not normal)• Number of items and number of subjects

Structure of an empirical research paper

Introduction

Background and preview

• Why do you conduct the study?• What makes the study interesting?• What is the goal/purpose of your study?

Introduction

Linguistic phenomenon

• Introduce the phenomenon you want to investigate (e.g. relative clauses, acquisition of verb-particle constructions).

• Characterize the most important properties of the phenomenon (notably the properties that are relevant to your hypothesis).

• Provide examples.

Introduction

Literature review

If there are previous studies, summarize their main findings and state what you intend to do in your study:

If there are no previous studies, make that clear: “This is the first study to investigate ….”

1. Look at a new aspect2. Challenge a hypothesis3. Test/replicate a previous study

Introduction

Preview / explicit hypothesis

• Introduce the phenomenon you want to investigate (e.g. relative clauses, acquisition of verb-particle constructions).

• Characterize the most important properties of the phenomenon (notably the properties that are relevant to your hypothesis).

• Provide examples.

MethodCorpus study

• Characterize the corpus (size, type, etc.)• Characterize the data you have compiled from the

corpus (total number of ADV-clauses, number of children/subjects)

• Describe the treatment of problematic cases and cases you have excluded (e.g. ADV-clauses inserted in the main clause)

• Describe coding and provide examples of each category

MethodExperimental /questionnaire study

• Characterize subjects (e.g. how many, selection procedure)

• Describe the questionnaire• Describe materials you have used in your study

(e.g. toy animals)• Describe the procedure (i.e. how did you collect

the data)• Describe coding• Describe treatment of problematic cases

Results

Descriptive summary

• Present tables and figures• Describe the results in tables and figures

Results

Inferential statistics

• Say what type of tests you have used and present the relevant measures (e.g. p-value, F-value, degrees of freedom, effect size, confidence intervals).

• If necessary, explain your choice of test.• State what the result of the statistical analysis

suggests.

Discussion

• Provide a short summary of your results.• If possible consider your paper from a broader

theoretical perspective and mention implications of your study for related questions.

• Future direction of research: Mention open questions: What should be done in the next step? Ideas for an experiment. etc.

Appendix

If the data are too comprehensive to be included in the text, include them in the appendix. If the data are very comprehensive, you might only present samples of your data in the appendix.