basic statistical tests

24/10/2013INTRODUCTION TO SCIENTIFIC THINKING AND RESEARCH PRACTISE

BASIC STATISTICAL ANALYSES AND INTERPRETATION

Takumi [email protected]

University of Lapland, Finland

This slide is available at...http://libero86pooh.wordpress.com/category/research/

mailto:[email protected]

mailto:[email protected]

http://libero86pooh.wordpress.com/category/research/

http://libero86pooh.wordpress.com/category/research/

OVERVIEW

OVERVIEW•Basic Statistical Tests

‣ Compare 2 groups‣ Compare 3+ groups

• Interpretation of the Results‣ Significance and p-value‣ p-value and sample size‣ Effect size

BASIC STATISTICAL TESTS

BASIC STATISTICAL TESTS

Compare

• T-test

• ANOVA• Multiple comparison

Correlation• Pearson correlation

• Spearman correlation

‣Statistical tests have two main groups.

T-test‘Compare means of two variables’

‣Independent t-teste.g.) TOEFL score in JPN and FIN

‣Dependent t-teste.g.) TOEFL score before and after preparation

Compare

ANOVA (Analysis of Variance)

‘Compare means of three+ variables’‣ Find out the difference in various groups

- e.g.) TOEFL score in JPN, FIN and CHN

※H1= There are difference somewhere in the variables.

cannot find out ‘which group’ have the difference

use Multiple comparison to clarify

Compare

Multiple comparison‘Compare means of three+ variables’

-However, it is different from ANOVA

Compare

0255075

100

JPN FIN CHN0

255075

100

JPN FIN0

255075

100

FIN CHN0

255075

100

JPN CHN

ANOVA Multiple comparison

DATA INTERPRETATION

Statistical significance

p=.0143


p=.0143?

p-value-Probability of the error

‣so, if p is very low...

possibility for error is very low

➡ The result will be reliable



Generally...

p<.05Significant

p-value


H0H1

p-value

p-value


H0H1

p= .35 → Not significant

p-value


H0H1

p= .05* → Significant

p-value


H0H1

p= .01** → Significant

DATA INTERPRETATION

n M SD Diff. p

AGroup1

p=0.137AGroup2 p=0.137

BGroup1 p=0.015BGroup2

p=0.015

These are 2 different results of t-test.

Q. Which of the tests has larger difference?A? B? or same?

Takeuchi and Mizumoto(2011)

DATA INTERPRETATION

n M SD Diff. p

AGroup1 50 40 10

3.0 p=0.137AGroup2 50 43 10

3.0 p=0.137

BGroup1 100 40 10

3.0 p=0.015BGroup2 100 43 10

3.0 p=0.015

The answer is... the same

because p is just the probability- p is NOT the actual difference


DATA INTERPRETATIONn M SD Diff. p

AGroup1 50 40 10

3.0 p=0.137AGroup2 50 43 10

3.0 p=0.137

BGroup1 100 40 10

3.0 p=0.015BGroup2 100 43 10

3.0 p=0.015

‣ M, SD, and Diff are the same score



AGroup1 50 40 10

3.0 p=0.137AGroup2 50 43 10

3.0 p=0.137

BGroup1 100 40 10

3.0 p=0.015BGroup2 100 43 10

3.0 p=0.015

‣ M, SD, and Diff are the same score‣ The different thing is sample size➡ p is dependent on sample size



AGroup1 50 40 10

3.0 p=0.137AGroup2 50 43 10

3.0 p=0.137

BGroup1 100 40 10

3.0 p=0.015BGroup2 100 43 10

3.0 p=0.015

The formula of t (independent)

t = x1 − x22S1

n1+

2S2n2


DATA INTERPRETATION The formula of t (independent)

‣ if n becomes larger, t will get larger

➡ p also gets lower

t = x1 − x22S1

n1+

2S2n2

DATA INTERPRETATION‣ Depending too much on p-value may lead to

FALSE INTERPRETATION➡ Actual difference is also important

So we need the value...- which shows the actual difference- which is independent of sample size

It is important to be able to tell whether it is the sample size or the coefficient that is making the difference. The effect size can do this.

(Cohen, Manion and Morrison, 2007: p.520)

Effect size

• There are several different calculations of effect size, for example (Richardson 1996; Capraro and Capraro 2002: 771): r2, adjusted R2, η2, ω2, Cramer’s V, Kendall’s W, Cohen’s d, and Eta. (Cohen, Manion and Morrison, 2007: p.51)

e.g.) Cohen’s d (frequently used for t-test)

Effect size

Independent of Sample size

d = 1M − 2MpooledSD

Effect sizeIn this result, effect size is...

n M SD Diff. p

A Group1 50 40 103.0 p=0.137A Group2 50 43 103.0 p=0.137

B Group1 100 40 103.0 p=0.015B Group2 100 43 103.0 p=0.015

d = 1M − 2MpooledSD

=310

= 0.3

The ES of A & B are same= Actual difference is same


Effect size

n M SD Diff. p

A Group1 50 40 103.0 p=0.137A Group2 50 43 103.0 p=0.137

B Group1 100 40 103.0 p=0.015B Group2 100 43 103.0 p=0.015

In this result, effect size is...

In Cohen’s d...‣ d=0.3 weak‣ d=0.5 moderate‣ d=0.8 strong

ES is ‘weak’


Effect sizeIn this result, effect size is...

n M SD Diff. p

A Group1 50 40 103.0 p=0.137A Group2 50 43 103.0 p=0.137

B Group1 100 40 103.0 p=0.015B Group2 100 43 103.0 p=0.015

In this result, effect size is...

In Cohen’s d...‣ d=0.3 weak‣ d=0.5 moderate‣ d=0.8 strong

ES is ‘weak’


Summary

• In the quantitative analysis, choosing appropriate test according to the research design will make the result better.

• Statistical analysis is conducted mainly by computer programme (SPSS, etc...), but it is important for the researchers to

ReferencesAmerican Psychological Association. (2010). Publication

manual of the American Psychological Association. Washington, DC: American Psychological Association.

Cohen, L., Manion, L. & Morrison, K. (2007). Research methods in education. London New York: Routledge.

Marczyk, G., DeMatteo, D. & Festinger, D. (2005). Essentials of research design and methodology. Hoboken, N.J: John Wiley & Sons.

Seale, C. (2012). Researching society and culture. London: SAGE.

Takeuchi,O. & Mizumoto, A. (2011). Introduction to effect size and power analysis. Reports of 2011 Studies in Japan Association for Language Education and Technology, Kansai Chapter, Methodology Special Interest Groups (SIG), 47-73.