04/21/23 HK 396 - Dr. Sasho MacKenzie 1

Comparing Means from Comparing Means from Two Data SetsTwo Data Sets

The t-testThe t-test

04/21/23 HK 396 - Dr. Sasho MacKenzie 2

Research QuestionsResearch Questions• To improve muscular power, should an athlete To improve muscular power, should an athlete

perform heavy resistance exercises, or light perform heavy resistance exercises, or light plyometric exercises?plyometric exercises?

• Is it better to imagine the flight of the ball or the Is it better to imagine the flight of the ball or the actions of your swing prior to striking a golf ball?actions of your swing prior to striking a golf ball?

• Is running 5 km or walking 5 km better for Is running 5 km or walking 5 km better for burning calories?burning calories?

• Do golfers sink more putts if they focus on the Do golfers sink more putts if they focus on the hole or on the ball during a putt?hole or on the ball during a putt?

• Will squatting to a lower depth during a vertical Will squatting to a lower depth during a vertical jump improve performance?jump improve performance?

04/21/23 HK 396 - Dr. Sasho MacKenzie 3

The t-testThe t-test• All of the questions posed on the previous slide can All of the questions posed on the previous slide can

be statistically addressed using the be statistically addressed using the t-test.

• A t-test determines if two groups of data are A t-test determines if two groups of data are significantly different (not meaningfully different).significantly different (not meaningfully different).

• A t-test is the ratio of the actual difference between A t-test is the ratio of the actual difference between two means to the difference that is expected due to two means to the difference that is expected due to chance alone.chance alone.

• The bigger the actual difference is compared to the The bigger the actual difference is compared to the expected difference due to chance, the more expected difference due to chance, the more statistically significant the t-test.statistically significant the t-test.

04/21/23 HK 396 - Dr. Sasho MacKenzie 4

The t-testThe t-test

• A t-test calculation produces a value (t-statistic) A t-test calculation produces a value (t-statistic) that is similar to a that is similar to a z-score..

• The The t-distributions, are also very similar to the , are also very similar to the z-score distribution (normal distribution).z-score distribution (normal distribution).

• The t-distribution changes depending on the The t-distribution changes depending on the sample size.sample size.

04/21/23 HK 396 - Dr. Sasho MacKenzie 5

The t Distributions (3 The t Distributions (3 examples)examples)

-3-3 00-1-1-2-2 11 3322-4-4 44

tt

N = 60 (same as normal curve)N = 60 (same as normal curve)

N = 10N = 10

N = 3N = 3

E.g., Area beyond t=3 E.g., Area beyond t=3 increases as N decreasesincreases as N decreases

04/21/23 HK 396 - Dr. Sasho MacKenzie 6

Let’s use an ExampleLet’s use an Example• Question: Do HK students drink more or less Question: Do HK students drink more or less

alcohol than the average St.FX student?alcohol than the average St.FX student?• Assumptions:Assumptions:

– Every student on campus honestly completed a form Every student on campus honestly completed a form and the average drinks/week is known.and the average drinks/week is known.

– Therefore, we know the mean of the population.Therefore, we know the mean of the population.

• Methods:Methods:– Determine the drinks/week for a sample of HK Determine the drinks/week for a sample of HK

students.students.– Determine if the sample mean is “different” than the Determine if the sample mean is “different” than the

population mean (perform a t-test).population mean (perform a t-test).

04/21/23 HK 396 - Dr. Sasho MacKenzie 7

What is “different”?What is “different”?

• Before the t-test, we must set a standard for Before the t-test, we must set a standard for statistical significance.statistical significance.

• This means determining the chance of error we are This means determining the chance of error we are willing to have in our final decision.willing to have in our final decision.

• I.e., How confident do we want to be in our I.e., How confident do we want to be in our decision that HK students drink a different amount?decision that HK students drink a different amount?

• This decision is represented by alpha (This decision is represented by alpha (), which is ), which is typically set at .05 (5%). This value is arbitrary.typically set at .05 (5%). This value is arbitrary.

• Assume no difference and that the study is Assume no difference and that the study is repeated 100 times. On 5 occasions, due to repeated 100 times. On 5 occasions, due to chance, we would incorrectly find that HK students chance, we would incorrectly find that HK students drink more.drink more.

04/21/23 HK 396 - Dr. Sasho MacKenzie 8

One-sample t-testOne-sample t-test

• We will use what’s called a one-sample t-test.We will use what’s called a one-sample t-test.• This compares the mean of a sample to the This compares the mean of a sample to the

mean of a population.mean of a population.

SEM

Xt

XX = = sample meansample mean• = population mean= population mean• SEMSEM = standard error of the mean= standard error of the mean

Actual differenceActual difference

Expected difference due to chanceExpected difference due to chance

04/21/23 HK 396 - Dr. Sasho MacKenzie 9

The HypothesisThe Hypothesis

• In statistics you must clearly state a testable In statistics you must clearly state a testable hypothesis.hypothesis.

• Typically the hypothesis tested is opposite to Typically the hypothesis tested is opposite to what you expect and is referred to as the what you expect and is referred to as the null hypothesis..

• Our null hypothesis is that HK students do not Our null hypothesis is that HK students do not drink a different amount than the average drink a different amount than the average university student.university student.

XX = = or or XX - - = = 00

04/21/23 HK 396 - Dr. Sasho MacKenzie 10

The CalculationThe Calculation• University PopulationUniversity Population

– Average drinks per week = 10Average drinks per week = 10

• HK Sample of studentsHK Sample of students– Mean = 12, SD = 5, N = 30Mean = 12, SD = 5, N = 30

19.2305

1012

SEM

Xt

• The odds of getting a t stat this big, or bigger, The odds of getting a t stat this big, or bigger, due to chance would then be determined by due to chance would then be determined by calculating a p-value. calculating a p-value.

04/21/23 HK 396 - Dr. Sasho MacKenzie 11

The P-valueThe P-value

• In Excel, the function TDIST() can be used to In Excel, the function TDIST() can be used to calculate the p-value. calculate the p-value.

• The degrees of freedom are N-1.The degrees of freedom are N-1.• Our example is a two-tailed test because HK Our example is a two-tailed test because HK

students may drink more, or less, than the students may drink more, or less, than the average. I.e., the sample mean could be either average. I.e., the sample mean could be either more or less than the population.more or less than the population.

• Since we set alpha = .05, if the p-value is less Since we set alpha = .05, if the p-value is less than .05, we will state HK students are than .05, we will state HK students are statistically different.statistically different.

04/21/23 HK 396 - Dr. Sasho MacKenzie 12

Graphic of two-tailed one sample Graphic of two-tailed one sample t-testt-test

-3-3 00-1-1-2-2 11 3322-4-4 44

tt

t distribution for N = 30t distribution for N = 30

Combined area beyond Combined area beyond t=2.19 and t = -2.19 is .037t=2.19 and t = -2.19 is .037

From TDIST, p = .037

04/21/23 HK 396 - Dr. Sasho MacKenzie 13

ConclusionConclusion

• Since p=.037 is less than alpha = .05, we reject the null Since p=.037 is less than alpha = .05, we reject the null hypothesis and conclude that HK students consume hypothesis and conclude that HK students consume significantly more drinks per week.significantly more drinks per week.

• The following shows how this would be explained in a The following shows how this would be explained in a study.study.

• It was determined that the average number of alcoholic It was determined that the average number of alcoholic drinks consumed by HK students (12 drinks), per week, drinks consumed by HK students (12 drinks), per week, was significantly more than the typical university student was significantly more than the typical university student (10 drinks), t(29) = 2.19, p = .037.(10 drinks), t(29) = 2.19, p = .037.

04/21/23 HK 396 - Dr. Sasho MacKenzie 14

Independent t-testIndependent t-test

• Determines if two sample means are Determines if two sample means are statistically different.statistically different.

• The null hypothesis is that the means come The null hypothesis is that the means come from the same population, from the same population, XX11 - -XX22 = 0. = 0.

• The bottom part of the t-stat now reflects the The bottom part of the t-stat now reflects the SEM for both samples, but is still a measure SEM for both samples, but is still a measure of how much you could expect the means of of how much you could expect the means of two samples from the same population to two samples from the same population to differ due to chance.differ due to chance.

04/21/23 HK 396 - Dr. Sasho MacKenzie 15

The EquationThe Equation

• The stuff on the bottom of the equation is The stuff on the bottom of the equation is called the standard error of the difference.called the standard error of the difference.

22

21

21

SEMSEM

XXt

04/21/23 HK 396 - Dr. Sasho MacKenzie 16

Independent t-test Independent t-test exampleexample

• Do HK students drink more or less than Do HK students drink more or less than Chemistry students?Chemistry students?

• Null Hypothesis: HK students and Null Hypothesis: HK students and Chemistry students drink the same Chemistry students drink the same amount of alcohol per week.amount of alcohol per week.

04/21/23 HK 396 - Dr. Sasho MacKenzie 17

The CalculationThe Calculation• HK sample of studentsHK sample of students

– Mean = 12, SD = 5, N = 30Mean = 12, SD = 5, N = 30

• Chemistry sample of studentsChemistry sample of students– Mean = 10, SD = 3, N = 30Mean = 10, SD = 3, N = 30

• The odds of getting a t stat this big, or bigger, The odds of getting a t stat this big, or bigger, due to chance would then be determined by due to chance would then be determined by calculating a p-value. calculating a p-value.

88.1

55.91.

1012222

221

21

SEMSEM

XXt

04/21/23 HK 396 - Dr. Sasho MacKenzie 18

Graphic of two-tailed Graphic of two-tailed independent sample t-testindependent sample t-test

-3-3 00-1-1-2-2 11 3322-4-4 44

tt

t distribution for N = 60t distribution for N = 60From TDIST, p = .065

Combined area beyond Combined area beyond t=1.88 and t = -1.88 is .065t=1.88 and t = -1.88 is .065

04/21/23 HK 396 - Dr. Sasho MacKenzie 19

ConclusionConclusion

• Since p=.065 is greater than alpha = .05, we Since p=.065 is greater than alpha = .05, we cannot reject the null hypothesis. There is cannot reject the null hypothesis. There is not enough evidence to suggest HK not enough evidence to suggest HK students drink more or less than Chemistry students drink more or less than Chemistry studentsstudents

• In a study it would be written as:In a study it would be written as:• It was determined that the average number It was determined that the average number

of alcoholic drinks consumed by HK of alcoholic drinks consumed by HK students (12 drinks), per week, was not students (12 drinks), per week, was not significantly different than the Chemistry significantly different than the Chemistry students (10 drinks), t(58) = 1.88, p = .065.students (10 drinks), t(58) = 1.88, p = .065.

04/21/23 HK 396 - Dr. Sasho MacKenzie 20

Dependent (Paired) t-testDependent (Paired) t-test• Determines if two Determines if two correlatedcorrelated sample means are sample means are

statistically different.statistically different.• Required when the same subjects are Required when the same subjects are

measured twice. E.g., Pre-test, Post-test study.measured twice. E.g., Pre-test, Post-test study.• Adjustments are made in how the variability Adjustments are made in how the variability

(SD) in the sample data is calculated. This (SD) in the sample data is calculated. This reduces the denominator in the t-statistic and reduces the denominator in the t-statistic and therefore increases the t-statistic.therefore increases the t-statistic.

• This accounts for reduction in the t-statistic due This accounts for reduction in the t-statistic due to the fact that the same subjects measured to the fact that the same subjects measured twice will show a smaller mean difference than twice will show a smaller mean difference than two completely separate groups. two completely separate groups.

04/21/23 HK 396 - Dr. Sasho MacKenzie 21

The DifferenceThe Difference

• Before any group means or standard deviations Before any group means or standard deviations are calculated, the difference scores between the are calculated, the difference scores between the two measurement times is determined.two measurement times is determined.

• For example, if you have a column of pre-test For example, if you have a column of pre-test scores and a column of post-test scores, then scores and a column of post-test scores, then generate a third column of post minus pre scores.generate a third column of post minus pre scores.

• The t-statistic is then calculated using information The t-statistic is then calculated using information from the column of difference scores.from the column of difference scores.

04/21/23 HK 396 - Dr. Sasho MacKenzie 22

The EquationThe Equation

• The variables in this equations come from a The variables in this equations come from a single column of difference scores.single column of difference scores.

pairsDiff

Diff

NSD

Xt

/

04/21/23 HK 396 - Dr. Sasho MacKenzie 23

Dependent t-test exampleDependent t-test example

• Do HK students drink more alcohol on the Do HK students drink more alcohol on the Saturday prior to a Biomechanics midterm, or Saturday prior to a Biomechanics midterm, or on the following Saturday?on the following Saturday?

• Null Hypothesis: HK students drink the same Null Hypothesis: HK students drink the same amount or less on the following Saturday, amount or less on the following Saturday, compared to the Saturday preceding a compared to the Saturday preceding a Biomechanics midterm.Biomechanics midterm.

04/21/23 HK 396 - Dr. Sasho MacKenzie 24

The Data: Number of The Data: Number of DrinksDrinks

10NSubjectSubject Sat. BeforeSat. Before Sat. AfterSat. After DiffDiff

11 1616 2121 5.05.0

22 66 99 3.03.0

33 1818 2525 7.07.0

44 1111 1010 -1.0-1.0

55 77 77 0.00.0

66 77 44 -3.0-3.0

77 88 1212 4.04.0

88 88 1010 2.02.0

99 1414 1515 1.01.0

1010 1212 1515 3.03.0

1.2DiffX

0.3DiffSD

04/21/23 HK 396 - Dr. Sasho MacKenzie 25

The CalculationThe Calculation• Difference Scores (Before – After)Difference Scores (Before – After)

– Mean = 2.1, SD = 3.0, N = 10Mean = 2.1, SD = 3.0, N = 10

• The odds of getting a t stat this big, or bigger, The odds of getting a t stat this big, or bigger, due to chance would then be determined by due to chance would then be determined by calculating a p-value. calculating a p-value.

24.2100.3

1.2

pairsDiff

Diff

NSD

Xt

04/21/23 HK 396 - Dr. Sasho MacKenzie 26

The P-valueThe P-value

• In Excel, the function TDIST() can be used to In Excel, the function TDIST() can be used to calculate the p-value. calculate the p-value.

• The degrees of freedom are (NThe degrees of freedom are (Npairs pairs - 1).- 1).

• Our example is a one-tailed test because we are Our example is a one-tailed test because we are assuming HK students drink more following a assuming HK students drink more following a midterm. This may not be a good assumption, midterm. This may not be a good assumption, but I needed a one-tailed example.but I needed a one-tailed example.

• Set alpha = .05, if the p-value is less than .05, Set alpha = .05, if the p-value is less than .05, we will state HK students drink significantly we will state HK students drink significantly more following a biomechanics midterm.more following a biomechanics midterm.

04/21/23 HK 396 - Dr. Sasho MacKenzie 27

Graphic of one-tailed Graphic of one-tailed dependent sample t-testdependent sample t-test

-3-3 00-1-1-2-2 11 3322-4-4 44

tt

t distribution for N = 10t distribution for N = 10From TDIST, p = .0258

The area beyond The area beyond t = 2.24 is .0258t = 2.24 is .0258

04/21/23 HK 396 - Dr. Sasho MacKenzie 28

ConclusionConclusion• Since p=.0258 is less than alpha = .05, we Since p=.0258 is less than alpha = .05, we

reject the null hypothesis and conclude that HK reject the null hypothesis and conclude that HK students consume significantly more drinks on students consume significantly more drinks on the Saturday following a midterm.the Saturday following a midterm.

• The following shows how this would be The following shows how this would be explained in a study.explained in a study.

• It was determined that HK students consume It was determined that HK students consume significantly more alcoholic drinks (2.1 more) on significantly more alcoholic drinks (2.1 more) on a Saturday after a midterm than on a Saturday a Saturday after a midterm than on a Saturday before a midterm, t(9) = 2.24, p = .0258.before a midterm, t(9) = 2.24, p = .0258.

04/21/23 HK 396 - Dr. Sasho MacKenzie 29

What if the t-test was two-What if the t-test was two-tailed?tailed?

• The null hypothesis would The null hypothesis would notnot be: HK be: HK students drink students drink the same amount or lessthe same amount or less on on the following Saturday, compared to the the following Saturday, compared to the Saturday preceding a Biomechanics midterm.Saturday preceding a Biomechanics midterm.

• But rather it would be: HK students drink But rather it would be: HK students drink the the same amountsame amount on the following Saturday, on the following Saturday, compared to the Saturday preceding a compared to the Saturday preceding a Biomechanics midterm.Biomechanics midterm.

04/21/23 HK 396 - Dr. Sasho MacKenzie 30

Graphic of two-tailed Graphic of two-tailed dependent sample t-testdependent sample t-test

-3-3 00-1-1-2-2 11 3322-4-4 44

tt

t distribution for N = 10t distribution for N = 10From TDIST, p = .0516

Combined area beyond Combined area beyond t=2.24 and t = -2.24 is .0516t=2.24 and t = -2.24 is .0516

Whoa! We no longer have significance at

=.05

04/21/23 HK 396 - Dr. Sasho MacKenzie 31

Interpreting the P-valueInterpreting the P-value• In an experiment of this size, if the populations really

have the same mean, what is the probability of observing at least as large a difference between sample means as was, in fact, observed?

• There is a p% chance of observing a difference as large as you observed even if the two population means are identical (the null hypothesis is true).

• Random sampling from identical populations would lead to a difference smaller than you observed in 1-p% of experiments, and larger than you observed in p% of experiments.