Mary Nelson, Micah MysiukGeorge Mason UniversityDepartment of MathematicsAccelerator Math Lead

Noyce Conference, May 2013Funded by NSF Grant: Robert Noyce Scholarship Program

Recruiting Noyce Scholars through a

Learning Assistant Program

Only four undergraduate STEM majors were licensed through GMU since 2004

All four pre-service teachers were Earth Science majors

President Obama has challenged us to educate 100,000 new teachers

In the current financial crisis in the US, we need to help promising new STEM teachers

Why Noyce in Northern Virginia?

Twice a year we accept applications from faculty for learning assistants

Advertise student learning assistant positions through posters and College of Science broadcast emails

Spring semester we had more than twice as many requests for LAs as we could fund

Spring semester we had 90 student applications for 28 positions.

Recruiting Method for Noyce Scholars Learning Assistant Program

New LAs must attend the Teaching and Learning seminar once a week

All LAs meet weekly with their supervisor All LAs are provided time for preparation,

which may include the following: Attend the class for which they are LAs Work through homework assignments that students in the course are doing Ask mentors questions about any topic that

they do not understand

Responsibilities of LAs

• Participate in weekly LA Seminar in their first semester as an LA• Meet weekly with their course mentor• Facilitate math oral reviews for Business Calculus, Calculus with Algebra, Calculus I, Calculus II,

Quantitative Reasoning• Facilitate Biology oral reviews for Cell Structure and Function• Conduct on-line peer tutoring• Create on-line modules to assist in student learning• Provide review sessions prior to tests• Provide peer to peer instruction in help sessions• Work with small groups during classes• Assist students in labs• Teach mini-lessons

LA Activities - Year 1

Sexual Harassment Prevention Training Learning Styles Constructivism Importance of Discourse Student Centered learning Wait time How to write and use rubrics How to facilitate oral reviews

Seminar Topics

LAs have genuine teaching experiences LAs work with faculty who are excited about

teaching and learning Students learn some basic educational

principles at the Teaching and Learning Seminar

LAs become an integral part of their department’s teaching effort

How is the LA Program a Recruitment Tool?

Grant pays for 10 LAs/year Accelerator funds 12-18 additional Community college has 11-12 LAs

Year 1: Began with 1 math Noyce – looking for a job

now Second semester – added Chemistry Noyce Next fall – 7 confirmed Noyce, possibly 3

more – all former LAs

Results from Year 1

When asked “what would you tell a student who was thinking about being an LA?” One LA answered,:

“Go for it! If you believe you have any future in education this is a must have collegiate experience!”

Another claimed: “I would tell them to definitely try it. The experience was great and taught me a lot; I really think it educated me an equal amount that I educated others. Being an LA gives you experience, patience, knowledge, and public speaking skills.”

Another explained that as a K-12 student, he wanted to be tennis pro and “anything but math.” He still has visions of being a tennis pro, but now his aspirations include being a mathematician and a teacher.

Comments from LAs/Noyce

The result of having Taylor in the lab was an increase in the number of A grades compared to Fall 2012. In Fall 2012 37% of the enrolled students earned A’s. In Spring 2013, when Taylor was assisting, the percentage of students earning A’s increased to 42%. Similarly, in Fall 2012, 27% of enrolled students failed to pass the course. In Spring 2013, this number dropped significantly to 15%.”

“Stephen was phenomenal. He arranged weekly oral reviews to help the Business Calculus students, and his attendance was amazing. Students really appreciated the help in understanding the material.”

Faculty comments:

Ungraded, voluntary Often cited by students as most important aid to

learning Small groups of 3-5 students for an hour Emphasis on conceptual questions

◦ Why would you use linearization? ◦ What does it look like on a graph? ◦ From the graph, what kind of functions will give the best

results?◦ Does it matter where you center the linearization?

Intervention: Oral Reviews

1. Vehicle for getting students to discuss mathematics and other sciences

-typically pattern match without understanding-need to put understanding in their own words -need others to correct and clarify

misconceptions-then they need to “say it again!” to convince

themselves that they understand-teachers often have ah-ha moments

- excellent training for LAs in student-centered teaching

Orals - What are they?

Placement Groups N Mean Std. Deviation

0-18 622 1.0400 1.13777

19-21 639 1.7236 1.21572

22-26 1245 2.3332 1.17252

27-30 372 3.0946 .97526

Placement Scores predict grades

Compare Exam Scores N Average St. Dev No Orals Exam 1 333 75.1 15.0

Orals Exam 1 134 81.6 10.4 No Orals Exam 2 298 74.5 15.4

Orals Exam 2 162 79.8 12.6 No Orals Exam 3 318 64.4 19.1

Orals Exam 3 138 73.9 15.7

Looking at control vs. treatment in Calculus I exam 1 N>600

Course Grades compared by number of oral

assessments

Fall 2007 – Course Grade (4 point scale) by Number of Orals

No. of Orals No. of Students Mean Standard Deviation 0 243 1.935 1.33 1 63 2.089 1.15 2 81 2.542 1.03 3 69 2.841 .85

Average Grades by placement score and number of orals

Three GMU Classes

TEST 1 Instructor 1 Instructor 2 Instructor 3 Average Median Average Median Average Median No orals 79 81 73 80 74 80 orals 83 86 82 86 87 90

Email questions: mnelso15@gmu.edu

Questions?

Students learn the importance of understanding the basic concepts in order to be able to apply those concepts to novel situations

Students learn better ways of studying Students work harder because they believe their

instructors are invested in their success. All of the above improvements increase with the

number of orals in which students participate

Effects of Orals

Students agreement increased significantly on: Item 8 – I am not satisfied until I understand why something works

the way it does. (p=.042) Item 11 – I study math to learn things that will be useful in my life

outside of school. (p=.012) Item 16 – To understand math I talk about it with friends and other

students. (p=.002) Item 23 – Mathematical formulas express meaningful relationships

among measurable things or amounts. (p=.001) Item 36 – When studying something new in math, I compare it to

what I already know rather than just memorizing the way it was presented. (p=.028)

CLASS Results: 7 items had significant pre/post differences

Item 7 – There is usually only one correct way to solve a math problem. (p=.037)

Item 18 – If I don't remember a mathematical method needed to solve a problem on a test, there's nothing else I can do. (p=.007)

*Students answers to all other questions were not significantly different pre/post

Students disagreed more strongly

University of Colorado, BoulderPenn State UniversitySeattle UniversityShippensberg UniversitySanta Clara UniversityGeorge Mason University

Universities Using Orals

Calculus I, II and IIIMatrix MethodsComplex AnalysisPDEsStaticsComponent DesignDynamicsHigh school algebra

Courses Using Orals

More time/slower pace

◦ Comprehensive exam after two semesters◦ Workshops add 2 hours/week

Motivation

◦ 1. Workshops◦ 2. Review sessions

Counterfactuals

0

5

10

15

20

25

30

35

Calculus I Calculus 2

9 year AVG, Calc1 and 2 2007-2008

Decline in Failure Rates

Helps me understand the hard concepts Helps me determine what I know and don’t

know for the upcoming test It clarifies things I was unclear about It gives me confidence before the written

test It helps to hear how other students think

about some of these things

Students’ Reactions

QUESTIONS?

Developing better motivation measures Examining the “caring” effect Using orals in other venues

◦ Mechanical Engineering: Component Design◦ Aerospace: Statics◦ High school algebra

Teaching students to run their own orals in Calculus III

Future Directions

GRADE

4.03.02.01.00.0

16

14

12

10

8

6

4

2

0

Std. Dev = 1.02

Mean = 2.5

N = 34.00

GRADE

4.03.02.01.00.0

60

50

40

30

20

10

0

Std. Dev = .79

Mean = .4

N = 69.00

One-semester vs. Two-semesterCourse Grades - Fall 03

Scatterplots of Final Exam Grades

Regular students Treatment students

REGVS134: 2.00

placement score

403020100

Fina

l Exa

m

140

120

100

80

60

40

20

0

REGVS134: 1.00

placement score

3020100

Fina

l Exa

m

140

120

100

80

60

40

20

74/150 was the average grade of the students in the one-semester class

97/150 was the average grade of students in two-semester class (treatment group) on the identical exam.

Comparison of Common Final Exam Scores

Comparison groups

Mean Exam Score

Standard deviation

Mean difference

Effect size

Treatment 97.37 24.221 Regular 73.98 27.809

23.39 .84 st. dev.

Group Subgroup Final Exam Conceptual Procedural Placement At-risk 93 9.2 42.6 14.15 Not-at-risk 98.2 8.4 44.7 20.3 Treatment All students 95.4 8.9 43.4 16.5 At-risk 52 3.25 22 13.99 Not-at-risk 79 6.7 33.3 23.4 Regular All students 74 6 31 21

Comparisons

Effect Size Differences for Final Exam Scores

GROUPS Mean ExamScore

StandardDeviation

MeanDifference

Effect Size

Treatment At-risk 93.13 27.41 40.9 1.49

Control At-risk 52.23 24.41

Treatment Not At-risk 98.30 25.30 20.59 .77

Control Not At-risk 77.61 26.69

Treatment At-risk 93.13 27.41 15.52 .57

Control Not At-risk 77.61 26.69

Success in Calculus II:At-risk in Both Groups

GROUP At-riskstudentstakingfinal

Meancoursegrade

Standarddeviation

% at-riskwho tookCalculus II

Of the at-riskwho took

Calculus II, %

who passed

Treatmentat-risk

16 2.34 (C+) 1.30

56% 89%

Regularat-risk 61 .79 (D-)

1.0620% 80%

Retention Rates for At-risk Students in Treatment and Control Groups

GROUPS Percent of StudentsAt-risk

Percent of at-riskNo longer at CU

Treatment N = 34

62% 30%

Control N = 615

16% 45%

Randomly selected 1 of my 2 large classes – coin flip before semester began

Trained all Calculus I TAs and 2 Noyce to do orals

Provided orals questions each time Each TA did 1 and each Noyce fellow did 2 I facilitated the rest About 50% of the class participated

Study Extended to large classes-Fall05

Preliminary Results

GROUPS Control Treatment

Test 1 74 82

Test 2 65 67

Test 3 65 72

Differences in AttendanceBased on Quiz Participants

GROUPS Control Treatment

Percent taking Quiz

68.5 85

As reported by a student from control class “It’s like a different class…I want to be in

that class next semester. Which class will get those things next semester? I want to be in there.”

When asked why, “They are really into it. Everyone is answering your questions. They’re really excited about it. It’s not like our class.”

Participation

Offered orals to all APPM 1350 students My class had over 50% attendance Some classes as low as 20% Analyzed results using Answer Tree Complications due to 30 students took final-

because of snow storm Major question is effect of motivation –

compare to Workshop and Review sessions

Fall 07 – Great Calculus Orals Experiment!

TEST NO ORALS Failure Rate

ORALS Failure Rate

NO ORALS Average

ORALS Average

1 12.5% 10% 75 81 2 13% 9% 74 79 3 13.1% 8.5% 64 73

Orals versus No Orals

Failure Rates for Calculus I

Typical Failure Rate 30-33%

Failure Rate for Fall 2007 22%

Failure Rates in Spring Calculus II

Year Failure Rate

2006 31%

2007 27%

2008 17%

We have been given CCLI Phase II grant Implementation in all Calculus I and II classes Implementation in UCCS Calculus classes Implementation in high school algebra classes Implementation in Mech E Component Design

class Implementation in Aerospace E classes Fall 09 Broader participation by TAs and LAs in

facilitating orals Observations of TAs and LAs to ensure fidelity of

treatment

Next Steps

No Orals versus Three Orals

Research Results

For years, failure rate for Calculus I has wavered between 30-33%

Last semester, fail rate was below 20%

Improvement in Pass Rate?

Is control group same as all previous Covariance due to placement scores Counterfactuals

◦ Class size: 35-42 vs 48-142 Compared to 48 person class

◦ Time on task Workshop students had same time-on-task Treatment had entire year’s material on final

Common final exam Enrollment and success in Calculus II Retention at the University

Plan of Analysis

◦ At-risk determined by 30 question placement test

◦ Students scoring below 18 are considered at-risk

◦ All but two treatment students who were not designated “at-risk” by the placement test were in the class because they failed the first test in the regular class (20-30%) and dropped back to the treatment class

Treatment Students

Control Students

34 person classes

48-142 person classes

Two -semester One –semester

62% at-risk 16% at-risk

Mean Place 16.5

Mean Place 21.4

Comparison of Groups

We hope to f’ll scale up to all Calculus I classes Orals will take place in recitations and

workshops AND before each midterm On-line homework will free TAs to contribute

more time to orals Analyses will examine effect on

◦ Overall class◦ Women and minorities◦ Students whose placement scores designate them at-risk

Plans (HOPES) for next year

Is the conceptual framework basically there?

What needs to be eliminated? What needs to be reworked? Suggestions PLEASE!

What do you think?

Comparison of Groups

◦ At-risk determined by 30 question placement test

◦ Students scoring below 18 are considered at-risk

◦ All but two treatment students who were not designated “at-risk” by the placement test were in the class because they failed the first test in the regular class (20-30%) and dropped back to the treatment class

Treatment Students

Control Students

34 person class

96-140 person class

Two -semester

One –semester

62% at-risk 16% at-risk

Mean Place

16.5 Mean Place

21.4

Study based on constructivist view of learning

Mathematics reform movement is an embodiment of constructivism◦ Emphasis on:*Vygotsky’s notion of ZPD*Scaffolding*Discourse*Formative Assessment / misconception theory

Theoretical Framework

Participation

Participation

TEST Workshop Non-workshop

1 47% 25%

2 52% 26%

3 43% 26.9%

Great Calculus Experiment –Fall 07

TEST ORALSFailure Rate

NO ORALSFailure Rate

1 10% 12.5%

2 9% 13%

3 8.5% 13.1%