FIRST ROBOTICS AS A MODEL

FOR EXPERIENTIAL

PROBLEM-BASED LEARNING:

A COMPARISON OF STUDENT

ATTITUDES AND INTERESTS IN

SCIENCE, MATHEMATICS,

ENGINEERING, AND

TECHNOLOGYDonald Sanford Griffith, Jr.

May 2005

INTRODUCTIONSignificance of the Study

• As reported by the Third International Mathematics and Science Study (TIMSS), students in the United States are lagging behind other nations in the areas of mathematics and science (U.S. Department of Education, 1999).

• Undergraduate admissions requirements for engineering schools require students to have a solid background in mathematics and science (U.S. Department of Labor, 2002).

• United States Department of Labor statistics also indicate that there will be the need for 20 percent more engineers by the end of this decade (Southern Regional Education Board, 2001).

INTRODUCTIONSignificance of the Study

• The number of students who plan to major in engineering upon college entrance has continued to decrease, ACT Policy Report (Noeth, Cruce & Harmston, 2003).

• Students enrolled in engineering programs hit a 17-year low in 1999, according to the National Society of Professional Engineers.

• Enrollment in engineering technology programs in 2000 was about half what it was in 1990 (Southern Regional Education Board, 2001).

Statement of the Problem

The problem addressed in this study was whether student

participation in the six-week challenge of the FIRST

Robotics Competition had an effect on high school

students’ attitudes and interest in science, mathematics,

engineering, and technology. The possible relationships

between the six-week challenge of the FIRST Robotics

Competition and high school students’ attitudes and

interest in science, mathematics, engineering, and

technology is uncertain and may be important to the

overall success of students in high school courses, post-

secondary education and related careers.

Purpose of the Study

The purpose of this study was to investigate whether

student participation in the six-week challenge of the FIRST

Robotics Competition, positively changed high school

students’ attitudes and interest in science, mathematics,

engineering, and technology after their participation. It

was anticipated that gender and race of the high school

students, in the context of the FIRST Robotics experience,

would influence attitudes and interest in science,

mathematics, engineering and technology in either a

positive or negative manner.

Research Questions

Primary Questions1. Do high school students that participate in the six-

week challenge of the FIRST Robotics Competition have more positive changes in their attitudes regarding science, mathematics, engineering, and technology after their participation than students who do not participate in the program?

1. Do high school students that participate in the six-week challenge of the FIRST Robotics Competition have more interest in science, mathematics, engineering, and technology after their participation than students who do not participate in the program?

Subsidiary Questions

1. Does the gender of high school students have an effect on their interest in science, mathematics, engineering, and technology?

1. Does the race of high school students have an effect on their interest in science, mathematics, engineering, and technology?

Null Hypothesis

1. There is no difference in the attitudes of high school students that participate in the six-week challenge of the FIRST Robotics Competition regarding science, mathematics, engineering, and technology after their participation, than students who do not participate in the program.

1. There is no difference in the interest level of high school students that participate in the six-week challenge of the FIRST Robotics Competition regarding science, mathematics, engineering, and technology for students after their participation, than students that do not participate in the program.

Assumptions

For this study the following was assumed:

1.     high school students participating in the research study would provide accurate and valid responses on the pre- and post- survey instruments.

2.     the variation of science, mathematics, engineering, and technology teachers students have experienced, and courses students have completed was equally distributed in the population.

3.     the variation of science, mathematics, engineering, and technology extra- curricular activities students have experienced were equally distributed in the population.

4. the variation of students with previous FIRST Robotics Competition experience were equally distributed in the experimental group sample.

Literature Review

FIRST has participated in 7 research studies since 1996

seeking to answer the question, “Is the program

working?” In each study, data were collected from

participants involved in FIRST. In each of these studies,

FIRST students were not compared to other student

populations; therefore there was not conclusive evidence

that the change in students’ attitudes or interests was due

to their participation in FIRST.

Literature Review

Educational philosophies, theories, and methodologies

supporting the FIRST Robotics Competition were

considered as the rationale for identifying topics of inquiry

in this review. The topics of inquiry identified were as

follows: philosophies of education, experiential learning,

constructivism, situated cognition, problem-based

learning, and project-based learning within the context of

high school students’ attitudes and interests in science,

mathematics, engineering, and technology.

Research Locations

• High school sites were located in urban and rural locations throughout the state of South Carolina.

• High schools sites had students participating as team members with a FIRST Robotics team that was registered to participate in the 2005 FIRST Robotics Competition.

• High schools and FIRST Robotics teams selected had participated in the 2004 FIRST Robotics Competition.

343

415

845

342

1026

804

1319

1051

1102

1320

1398 128

7

Distribution of FIRST Robotics Teams Participating in the Study

Distribution of Sponsoring High Schools Participating in the Study

Research Locations

During the six-week period of the study, students

participating in FIRST Robotics were involved designing,

developing, and producing a robot for the 2005 FIRST

Robotics Competition while students at the sponsoring

high schools were enrolled in a science, mathematics,

engineering, or technology course.

Sampling

• A convenience sample was drawn from the assessable population of 2005 FIRST Robotics teams and sponsoring high schools in the state for this study.

• Students selected were attending public high schools during the 2004-2005 school year.

• Students participating in the study represented twelve of the nineteen 2004 FIRST Robotics teams and twenty of the thirty- five sponsoring high schools.

• The twelve FIRST Robotics teams had a total of one hundred eighty- three participating students.

• The sponsoring high schools had a total of five hundred fifty participating students.

Instrumentation

• Copied segments of the What Do YOU Think? survey designed by the Goodman Research Group in 2000 for FIRST, to investigate attitudes and interest in science, mathematics, engineering, and technology of the study participants.

• Administered pre- and post-surveys to participants based conclusions and recommendations included in the Goodman Research Group 2000 Final Report To FIRST :

“…any data being collected from students as part of a program evaluation be accomplished via written mail surveys not web- based surveys.”

“…future research in the direction of tracking students over time and comparing them to students who do not participate in the program.”

• Administered the What Do YOU Think? pre- and post-surveys using Optical Mark Reader (OMR) forms to record participant responses.

Procedures

• Public high schools in a southeastern state identified.

• Letters of support received.

• Procedures for data collection detailed.

• Approval to conduct study received from Clemson University IRB.

• Pilot study performed.

• Study pre- and post- surveys administered.

• Statistical procedures performed.

• Findings compared to findings reported by the Goodman Research Group, Inc. 2000 Final Report to FIRST.

Statistical Procedures(OMR OS/390 Data Set Analysis: 9.1 SAS Software©)

•the frequency for responses to each question was examined.

•chi-square procedure was used to examine the distribution patterns of the independent demographic characteristics.

•ANOVA was conducted to determine if differences existed between the pre- and post-survey responses to each question within each group, and if differences existed between these two groups.

•ANOVA was conducted to determine if differences existed between the pre- and post-survey response averages within each group, and if differences existed between these two groups.

Microsoft Word Document

Participants

Group Pre-survey Post Survey Loss Participant %

Experimental 181/183 131/132 (52) 72.38(FIRST)

Control 546/550 373/375 (177) 68.32(SMET)

Totals 727/733 504/507 (229) 70.35

Microsoft Word Document

Gender and Race Pre-survey Post-survey

GenderPercentages

Group Female Male Female MaleExperimental 24 76 30 70(FIRST)Control 37 63 40 60 (SMET)

RacePercentages

Minority White Minority White

Experimental 23 77 30 70(FIRST)Control 31 69 33 66 (SMET)

Microsoft Word Document

Microsoft Word Document

Survey Instrument

7 Attitudinal Categoriesteamwork

science and mathematics knowledge of science and math careers,

persist in science, mathematics, engineering, and technology enjoyment of high school science and math classes

the working world self-image

3 Interest Categoriescareer field interests

students’ expected activities for the year after high school graduation

students’ highest level of education

Survey Response SelectionFour items are based on a 5 point Likert type scale from (1) a lot to (5) doesn’t apply (to the item). Twenty-three items of the sixty survey items are based on a 5 point Likert type scale from (1) strongly agree to (5) strongly disagree (with the item).

A B C D E

Strongly Agree Neither Agree Disagree Strong Agree nor Disagree Disagree 14. I enjoy participating in team- 1 2 3 4 5 orientated activities.Ten items are based on a 4 point Likert type scale from (1) very likely to(4) very unlikely (with the item). Two items are based on a 4 point Likert type scale from (1) a lot to (4) nothing (with the item).

A B C D

A lot Some A little Nothing

33. Science? 1 2 3 4

The remaining twenty-one items required the student to select one from multiple choices.

Microsoft Word Document

Research Question 1

Do high school students that participate in the six-week challenge of the FIRST Robotics Competition have more

positive changes in their attitudes regarding science, mathematics, engineering, and technology after their

participation than students who do not participate in the program?

A one-way analysis of variance (ANOVA) was conducted to examine differences between pre- and post-survey dependent variables within the experimental group (FIRST), and the control group (SMET).

A two-way ANOVA was conducted to examine findings specific to the participants by comparing between the FIRST and SMET groups.

Test statistic used to determine level of significance for each dependent variable in experimental and control group ( p-value α=.05 ).

Microsoft Word Document

One-way ANOVAExamine differences between pre- and post-survey

dependent variables within each group

Dependent variable 54: Managing your time effectively is an important part of being successful in your job. Source df Sum of Squares Mean Square F Value Pr > FModel 1 4.5989011 4.5989011 7.54 0.0062Error 649 395.7820513 0.6098337

Corrected Total 650 400.3809524

Two-way ANOVA

Examine differences between pre- and post-survey dependent variables between each group

Control * Test Group (SMET 546 / FIRST 181)

Examine category means, pre- and post-survey question averages in each group

FIRST Group Category 6: Statements about the working world.

Source df Sum of Squares Mean Square F Value Pr > FModel 1 1.6037269 1.6037269 3.67 0.0557Error 648 282.8072133 0.4364309Uncorrected Total 649 284.4109402

Research Question 2

Do high school students that participate in the six-week challenge of the FIRST Robotics Competition have more

interest in science, mathematics, engineering, and technology after their participation than students who do

not participate in the program?

Response frequencies for the experimental group (FIRST) and control group (SMET), pre- and post survey responses were explored in order to address research question 2.

The response frequency for each question on the pre- and post-survey was calculated as a percentage for each group.

Increases in career field interests between pre-survey and post-survey, FIRST experimental group, reported by gender.

Percentage

Field Gender Gender Percentage Boy Girl Pre-Survey Boy Girl Post-survey Change

Architecture 6.47 1.90 8.37 8.46 4.76

13.22 +4.85

Business, Finance 1.67 2.53 4.20 3.08 5.56

8.64 +4.44

Math 12.32 4.43 16.75 10.38 9.52

9.90 +3.15

Engineering 18.37 10.76 29.13 22.31 7.94 30.25

+1.12

Decreases in career field interests between pre-survey and post-survey, SMET control group, reported by gender.

Percentage

Field Gender Gender Percentage Boy Girl Pre-Survey Boy Girl Post-survey Change

Architecture 6.80 2.55 9.35 6.71 1.72

8.43 -0.92

Business, Finance 3.80 6.75 10.55 4.52 5.15

9.67 -0.88

Math 8.26 5.10 13.36 8.63 2.94

11.57 -1.79

Engineering 11.34 5.10 16.44 11.37 4.66

16.03 -0.41

Increases in career field interests between pre-survey and post-survey Gender differences of the SMET group

Percentage

Field Gender Gender Percentage Boy Girl Pre-Survey Boy Girl Post-survey Change

Arts, Design 4.45 6.75 11.20 5.07 10.05 15.12 +3.92Social Science 1.38 2.70 4.08 1.78 4.90 6.68 +2.60Medicine 2.27 6.75 9.02 3.70 7.84 11.54 +2.52Law 3.08 5.40 8.48 3.84 7.11 10.95 +2.47

Comparison of FIRST and SMET post-survey career field interests between students, positive difference of interests for FIRST students. Gender differences are reported for each group.

Percentage

Field Gender FIRST Gender SMET Percentage Boy Girl Post-Survey Boy Girl Post-survey Change

Engineering 22.31 7.94 30.25 11.37 4.66 16.03

+14.22

Math 10.38 9.52 19.90 8.63 2.94 11.57

+ 8.33

Computers 13.08 9.52 22.60 9.04 7.35 16.39

+ 6.21

Architecture 8.46 4.76 13.22 6.71 1.72 8.43

+ 4.79

Science 4.62 7.14 12.03 3.97 6.37

10.37 + 1.69

Comparison of the FIRST and SMET groups, pre-survey and post-surveyinterest in attending a 4-year college, service academy, or university after leaving high school. Gender differences are reported for each group.

Percentage

Pre-Survey Post-survey Net ChangeGroup Gender Gender Gender Boy Girl Boy Girl Boy Girl

FIRST 50.47 51.35 59.06 61.02

+8.59 +9.67

SMET 31.14 44.62 32.78 45.50

+1.64 +0.88

Comparison of the FIRST and SMET groups, pre-survey and post-survey interest in attending a 2-year junior or community college after leaving high school. Gender differences are reported for each group.

Percentage

Pre-Survey Post-survey Net Change Group Gender Gender Gender Boy Girl Boy Girl Boy Girl

FIRST 14.62 25.68 18.11 23.73

+3.49 -1.95

SMET 18.32 20.25 17.78 18.48

-0.54 -1.77

Although there was a decrease of 1.95% indicated by girls in the FIRST group regarding their interest to attend a 2-year junior or community college after leaving high school there was an increase of 9.67% by girls in the FIRST group between pre- and post-survey responses, regarding their expectation to be attend a 4-year college, service academy, or university.

Comparison of the FIRST and SMET groups, post-survey responses, students’ highest expected level of education. Gender differences are reported for each group.

Percentage Post-survey

Expectation Group Gender Total Boy Girl

4-year college degree FIRST 19.55 20.34 39.89 SMET 27.54 18.69 46.23 Master degree FIRST 22.56 22.03 44.59 SMET 13.37 19.63 33.00

Ph.D. or M.D. FIRST 12.78 13.56 26.34 SMET 6.42 10.28 16.70

Comparison of the FIRST and SMET groups, post-survey responses, students’ expectation of a 4-year college degree. Gender differences are reported for each group.

Percentage

Expectation: Race Gender 4-year college degree White Minority Boy Girl Pre Post Pre Post Pre Post Pre Post

FIRST 17.41 13.59 6.30 6.52 22.22

19.55 16.44 20.34

SMET 6.29 16.05 5.72 7.77 22.95

27.54 18.27 18.69

Null Hypothesis 1

There is no difference in the attitudes of high school students that participate in the six-week challenge of the FIRST Robotics

Competition regarding science, mathematics, engineering, and technology after their participation, than students who do not

participate in the program.

The null hypothesis concerning attitude differences of high school students that participate in the FIRST Robotics six-week challenge as compared to students that do not participate in the program was not rejected.

High pre-survey dependent variable scores provided by students in the FIRST group did not allow for significant gain in each of the seven-attitudinal categories.

Findings indicated that there were significant attitudinal differences between students in the experimental group (FIRST), and students the control group (SMET).

Students in the FIRST group had statistically significant higher attitude means than students in the SMET group on both pre- and post-survey’s in the seven-attitudinal categories.

Null Hypothesis 2

There is no difference in the interest level of high school students that participate in the six-week challenge of the FIRST Robotics Competition regarding science, mathematics, engineering, and technology for students after their participation, than students

that do not participate in the program.

The null hypothesis concerning interest differences of high school students that participate in the FIRST Robotics six-week challenge as compared to students that do not participate in the program was rejected.

The frequency for responses to each question in the three interest categories of the pre- and post-survey were calculated for the experimental and control group to evaluate differences.

The results were expressed in percentages and indicated that there were significant differences in the respondent scores for the pre-survey versus the post-survey in the FIRST group.

Subsidiary Question 1 and 2

Does the gender of high school students have an effect on their interest in science, mathematics, engineering, and

technology?

Does the race of high school students have an effect on their interest in science, mathematics, engineering, and

technology?The chi-square procedure was used to examine the distribution patterns of the independent demographic

characteristics along the dependent variables.

Descriptive analysis was conducted for the data on race and gender. Subsidiary questions 1 and 2 were addresses

within the independent samples t test analysis demonstrating no significant differences on pre-survey or

post-survey scores by either gender or race.

The Importance of This Study

The findings of this study suggest that FIRST students’ have, and continue to have, high positive attitudes and

interests in science, mathematics, engineering, and technology.

Positive attitudes and interest in these subjects lead to higher academic achievement.

The consequences are that students tend to enroll in advanced courses and pursue careers related to science,

mathematics, engineering, and technology.

Enjoyment Science and Math

The post-survey reported increases in enjoyment of science and mathematics for students in the experimental group

(FIRST).

The post-survey indicated that more than 79% of the FIRST respondents enjoyed science, and 81% enjoyed math a lot,

compared to 54% and 59% respectively reported by students in the control group.

Develop Interest in Science and Math

FIRST respondents, 84% were either likely or very likely to have plans for taking advanced high school or college level

classes in each of the four fields.

Students indicated the greatest likelihood of taking college level classes in science, mathematics, and computers.

In contrast, the control group respondents reported on the pre- and post-survey that 63% were either likely or very likely to have plans for taking advanced high school or

college level classes in each of the four fields.

Immediately After High School

The post-survey reported that over 60% of the students in the FIRST group planned on attending a four-year college,

service academy, or university immediately after graduating high school, an increase of

9% from the pre-survey.

These post-survey results are compared to 39% for the SMET group of students, an increase of 1.5%.

Over 60% of FIRST students responded on the post-survey, that over the course of their educational careers, they would earn at least a Bachelor’s degree, 22%

expected a Master’s degree, and 21% an advanced professional or terminal degree.

Educational Expectations

The study indicated that there was a difference in white and minority students’ expectations about their highest

level of education.

Minority students in both groups indicated on the pre- and post-surveys that they were less likely than white students

to expect a 4-year college degree.

The same trend was observed regarding girls with one exception;

girls in the FIRST group responded on the post-survey that they were more likely than boys to expect a 4-year college

degree after participating in the FIRST Robotics Competition six-week challenge

experience.

Consistent With Previously Reported Findings

Study findings that were consistent with findings reported in the Goodman Research Group’s Final Report to FIRST

(Goodman Research Group, 2000)

Differences in study findings were reported in two areas.

students’ perceived knowledge of science and mathematics careers, and students’ intentions to persist in science, mathematics engineering, and technology courses

and related careers.

Summary

Results of the study offer some evidence that the FIRST Robotics Competition six-week challenge experience has

short-term attitudinal and interest impact on students.

Students in the FIRST group:

reported high positive attitudes regarding science, mathematics, teamwork, and the working world.

were more interested and felt positive about their abilities, in science and mathematics compared to students in the control group.

attitudes were more positive regarding teamwork and the world of work compared to students in the control group.

likely to have plans for taking advanced high school or college level classes in science, mathematics, engineering, technology.

Recommendations for Future Study

Investigate the long-term outcomes of student participation in FIRST Robotics.

An investigation of student participation in FIRST Robotics Competition regional and championship events. FIRST Robotics Competition event experiences involve students in the performance phase of the competition.

Expanding the study to include FIRST Robotics Competition student participants enrolled in other than public high schools, e.g., private, parochial, charter, etc., may provide very useful insights.

Follow-up studies in various geographic areas of the country may provide additional information and prove useful in forming the basis for future inquiry regarding the study variables.