Math/Physics Competency Test as a Diagnostic Tool in Undergraduate Mechanical Engineering Curriculum

C. Shih and N. Chandra*

Department of Mechanical EngineeringFAMU-FSU College of Engineering

Florida A&M University and Florida State University

2006 ASEE Annual Conference, Chicago, June 18-21

* Lead faculty, soon will be the Associate Dean, CoE, U. Nebraska, Lincoln

Math/Physics Preparation-Historical Background

Levels of Preparation are not uniform among Students

Unique two-institution affiliation + transfer students

Some students do not retain necessary Math/Physics concepts

They cannot formulate word problems in math form

Significant time wasted in repeating the same concepts in a number of sophomore and even higher level classes

Early intervention is necessary so we can identify deficiencies early in order to help students to gain the required knowledge.

Collect enough data and frequently revisit the subject with the ME faculty to decide on the future course of action - involving math/physics faculty and other engineering faculty.

Recent curriculum reform further highlights these deficiencies

Integrated ME Curriculum

Initiate the curriculum change in 1997 Begin professional preparation/higher-level skills early in the

curriculum ME Tools: introduce all relevant ME tools/skills including communication,

Mathematical/programming tools, machine shop experience, CAD, and fundamentals of thermodynamics (Stirling Engine)

Introduction to ME: introduce engineering profession, ethics, design process/tools, and engineering mechanics

Streamline related classes into a class sequence to emphasize connectivity between topics, design considerations, project work, team work, etc.. Ex: Combine Thermodynamics + Heat Transfer + Fluid Mechanics

Thermal/Fluids I & II with lab/workshop components Increase technical electives to four

Culminating into a two-semester senior capstone design project (presented in session 2125) Integrated curriculum diagram

Introduction to Mechanical Engineering

The first ME class students take after formally be admitted into the engineering school (sophomore-level) Passed engineering pre-requisites (Calculus I & II, Physics I,

and Chemistry I with C or better in all four) Outlines: introduce ME as a profession, ethics, design

process/tools, and concepts of mechanics (mostly Statics) Consider one of the gatekeeper courses for ME

About 25% of students received either “D” or “F” Perfect time to exam students’ proficiency in pre-required

knowledge in math and physics Early warning and intervention tool

Math and Physics Requirements

Calculus I and II and Physics I as pre-requisites

Math covers functions (exponential, logarithmic, trigonometric), calculus (derivatives, integral, ODE), matrices, word problem formulations.

Physics: static/dynamic equilibrium, Newton’s law of motion, friction, and free-body diagrams.

The test is based on these pre-requisite Math and Physics courses using the specific text books (equivalent to high school AP Calculus/Physics).

To establish the baseline competency for students to take more advanced mechanical engineering courses.

Test Format

A large data base of multiple choice questions in math and physics extracted or developed directly from the text books.

The test (35 from math+15 from physics) is given during the second week of Intro to ME course.

A score of 80% gives the student a full 10 (15) points out of the course maximum of 100 points.

Students who fail the first time can retake the test in the eighth week and get a reduced score should they pass with 70% or higher; zero is given to those who did not score 70% or higher.

Remedial classes conducted between the first and second tests.

Test Evolution

Problem quality control Statistical data taken from each test is used to weed out inappropriate

questions (either too easy/difficult or ill-posed) A question could be reviewed for modification or deletion should the

percentage of students answer correctly fall outside an acceptable value (60-80%) good reference for remedial lectures

Grading scheme Grade bias correction: Instead of awarding full 10% credit, the actual test

grade will be prorated based on the percentage of questions answered correctly

To emphasize the importance of the Math/Physics preparation, the test percentage has been increased from 10 to 15% in recent semesters.

Add an on-line practice test that exactly simulates the format (and timing) of the test, available starting Fall 2003.

Online Practice Test on Blackboard

The online practice test (through Blackboard) is identical to the in class test, with the same number of questions: 35 math and 15 physics.

The online practice test has questions selected from a database with over 150 questions on each subject. This allowed the students to take the practice test as many times as they choose; they get a different test online each time. .

Exam is graded right away with the right answers indicated.

There is an excellent correlation between the online test scores (automatically recorded for reference purpose) and the actual test scores.

Snapshot of the Online Test (graded)

Student Data, Fall 2002 to Spring 2006

A total of 378 students over a period of eight semesters had taken the exam. Competency scores recorded along with their SAT/ACT, pre-requisite GPA, post ME GPA

In general, data indicates a useful tool for both diagnosis and future performance prediction

49% passed the first time and 75% passed for the second time

For those who received a “F” course grade, more than 75% of them did not pass the test

Correlate math/physics skills to readiness for mastering engineering subjects

>80% of those who did not pass the test received either “D” or “F” course grade

The test serves effectively as a gatekeeper (drop one letter grade if not passed)

The test percentage has been increased to 15% in recent semesters

Diagnostic Tool: Test Score vs. SAT & ACTAll Semesters

0

10

20

30

40

50

60

0 5 10 15 20 25 30 35

Math ACT Score

Rev

iew

Sco

re

Math ACT Score vs. Review Score Linear (Math ACT Score vs. Review Score)

All Semesters

0

10

20

30

40

50

60

0 100 200 300 400 500 600 700 800 900

Math SAT Score

Rev

iew

Sco

re

Math SAT Score vs. Review Score Linear (Math SAT Score vs. Review Score)

• High ACT seems to correlate very well with high test score

• Significantly more scatter for SAT correlation• Math test score only shows similar correlation

Test Score verse the Pre-requisite GPA

All Semesters

0

10

20

30

40

50

60

0 0.5 1 1.5 2 2.5 3 3.5 4

Pre- Reg GPA

Re

vie

w S

co

re

Pre-Req GPA vs. Review Score Linear (Pre-Req GPA vs. Review Score)

70%

Pre-Req GPA 2.5

How Good the Test as a Predictor?

All Semesters

0

10

20

30

40

50

60

0.000 0.500 1.000 1.500 2.000 2.500 3.000 3.500 4.000

GPA

Rev

iew

Sco

re

GPA vs. Review Score Linear (GPA vs. Review Score)

GPA is calculated based on Mechanics & Materials I, Mechanics & Materials II, Thermal/Fluids I, Thermal/Fluids II, Mechanical Systems I, Mechanical Systems II, Dynamic Systems I and Dynamic Systems II.

Good preparation alone may not guarantee good GPA

However, in order to have good engineering grade you have to have good preparation

Test Score vs. Final ME GPA

All Semesters

0

10

20

30

40

50

60

0 0.5 1 1.5 2 2.5 3 3.5 4

Post IME GPA

Re

vie

w S

co

re

Post IME GPA vs. Review Score Linear (Post IME GPA vs. Review Score)

Summary

The test can be used effectively as a diagnostic and prognostic tool

Also supported by anecdotal information from senior exit interviews as well as informal conversations with students/faculty

More systematic analysis on its effectiveness is needed to improve the test

Inform students early on (during freshman year), and allow students to take practice tests prior to taking Intro to ME course.

Coordinated effort in early intervention and remedial assistance is needed.

Will share the data with FAMU and FSU Math/Physics Departments.