using student data to inform design force and motion tom regan session c, strand 6: assessment ksi...

Using Student Data to Inform DesignForce and Motion

Tom Regan

Session C, Strand 6: Assessment

KSI 2006

Copyright © 2006 AAAS Project 2061 2

Outline

• Generic Items

• “Balanced” & “Unbalanced”

• Speed Table

• Arrows v. Words

• Logical Operators


Pilot Test Sites

• 6th grade; four classrooms

• Suburban

• 10% Free & Reduced Lunch

• Had studied “Forces’ Effect on Matter”

• 8th grade; four classrooms, one G/T

• Small town

• 33% “economically disadvantaged”

• Had 15 days’ exposure to forces and motion


Interview Sites

• 8th grade; three students

• Urban school

• 78% FRE

• Had studied motion

• 7th & 8th grade; ~ 10 students. Most students 8th grade G/T

• Suburban

• 11% FRE

• Had one year of physical science, including forces & motion


Generic Items

Item 36-2, Classroom 2A6B1 (6th grade; n=34)An object is moving. A force acts on the object in the direction of

the object's motion. What will happen to the object's motion while this force acts?

A. (47%) The object's motion will depend on the type of force.B. (6%) The object's speed will stay the same.*C. (21%) The object will speed up.D. (15%) The object will slow down.

Items that used generic “force” and “object” rather than, for example, “box” and “push”


Selected Student Comments on Answer Choices (1 of 2)

• Opposing Force (8 of 34 students)– Students thought that the force is in the opposite direction to the

object’s motion.

– Ex. “Why would it speed up if the force is in the way of its motion?”

• Two Forces (4)– Students thought that there are two forces acting on the object.

– Some of these were “force of motion” misconception

• Introduction of Extraneous Forces (2)– Students introduced the forces of gravity and friction into the problem

– These students don’t realize that the force mentioned in the stem is the only force on the object


Selected Student Comments on Answer Choices (2 of 2)

• Type of Force (3 of 34)– Students thought that Hard/Soft and Big/Little are types of forces

– Item writer’s intent: “Type” includes everything about force besides strength, direction, and duration

– Popularity of choice A suggests that this was not students’ interpretation.

• Force “hits” (4)– Students used the word “hit” where we would say “acts on”

– Could this reflect/lead to confusion with an impulsive force?

• No students commented on the word “object”


Generic Item Interviews

Suburban School: ~6 students, mostly 8th grade G/T

• (two students) prefer generic to particular

• (three students) compare situation to pulling-cart-lab (and get the right answer).

• (one student) Is it a “tap” force or a “constant” force?

• 7th grade G/T, hadn’t taken physics: prefers particular to generic

Urban School

• 8th grade: either generic or particular is okay (gets right answer)


“Type of force” Distractor

Item 36-2, Classroom 2A6B1 (n=34)

An object is moving. A force acts on the object in the direction of the object's motion. What will happen to the object's motion while this force acts?

A. 47% The object's motion will depend on the type of force.B. (6%) The object's speed will stay the same.

*C. (21%) The object will speed up.

D. (15%) The object will slow down.

Many students selected answer choice A

It is an effective distractor and should be retained.

It distracts students from the targeted learning goal (Newton’s 2nd Law) and should be removed.

?


“Balanced” and “Unbalanced” Forces

Item 38-2, Classroom 2A6B2 (6th grade; n=14) and

Item 57-1, Classroom 2A6A2 (6th grade; n=26)

An object is [38-2] speeding up / [57-1] slowing down. Which of the following statements about the forces acting on the object is TRUE?

A. (29%) / (35%) Balanced forces are acting on the object.

B. (*50%) / (12%) An unbalanced force is acting on the object in its direction of motion.

C. (14%) / (*35%) An unbalanced force is acting on the object in the direction opposite to its motion.

D. (7%) / (8%) No forces are acting on the object.


• Whether an object speeds up or slows down depends on the direction of the unbalanced force relative to the motion

• But some students think that speeding up/slowing down depends on whether the force is balanced or unbalanced:

Forces are balanced object speeds up 3 of 14 students

Forces are unbalanced object slows down 2 of 14 studentsForces are balanced object stops 3 of 14 students

Forces are balanced object slows down 3 of 26 studentsForces are unbalanced object speeds up 1 of 26 studentsForces are balanced object stops 2 of 26 students

Some Student Ideas about Balanced/Unbalanced

Spe

edin

g up

item

Slo

win

g do

wn

item

Your experiences with balanced/unbalanced?


Speed TableFM64-1In the drawing below, the arrow labeled "FORCE" represents a force acting on an object. The length of the arrow represents the strength of the force, and the direction of the arrow shows the direction of the force. The arrow labeled "Motion" shows the direction of the object's motion.

Every second, starting when the clock reads "0 seconds," the speed of the object is measured. Which row of the table could be a correct representation of the object's speed between 0 seconds and 6 seconds?

Clock► 0 sec 1 sec 2 sec 3 sec 4 sec 5 sec 6 sec

*Row A (~40%) 10 mi/hr 11 mi/hr 12 mi/hr 13 mi/hr 14 mi/hr 15 mi/hr 16 mi/hr

Row B (~10%) 10 mi/hr 10 mi/hr 10 mi/hr 10 mi/hr 10 mi/hr 10 mi/hr 10 mi/hr

Row C (~5%) 10 mi/hr 10 mi/hr 10 mi/hr 13 mi/hr 13 mi/hr 13 mi/hr 13 mi/hr

Row D (~15%) 10 mi/hr 11 mi/hr 12 mi/hr 13 mi/hr 12 mi/hr 11 mi/hr 10 mi/hr


Speed Table Results

Pilot Testing

• This item, and a similar item with force in opposite direction, were administered to 2 sixth-grade and 2 eighth-grade classes (total n=81)

• 33% indicated confusion (but not its cause)

• 41% chose the correct answer

Interviews (~5 8th grade G/T students),

• I asked students to describe each answer choice before choosing an answer

• Students chose correct answer


Improving Distractor C

• Choice C is incorrect because the speed should change the entire time that the force acts, not just between 2 sec and 3 sec.

• Students wrote that it was incorrect because of the abruptness of the change in speed. Not exactly right. Perhaps abruptness is implausible.

• Alternate distractor

• Still incorrect, but perhaps more plausible


Row C (~5%) 10 mi/hr 10 mi/hr 10 mi/hr 13 mi/hr 13 mi/hr 13 mi/hr 13 mi/hr


10 mi/hr 10 mi/hr 11 mi/hr 12 mi/hr 13 mi/hr 13 mi/hr 13 mi/hr


Speed Table Next Steps

• Specify the time-dependence of the force. Example: “The force is constant & acts for all six seconds.”

• Change distractor C.

• Pilot equivalent item with all-word stem (no force & motion arrows) to isolate source of confusion

• Compare to items using speed-time graphs.


Arrows vs. Words: Arrow Item

FM11-2: In the drawing below, the arrows labeled "WIND" and "WATER" represent forces acting on the sailboat. The length of the arrow represents the strength of the force, and the direction of the arrow shows the direction of the force. The arrow labeled "Motion" shows the direction of the sailboat's motion.

Which of the following statements correctly describes the sailboat's motion while these forces act?*A. The sailboat will speed up.B. The sailboat will speed up for a short time and then slow down.C. The sailboat will move at a constant speed.D. The sailboat will speed up for a short time and then move at a constant speed.


Equivalent Word Item

FM11-3: A sailboat travels on a lake. Two horizontal forces act on the sailboat: the force of the wind on the sailboat and the force of the water on the sailboat. The force of the wind on the sailboat is in the same direction as the sailboat's motion, and the force of the water on the sailboat is in the opposite direction to the sailboat's motion. The force of the wind on the sailboat is stronger than the force of the water on the sailboat. Which of the following statements correctly describes the sailboat's motion while these forces act ?

*A. The sailboat will speed up.B. The sailboat will speed up for a short time and then slow down.C. The sailboat will move at a constant speed.D. The sailboat will speed up for a short time and then move at a constant speed.

There’s also a pair of equivalent items, 55-1 & 55-2, in which both forces oppose the motion


Sixth Grade Results

Item P-A n *A B C D No choice

Confusing

Arrows (FM11-2; classroom 2A6A1)

33% 26 19% 8% 58% 8% 7% 4%

Words (FM11-3; classroom 2A6A2)

52% 27 19% 15% 37% 15% 14% 33%


Confusing

Arrows (FM55-1; classroom 2A6B1)

54% 33 64% 0% 21% 12% 3% 9%

Words (FM55-2; classroom 2A6B2)

40% 12 33% 8% 17% 17% 25% 33%

“P-A” is a measure of the classroom’s force & motion preparation and/or ability. It is the average of the classroom’s scores on two common items.


Eighth Grade Results

Conclusion: Arrow items easier & clearer than word items


Confusing

Arrows (FM11-2; classroom16A8A1)

23% 17 24% 0% 41% 18% 17% 24%

Words (FM11-3; classroom16A8A5)

16% 15 13% 20% 20% 13% 34% 40%


Confusing

Arrows (FM55-1; classroom16A8B5)

29% 28 68% 4% 4% 11% 13% 14%

Words (FM55-2; classroom16A8B4)

30% 24 25% 4% 33% 13% 25% 33%


Logical Operators: Must Item

FM73-4, classroom 16A8A1 (8th grade; n=18)A person pushes a box across the floor. There are two horizontal forces on the box: the force of the push, and the force of friction. The force of friction is in the opposite direction to the box's motion. The speed of the box is increasing. Which of these statements must be true?A. (11%) The force of friction is weak.B. (6%) The force of the push is strong.C. (6%) The force of the push is the same strength as the force of friction.*D. (61%) The force of the push is stronger than the force of friction.

A & B could be true; D must be true


Logical Operators: Goal of Item

• Targeted Learning Goals:– Newton’s 2nd Law

– Multiple Forces

• Goal of Piloting & Interviews: Determine whether the concept “must” can be used in items at the middle level.

• If students cannot handle “must,” then the item will not yield clear information about science content knowledge


Logical Operators: Results

• Most students (11 of 18) got question right

• 9 students’ comments indicated some realization of the logical issue (7 of 9 got question right)A. The force of friction is weak

– “The force may be weak but I think there is a better answer.”

– “It could be, but it doesn’t explain what was happening, correctly.”

B. The force of the push is strong

– “That doesn’t mean it’s stronger than friction.”

– “It doesn’t give enough information.”

– The best comment, in response to A: “The force of friction may not be weak, it just may be weaker than the force of the push.”


Logical Operators: Interviews

• 8th grade G/T

• Students answer question correctly

• Students preferred D to A & B because it’s the “best answer.”

• Students were comfortable selecting the best answer.

• No mention of “must.”


Logical Operators: Discussion Questions

• Which (if any) student comments demonstrate facility with “must” ?

• How to determine student’s engagement with “must” ?– With an item?

– In an interview?

• Is it really an issue of science content?– “Students should know that an object’s change in motion depends

on all the forces acting on the object.”

• Research Question: What if “Not Sure” were not an answer option? – # Not Sure per Student = 1.17


End


Arrow v. Words: “Not Sure”

Item n “Not Sure” per student

Arrows (FM11-2; classroom 2A6A1)

26 0.88

Words (FM11-3; classroom 2A6A2)

27 0.77


Arrows (FM11-2; classroom16A8A1)

17 0.71

Words (FM11-3; classroom16A8A5)

15 1.2


Arrows (FM55-1; classroom 2A6B1)

33 0.48

Words (FM55-2; classroom 2A6B2)

12 0.58


Arrows (FM55-1; classroom16A8B5)

28 0.96

Words (FM55-2; classroom16A8B4)

24 1.25

using student data to inform design force and motion tom regan session c, strand 6: assessment ksi...

Documents

students students

students forces

generic force

force acts

forces motion slide

design force

tap force

impulsive force