ITEAMSITEAMSInnovative Technology-Enabled Astronomy for Middle Schools

An ITEST Strategies ProjectOf the Science Education Department

At the Harvard-Smithsonian Center for Astrophysics

Philip M. Sadler, Principal InvestigatorR. Bruce Ward, Project Director and Co-PI

Jaimie Miller-Friedmann, Associate DirectorFrank Sienkiewicz, Project Engineer

Paul Antonucci, Curriculum Specialist and EngineerWendy Berland, Department Administrator

ITEAMS Assessment:Using DDMC Questions

In an Online System

•Diagnostic pretests

•Gain analysis on pre-post tests

•Longitudinal data for students and teachers

•Identification and tracking of misconceptions

ITEAMS Assessment:Using DDMC Questions

In an Online System

•Online Testing System

•Automated reporting to test-takers

•Immediate gratification for PD Providers!

•Decrease in human error

DDMCDDMC::The History

•Misconceptions

•Higher Discrimination than traditional MC tests

•Ability to reach a wider audience (vs. interview)

•The J-Curve, misconceptions strengthen first, before moving to the scientifically correct conception

Sample Item:

How long does it take for the Moon to go around the Earth?

a. One day.b. One week.c. One month.d. One year.e. It never happens.

Sample Item:

How long does it take for the Moon to go around the Earth?

a. One day. 27.3%b. One week. 6.8%c. One month. 46.2%d. One year. 15.2%e. It never happens. 3.0%

Pretest

How long does it take for the Moon to go around the Earth?

a. One day. 27.3%

16.2%b. One week. 6.8%

7.4%c. One month. 46.2%

57.4%d. One year. 15.2%

11.8%e. It never happens. 3.0%

5.9%

Sample Item:

Pretest Post-test

How long does it take for the Moon to go around the Earth?

a. One day. 27.3% 16.2%31.5%

b. One week. 6.8% 7.4% 8.9%

c. One month. 46.2%57.4% 30.8%

d. One year. 15.2%11.8% 15.6%

e. It never happens. 3.0% 5.9%13.2%

Sample Item:

Post-testPretest National Sample

How long does it take for the Moon to go around the Earth?

a. One day. 27.3% 16.2% 31.5%0.0%

b. One week. 6.8% 7.4% 8.9%22.2%

c. One month. 46.2% 57.4%30.8% 77.8%

d. One year. 15.2% 11.8%15.6% 0.0%

e. It never happens. 3.0% 5.9% 13.2%0.0%

Sample Item:

Post-testPretest National Sample

Teachers

Significant Gender Differences in Affective Survey

0

1

2

3

4

5

6

7

Interest inSTEMCareer

DevelopingNew

Knowledgeand Skills

Having anEasy Job

watchingscience

programs

GirlsBoys

Important factors in choosing a careerImportant factors in choosing a career

•30% of Variance Explained by Factor 1:

•Having others working under my supervision •Working with people rather than objects •Developing new knowledge and skills •Having lots of family time •Having lots of time for myself/friends •Making my own decisions •Working in an area with lots of job opportunities

Affective Predictors for Content Affective Predictors for Content Knowledge Knowledge

Factors in Choosing a Career:

(+) Working in an area with lots of job opportunities(+) Working with people rather than objects

(-) Helping other people (-) Making my own decisions (-) Having an easy job

Linear RegressionOutcome Variable: Total Score Int Test

Technology Use and Expertise:

(+) Web browsing and information searches (-) Creating graphics, photos, graphs or tables on a computer

STEM Interest:

(+) I could see myself being a scientist or engineer. (-) I would like to work with people who make discoveries in science

Affective Predictors for Content Affective Predictors for Content Knowledge Knowledge

Linear RegressionOutcome Variable: Total Score Int Test

The Big Idea… The Big Idea…

•ITEAMS modeled after protocols used for larger studies:•SPARCS•MOSART•PRiSE

•Anyone can access SED designed DDMC tests•Answer Keys•Data

http://www.cfa.harvard.edu/smgphp/mosart/index.html