WELCOME

THE 12TH ANNUAL MARCES/MSDE CONFERENCE:

Value Added Modeling and Growth Modeling with Particular Application to Teacher and

School Effectiveness

Big Challenges

• Statistical Modeling• Policy Determination• Psychometrics

They all come together along with high stakes decisions and a great deal of ignorance combined with being well meaning

The Federal Government Is At It Again• NCLB succeeded in making all students

proficient• Now RTTT will succeed in making all teachers

the same• First, we need more longitudinal data• Then we can decide how to analyze the data• Then we need to decide what to do with the

information that results

1. We know what teachers should do to make student’s knowledge increase. Do we?

2. We are trying to measure something that, if it exists at all, seems to exist in very small amounts (teacher effectiveness)– Rockoff (2004): Teachers account for 5 to 6.4%,

schools 2.7 to 6.1%, students 59 to 68%. – Not sure about how much school context

accounts for?

Assumptions

3. How do we demonstrate causality in a natural environment?It would be nice to have a theory. Rubin and others (2004) say theory is critical. Bummer!We seem to assume that teachers teach and students learn, like pouring water into a vase with a seed in it and the seed grows.Not so easy to study unless we do experiments.Maybe students cause good teaching rather than the other way around? Or, Maybe it is bi-directional.

Assumptions

4. Data related issues. Issues include sample size (small for teachers),

missing (student mobility and sickness, etc.) not at random, quality of assessment data (OK), and appropriateness of assessment data (questionable).

5. Teachers involved in instruction whose subject matter is untested. Florida (Prince, et al., 2009) and Memphis (Lipscomb, et al., 2010) found that about 70% of the teachers are involved in areas that are not taught at all.

Assumptions

6. How do we isolate the effect of a single teacher’s role in student learning?

Co-operative teaching, parent volunteers, librarians, etc. all confound the teacher effects. Teaching is a team effort and the work is confounded. We are looking at past teachers, but not much about others in the same year. How to isolate context effects from school practice? Cross-classified models might help a little, perhaps.

Assumptions

7. How do we validate what we are doing?Reliability is relatively easy. Validity requires, again, a theory of what is happening or what should be happening.

8. What about interactions between students and teachers? Are teachers a main effect? Teachers are good with all students? Doubtful.

Assumptions

9. The Assessments are NOT designed to detect change across years. State summative testing is unlikely to do that. PARCC and Smarter Balanced are talking about mid year testing. Probably not sufficient.Use of frequent computer based testing could accomplish this, if we designed the curriculum within a year and linked the curriculum across years in a way that defined the desired growth and permitted its measurement.

Assumptions

10. The scale issue. How should we scale the data? Vertical scales might help but often encourage more confidence than warranted. Does change at the high end of a vertical scale mean the same thing as change at the low end of a scale? How about comparing scales in different subject matter. Is there really a common vertical scale? Doubtful. Bi-factor model can help. How would we even know? Back to the validity issue.

Assumptions

READY TO BE TAUGHT?• I have invited 12 experts to present their work• These are some of the best and the brightest

working in this field• Let’s get started