Value subtracted

This could soon be coming to Florida. -cpg

By Aaron Pallas

The theory is simple, although the calculations are complex: Students learn more from some teachers than others. If we can measure students' achievement after they've spent a year with a teacher - and how much these students have gained from where they started at the beginning of the year - we can isolate their teacher's impact. Then, we can rank the "value" added by that teacher compared to other teachers whose students started at the same place.

Value-added measures, in conjunction with other measures of teachers' skills and abilities, shed useful light on a teacher's performance. Taken in isolation, however, these measures fall far short of telling us all that we want to know. For example, the Teacher Data Reports are based entirely on the New York State standardized tests of English Language Arts and mathematics, which capture a very narrow slice of what students learn in school. Also, each classroom is unique, and it is not possible to statistically treat two different teachers' classrooms as though they were identical.

Here, more specifically, is what the Teacher Data Reports won't tell you:

If a teacher rated highly one year is likely to be rated highly again the following year. Among seventh-grade teachers of English in the city, two-thirds in the top quarter in 2007-08 dropped to a lower quartile in 2008-09 - and 17% in the top quarter of all teachers that year plummeted to the bottom quarter the next.

A teacher's rating on other commonly-used student achievement tests. A researcher found that teachers in a large Northeastern urban school district who had been rated most effective in teaching reading based on a state reading test were ranked lower on two other prominent reading assessments. Among teachers in the top quarter of all teachers on the state reading test, 25% were in the bottom half on the other tests.

How a teacher would be rated with a different value-added formula. Value-added statistical methods vary, as do measures of students, classrooms and schools that seek to isolate teacher impact. For example, the Teacher Data Reports use students' eligibility for free or reduced-price lunch as an indicator of family disadvantage, which is less informative than family income or living in a single-parent home. Each model affects teacher rankings: One study showed that just half of teachers are ranked in the same quartile by two different value-added models applied to the same test results.

A teacher's contribution to students' development of intellectual curiosity, creativity, critical thinking, citizenship or self-direction. These outcomes, so central to our concept of a good education, cannot be gauged by standardized tests. Researchers studying a national sample of elementary schoolchildren and their teachers found that the teachers most effective in enhancing children's social and behavioral skills were not the most effective at teaching reading and math.

Whether a teacher's contribution to student learning will persist into the future. A teacher's impact can range from boosting a standardized test score to improving future learning. Value-added measures reward short-term learning. A recent analysis of fourth-grade and fifth-grade teachers in North Carolina showed that about 20% of a teacher's effect on a student's year-end tests was still present two years later.

Whether a teacher's principal thinks the teacher is highly successful or not. One study of elementary school principals and teachers in a Western school district found that when a principal ranked a teacher in the top half of all teachers in the school at raising student achievement in reading or in math, there was a 40% likelihood that the value-added measure put that same teacher in the bottom half of teachers at that school in the same subject.

Whatever the courts decide, it should be clear that Teacher Data Reports provide little more than crude information about which teachers are more successful than others in promoting learning. We need a new generation of tests that provide a broader picture of student learning and are less susceptible to gaming by schools and school systems alike.

Judging a teacher on the basis of reports with so many limitations is unfair, harmful to the profession and unhelpful to anyone who cares about educating our children.

Aaron Pallas is professor of sociology and education at Teachers College, Columbia University.

http://www.nydailynews.com/opinions/2011/01/03/2011-01-03_what_the_data_wont_tell_us.html?page=1