America's Obsession with Ed Reforms that don't work

From the Washington Post's Answer Sheet

By Carol Corbett Burris

My three daughters loved Miss Levin. Each of them had Gail Levin in elementary school. Whenever she was their teacher, I knew I was taking the back seat from September to June. I did not mind. She made them feel very smart. That was because she challenged them to think deeply and critically about what they learned. She reminded me of the very best teachers that I have enjoyed at all levels of my own schooling — those who would not accept opinion as fact and challenged my thinking with the opposite point of view.

Teaching students to carefully consider evidence and distinguish opinion from fact is so important. It helps them become good citizens. We depend on our teachers and our leaders to clearly communicate what is fact from wishful thinking, modeling good decision making for us all. The importance of evidence should never be disregarded, even when — particularly when — it gets in the way of an agenda. This is true when we are bringing a new drug to market or bringing a medical procedure to scale, and it should be true as well of educational change.

Yet in 2009, when Education Secretary Arne Duncan was asked by an Education Week reporter about the evidence base for the policies of his department, he replied, “So I would argue the whole turnaround stuff is relatively new but I think there’s a lot of scientific evidence that the status quo doesn’t work and that’s the evidence that I’m looking at.” That is akin to saying, “I know my child is ill so I will give her any new medicine I happen to have in my cabinet.”

Yes, we can agree that there is a great deal about the status quo in schools that deserves reform, but that tells us absolutely nothing about whether any given reform is helpful, harmful or simply useless. (As an aside, my hunch is that we might not all agree with Secretary Duncan if we separately compiled our list of grievances with the status quo.)

This same minimizing of the importance of policy being guided by high-quality evidence is being played out at the state level, where misleading information is presented as fact. Allow me to present two troubling examples.

Take a look at slide number 20 contained in New York State Education Commissioner John King’s PowerPoint presentation. The graph purports to demonstrate that if we invest in technology, student achievement will accelerate at fantastic rates. In addition, we will get an outstanding return for our tax dollars if money is invested in “teacher effectiveness” programs. One is left to imagine what would happen if these two reform strategies were combined.

But where is the evidence in research to support these claims? As Bruce Baker of Rutgers University argues, there is no evidence at all. Baker did an outstanding job of debunking this bogus graph on his blog. His critique, which is a must read, can be found here.

The graph, labeled “illustrative” (which seems to be a new synonym for “fabricated”) in the PowerPoint, was also included in a presentation to the New York State School Board Association, along with an equally misleading slide, Teachers Matter Most, which followed it. See slide 18 here. This second slide makes two claims: (1) Research shows that an effective teacher is the most important contributor to student learning, and (2) Students with effective teachers three years in a row will bridge the achievement gap.

Let’s take these claims one at a time.

Research shows that of all of the IN-SCHOOL factors that affect learning, teachers contribute the most. In-school factors contribute about 20% of the variance of student scores — an equal amount is unexplained (error), which researchers often refer to as noise. Most of the variance is captured by out-of-school factors, generally linked to poverty and wealth.

Of those in-school factors, roughly half is attributable to teachers. If the slide were accurate it would say: Research shows that in-school factors contribute 20% to student learning. Of that 20%, at least half is attributable to teacher quality. Matt Di Carlo who writes for the Shankar blog provides an excellent explanation of the above breakdown between in- and out-of-school factors here. Similarly, the Educational Writers Association recently explained the limits of teacher contribution here.

The first claim would not be half as worrisome if it were not for the second claim — that three effective teachers in a row will bridge the achievement gap. As one who has worked relentlessly (and successfully) to bridge that gap in my high school, I know that this claim is more than an oversimplification — it is deceptive. It’s worth turning to Di Carlo again; he clearly explains the origins and problems with that claim. Moreover, he was disputing the claim that FIVE effective teachers in a row can close the gap, not three as presented by the New York State education commissioner and as posted on the State Education Department website.

When the state puts the entire burden of closing the gap on the backs of teachers, while ignoring such gap-producing factors as poverty and underfunded schooling, it hides the truth from the public regarding the complexity of the issues that must be addressed. Perhaps most importantly, it takes the burden off society to address issues such as poverty and inequitable school funding. It gives politicians permission to not address serious problems that affect learning such as teenage pregnancy, truancy, illegal drug use, gangs, and uneven access to health care. And it ignores the effects of racially and socio-economically isolated schooling and classrooms and pretends that separate and unequal are just fine — if only those teachers would do their jobs better.

Jonah Rockoff of Columbia University, who consults with the New York State Education Department on VAM (value-added modeling) scores and teacher evaluation, recently made a presentation at the Nassau Boards of Cooperative Educational Services. He included a slide, far more modest in its claims, regarding effective teaching and the achievement gap.

When I questioned him about it, he was quite honest and admitted that there exists no empirical proof that three effective teachers in a row would close the achievement gap. It is merely a hypothetical extension of results from a model. He also honestly admitted that there exists no study that demonstrates that evaluating teachers using student test scores results in gains in achievement.

Yet the New York State Race to the Top budget for implementing the new evaluation system is $14,500,000. As you look through the rest of the budget, (see pages 59 and 60), you will be staggered by the costs of unproven reform. Then look for a category called “offset local costs of education during a recession. You will not find it. The money that goes to districts will be quickly spent on mandated training for this unproven system and its troubling incentives and disincentives. Surely there will also be additional costs at both the state and local level.

During the past decade our school has worked hard to bring challenging learning and success to all students, and this year all of our 11th graders (with the exception of students who are developmentally delayed) are taking IB English. Anyone who knows the rigor of the IB knows that is quite an accomplishment.

It required hard work on the part of our teachers and students to get where we are today, but we have made it work for students — those from million dollar homes and those (now nearly 16% of our students) who receive free or reduced price lunch. It required time, study, professional development, curriculum development, ample community support and dedication. It happened in an integrated community willing to devote resources to level the playing field for all kids. It required the leadership of a strong superintendent who let his principals and teachers try new ideas without the fear of being fired over test scores. Yes, it was not the exciting “turnaround stuff” that sorts and selects teachers into four groups while having instruction driven by the results of state tests. Instead, it was based on evidence and hard work.

Because I am in the final years of my career, I will never personally be harmed by this new evaluation system. But I, like my fellow Long Island principals understand how this new system will slowly undermine true reforms like the one at my school. That is why we care so deeply and speak with one voice.

When my daughters and I found out that Gail Levin retired and was very ill, four grown women cried. So many years later, we still felt a warm bond with this wonderful teacher. Teachers like Miss Levin will become rare birds in the test prep schools to come. They will fly to other professions or flock to the private schools rather than parrot the drill needed for students to correctly answer the multiple choice questions on the state exam. But to honor her and all the great teachers who have graced our own learning, I hope and believe that educators, parents and others will not give up the fight to fend off unproven “turnaround stuff.”

http://www.washingtonpost.com/blogs/answer-sheet/post/proof-there-is-no-proof-for-education-reforms/2011/11/13/gIQAAeVWJN_blog.html?mid=52&utm_source=feedburner&utm_medium=email&utm_campaign=Feed%3A+GLCWorthARead+%28Worth+A+Read%29