Blaming teacher's unions for education problems has no basis in fact

From the Shanker Blog, by Matthew DiCarlo

For the past couple of months, Steve Brill’s new book has served to step up the eternally-beneath-the-surface hypothesis that teachers’ unions are the primary obstacle to improving educational outcomes in the U.S. The general idea is that unions block “needed reforms,” such as merit pay and other forms of test-based accountability for teachers, and that they “protect bad teachers” from being fired.

Teachers’ unions are a convenient target. For one thing, a significant proportion of Americans aren’t crazy about unions of any type. Moreover, portraying unions as the villain in the education reform drama facilitates the (mostly false) distinction between teachers and the organizations that represent them – put simply, “love teachers, hate their unions.” Under the auspices of this dichotomy, people can advocate for changes , such as teacher-level personnel policies based partially on testing results, without having to address why most teachers oppose them (a badly needed conversation).

No, teachers’ unions aren’t perfect, because the teachers to whom they give voice aren’t perfect. There are literally thousands of unions, and, just like districts, legislatures and all other institutions, they make mistakes. But I believe strongly in separating opinion and anecdote from actual evidence, and the simple fact is that the pervasive argument that unions are a substantial cause of low student performance has a weak empirical basis, while the evidence that unions are a primary cause of low performance does not exist.

In a recent article, Brill claims that he has “read all the white papers and commission reports.” Yet his book contains only about a dozen research citations, and not a single mention of the work that’s been done on the relationship between unions and achievement.

In a prior post, I discussed the state of this research. In a review of the literature, published as a chapter in this 2006 book, economist Dan Goldhaber noted:

Despite the strength of the heated rhetorical debate about unions, from a theoretical perspective there is no definitive reason to believe that unions would either promote or hinder student achievement. […] Furthermore, the causal impacts of teachers unions on students are difficult to establish since unions do not arise randomly. […] The evidentiary base for drawing conclusions about the student achievement impacts of teachers unions is quite thin.
In other words, isolating the effect of unions or collective bargaining on achievement is extremely difficult, and there’s no empirical or even theoretical basis for strong claims either way.

Similarly, a 2002 review of 17 studies on this topic concluded that most found no effect of unions/bargaining on achievement (and several even found a positive association), but also that there was contextual variation in their impact – for example, one paper found slightly negative union effects on low-performing students.

This more recent national analysis of collective bargaining’s effect on achievement in grades 8-10 found no discernible impact, while this paper (published this year) found that “contract restrictiveness” was associated with slightly lower achievement levels in California, but that there was no association between contracts and achievement growth.

In general, the existing research is mixed, and suggests that there is only a weak relationship between unions and achievement scores. Given the methodological difficulties (for example, non-random assignment of unionization and the fact that some policies, such as tenure, exist in all states), as well as the fact that the research is still developing, we cannot say for certain whether unions exert some large-scale influence over student outcomes.

But it is reasonable to suggest that if unions really warranted their mythical position as the primary cause of low educational performance, we might expect to find at least modest negative effects, on some consistent basis, within the body of evidence on the topic. This is not the case.

There are also more observational forms of evidence on this score, which the generally-on-point reviews of Brill’s book pointed out. For instance, as I explained in a this post, there are ten states in which there are virtually no binding teacher contracts. If unions, mostly through the bargaining process, are the major cause of low performance, scores in those states where unions’ actions are limited or proscribed might be at least marginally higher. They are not.

A similar point might be made about charter schools, which receive a great deal of attention in Brill’s book. Most of these schools are not unionized and free to determine pay and working conditions as they see fit. If the personnel policies common in regular public schools were indeed the primary causes of poor student outcomes, charter schools should, on the whole, outperform comparable public schools. Yes, there would still be variation – some good, some bad. But it would be reasonable to expect to see some consistent positive impact overall. We do not. A few charters get exceptionally good results, for reasons that remain an open question (school time, tutoring and funding are among the more compelling possible factors). But, on the whole, regular public schools perform just as well or better (also see here).

Finally, although cross-national comparisons are even more awash in caveats, it is worth noting that teachers in many of the highest-performing nations in the world, such as Finland, Canada, Singapore, and South Korea, are heavily-unionized. Now, it is certainly true that other top-scorers, such as the Chinese city of Shanghai, have weak teachers’ unions, but this only reinforces the conclusion from the research, and that suggested by the results among charter schools and non-union states: That the connection between unions and performance is not obvious.

So, again, the empirical and observational evidence on the union/achievement relationship is far from definitive, and it’s not yet possible (if it ever will be) to make strong statements about it. But the claim that unions are the big reason for poor educational outcomes is really just speculation.

It is also a distraction. Most responsible people, including the secretary of education, acknowledge that changing education policy pertaining to teachers cannot work – will not work – without the support of teachers and the organizations that represent them. Brill himself makes a similar argument toward the end his book (and holds up a few examples of locals that are, in his view, acting positively).

It’s an important point. Unfortunately, it’s preceded by a few hundred pages of unsupported blanket statements that suggest, if anything, the opposite. I assume Brill believes that unions must be “forced” to change, and that his rhetoric makes him an agent of such change. Moreover, he probably thinks that his failure to review the relevant literature is due to the fact that he’s a journalist, not an academic, and that his research came in the form of extensive interviews with dozens of figures in education policy.

That’s all fine – he’s entitled to his opinions and his methods. I am even inclined to agree with him on a certain issues (if not the specifics), including using test-based productivity measures in teacher evaluations and thoughtful new compensation systems.

But none of these and other interventions for improving teacher quality has anything resembling a consistent track record of success. Much of this is relatively new. Getting these policies right will require not only patience and attention to detail, but also collecting evidence from rigorous policy evaluations. There is no basis for certainty, and identifying good guys and bad guys, while it might make for good bedtime reading (I personally enjoyed the book), is not a useful part of the policymaking process.

That said, if Brill and those who agree with his portrayal still insist on political theater in which it’s unions versus the rest of the world, then their show can go on. There’ll always be an audience for that production. But, in the end, it’s not very productive.

http://shankerblog.org/?p=3509