From Ed Wize
by Jackie Bennet
When it released the 2011 Progress Reports to the public last month, the DOE made a point of noting that charter schools received more A’s than did their regular public school counterparts.
Technically that’s true, but technically is about as far as it goes. When we compare the charter middle school A’s to the public middle school A’s for example, we see that the Progress Reports offer little evidence of better student achievement. In fact, in spite of an uneven playing field that should have tilted the scores in favor of the charters, the Progress Reports actually indicate that when it comes to academics, the middle school charters that got A’s did not do that well.
Leaving aside demographics1, let’s take a look at the things that should have tilted the advantage to the favor of the middle school charter A’s. Then, let’s look at the results and some possible implications for our schools. We often hear that our public schools need to learn something from our charters. But it seems the charters may have a lesson they need to learn as well.
One note: throughout this post, charter A’s and public A’s refers to A-rated middle schools.
Uneven Playing Field: Grade levels
Though 12 of the 23 middle school charters earned A’s last year, only one had the same grade configuration as the regular publics against which it was graded. The different grade-levels should have given the charters a comparative advantage.
First, 9 of the 12 charter A’s included a fifth grade and the vast majority of regular public schools (over 95%) do not. That’s a statistical problem because for a variety of reasons, fifth graders citywide — and probably nationwide — outperform students in grade 6-8. Passing rates on the Math state tests, for example, drop 7 points between 5th and 6th grade. In ELA the drop is 5.5 points. When the performance of schools with 5th graders is compared to schools without 5th graders, the schools that serve them ought to do better on the Progress Reports, which measure exactly these things.
That alone should give the charters with a 5th grade (3/4 of the charters) a significant advantage, but there is more. Five of the charter A’s schools, did not have an eighth grade (one didn’t have a 7th either). Citywide 8th graders score lower than students in grades 5, 6, or 7 for a total drop of 14 points in ELA and 10 in math. Charter A’s without 8th graders were compared to public schools that included them, and again, that should have given the charter A’s higher passing rates, as well as better scores on the Progress Reports.
Finally, three of the schools had a double advantage — 5th grade classes but no students in grade 8.
Uneven Playing Field: Testing Cohort Attrition
Five of the 12 schools have high testing-cohort attrition.
As I noted in a more thorough attrition report last year, the middle school testing cohorts shrink at alarming rates. As students disappear, passing rates tend to rise proportionately, leaving one to suspect that it is the students with the greatest struggles who disappear.
Not all of the charter A’s had high attrition, but those that did not tended not to do as well. In any case, between the 2010 and 2011 testing cycles, for example:
•Five of the 12 A-rated middle school charters saw their 5th grade class shrink by anywhere from 10 to 20 percent by the time the students reached grade 6.
•Three of these five also had high attrition between grades six and seven (a additional 24%, 22% and 14%)
A few of the charter students might be disappearing into a different cohort (if they were left back, in other words). However, since virtually all the testing groups shrink, including the groups into which students could have been left back, it is logical to assume that most have simply left the school.
Meanwhile, a check of the public middle schools reveals no such trend. But just as important, the vast majority of the public schools take in new students at every grade level. Students might arrive at a regular middle school at the middle of 8th grade, or the start of 7th, or any time in between, but all — or virtually all — of the charter middle schools will accept students only in the first year (i.e., grade 5) and continue to educate only those who stay.
Results: Academic Achievement
In spite of attrition and the statistical advantages the middle school charters had because of the grade configurations, academic achievement was not any better in the charters than the publics. In fact, according to the Progress Reports, it was worse.
After factoring in demographic differences, the DOE measures academic achievement on state tests as both progress and performance (performance is essentially passing rates) against ostensibly similar schools. Then it gives the schools a letter grade for each category (one for progress and one for performance). These sub-grades (along with another sub-grade for the school environment) are then rolled into the final grade. Here are the sub-grade results for academic progress and performance for the A-rated charters:
•58% of the charter A’s did not get an A for progress and also did not get an A for performance. Only 19% of the public A’s were in a similar situation.
•67% of the charter A’s did not have an A in progress, compared to 27% of the regular public A’s.
•The charter A’s slightly outpaced the publics in performance, but only slightly. Thirty-three percent received A’s for performance vs. 26% of publics. Keep in mind, however that performance means passing rates, and the different grade configurations would have favored the charter A’s.
Together, the progress and performance achievement measures account for 85% of each school’s final grade.
So if 58% of the charter A’s were not getting an A in either progress or performance, how did they get an A for a final grade? According to the Progress Reports, these schools scored an A for their school environment.
And that was enough to bring them up.
Results: The Learning Environment
Every single charter A received an A on the “environment” section of the Progress Report, which is a measurement of the schools’ attendance rates and survey results. These high scores in environment indicate high rates of attendance. They also mean that parents, students, and teachers have said on the DOE’s annual survey that the school is safe and has a good overall school culture.
Not only did the charter A’s score well; they also scored better than the public A’s. And, while 100 %of these charters got an A for their environment, 35% of the A-graded regular publics did not.
Implications
If some charters have indeed created safer, more collaborative environments with a high degree of trust and respect among parents, teachers, administration, and students, then we ought to find out what factors have made that possible. After all, good school environments matter greatly for academic achievement, and they are also a worthy outcome (or, outputs, as the charter folk like to say) in and of themselves.
But if we may have something to learn from the charters, the charters have something to learn from us too. Keep in mind that, 58% did not get an A in either category for academic achievement whereas that was the case for only 19% for the public A’s. What’s more, while 58% of these charters got an A solely because of a high grade on their environments, 35% of the public schools received their overall A in spite of lower environment scores. Academics pulled them through.
And that’s a finding worth stumbling over. It also leads us to ask a question: what advantage did the public A’s have that allowed them to overcome not only a statistical deck that seemed to be stacked against them, but also an environment that was somewhat less conducive to learning?
Hmmm…could it be…the level of the teachers’ experience at the public schools? At the schools that pulled an overall A even though they did not have an A for environment, teachers had relatively high levels of experience. The median number of years of experience in all public middle schools is 7, but two thirds of the public A’s had a median even higher than that. We don’t know much about experience in the charter schools, which (in spite of being public offer no transparency in this regard), but we can make some estimates. In the very limited but random DOE data sets I have at my disposal, it appears that the mean experience level at New York City charters is just 2 years.2
Teacher experience, as well as the stability that comes with it, makes a difference in schools. Teaching is a hard task to master even for very promising new teachers who must make hundreds of high-stake decisions in their classrooms every single day. Every serious researcher that has studied the impact of teacher experience has concluded in one form or another that you cannot improve student learning on the backs of teachers who are essentially at the apprentice stage of their careers.
But as important as experience is, stability matters too. I’m very old fashioned when it comes to schools and teaching. I believe that children like and need structured, stable communities where they can see familiar faces and become a part of real school traditions. And I intentionally draw a distinction here between “real” school traditions and the phony, pop-up cultures that charters sometimes impose. True academic cultures are shaped by the idiosyncratic individuals who have committed themselves to the institution for the long haul. Teachers need to be good, yes. But they also need to be there, year, after year, after year.
Conclusion
I know that this post rests on the assumption that the Progress Reports can tell us something real about the schools. Maybe they can, and maybe they can’t. But so long as the DOE continues shut our schools based upon what these reports sometimes say, then we too — and our colleagues in the charter schools — should consider those lessons we can draw from them that the DOE is less anxious to publish in the press.
That lesson is two-fold.
For the public schools the lesson should be about the importance of a good environment. Right now, we have a situation where good teachers earn A’s for their schools in spite of what happens in the administrative cabinet and outside the classroom door. That’s not good enough. Rather we must work to become schools where the entire environment supports good teachers, and their students, in their work.
The charters, meanwhile, need to learn to attract experienced teachers and retain the new teachers that they have.
And that is something we can show them how to do.
http://www.edwize.org/sharing-best-practices-a-lesson-for-the-charters
No comments:
Post a Comment