What is "true" about Florida Schools Improvement

By Michael Martin

Many efforts to promote ideas taken from the school reforms in Florida often are a classic example of “truthiness,” defined as believing what one wants to be true instead of what is actually true.

Florida did achieve considerable improvements in student achievement during the last decade. But it is important to see when the gains occurred and where they occurred.

Florida had an overall Grade 4 reading score of 205.7 in 1998, which was almost the same as the 204.9 score in 1994 on the National Assessment of Education Progress (NAEP) tests. The score in 2002 was 214.4, rising to 219.5 in 2005, and 223.5 in 2007, and 225.7 in 2009. Thus from a less than one-point change in 1998, Florida saw an almost 10-point change in 2002, a five-point change in 2005, a three-point change in 2007 and a two-point change in 2009.

Clearly something significant occurred to improve reading test scores in Florida between 1998 and 2002 that continued at a much lower level in subsequent years.

The No Child Left Behind Act could only have had a role in diminishing the gains, because its implementation occurred after 2002. Similarly, the voter-passed class size reduction initiative in Florida occurred in November of 2002. Mandatory Grade 3 retention was a law passed by the Florida legislature in 2002 and began in the 2002-03 school year. These reforms can only be associated with the declines in test score gains.

Averages can sometimes be deceptive. Looking at the NAEP scores marking the 10th percentile low end versus the 90th percentile high end provides an insight into what occurred.

Grade 4 reading scores at the 90th percentile went up 4 points from 1998 to 2002, but at the 10th percentile scores went up nearly 15 points. Thus the 10 point overall average gain in Grade 4 reading scores on the NAEP from 1998 to 2002 was primarily a gain at the lowest scoring levels, and Hispanic scores at the 10th percentile went up 19 points and African-American scores went up 17 points.

Something did happen in Florida in 1999 that specifically targeted low scoring students. In 1999 Florida created its “Assistance Plus” program targeting additional resources at schools rated as “D” or “F” in student test scores.

In an October 2002 status report on the Assistance Plus program, it noted that 64 schools were targeted for managerial assistance, including training teachers for “Effective Reading Practices” and funding reading coaches as well as coordinating activities with local college Assistance Plus Teams. The status report noted that “over $25 million statewide” in fiscal assistance was provided to these 64 schools, or about $390,000 per school.

The state created regional school improvement facilitators who were assigned failing schools to work with. These failing schools were paired with higher performing schools to provide peer models and assistance. In addition, colleges provided research-based assistance for implementing school improvement programs.

The schools also began working with other community organizations and administered community outreach programs, including climate surveys in languages of the parents. The Achievement Plus program perfectly fits the time and target of when and where Florida’s test scores improved.

From 2002 to 2005 the gains at the lowest scoring levels in Florida continued. At the 90th percentile NAEP Grade 4 reading scores increased by only 3 points, but at the 10th percentile scores increased over 10 points, with Hispanic scores at the 10th percentile increasing almost 15 points and African-American scores increasing over 12 points.

It is likely that the de-emphasis of the Assistance Plus program in later years accounts for the diminishing increases in Florida reading scores in more recent NAEP tests.

After 2005, the 10th percentile Grade 4 reading scores continued to show improvement, but at about half the levels of previous years. In fact, 90th percentile scores in 2009 had actually fallen by about half a point, but 10th percentile scores increased over 5 points while the overall average increased only 2 points.

People who claim various programs were responsible for the improvement in NAEP test scores in Florida over the past decade must explain why their improved NAEP reading scores primarily occurred among the lowest scoring students while other student scores largely stagnated, and why those increases were most dramatic from 1998 to 2002, diminishing afterward.

Targeting the lowest scoring schools in Florida for funding increases through Assistance Plus also meant targeting these funds at low-scoring minority students, and low-scoring minority students had the largest test score improvements. It is the large increases in 10th percentile test scores that created Florida’s much touted reduction in the achievement gap between minorities and majority students.

Targeting additional resources toward increased management training and professional development among teachers dealing with the most difficult students to teach makes simple common sense. Helping schools and teachers dealing with significant problems in Florida by providing additional human and fiscal resources fulfills Occam’s Razor for the best explanation of Florida’s success over the past decade.

From the Answer Sheet: http://voices.washingtonpost.com/answer-sheet/guest-bloggers/what-really-helped-floridas-te.html?wprss=answer-sheet

Is Jeb Bush stuck in fourth grade?

A Sherman Dorn twofer. -cpg

By Sherman Dorn

The title above is a bit flippant, but since Jeb Bush's op-ed in the Wall Street Journal yesterday focused (again) on fourth-grade achievement data in reading, it's important to keep in mind that there are students in other grades, too. Florida's fourth-grade reading NAEP data is where you see the strongest evidence on the Florida policy effects of the past decade. PolitiFact rated the narrow claim about fourth grade as mostly true, using NAEP. There is an active debate about the extent to which third-grade retention is polluting fourth-grade achievement data, but I think there are broader and equally important perspectives:

■In the op-ed, Governor Bush ignored his own advocacy and support for a state technical assistance agency specifically for reading and for reading coaches in Florida's elementary schools.
■Governor Bush ignored possible effects of the voter-approved class-size reduction mandate since 2002 and the benefits the state budget and local school taxes received from the early-aughts real-estate boom in Florida.
■The evidence of achievement improvement in Florida, and of narrowed achievement gaps, is weaker in other subjects and for other grades.

Four fading fads and four future fetishes, 2011 version

By Sherman Dorn on January 4, 2011

Belated happy New Year! In the spirit of renewal, here is some clearing out of old cobwebs and identification of likely new ones. Fading and (briefly) the reason why I have some hope on these fronts:

1.School uniforms as panacea: Pinellas County turned a silly idea by the superintendent into a more sensible focus on what's appropriate dress in general.

2."Brain-based" nostrums: didn't read this much in 2010, that I can recall. (For those who are curious: all learning in humans is brain-based. There is, unfortunately, no direct line from the research in cognitive psychology to specific instructional practices. If there were, I'd be looking for lessons on how to teach history in the research on chronosthesia, or subjective time.)

3."Mozart effect" marketing: I saw "uterophones" in Wired recently (though I couldn't find the item this morning when searching). With luck, that'll be the last silliness along these lines for a few years.

4."65% solution" (aka mandating a certain proportion of education spending "in the classroom," vaguely defined): haven't heard word of this in many moons.

But don't worry, magic-bullet seekers! We are sure to get or continue new fetishes that will turn out to be as unreality-based as the ones above:

1."At least 50% of teacher evaluation should be reserved for test scores and their derivatives." I'm one of those folks who think student outcomes should matter in evaluation, but the direction on this is about as realistic as the key NCLB drivers. I agree with Bill Tucker's guesstimate of 25-30% as a realistic proportion of evaluation for student outcomes. I may be wrong in predicting that 50% will turn out to be wildly unrealistic in practice, as the "65% solution" is (and at least the latter was largely political pandering). But regardless of your first guess as to an appropriate weighting, this level of specificity is policy chutzpah; a specific target number targeted by jurisdictions all over the country isn't based on any existing research, and attempts to put it in stone (i.e., statute) constitute a fetish.

2."Technology can individualize everything," or tech lust, software version. The Digital Learning Council's manifesto at the end of November overgeneralized from the experience of intense computer-assisted instructional planning in School for One. Yes, there are subjects where assessment-guided instruction can be far better pinpointed. The development costs for that can pay off for some subjects, but not for everything we care about.

3.Tech lust, hardware version. I use my iPad for a number of tasks, and I'll be trying the Attendance app this semester (hat tip). First, this is a personal iPad, not a university device. Moreover, I am skeptical that a tablet is a wise first choice for student use, though we're seeing some experiments by wealthier suburban districts and a brand-new story in the New York Times for the faux-trend aspect. More generally, school districts are vulnerable to making gigantic tech purchases without thinking through the consequences, such as the use of federal stimulus money for laptops for all Detroit teachers and 40,000 students even while laying off bus aides for special-education students (hat tip). Even apart from the person-vs.-machine issue,* think about the books you can buy with the cash that otherwise goes to a single iPad or Dell laptop. Putting effort into making websites friendly for cell phones and tablets? Absolutely! (This blog uses the "WPTouch" WordPress plug-in so anyone with an iPhone or an iPod Touch sees the blog with iPhone-specific formatting.) Buying a limited number of tablets for use in specific circumstances?Yes: I can imagine some special situations such as Lindamood Bell tutoring where the next-generation iPad could provide instant feedback in speech-and-language therapy. But the broader tech lust? It's a perennial problem.

4."Learning styles." I wish I could consign this term to the dustbin of education fad history, but I can't. The District of Columbia's evaluation system (IMPACT) has a specific observational component tied to teachers' accommodations of learning styles, and now Florida's new official list of Accomplished Practice for educators requires that assessment accommodate learning styles. I'm just waiting for the first lawsuit stemming from the new Florida regulations: "My kid would've had a higher grade if you had accommodated his learning styles." Not documented needs, as in a 504 accommodation plan or an IEP, but learning styles.

Three of the eight items above owe something to people who do work in cognitive psychology, or at least do enough to translate it for those of us who are psych laypeople. In that vein, I think the public owes a certain debt to Daniel Willingham, who is hanging up his boots at least for now as guest columnist for Valerie Strauss. It's a shame, because he's good at translating research into appropriate caveats for practice. So go reread what's there by him…

* As far as I am aware, there is only one suggestion for how to get a laptop to control human behavior on a moving vehicle, and the theory of action is not empirically tested, or at least it hasn't been reported yet in a refereed publication.

http://shermandorn.com/wordpress/