Miami's Superintendent Carvalho's Thoughts on Florida's System of Educational Accountability and Assesment

By Alberto M. Carvalho

It is not lost to anyone these days that when it comes to educational accountability and more specifically, assessment, public opinion is being influenced by a number of factors and entities that, in some cases, merit consideration, understanding, and acceptance, and, in others, outright rejection. First, we need to recognize that assessment, as a legitimate tool of accountability, exists only to inform and improve the teaching and learning process. Its use beyond that most legitimate purpose lends itself to misinterpretation of results, erroneous conclusions, perversion of the system itself, and potential harm to students, teachers, schools, and communities. These potentially unintended consequences are accentuated further when the accountability system itself is not inclusive of factors that intuitively and scientifically influence student and teacher performance.

Florida is currently one of many states facing educational reform debates, seen by some as driven too far by policy and ideology more imbedded in influenced think-tank pronouncements than in common sense and peer-reviewed research-based findings. Like a pendulum swung too far, the bounce back from that intentional push is now being felt with equal vigor and repercussion. In question here is nothing less than the validity, reliability, and even viability of the state's accountability system.The litany of changes in just a few years with irresponsible and uninformed implementation, devoid of consideration for the multi-stacked impact of so many simultaneous modifications, has left the public confused about the true performance of students. Meanwhile, educators at all levels are concerned, doubtful, and skeptical about both policy and its rollout timeline. The recent disconnect between reports of sinking school-grade performance, alongside improved 

student outcomes, has only added to the confusion and the heartbreak; particularly as it is the result of a simplistic view of student achievement amidst implementation of new standards, scale and cut scores, and end-of-course exams, all either introduced or deliberately modified with predictable consequences.

Some, in an expected defensive position, will say that this is a necessary evolution for the sake of educational and economic competitiveness. Others will even suggest that the push back is driven by special interest, or fear of change. To that, I submit that what is in question is not the need for better and more complex standards or assessments better aligned with the needs and demands of the new economic reality. The debate is not reform, but the form and vehicle of this reform, and even the agents behind it. If there ever was a case of the "ends justifying the means," or better, "by all means necessary," one would find it through an honest observation of the educational policy, standards implementation, and assessment decisions made in Florida over the past few years. Simply put, the “What” has trumped the “How” with dramatic and unfortunately, avoidable consequences. 

So, on the eve of the most dramatic shift since the inception of the FCAT or the transition to FCAT 2.0 in terms of standards and assessment, we ought to pause. We need to take time to honestly reassess previous and recent decisions and their consequences and have the courage to proceed on a path that is student- and teacher-centric. A path that can and must be a both and not an either-or proposition, unlike what the for-hire pundits say. A path that excludes politics, influence, ignorance, and extremism.

How shall we proceed then? Abandon accountability and assessment altogether? Well, for starters, you cannot manage what you cannot measure. Without understanding the cruel reality of achievement gaps, which are often rooted in previously existing conditions known as opportunity gaps, we cannot develop strategic and differentiated solutions to reach and teach all children. Respectful accountability is a data tool of truth that enables and empowers appropriate intervention. Simply relying on an "impression" of achievement is not enough as history has taught us. We cannot go back to the days when majority performance, albeit often mediocre, cloaked entire pockets of underperforming students even in higher performing schools or communities. In education, the loudness of a performing majority cannot drown out the plea of the struggling child whose poverty, disability, linguistic proficiency, or societal disconnect create known school readiness deficits. We cannot return to the days of going along to get along, by eliminating the uncomfortable realities of today's debate, by simply deciding to not examine whether children – ALL children – are learning, and to what level, as a result of our efforts. Do we as parents, not weigh and measure our children as we proudly and lovingly monitor their physical development? Equally, we have a moral responsibility to respectfully and reasonably do the same from an educational perspective to ensure their cognitive development. 

Once again, the issue here is not one of accountability and assessment "all or none." It is one of how, by whom, how often, at what expense, and most importantly, toward what end? We need to recognize that we are not going to assess our way to excellence, but failing to assess altogether is guaranteeing we will never get there. I choose to look at the ongoing debate surrounding this issue as, by and large, healthy and revealing of greater awareness and engagement. We cannot, however, allow the silence of a common sense majority to be hijacked by disconnected and often misinformed or politically influenced voices of dissension.

Nor can we expect that someone other than those who care for and understand the true democratic value of public education will step up and legitimately provide the needed fixes. Maybe the first thing we need to do is recognize that the true regression to reason will not be easy or swift. To devise the right and respectful accountability model that serves our children and teachers well, we must dedicate more time than we have previously before this school year's accountability demands are applied. A prudent course of action relies on following a number of reasonable and sequential steps.

First, we must reassess the impact that an inflexible and flawed federal educational policy is having on our state and nation. Policy that demands, among many other requirements, equal application of egregious consequences on fragile student populations from the English-language learners, who are assessed well before they are able to decode in English, to children with disabilities regardless of severity of the disability itself. To that end, we support the Governor’s and the Commissioner’s efforts to have the ESEA waiver appealed by allowing additional flexibility in regards to these learners. We need to debate the cost/benefit analysis of Florida's much celebrated and recently maligned No Child Left Behind waiver, which is seen by many as having exchanged unreasonable federal policy for arbitrary state rules.

Second, statutory timelines for accountability implementation must be adjusted, not in alignment with arbitrary assessment mandates, but with the readiness/awareness level of standards with educators, parents and communities. In light of what is being decried as a rush to roll out new assessments to comply with statutorily mandated accountability timelines, an additional year of standards implementation should be granted. This will enable sufficient community awareness and understanding as well as optimal statewide professional development of teachers and leaders. 

Third, the state must ensure that as it constructs a new assessment, the private entity contracted to develop it has the experience in educational assessment to do so. The state also must demand that all statistical requirements of validity and reliability, inclusive of field-testing sample questions and the whole exam, be performed in Florida with Florida’s students – not in non-demographically representative states with homogeneous populations dissimilar from our own. The current field-testing done in Utah does not pass the proverbial smell test sensibility, as item bias determination will never be believable. It would make sense, also, to review all statutorily mandated accountability rules that directly or indirectly result in required assessments. Over-assessment is a reality that the state must own and address. Instructional time is too precious to spend it assessing students on duplicative measures. Additionally, assessment of students should serve the strict purpose of informing instruction, not to simply provide a variable into a teacher’s performance evaluation formula, as is the case of the new state-mandated, district-designed end-of-course exams in grades K-12.

Fourth, the state must abandon its plan to use baseline assessment data (data generated by first-ever administration of an exam) as a means of generating school grades. The reason is simple; accountability in Florida relies on an analysis of both performance as well as improvement over time, otherwise known as learning gains. The measure of performance over time demands comparable data preceding the year in question. That will be impossible in 2014-2015 without statistical perversions as baseline assessment data, which by definition, cannot legitimately be compared to previous performance data. Simply put, indefensible or baseless grades are worse than no grade at all. One alternative for this transitional year could be reporting school performance through an objective report card that informs communities of all relevant educational indicators, without having to assign an unreliable school letter grade. Next, the state must not only sustain its earlier and rightful decision of suspending accountability consequences during this transitional year, but also expand it to a second year. The wisdom that justifies this decision, which is rooted in not advancing with consequences during a year of change and inconclusive results, stands as the basis for the previous recommendation. How can results that are potentially inconclusive and not trusted to drive consequences be trusted to produce school grades with all the implications they are known to carry from something as subjective as school prestige to school recognition funding? 

Lastly, the reasonableness of tying inconclusive and statistically unreliable achievement data to teacher evaluation and performance-pay scales based on learning gains or Value-Added Models, when those very same gains that are indispensable to the model are non-existent, is questionable at best, unethical at worst.

Ultimately, what is at stake here is not only the strength and trustworthiness of our state's accountability system, but the system itself. It should be evident that a rush to protect legacy or policy status quo could prove to be not only disingenuous but also potentially disastrous. Simply getting it done cannot be a cheap, expedited substitute for getting it done right. There is a high moral imperative at play here, which if not addressed and reasonably responded to, will define not only who we are, but also what we value, how we tackle challenges when we know better, and how we value our children and those who teach them. Oh yes, and as we, the adults, consider, debate, disagree, argue, and unfortunately sometimes disrespect each otherin this process, we should not ignore the fact that our children are observing us and, as smart as they are, they are learning…or being confused by our confusing ways.