Last year the Oregon legislature passed a bill, introduced by Representative Lew Frederick, requiring a state audit of the SBAC testing process. The bill instructed auditors to look at all costs (not merely state line item costs), impact on instructional time, and much more.
This week the Oregon’s Secretary of State’s office released the audit. And it's a conundrum. Here’s a few bits of that conundrum.
First, it finds very serious problems. Just buying the SBAC tests is far more expensive than OAKS was. There’s major confusion about why these tests are given and whether the results have meaning. There’s critical concern that the tests punish certain populations (a typical problem of this type of testing) and that results are used inconsistently. And serious concerns that SBAC isn’t comprehensive - it focuses on only a narrow range of education.
Then, the audit regurgitates SBAC marketing material without challenge. I thought auditors were supposed to be critical thinkers. Instead, for example, they accept at face value the claim that these tests evaluate critical thinking. Experts in testing don’t think it does. Truth is that the only thing known about SBAC is that it CLAIMS to evaluate critical thinking.
Third, the audit ignores critical issues. For example, I’ve estimated these tests drain educational time worth over $200 million annually from local schools and districts. Yet the audit only considers line item state costs of around $10M. Penny wise and pound foolish?
The audit also doesn’t challenge the idea that individual scores have accuracy. Established fact is that tests like these have incredible error when looking at individuals or small groups. That error is only controlled once 300 or so tests from similar students are grouped together. Yet the auditors seem to accept at face value the claim that SBAC evaluates individuals.
Finally, unbelievably, the audit makes recommendations that ignore what it found. The audit merely recommends, gosh golly, that ODE communicate better and administer the tests more aggressively.
SBAC failure is NOT a problem of communication but of poorly designed testing theory and educational theory. Communication can’t solve that problem. Even worse, “communicate better” is what Bill Gates has said as it has become clear his Common Core State Standards vision is a failure. So I gotta ask, who’s in charge of Oregon’s education policy?