Sunday, January 13, 2019

Beaverton Business Man, Parent Responds to Oregonian Editorial Board: "SBAC tests are neither good data nor useful data."

The Oregonian recently published an editorial suggesting that parents who opposed high stakes standardized tests are fighting against the community good. But I was disappointed by the editorial’s logic and lack of respect for the problem of relying on SBAC test data. 
For background, I’m a trained and experienced applied mathematician and have used data to manage human oriented programs for 25 years (advertising, marketing, etc). In my role, I’ve had to rely on data to deliver $1B in revenue for clients and my clients have included a large number of the Fortune 100 companies. 

The error is that the editorial presumes that SBAC tests are “good data”. There’s little evidence to support this. 

Standardized tests have become a focus because they are the ONLY data available in standardized ways - not because they are the data that’s needed.

Just a few brief points about the data SBAC produces:

Tests of this sort are able to evaluate (at most) only about half of what we need kids to learn in any subject area. Consider writing.
  • We need kids to learn to communicate effectively in writing.
    • While grammar, spelling, and word knowledge are important they are barely a start.
    • To write effectively they have to (a) know what they think, (b) be able to choose effective ways to communicate it and (c) be able to put that into understandable written form. 
    • Grammar, spelling and word knowledge are only part of the (c) - and only part of item (c).
  • While there have been claims that SBAC evaluates this, it doesn’t. 
  • To effectively evaluate how well our schools teach students to write would require:
    • Students be given a subject that interests them.
    • That they be given time to think and explore the topic to arrive at a point of view.
    • That they be given time to ponder that point of view and sort out how to communicate in writing about it.
    • That their writing be evaluated by a real human being who spends 20 to 30 minutes reading what they write in order to establish an effective grade.
    • Unfortunately, SBAC is primarily graded by machine. When it’s graded by a human the standard appears to be 2 minutes grading per answer and graders are given incentives to do that grading faster. 
  • In other words, SBAC doesn’t even come close to effectively grading writing. It’s a superficial test - which means it’s bad data - not good.

ALL subject areas (tested or not) are generally about half of what we need students to learn at school. 
  • It’s far more critical that they “learn to learn”, “learn to study”, “learn to explore issues”.
  • Even more critically, we need them to take issues and areas where there are NO answers known and be able to construct effective ways to explore those.
  • We need phys-ed, music, art, drama, business, etc.
The SBAC tested subjects only reflect about 25% of what we need schools to deliver. Given that only about 1/2 of the subject matter can be tests, this means:

SBAC tests evaluate about 12.5% (1/8th) of what we need schools to deliver.

SBAC tests are neither good data nor useful data. They are simply data and they fill vast computers in ways that allow silly little superficial reports to be sent home. Administrators probably love the data - because administrators don’t demand data be good - only that it be present (this observation is about every industry including, I expect, the newspaper business).

The role that your newspaper and the Oregon Department of Education give them in discussion misleads the public into believing they are highly meaningful.

My bottom line is we are in a position where society (and PPS administrators) WISH there was an objective way to rely on data to evaluate schools. There is not that opportunity. Unfortunately, PPS has chosen an Orwellian claim that bad data is good. 

There are other ways to evaluate schools. And parents, despite complaints you might hear, do know how well their kids are doing. But parenting is difficult because in middle school years, there’s no way to be certain that the hormone driven mess who is their child will turn out to be successful in life. 

What other ways? Grades and teacher evaluation of children. Many colleges, a constantly increasing number, have dropped any use of SAT or ACT in admissions and rely on grades. They have found that grades are a far better predictor of success than standardized tests. This is a REAL WORLD example and they have made this choice based on observing what happens.

Doug Garnett
Beaverton School District Parent and OSOS Member


From Flickr: "When my son takes standardized tests" 

No comments:

Post a Comment