Any question on a fill-in-the-bubble test provides all the data necessary to come up with the correct answer. Students are then supplied with four or five possible responses. By their very nature, standardized tests inflate the scores of students on the low end of the scale. The only students who score lower than 20 percent on a fill-in-the-bubble test are victims of bad luck, since entirely random responses should raise you at least that high.
Just the appearance of a correct answer printed on a test booklet should increase scores across-the-board. Some percentage of students who cannot correctly answer the question "Who was President of the United States during the Civil War?" with no further prompting, will no doubt choose the correct answer when it is placed next to George Washington, Woodrow Wilson and Bill Clinton. There is good reason to believe that scores would plummet if tests were "fill in the blank" instead of "fill in the bubble."
Student assessments can also include essays, projects, or oral interviews. These allow students to demonstrate a deeper and wider knowledge of a particular subject than can be measured by a "fill in the bubble" test. However, using the previous example, it's hard to imagine a student who can write a meaningful and exemplary essay about any aspect of the Civil War if he or she can't answer the "bubble" question of who the President was.
So why would educators and their political allies criticize measurements that cast them in a better light than the alternatives?
And the answer is:
Because of the political battles over education and the presence of standardized tests, the tendency of school systems is to evaluate students more generously in alternative assessments. In the absence of standardized tests, very few students "fail," receive Fs, are retained, or are denied diplomas.
8 comments:
If we accept the assertion that absent standardized tests, schools dumb down and pass kids who are not deserving of passing, we need to ask why this might be so. Merely due to the lack of standardized tests? Unlikely. Could it be because schools, all of the schools, are full of incompetents who pass the undeserving because they don't know any better, or knowing better, do it to hide their incompetence from the public? Nah.
Perhaps we might entertain the idea that the schools are a reflection of the expectations of society, and that as such, society sees high school graduation not merely as an indication of the attainment of specific academic standards such as may be found on a state education agency website, but as a social milestone that says as much or more about growth and coming of age. Can we really believe that the public thinks that everyone with a high school diploma has attained the same academic knowledge, growth and acheivement? No, only those who buy NCLB-like assertions such as every American child being able to perform on grade level by a date certain could believe that every child with a diploma is average, to say nothing of above average. It's just not that cut and dried.
And what can standardized tests do for us? Do they tell any competent teacher anything they don't already know? Can they help a student who doesn't read well do better? Can they help students who already do well academically improve? Can they help poor teachers teach better? Tell principals which of their teachers aren't doing well? If the only mechanism, or even the primary mechanism we have for improvement is standardized test scores, we have far, far deeper problems indeed.
Standardized tests are a political solution to educational problems that, in many schools, don't exist. They are useful primarily for giving politicians convenient sound bites with which to whack political enemies, but they have no ability to solve educational problems that do exist. Nor do they tell any competent teacher anything they didn't already know about their students the first few weeks of a given school year. And if a teacher isn't competent, no single test score delivered toward the end of an academic year is going to be helpful.
Testing has limitations and it has value, but not when it's employed for political purposes, and that, after all, is why we're all chatting about it, is it not?
No, that's not why *we* chat about it. But it *is* why the presidential candidates talk about it.
Is there a clear guideline for differentiating sincere political opinions from empty rhetoric aimed at strengthening a base that is in no danger of going anywhere? I haven't found one.
Schools don't have to full of incompetents in order to fail their duty to the public. If educators don't have some reliable measure of movement toward or away from the goal of an educated electorate there's no reason to expect them to move toward the goal. There are however reasons to expect them to move away from the goal.
Since there are an infinite number of ways to do a task improperly and rather fewer ways to get it right, not measuring whether you're getting closer to or farther from your goal makes it much more likely that you're getting farther away.
For compentent teachers that's not an issue. They're competent so an objective measure of performance will highlight that fact. A fact which, being unmeasured is also unvalued. For incomptent teachers the advent of objective measures of performance would be a disaster but that's understandable. What's somewhat less understandable, although not unexplainable, is the antipathy to testing in ed schools.
If nobody's measuring anything then a half-baked scheme is just as good as an effective idea. Since education hasn't changed much since Socrates' time it's tough to come up with new, effective ideas while crack-brained foolishness springs up like mushrooms after a rain if given half a chance. Objective testing would put an end to the vigorous cottage industry of education ideas that produce nothing but PhDs and grants which would not benefit ed schools.
You're right Mike that standardized tests are a political solution but it's a political problem, public education being a political creation. If you want to get the politics out of education then you have to take the education out of politics.
Not that good a slogan since it's too big to put on a bumper sticker but easily remembered with the added benefit of being true.
Dear Allen:
One of the problems that seems to crop up in discussions of this issue is the idea, nearly always implied if not explicit, that absent mandatory, high stakes tests, measurement, assessment, testing, or whichever term is current this week, is not being done. Or in the alternative, the only, best way to assess student and school competence or progress is the mandatory, high stakes test. I don't believe that either idea holds up to examination.
During an average school year, I manage to give and grade an average of 150 assignments, or a bit more than three a week. For a high school English teacher, that's a substantial load of writing, which is necessary if I'm going to give my kids the practice and feedback they need. I can tell within a few weeks after meeting a class for the first time exactly where each student is strong and weak and what I need to do to improve their abilities. I do that because apart from their written work, I'm assessing them each and every day by listening to their speech, by speaking with them about their work, and simply by interacting with them. In this, I'm nothing special. Any competent teacher can do this, indeed, this ability to assess student needs and progress should rightly be considered the minimum basic equipment for any competent teacher.
What does a single score on one test that my students take in February, and that I receive only in May tell me, particularly when compared to my daily assessments and interactions? Nothing at all. Preparing for the test does, however, kill at least two months of my school year doing lower level, formulaic rote teaching that I would spend little time on absent having to take the test.
But what about teachers who lack that assessment ability or don't do it well? They, like students, can learn, and that's a supervision issue; the high stakes tests can't address that, can they? And if our principals are relying on test scores of that kind to tell them if a teacher is doing a good job, it is the principals who need to learn how to properly assess teacher performance. Again, those test scores don't reveal, in any meaningful way, that kind of information.
You are correct in asserting that political cosiderations are a part of public education, but we must all draw the line at some point. Most rational folks would agree that we should not uproot well established, proven educational practices simply because the political winds are momentarily blowing in a different direction. As educators, we need to be clear in telling the public that educational problems, to whatever degree they exist, require educational solutions, and the degree to which those solutions make a given politician happy should have little or no bearing on their adoption or rejection.
We should never fix that which isn't broken, particularly with tools or materials that can't give the desired result.
You're certainly entitled to your opinion about the value of testing but outside the hothouse atmosphere of public education the value of testing isn't controversial. The way to determine whether value's being received is to set an objective standard and measure where, in relation to that standard, the subject of measurement falls. You can decide for yourself that the notion of objective standards of performance has no place in public education but I can assure you that the idea is gaining credence now that its necessity is accepted.
One outcome of the establishment and utilization of objective standards to measure educational attainment is objective standards for practitioners. If those standards don't capture all the subtle skills that go into teaching they do set defined and achievable measures of performance which means that the substandard performers can be dealt with as deemed best in the pursuit of the organization goals. If one of those goals is the continued employment of ineffective teachers against the hope that with sufficient help they'll develop into competent teachers then objective standards aren't a problem, they're a solution pointing out efficiently which teachers need help. Or, it tells you which teachers ought to get pink-slipped if carrying them on the payroll while they slowly develop a conscience or teaching skill isn't an organization goal.
You are correct in asserting that political considerations are a part of public education, but we must all draw the line at some point.
Wrong again. If "we must draw the line at some point", who does the drawing? You? Teachers in general?
If the answer is "the voting public" then we've drawn the line precisely where it is right now. If you think the line ought to be drawn somewhere else or by someone else then be less vague and we can see whether you're offering a viable alternative to the political process in public education.
As educators, we need to be clear in telling the public that educational problems, to whatever degree they exist, require educational solutions,
Trouble is, as an interested party in a political struggle the objectivity of educators is suspect. Any proposed solutions will be viewed through the lens of self-interest and all the prickly outrage in the world isn't going to change that.
the degree to which those solutions make a given politician happy should have little or no bearing on their adoption or rejection.
How about the degree to which a solution makes the voting public happy? If it's a public institution run for the benefit of the public making a given politician happy means that there's a significant likelihood of making that politician's constituents happy. If you're dismissive of satisfying the public then you're just posturing or unrealistic.
Dear Allen:
Thanks for your reply. I agree that in many ways, the voters determine policy, but not quite in the same way that you seem to see that process. I'm afraid that I must also disagree that mandatory, high stakes testing is not controversial outside public education. You need only look to Texas to see what I mean.
You may recall that Texas was, if not the first, among the first states to get on the testing bandwagon. In that, we saw a slight departure from educational faddism. Usually, it is members of the education establishment who come up with and mandate fads. With high stakes testing, it was the Governor and legislature behind the fad. The fads, such as the open classroom concept of the 70s and 80's, promise to solve every extant problem, cost huge amounts of money, and go through several consistent phases: (1) The fad is implemented despite the outcries of teachers who assert that the fad can't accomplish what its backers claim and that it is the dumbest thing anyone ever heard. (2) After about 4-5 years, it's obvious the teachers were right, but too many people are too invested in it and too much money has been spent to openly admit it, so the status quo is maintained for another 3 or so years (even as the schools begin to quietly back away from the fad) until...(3) Even the public figures out the fad is incredibly stupid and the pendulum swings back to rationality, leading to the final step...(4) The fad is dismantled at great cost, taking another year or so. Of course, then the new fad comes along and there we go again.
So it is in Texas with the TAKS test, the mandatory, high stakes tests that were state of the art, would solve every problem, were unassailably valid, could not be improved upon, were the pinnacle of accountability, yadda, yadda. So what's up? TAKS is on the way out. Within 2-3 years it will be gone to be replaced by something else, something quite unlike it. No one is absolutely sure what that will be yet, but the vague outline looks more or less, for the moment, like a final exam. Imagine that.
Regarding objective standards of and evaluation of teachers, I have absolutely no problem being evaluated on my work and accomplishments. But mandatory, high stakes tests given to students do not do that. They cannot do that. As I mentioned in my last post, the scores of my students on any single test, on any 10 tests, say virtually nothing about my abilities and practices because those scores are only minimally reflective of me. Evaluating the performance of a teacher is much more complex and requires much more time, knowledge and expertise on the part of a principal.
Here's just one example of what I mean. Each year, some of my students who can barely string together coherent sentences will pass the TAKS test, and some of my most intelligent students, students who are writing on college level or above will fail. What do we conclude from those scores? That I'm a half bad teacher? Half good? If that test score is the determining factor, that's about as good as we can do in evaluation. But if my evaluator is a professional, as mine thankfully are, they will quickly and easily discover that I am adept in teaching the basic tricks--and I do mean tricks--necessary to pass the test in such a way that just about anybody can master them and pass. They will also learn that really intelligent and capable kids often see such tests as meaningless and insulting and will, instead of playing by the rules and using the tricks, write something very good, but something the people grading the test won't appreciate or accept (such folks being notorious for having little sense of irony, satire or humor in general).
And yes, teachers should indeed draw some of the lines in determining educational practice. The public's input in democracy is in electing school boards and in monitoring their performance. It is the job of the school boards to hire qualified administrators who then hire highly educated, qualified teachers. If anyone in that system is not doing their job, it is ultimately the school board's responsibility to deal with it. If they will not, it's the voter's turn. Under the mandates of NCLB, I'm a Highly Qualified Teacher. I have the experience, skill, knowledge and ability to do my job. Shouldn't the public listen to me on education matters rather than to someone who doesn't have my experience, skill, knowledge, ability and federally and state mandated certifications and qualifications? Just a little bit? After all, you can't love NCLB and all that it mandates and not support your Highly Qualified Teachers!
Don't get me wrong; I'm not the slightest dismissive of the public. In fact, I love accountability, but real accountability, not the political version represented by TAKS. I love to have members of the public (and my principals and administrators) come into my classroom for a few hours or as long as they wish, as often as they wish, and find out what is really going on so that they can make truly informed decisions on matters of educational policy. I want them to come and see what I do, to ask questions, to read my handouts, to read my student's work, to get an informed feel for things rather than trying to make decisions by listening to opposing politicians tossing sound bites at each other or through single test scores. I wouldn't call that unrealistic or posturing.
Is every teacher equally qualified or is their opinion sacrosanct in matters of education policy? Nah. We have to hire teachers from the human race, after all. But we wouldn't be too bright if we didn't listen to those most knowledgable and capable in a given field, would we?
The majority of Texas politicians were very happy about TAKS some years back. Not anymore. Reality (in the form of the public and actual practice in the real world) has reared its ugly head. Teachers were right after all. Imagine that. But then again, considering that they're the people that the public have hired, considering that they are highly educated, qualified and knowledgable about education, maybe that's not such a bad thing.
Thanks again, Allen, for the discussion. I hate to turn these posts into private chatrooms, so I'll make this my final post on this topic and let readers judge based on what we've written. You have the last word if you'd like. Take care and Godspeed.
Clearly, tests should test what they purport to, and we should draw no conclusions beyond those the test is able to provide. Psychometricians know how to make good tests, and they know what the strengths and limitations are of the tests they make. From everything I've heard, the TAKS was not well constructed in the first place.
And I wouldn't use any one test to evaluate *you* as a teacher. But if we look at individual student performance over time, and we look at the your classes' performance over a few years, we should start to see some pattern forming of your effectiveness as a teacher.
Of course, is anyone looking at test scores that way?
Post a Comment