Mississippi had a problem born of the age of soaring student testing and digital technology. High school students taking the state’s end-of-year exams were using cellphones to text one another the answers.
With more than 100,000 students tested, proctors could not watch everyone — not when some teenagers can text with their phones in their pockets.
So the state called in a company that turns technology against the cheats: it analyzes answer sheets by computer and flags those with so many of the same questions wrong or right that the chances of random agreement are astronomical. Copying is the almost certain explanation.
Since the company, Caveon Test Security, began working for Mississippi in 2006, cheating has declined about 70 percent, said James Mason, director of the State Department of Education’s Office of Student Assessment. “People know that if you cheat there is an extremely high chance you’re going to get caught,” Mr. Mason said.
As tests are increasingly important in education — used to determine graduation, graduate school admission and, the latest, merit pay and tenure for teachers — business has been good for Caveon, a company that uses “data forensics” to catch cheats, billing itself as the only independent test security outfit in the country.
Probability is a field of math that was created to describe issues in gambling. From probability came statistics. And now statistics is being used to stop people from "changing the odds" in testing, there's a sort of justice in that.
Although its data forensics are esoteric and the company operates in the often-secretive world of testing, Caveon’s methods are not without critics. Walter M. Haney, a professor of education research and measurement at Boston College, said that because the company’s methods for analyzing data had not been published in scholarly literature, they were suspect.
“You just don’t know the accuracy of the methods and the extent they may yield false positives or false negatives,” said Dr. Haney, who in the 1990s pushed the Educational Testing Service, the developer of the SAT, to submit its own formulas for identifying cheats to an external review board.
David Foster, the chief executive of Caveon, said the company had not published its methods because it was too busy serving clients. But the company’s chief statistician is available to explain Caveon’s algorithms to any client who is curious.
I wonder how many people understand his explanations.
Continuing on, I like Caveon president Fremer's unapologetic view of standardized testing:
Dr. Fremer has little patience with critics who say standardized tests do not accurately measure academic prowess.
“Fundamentally,” he said, “testing is a way of ascertaining what you know and don’t know and developing ranks, and the critics go right to the ranks. Well, it does rank, but on the basis of knowledge of the subject, and if you think that’s not important, there’s something improper about the way you think.”
At the very least, doing well on a standardized test usually indicates some level of mastery of the material.
No comments:
Post a Comment