“The thing is, as long as the test organizations are following their own policies, however unreasonable they may be, there’s nothing the courts can do about it. They’re not public institutions, so “innocent until proven guilty” doesn’t apply and the Terms and Conditions do spell out that they do not have to “prove” that a student has cheated in order to cancel a score and that the reasons for cancelling a score are not limited to things that are within the student’s control (ie proctor messes up the timing).”
“I’d love to know your knowledge about that–please share!”
See my post #14. Based on my reading about this in the past and friends I know in the field of psychometics and probability theory, not to mention education. As to the quality of the modelling - unknown.
“I don’t have a strong opinion on that, to be honest, but from a pragmatic perspective, several of the other problems could be ameliorated or even solved if test recycling were ended. Obviously stopping cheating is important, but the CB/ETS/ACT have shown little to no ability to do so, and their continuing reliance on test recycling suggests they don’t really even care all that much or at least aren’t willing to put their money where their mouths are. As a teacher, if I have cheaters in my class, I catch and punish them when I can, but I also have a duty to enact procedures to make cheating as difficult or unrewarding as possible. The two go together hand in hand.”
What procedures can ETS/ACT put in place to catch cheaters more effectively? Do you agree on a "no do-over" policy if someone has a credible case against him/her? How can they better tweak their internal reviews so that they don't punish the innocent? We can both agree on opaqueness, as my post #41 points out.
Okay, so you don’t really know and can either take their word for it or not. That’s fine and it’s the same position I’m in, and as I’ve made clear, I don’t take their word for it for several reasons that may or may not move the needle for you or others.
Dunno. Not my job, and I don’t think my opinion matters to the CB anyway. That said, I do know that in Korea, at least, to prosecute cheaters, the CB has to send a representative to testify in court, and in the last 16 years the CB has elected to do so a grand total of zero times.
Everything in that question hinges on the word “credible.” I’d like to see a credible case before making a blanket statement about something that is so far mostly hypothetical.
There’s no way to know this without actual detailed knowledge of how they undertake those reviews at present, and nobody knows that as far as I can tell.
@marvin100 - not employed by ETS or CB but the mathematics are pretty well known. As pointed out already, this information can - and would - easily be shared under oath. Not sure I’m in the same black hole as you are on this issue. A lot depends on one’s level of mathematics knowledge . . .
CB is reducing the number of testing centers in S. Korea and other parts of Asia. Testifying in court would be a waste of time because the problem is so prevalent.
@JBStillFlying Thanks for posting a copy of the demand letter. I wonder if that was just 'Civil Rights Lawyer speak" (that is what it feels like but it is hard to know) or if Crump really believes she was flagged due to race, gender, or some other protected class. All he would have to do is to come to CC and he would see that College Board is pretty much an equal opportunity score cancellation group when it comes to seeing large score increases within one sitting of the SAT. It would be nice to see all of the testing organizations do a better job at releasing scores when a score increase has occurred without cheating (especially with lots of proof by the test taker). It feels like they don’t like to admit that they are ever wrong, but the real lesson from this episode is to prep hard BEFORE taking a standardized test the 1st time and start early enough to not be cornered by application deadlines in a student’s senior year.
The Brendan Clare case against the ACT is interesting and IMO shows how sloppy the testing companies can be, if allegations like this are true:
And this is just CYA stuff on the part of the ACT I think - why should a student give up the right to go to court when agreeing to retest to prove innocence?
@ChangeTheGame - playing the Civil Rights card might well be great strategy. It’ll be interesting to see whether it causes these test companies to review and change some egregious policies in this matter. The big problem seemed to be ACT, however - wasn’t aware that flagging a test within 2 weeks after it’s taken (such as what ETS did in Ms. Campbell’s case) is a sign of being abusive. As I’ve pointed out, this case seems a bit different from others. CB claims that they sent along the first set of “evidence” earlier this month per standard procedure, and thought I read that Ms. Campbell opted for arbitration? One never knows if the reporting is accurate.
Your post is spot on about re-testing. It appears from their score-hold materials that ETS might be using the student’s history to gauge improvement and test irregularities, and it would certainly be common sense for them to do so. Keeping in mind that the “score increase” reason has been put forth officialy only by one side of the current case (in part because ETS doesn’t comment on these things as a matter of policy), it’s certainly feasible that an IRREGULAR score increase is scrutinized very carefully. In other words, these testing peeps might be looking at ALL your scores. A very good reason not to have too many of them.
Testing in senior year shouldn’t be problematic if the schools are willing to await the outcome of any flagging (ie not just one student or a protected class but anyone experiencing a score-hold; guessing FSU has had to deal with the flagged scores in the past given the number who apply there every year). Scores seem to be delayed or held every test administration based on the results threads for SAT and the impact seems to differ by stated policy of the university or scholarship agency. NMSC, for instance, warns semi-finalists well in advance not to put off getting a confirming-score till December. But many schools say they will review Oct. or Dec. scores when received as long as self-reported (if allowed) or sent directly from the agency when available. Only have anecdotal info. on this but it appears students with delayed scores can contact their AO and get advice on next steps. It’s a very small probability that your test will get flagged in the first place, and an even smaller probability that it was flagged incorrectly (although unfortunately one can’t discount the possibility of being victimized by another tester who was copying your score sheet). The large majority of seniors should be able to take a late summer or fall test w/o any worry other than the test center suddenly closing and the retest date having a major conflict (which is what happened to my kid last August!). A couple of schools my son applied to (Georgetown, CMU) even recommend Junior or Senior year testing.
@OHMomof2 - whether testers were in another room is a factual question that is settled one way or the other in court, although one hopes that the facts of the lawsuit are correct. And my knowledge of the law could be very wrong, but if one is of minor age (like a good majority of kids taking the SAT/ACT) how can one possibly sign away rights to a trial? Seems that’s the issue they should focus on first. But due process is going to be an issue in that suit. A private company may be able to set the terms, but neglecting to involve a parent or other adult signer seems very problematic.
I read in one of the old court cases that the magic number is 350 points combined for math and reading or 250 points on either one within an 18-month period (although those numbers are for the old SAT. Not sure if they were adjusted after the revised one came out). The computer flags that student as a potential cheater, and then they do their “investigation” (comparing handwriting between the tests, looking for similarities in answers with others nearby, etc). It’s possible, however, that this investigation will be concluded before the scores are scheduled to come out, so an honest student with a large score increase may never know that they were flagged in this way.
^ Regarding #67: Can totally see SAT/ACT registration getting cleaned up there. Of course, NO ONE reads the fine print from anything and not sure how that argument from a grownup can hold up in court or whether they can make a case that fine print itself is burdensome . . . one great potential outcome is that this one little change might lower the number of re-tests. One can only hope.
@mathhappy at #68 - my goodness, one of my kids went from a 26 to a 35 on ACT Science in six weeks . . . she was never flagged that we know of. I wonder if she underwent any sort of investigation. This was ACT so not sure if exactly similar (though their vague description of what they do sounds similar to ETS . .). Her score was released on time with everyone else’s and she graduated w/o incident. I’ve mentioned statistical analysis before and ETS used that terminology for its response to the lawsuit regarding the August re-cycled SAT. ACT makes the same kind of reference in its materials. I can see where a Monte Carlo simulation or other statistical process would be helpful in figuring out what’s improbable vs. merely unusual. Guessing a 9-point increase didn’t seem improbable for ACT science!
@milgymfam It may have been actually harder to flag such a high score, especially if there was no one with a score close to a 35 Science subsection in the room. The ACT Science has been a pain in the rear of many ACT high scorers and I would suspect that @JBStillFlying’s kid had pretty good scores everywhere else. My son made a similar jump on that section practicing at home within a short period of time, so some kids have to figure out that section before they can make the “jump”. I hope that we one day find out how testing companies look for cheating because it can not be an easy endeavor.
@JBStillFlying
On the ACT, an old lawsuit indicates that their “magic number” is 6 points over a 20-month period, although I believe that is only for the test overall. I don’t know if they have a cutoff for a single section, but I would imagine that high jumps are not uncommon in strategy-heavy sections like science and reading.
ETA: It seems that accomplishing a 6 point overall increase would be far more challenging than a 350 point increase on the SAT. It seems that SAT errs on the side of identifying more students but providing legitimate pathways to validate the score, while ACT identifies fewer but has already decided that you cheated before the first contact. Just my two cents.
Actually, it’s extremely easy. The issue is that is also very costly. Much easier to just flag a high score gain and make them retake – at no cost – to validate the newer score.
@ChangeTheGame at #72 - bingo. Solid performance on that first ACT but lower than she was hitting in practice. Second ACT was at least a +2 in all sections and a 35 in two of them; that outcome actually WAS consistent with her practice. It’s just one example of how a score improvement can be completely explained by getting used to the real test and also to the small tweaks that ACT was sticking in there precisely to throw kids who were perhaps a bit too complacent due to all that practice. She was admittedly thrown the first time so went in expecting the unexpected the second time.
My oldest was consistently hitting 28’s on ACT science in practice and ended up with a 33 on the real test (took just once). She claims it’s just a hit-or-miss section and there might be something to that, although I think perhaps our prep materials were a tad “hit or miss” as well
^@bluebayou’s comment at #74. Not sure it’s just an unusually high score gain that is flagged. We know kids who have gone from a 22 to a 32 in a few tests and over far less than a 20 month period, with no apparent problem by ACT. There needs to be something else present besides a high score gain. But you must be correct that the gain is an important part of the investigation, simply because one or two points is a very likely outcome whether you cheated or not, whereas a +6 might have far fewer paths pointing to a legitimate gain. Much easier to catch the +6’s.
My D went from a 23 to a 30 composite from one test to the next (sat out a round in the middle). She went from taking it cold to doing some prep to work on her timing. It never even occurred to us that she could be flagged for studying and bringing her score up.
My kid went up 6 points in under 20 months. Not much prep (none before the first test), she used a prep book from the library for a few hours before the last. I suppose I should consider myself lucky she wasn’t flagged.
[quote]
A New Jersey judge has found that the binding arbitration requirement everyone who takes the ACT must sign is "unconscionable" and may not be enforced in the case of a student suing the testing organization.
“Incorrect test score reporting can have devastating and life-long consequences for an innocent minor,” the ruling said. “Based on the timing of the score cancelations here, the damage is done and the harm possibly irreparable when caused by such and intolerable and lengthy delay. The available remedy at arbitration is ephemeral and illusory as the student watches the college admissions process pass by while the validity of the test scores is subject to limited and narrow review.”
