<p>Belkin39, I gather that psychometricians believe they can translate the scores between different versions of a test. It is not as simple as a harder test leading to lower scores. The scores are “scaled scores” because the psychometricians have taken each test’s difficulty into account.</p>
<p>Thank you for the reference, Periwinkle. People I spoke to who have been on the ISEE Board stated that the reduced number of scored questions had a direct correlation with lower stanines the last two years. And I know that TT schools in NYC are taking applicants with much lower stanines than in previous years (this happened last year and again this year). In 2009 most TT schools would not take a child with a stanine lower than 7, ever. I know personally of children admitted with lower than that the past two years, including stanine 5 in more than one sub-test. Sure, there might be clusters of children in the 50th percentile. But the point is that it is easier to fall to stanine 5 now than it used to be b/c if you miss a handful, you could be in stanine 5. That wasn’t the case in prior years. You are discussing scaled scores and I am discussing percentile rank. This isn’t an issue of prepping. My question was really more of a mathematical one. I was wondering more about the norming population’s relationship to setting percentiles. Anyway thank you for the reference. I will take a look at it.</p>
<p>One other thing I thought about after I posted the above. It is my understanding that the scaled score is set based on all the different versions of the test given on that particular year. In contrast, the percentiles are normed based on the <em>prior</em> three years. So scaled scores are made based on 2011’s different test versions. The 2011 percentiles on the other hand are based on the previous three year’s performances. It’s very different. Schools look at percentiles and stanines in the admissions process, from what I have been told by DOA’s.</p>
<p>
</p>
<p>That doesn’t necessarily mean that more students are falling into the 5th stanine. It could mean that more students in your social circle are falling into the 5th stanine. Do you want the cynical explanation for the NYC schools’ change of heart? Do you know any successful financial aid candidates who fell into stanine 5? And how many successful children of hedge fund managers? If I Google “ISEE tutor New York,” I find some tutors with very high hourly rates. Some parents can afford to pay hundreds of dollars per hour for tutoring, if the quoted rates are accurate. </p>
<p>Presumably, kids who weren’t tutored for the ISEE would not score lower on the new exam than on the older version. How many children are tutored for it? If they’re a small subset, they could fall into lower percentiles on the new exam than they would on the old exam, but the overall percentiles would be relatively stable.</p>
<p>You make some great points on the advantages of prepping Periwinkle. But I think we are talking past each other. My main question centers on why stanines fell across the board after re-norming. ERB acknowledges that stanines have fallen since the re-norming and I have not heard anyone say that it is correlated to an inability to prep for the test. How would a reduction in the number of questions affect the inability to prepare for the content of those questions? Having said that, I am tempted to talk with you about prepping and the unfairness to economically disadvantaged children. I could not agree with you more on that. But it is not really connected to my question. Many people affiliated with ERB have stated that the reduction in stanines correlates to the re-norming of the test since they reduced the number of scored questions and the setting of the percentiles against groups who took the prior test. I don’t think that is in dispute. I invite anyone to contact ERB to ask that question as I myself did a while back. I do think that Brother’s observation that once the norming pool becomes 3 years post re-norming, stanines should stabilize. That was what I got from ERB as well. If so, next year should be much better and 2013 should be stable.</p>
<p>Also, I just want to add that I don’t know any person’s ISEE scores, whether they be in my social circle or not. I don’t ask about it. I only know what I have learned from ERB and people connected to the ISEE Board. In terms of whether more people are falling into the 5th stanine, that actually leads back to one of my original questions. It has always been my understanding that the stanines are set on a bell curve. So you would expect that the scores would be evenly distributed each year as they always are. But post re-norming, you might expect that a child can fall several stanines without missing as many questions as they would have had to miss in prior years to get the same stanine score. And in fact, we know factually that was the case in 2010 for 5th grade. It’s been reported that there was no stanine 8 in Reading. One wrong question resulted in stanine 7. In 2011 the drop does not appear as steep but still much steeper than in 2009, for example. My question on the bell curves was whether the bell curve is a three year bell curve? It seems like it could be in order to correlate with the percentile ranks which are based on scores from the three prior years.</p>
<p>Perhaps the bell curve is still there, but it’s been shifted downward, its center now around the fourth or third stanine? It seems odd for a standardized test to make it impossible to distinguish between the top of the curve. That would seem to fly in the face of the purpose of a standardized test. </p>
<p>If it’s been re-normed, the earlier scores shouldn’t be used to determine percentiles. Perhaps the schools are ignoring them, and looking for good scores in their pool of applicants.</p>
<p>I think/suspect the rolling averages make stanines only good for the year in which they are reported. IOW, I suspect the bell curve is a three year curve and so for example what would have been a stanine of 7 in 2009 would have been an 8 in 2010 b/c when it (the child’s relative performance on the ISEE in 2009) is measured against 2010 child’s performance, the 2009 performance went up. Likewise, the stanines in 2010 and 2011 are lower than they would have been in a pre-renorming year. It’s very confusing. I guess the bottom line is that scores are lower than in year’s past and it is a function of the re-norming of the exam. This makes high performers like TOWINSOR’s kid really an outlier which is fantastic. It also means that kids who got 5’s shouldn’t feel so bad. They didn’t do worse than half the kids that took the test this year or last year. They did worse than half of a testing pool who took the test between 2008 and 2010 and 2/3 of that pool took an easier test than they did.</p>
<p>I just re-read what I wrote earlier. The bell curve should have an even distribution. There should be an equal number of 9’s and 1’s, maybe. So if that is the case, and if it is a three year bell curve, then wouldn’t rolling averages change the stanine assigned in 2009 to a higher value in 2010 once the lower performing scores from 2010 are factored in on the percentile rankings? I think so, but I am not sure.</p>
<p>The tests are only good for one year, though, so it doesn’t matter. </p>
<p>If they’ve introduced serious changes in the test, at some point it’s a different test. If the old and new versions are very different, the three year rolling average would be worse than useless. It would be comparing apples and oranges. I presume there are really two bell curves, one from the old data set, and one from the new data set. I still don’t know why there would even be stanines based on the previous years’ data.</p>
<p>My son didn’t take the ISEE, so this is all theoretical to me. There isn’t that much information published about the tests, for good reason.</p>
<p>Only some regions of the bell curve would likely be shifted downward. Low scorers would be at about the same level unless even the easiest questions were made harder with the revisions.</p>
<p>Periwinkle, yes it is apples and oranges and percentiles are set based on the prior three year’s data and percentiles determine stanine. So essentially, yes it is rather odd to use the data from a totally different test. I think it matters that the old test had a lot more questions that were scored too. The ISEE doesn’t penalize for wrong answers either.</p>
<p>Brother, I think if I had to guess I’d say it was the region between 7 and 5 that shifted down. There are so many 5’s and 6’s that you know a lot of those 6’s would have been solid 7’s in 2009. And likewise, the 5’s would have been 6’s.</p>
<p>My son is entering 6th grade and just took the ISEE. I thought his scores were meh, 5’s and 6’s. But all three schools said that those were very good scores for this year, especially in reading.</p>
<p>zp</p>
<p>my scores were 6,7,8,8 and i got into dalton, riverdale and poly prep. schools look for balanced kids besides trinity and horacee</p>