<p>Admission Test's Scoring Quirk Throws Balance Into Question </p>
<p>By DAVID M. HERSZENHORN
Published: November 12, 2005 (NewYork Times)</p>
<p>For weeks, Joshua N. Feinman had graded practice tests to help his daughter prepare for New York City's specialized high school exam. Then one day, he took a hard look at the scoring chart from her private test-prep class and was stunned by how the verbal and math scores added up. </p>
<p>''I took a look and said, 'Wow, this thing is really nonlinear,' '' said Mr. Feinman, the chief economist of Deutsche Asset Management. '' 'Wow, it's much better to score high in one and low in the other than to score good in both.' '' </p>
<p>Mr. Feinman had stumbled on a little-known facet of the test: because of the complex way it is graded, a student scoring extremely high on one part of the exam has a sharp advantage over a student with high but more balanced scores in each subject. </p>
<p>For Mr. Feinman's daughter, Amanda, and more than 26,000 other eighth graders who will get their results in February, the implications loom large. Last year, for instance, a student with a 99 percentile score in math and 49 percentile in verbal would have been admitted to Stuyvesant High School -- the most coveted specialized school -- but a student with a 97 in math and 92 in verbal would not. </p>
<p>As a result, test-prep tutors who understand how the test is scored advise their students to spend as much time as possible not where they are weakest, but on their stronger subject. Students are allotted two and a half hours for the exam and can divide the time as they wish. </p>
<p>City education officials and the company that has prepared the test since 1983, American Guidance Service, said that they were aware of the potential outcomes and that scoring for the exam had to be designed this way to identify the best test takers. They also said their hands were tied by state law, which they said required that admission to the specialized schools be based on a single combined score in math and verbal. </p>
<p>Officials also said that the test, which had not changed in at least 30 years, had served its purpose well, even as the number of children taking the test has soared. About 4,200 students were admitted to specialized testing high schools for ninth grade in September 2005. </p>
<p>''Parents should be really confident, and I am confident, that this is a scientifically valid and reliable test,'' said Lori Mei, the director of testing for the city schools. </p>
<p>But in an age of $325-an-hour test-prep tutors, and cutthroat competition for the best schools, some parents and testing experts said that even the slightest quirks in scoring should be publicized to ensure fairness. </p>
<p>''As taxpayers and parents, we should know how the test is graded -- not necessarily with an eye to changing it -- but certainly as a matter of public knowledge,'' said Mr. Feinman, who lives on the Upper East Side. ''It shouldn't be hidden or disclosed only to the select few who have the advantage of test prep.'' </p>
<p>Even some veteran test-prep tutors were surprised. </p>
<p>Barry Feldman, an owner of GRF Test Preparation, which tutored Mr. Feinman's daughter, said that in 24 years in the business, he has never focused on the scoring method. </p>
<p>''I just really never thought about it before,'' said Mr. Feldman, a retired junior high school math teacher and a 1964 graduate of Stuyvesant. ''What are the reasons? Why do they do it how they do it? I don't know. I really don't know, and I never really thought about questioning it.'' </p>
<p>Officials of American Guidance Service, a private company in Minnesota, said the test had been designed to the city's specifications. Principals of the six specialized schools are not involved in developing or grading the test, much as colleges are not involved in administering the SAT. </p>
<p>In essence, the scoring system rewards students with more points per question as they get closer to a perfect score on either math or verbal. </p>
<p>The impact of the scoring is potentially so great that a student with a perfect score in math last year could have answered fewer than half of the verbal questions correctly and still been admitted to Stuyvesant. But students who got 41 of 50 points on one part and 42 of 50 on the other would have fallen short of Stuyvesant's cutoff. </p>
<p>''Stuyvesant loves lopsided geniuses,'' said Naomi Bushman, a mathematics education consultant who runs a test-prep course for the exam. </p>
<p>The reverse was similarly true. A student with a perfect verbal score needed only 18 of 50 math questions for admission to Stuyvesant. The same student needed only 11 of 50 math questions correct for admission to the Bronx High School of Science and 9 correct to get into Brooklyn Technical High School. </p>
<p>Principals said they were aware that a super-high score on one part could substantially lift an applicant's chances, because many recent immigrants with extremely limited English skills had earned admission by posting exceptional math scores. But even principals said they were surprised to learn just how low a student could score on one part and still be admitted. The principals said they were typically not shown copies of the test or the score charts and were not asked their opinion of the results. </p>
<p>City officials acknowledged that they had not discussed the test with principals in recent years and that they had never conducted studies to gauge the validity of the test. But they insisted it was valid, as evidenced by the high graduation rates of the specialized schools. </p>
<p>But Regan Kenyon, the president of the Secondary School Admission Test Board, which publishes the admissions exam for 700 private schools and boarding schools, said validity studies, comparing students' test scores to their freshman year grades, were needed regularly. </p>
<p>''We don't know if the way they are selecting kids is the right way or not,'' he said of New York City. ''If you haven't done the study, then you don't have a transparent admission process.'' </p>
<p>Stanley Teitel, the principal of Stuyvesant, said that educators there had long wondered about establishing minimum cutoff scores, so no student could be admitted without scoring well on both verbal and math. But he said there should also be a place at Stuyvesant for students who are exceptional in one area. </p>
<p>Mr. Teitel said the school and its alumni would strongly resist using any criteria but the test for admissions. ''One of the things they love about the school is that it remains a true meritocracy,'' he said. ''You only get into Stuyvesant because you proved it on an exam.'' </p>
<p>But virtually all testing experts warned against using an exam as the sole basis for admissions decisions. The Boston Latin School judges applicants on test scores and grades. The Thomas Jefferson High School for Science and Technology, a specialized school in Fairfax, Va., uses test scores, grades, essays and teacher recommendations. </p>
<p>According to American Guidance Service, 52 students -- or one-fifth of 1 percent of the more than 25,000 eighth graders who took the test last year -- scored perfectly on one part. Of these students, one scored in the 50 percentile on the other part of the test; some scored in the 60's, 70's or 80's, but most were in the 90's. </p>
<p>The test, known by its acronym SHSAT (pronounced Sh-ZAT), consists of 45 verbal questions and 50 math questions, with each part worth 50 points (five verbal reasoning questions are worth 2 points each). </p>
<p>The total for each part is scaled and curved, based on the difficulty level of individual questions and the relative performance of all test-takers, to a score from 20 to 400. Then, these two numbers are added to get a final score. </p>
<p>To differentiate among high achievers, there are wider gaps between scores at the top of the curve. On one form of last year's test, a verbalscore of 49 converted to a 345, while a perfect math score of 50 converted to a 365, a 20-point gap. But from 39 to 40 correct, the gap was only 5 points. </p>
<p>The city's official guide to the test describes the scaling of scores. But the guide never specifically explains that a high score in one subject can lift a low score in the other, or that students might benefit from focusing on their stronger subject. </p>
<p>Several test-prep instructors said they learned how the test is scaled by accompanying students to review their results. Any parent can request to see his or her child's test paper and results, but not the scoring chart. </p>
<p>Ms. Bushman, the math education consultant, said that she explained the scoring to her students but that the concept was hard to grasp. </p>
<p>''My advice to a student is that catching an error on his stronger side is worth more to his total score than catching an error on his weaker side,'' she said. </p>
<p>The lower a school's cutoff score, the bigger the problem, said Mr. Feinman, the economist. For example, a student in the 95 percentile in math and the 79 percentile in verbal could have made Bronx Science last year, beating out a student in the 94 percentile in math and the 81 percentile in verbal. </p>
<p>Mr. Feinman said these results seemed arbitrary. </p>
<p>''There is a notion that the kids who didn't get in are clearly inferior to those who did,'' he said. ''But I think what the grading suggests is there may be a significant number of cases where that is really, really hard to say.'' </p>
<p>Mr. Feinman added, ''If you don't have to spend your time equally you should spend it more on the area in which you are stronger, which would be counterintuitive, I think, to most people.''</p>