USNews overperf.(+)/ underperf.(-)

<p>What does the overperf.(+)/ underperf.(-) category mean? I realize it is the difference between predicted and actual graduation rates, but could someone enlighten me on what this data actually means??? How does USNews get the predicted graduation rates?</p>

<p>I don't think anyone knows. I've assumed that it's based on GPA/class rank/standardized test scores in some manner. I suspect it doesn't take financial aid into account, but I don't know that for sure.</p>

<p>There are a lot of people who would like to know, so I hope someone answers this with good info.</p>

<p>This is what is says in the USNWR Methogology link: I'm still not enlightened. It seems to be measuring USNWR's model as much as anything.</p>

<p>"Graduation rate performance (5 percent; only in national universities and liberal arts colleges). This indicator of "added value" shows the effect of the college's programs and policies on the graduation rate of students after controlling for spending and student aptitude. We measure the difference between a school's six-year graduation rate for the class that entered in 1999 and the rate we predicted for the class. If the actual graduation rate is higher than the predicted rate, the college is enhancing achievement."</p>

<p>haha...now i know a + sign means good lol</p>

<p>Yes, but how does US News "predict" the graduation rate? That's what always seemed a little fishy to me.</p>

<p>fireflyscout:</p>

<p>I don't think it's necessarily fishy, but it is undisclosed to the best of my knowledge. If US News is using a factor analysis to predict grad rates, I think they should tell us what it is.</p>

<p>Goodness, you are an optimistic soul, Tarhunt!</p>

<p>ohio mom:</p>

<p>Oh, I don't think I'm optimistic. I suspect that the US News model leaves much to be desired and this is why they haven't disclosed. I remember when they went to this approach. They received criticism that raw graduation rate data were unfair to schools where the student population was unaccomplished relative to high selective schools. So, they went to a value-added approach. Overall, I suspect this approach is better than what they were doing before.</p>

<p>I doubt their factor analysis is very accurate. Even if they sampled college graduates and non-college graduates to get their numbers, there are almost certainly a number of factors influencing graduation that can't be easily accounted for. In addition, value-added calculations are bound to be less than accurate for schools with smaller populations.</p>

<p>Still, it's an attempt of some sort to get it close to right.</p>

<p>I've always assumed that they asked the colleges to predict their own graduation rates, then get the actual figures later.</p>

<p>Nope, that would be very corruptible. Every college would underestimate its graduation rate on purpose to get the extra points.</p>

<p>ferrisbueller:</p>

<p>I've read new articles from those representing colleges in which they were quoted as wondering how this was determined. I'm fairly sure it's not self-determined.</p>

<p>that makes sense, I was just speculating. At any rate, why does any number other than the actual graduation rate matter?</p>

<p>It is my understanding that US News gets the information for the predicted and then actual graduation rates from an annual survey that colleges are required to submit to the federal government called the Integrated Postsecondary Education Data System (IPEDS) Graduation Rate Survey. The predicted graduation rate should simply be the actual one from the year before, it is just showing improvement or decline from one year to another.</p>

<p>ferris:</p>

<p>Measuring only actual grad rate would be a bit unfair to schools that admit kids with average or below average skills. I think the idea is to figure out which schools do the best at helping students to graduate. An 85% grad rate at Harvard, given the skills of the undergrads there, might not be as impressive as, say, a 65% grad rate at some schools.</p>

<p>drusba: I hadn't heard that. As I said, I've seen newspaper articles with deans of this or that wondering where US News gets those numbers. If you're right, it clears that up.</p>

<p>I have assumed, but wsn't sure, that they used a methodology similar to that of Sandy Astin and crew. They did a big study on "expected graduation rate" where they ran a regression using variables that included test scores and HSGPA (or some similar measure of quality) and some other institutional measures. I don't have it in front of me. I don't know if USNews duplicated Astin exactly or just did something similar, but i am certain they did a regression.</p>

<p>Whatever their regression equation, it is not the case that the predicted rate is the one from the year previous, and that the difference is the change from one year to the next.</p>

<p>while I understand that you have to look at a grad rate within the context of the type of students admitted, that doesn't answer the question of why performance in relation to a predicted grad rate matters. Is it important because if they exceed the predicted grad rate it shows they are really working to help students graduate because their actual rate is better than one would predict based on admissions statistics? I suppose that makes sense...</p>

<p>I was wrong about it being the last year's rate. They use the IPEDS info for actual rate but have a regression analysis for a predictor. Though it is not easy to understand, the USNews predictor methodology is explained, analyzed, and criticized here: <a href="http://www.wesleyan.edu/ir/papers/AIRCASE99.PDF%5B/url%5D"&gt;http://www.wesleyan.edu/ir/papers/AIRCASE99.PDF&lt;/a&gt;&lt;/p>

<p>ferris:</p>

<p>Yep. That's about the rationale, as I understand it.</p>

<p>drusba:</p>

<p>Regressions, especially ones in which certain potentially relevant factors are unknown, are always controversial.</p>

<p>Thanks for finding and posting this link.</p>