With a Departure from Historical Criteria, U.S. News Appears Willing to Shuffle Its Rankings

Rather than average class size, USNWR ranks are based on reported percentage of classes in the following buckets <20, 20-29, 30-39, 40-49, and 50+. Classes are defined as "an organized course offered for credit, identified by discipline and number, meeting at a stated time or times in a classroom or similar setting, and not a subsection such as a laboratory or discussion session. Undergraduate class sections are defined as any sections in which at least one degree-seeking undergraduate student is enrolled for credit. "

In theory a college trying to game this ranking, might put caps on most class sizes choosing numbers that end in 9 (19 or 29) and make a minority of classes as large as possible. A 1000 person class is counted the same as 50 person class for the USNWR ranking. If the college has a grad school ,they might encourage undergrads to enroll in small primarily grad classes. There could also be some creative accounting of which courses are included and excluded. For example, the Columbia analysis found >35% of classes had >20 students, yet Columbia reported only 18% >20 students to USNWR.

4 Likes

When I was looking at Northeastern for S23, I noticed that they broke the classes out into the lecture sections and the smaller group sessions, with separate course registration numbers. I thought it was a funny way to do things, but I guess that is why.

For instance, this a recommended the plan of study for first year civil engineering majors. PHYS 1151 is the lecture course. PHYS 1152 is the lab. PHYS 1153 is the TA session.

Question: How will USNWR factor and weight this diversity measure?

Answer: In a way that doesn’t shake up the top 20 much.

———————

But seriously
 the reason USNWR needs those familiar names at the top is to retain perceived credibility. So I don’t expect this new methodology to cause much movement among the top 20. Where I think we’ll see changes, is in T26-75, with many public flagships moving higher and a few schools that had try to custom fit the previous methodology falling a bit. But, overall, nothing earth shaking.

3 Likes

Per the Common Data Set directions, both the main lecture and its associated lab/discussion sections should be reported separately, regardless of the numbering. (CDS labels the main lecture as a “section” and the associate lab/discussin as a “sub-section”.). So a Chem/Math class of 200 might be in the main lecture ("section), and 10 labs of 20 each (“sub-section”)

What/how USNews does with those two totals (1 section of 200, and 10 sub-sections of 20) is beyond my knowledge.

1 Like

It seems likely that colleges in the two national categories will move into and out of top-20 (or top-15, top-25, top-50, etc.) positions more than has been seen in recent years. This, by itself, would be notable.

Information from the Brookings Institution (based on IPEDS data) indicates how this could impact colleges in USN:

There’s a lot of talk around gaming rankings.

But from my admittedly naive perspective, surely every college does this to some extent?

Even Harvard has the z-list and sends letters out to kids who have no real chance of acceptance to get them to apply (according to the data that was released in the SFFA case, Harvard sent letters to kids who had virtually no chance of being accepted according to their admissions process).

2 Likes

There isn’t enough information to tell. For example, the article mentions that class size will be removed, which has a significant 8% weighting (far more than alumni giving or terminal degrees). That missing 8% has the potential to have a significant impact, but it depends on what replaces that lost 8%. If it’s largely replaced by things like 1=“marginal” / 5=“distinguished” college admin survey or financial resources per student, then I don’t expect much change. Colleges that game or lie about class size to a greater degree than typical might be effected, but typical colleges would not see much change.

The article also mentions a greater emphasis on “success in graduating students from different backgrounds.” It’s unclear what this means and whether the weighting given to this category is significant. For example, it could mean giving a small weighting to graduation rate for URMs, which is an available stat to USNWR. This would result in little change in rankings since graduation rate primarily relates to being selective enough to admit students who are extremely likely to graduate, offering good enough FA for students to pay for college, and not having 5-year programs (co-term / co-op / 
). The colleges that do best on this type of measure are Ivy+ and the most selective LACs – the same colleges that are already at the top of the rankings.

The weightings are arbitrary. There is no validation or verification of whether the formula for “best college” is reasonably accurate besides the output being what USNWR readers expect to see, with familiar names at the top. If the weightings used in the formula don’t do this, then I expect USNWR will adjust weightings until they do.

6 Likes

That’s the real heart of the problem, right? A lot of factors are used as a proxy for educational quality and the derived rankings become canon for “quality”.

4 Likes

That is common at many colleges and has existed for decades.

The weightings might not make much sense to us, the readers, but I would submit that they are anything but arbitrary. USNews has been accepting feedback and tweaking the numbers for years based on that feedback. For example, they’ve had plenty of discussions over teh years of Alumni Giving – which to me, just favors rich private schools – but it has persisted, at least until now. Heck, the cynic in me would say Alumni Giving was purposely left in to give the top privates a few easy extra points over teh publics. (If the NE schools do not predominate in the top 10, they won’t sell too many magazines.)

2 Likes

I would add that if the course is in a field that considers real-time discourse, verbal agility, facilitation, and presenting to be important academic traits, then a smaller class size allows students to practice those skills more frequently. It makes it easier for professors to evaluate their students’ demonstration of those skills.

On the issue of paying for class size, my daughter took two classes this year that were scheduled for the exact same time. This meant that she never attended one class live, a large lecture in which attendance was not mandated though (maybe sections were, I am not sure), and she always went to the small class instead. Originally, I was sort of aghast at this notion and wondered why pay for a class that you don’t have to attend? It ended up working out for her, and she did well in both classes. But I can see how for a student who had difficulty working independently and/or was struggling in other ways, it would probably help to have smaller class sizes in terms of motivation and accountability.

While my daughter has had a good experience so far, I remember struggling to get my work done during one particular semester in college. In retrospect, I think that I was probably somewhat depressed, but no one realized it including me --people were just as not as aware of that stuff at the time as now. If I didn’t have to go to class and have daily contact with my professors and TAs that semester would have been a disaster especially since I was in a single. I would have rarely left my room! In other words, small interactive classes probably helped me push through a rough patch without a hit to my GPA.

1 Like

That!

“Best” is completely subjective!

So the only “measure” if their ranking is “accurate”, is if they generally produce the results that readers expect to see.

The shifting of percentages/weightings back and forth, including/dropping criteria is nothing but alchemy. It’s just as valid as adding in the quotient of zip code and area code for good measure.

In reality there should be no “overall” ranking published - because it can never be “correct”.
The ONLY ranking should be produced by each individual user, by sorting by whatever individual criteria, and possibly combinations thereof, that are meaningful to them.

4 Likes

Or these new rankings will separate private and public schools, just like they currently separate “National Universities” and LACs. Seems to me that is a plausible outcome here, after all UCLA only cares about being the number 1 public university, not how it compares to Duke or Georgetown.

Not gonna happen. Kinda defeats the whole purpose of one ranking of Unis, which was the whole concept of USNews’ ranking in the first place.

btw: disagree. Cal and UCLA and Michigan and UVa definitely do care how they rank in comparison to the other R1’s.

2 Likes

I don’t see whats wrong with this. In America the top colleges are private, and thats just a fact. Graduating URM students and pell grant students is very noble. The only thing that will really help the publics in the rankings is increasing the weight of the reputation score, and im 95% sure US news wont do that.

Us oldsters may remember that the first USNews college ranking had 3 publics* (gasp!) in the top 10, all ranked higher than MIT, and a certain private college in Rhode Island was no where to be found.

*Berkeley #5, Michigan #7, & Illinois #8, tied with Cornell

A fact? And this is known how? From USNWR rankings?

That’s a circular argument.

4 Likes

Which public(s) do you see in top 10??

1 Like

I’m not big on overall rankings, because everyone’s weighting methodology varies based on what’s important to them.

The discussion above about small class size is just one example. To some there is a huge value proposition in small class size. To others, class size may be lower on the value list. Neither view is wrong, and neither is better for overall (vs personal) ranking purposes. There is no one-size-fits-all way to measure factors like this for all prospective students.

There is also a continuum of thought on valuing private education versus public education. Private school “bias” can start as early as preschool. But to some, a public school education has more value than a private school education. Neither view is wrong, and neither should be valued higher for overall ranking purposes.

3 Likes