WSJ & Forbes Rankings

[Nocreativity1]

Carino an honest question…Why do UC people insist on highlighting they really are happy? It is an incredible school on par with the absolute best school’s in the world, but seemingly its fans are constantly arguing it’s “unique” academic rigor and happiness of its students.

Who cares what theWSj and Forbes think about UC or anyone else for that matter, Prezbucky loves you!!

[nrtlax33]

@“Cariño” : For the link you provided at #58,your statement

According to this Forbes’s ranking, UChicago is #21 of the 200 colleges with the most grateful, successful, and happiest graduates.

is not consistent with the methodology used in the article.

The index ranks private not-for-profit colleges with more than 500 students using two variables: the median of total private donations per enrolled student over the last seven years, as reported to the Department of Education, and the alumni participation rate, or the percentage of graduates that give back in the form of donations to their colleges each year, regardless of dollar amount. The alumni participation rate, compiled by the Council for the Advancement and Support of Education (CASE), is averaged over three years. Total median dollar donations are given a 70% weighting in the index and the alumni participation is given 30%.

So a school got some big development cases will have the happiest, most successful graduates?? Read this article (this one is free) to find out – http://online.wsj.com/public/resources/documents/Polk_Rich_Applicants.htm

It is well known that some schools ask alumni to donate $1 to have the highest participation rate.

Do you believe “Colleges With The Happiest, Most Successful Alumni” can be measured by the index above? I do understand Univ of Chicago got some big donation recently. (some not paid actually) But that has nothing to do with happiness or success.

NYU got enough donation to offer all its med students free tuition. I do believe and I agree that if some company starts to rank med student happiness, NYU will be number one.

No wonder not many people are reading Forbes. Total waste of time. I choose to rebuke you to prevent more tragedies at the University of Chicago.

[simba9]

Why do UC people insist on highlighting they really are happy?

I thought the University of Chicago was where “fun comes to die.”

[Nomorelurker]

@Greymeer, thanks for taking the time to do that analysis on the three variables (measures of efficiency) on the WSJ ranking. Your analysis provides another important aspect of context of rankings. Your posts #34 and #35 on page 3 of this thread are fascinating. To see how a school such as UGA ranked at 240 by WSJ (which seems an anomaly) comes out in the top 25 for real world outcome and resources available to students, is telling. Many other public universities, in addition to Georgia, which you identified as “most efficient” were in the top 25 of universities hitting the mark, such as UVA, UCSB, UT, Michigan, and UF. By contrast, some small private universities like Tulane which have historically done very well with USNWR, FORBES, and others, came out in the bottom 100 out of 500 universities, on your 3 variable analysis. Very interesting.

[Mastadon]

The “engagement” metric in the WSJ/THE ranking is highly suspect given their description of their methodology.

It appears that they did not use the same sampling method across all schools, and one of the methods they used (using a student representative to facilitate the recruiting of respondents) is far from random and can introduce a significant bias in the sample.

Times Higher Education has captured student engagement across the US through its US Student Survey, carried out in partnership with two leading market research providers. For 2017 and 2018, we gathered the views of almost 200,000 current college and university students on a range of issues relating directly to their experience at college.

Students answer 12 core questions about their experience that are either multiple choice or on a scale from zero to 10, and also provide background information about themselves. The survey was conducted online and respondents were recruited by research firm Streetbees using social media, facilitated, in part, by student representatives at individual schools. We also worked with participating institutions who distributed the survey to random samples of their own students. Respondents were verified as students of their reported college using their email address. We used an aggregated group of respondents from both years (2017 and 2018 surveys). At least 50 responses in the 2018 survey were required for a university to be included.

The “outcomes” metric in the WSJ/THE ranking is highly suspect because:

1. It includes the a reputation survey of people in academia - which is neither objective nor a true outcome. It mimics the controversial USNews "peer survey" and is one reason for the relatively strong correlation between the two rankings.

An interesting point to note is that the results of the survey of academics and the survey of students (i.e. the engagement survey) have a strong inverse correlation for top ranked schools and that the schools that @prezbucky thinks are “undergraduate focused” (Princeton and Caltech) have some of the lowest engagement scores.

1. The Scorecard database (which is used for salaries) is based on federal student loan data which needs to be used with caution, particularly for "no loan" schools such as Harvard.
2. The "value add" statistical method they use to attempt to eliminate bias just ends up introducing a number of other biases that are less transparent to the consumer, and therefore more misleading.

The “resources” metric in the WSJ/THE is highly suspect because:

1. The student faculty ratio is not a good proxy for undergraduate focused resources because professors at research focused universities carry a lower teaching load so they can have more time for research.

For example, Caltech has a 3:1 student faculty ratio, but 9.6% of its classes have over 50 students. Princeton has a student faculty ratio of 5:1, but 10.1% of its classes have over 50 students. Clark University has a student faculty ratio of 10:1, but only 4.7% of its classes have over 50 students. WPI has a 13:1 student to faculty ratio and only 10.7% of its classes have over 50 students.

2.The number of “scholarly” research papers published per faculty member is a metric that rewards quantity rather than quality of research (which would include some notion of citations) and is likely to be inversely correlated with the amount of time a professor spends with undergrads.

1. The data that academic spending is based on does not always separate out research expenses from other academic expenses (it depends on how each school tracks its expenses) so it is neither self consistent nor representative of undergraduate spending. This mimics the USNews resource metrics and is another reason for the relatively strong correlation between the two rankings.

As an example,
Note that Caltech is amortizing the cost of the Jet Propulsion Laboratory (a major government funded research lab) over a relatively small student population - hence the appearance that they are “inefficient” in the @Greymeer calculation. Of course this also raises their WSJ and USNews rank, despite the fact that it could be argued that professors spending time at JPL are not spending time with undergrads (which would be consistent with Caltech’s low engagement score). For those who were not around in the 90’s - there was one year that Caltech was ranked number one by US News by a significant margin- because they removed the infamous “logarithmic adjuster” that was being applied to academic spending. The adjuster was re-instituted the following year, so that an Ivy League school could be returned to its rightful place at the top of the ranking.

So, the WSJ, USNews and Prezbucky’s rankings actually suffer from many of the same flaws, as well as their own unique flaws, but I would say that Presbucky’s is clearly the weakest of them all - because he likes to shamelessly promote it and I like to tease him about it

https://www.timeshighereducation.com/wall-street-journal-times-higher-education-college-rankings-2019-methodology

[nrtlax33]

@Mastadon : WSJ/THE methodology states

Times Higher Education has captured student engagement across the US through its US Student Survey, carried out in partnership with two leading market research providers. For 2017 and 2018, we gathered the views of almost 200,000 current college and university students on a range of issues relating directly to their experience at college…
The survey was conducted online and respondents were recruited by research firm Streetbees using social media, facilitated, in part, by student representatives at individual schools. We also worked with participating institutions who distributed the survey to random samples of their own students.

Only one company’s name is mentioned above – Streetbees. Streetbees counts Unilever, Vodafone, L’Oréal, Pepsi, BBC as its clients. They are specializing in commercial market research, unlike non-profit/academic institutions. I think using a student representative to facilitate the recruiting of respondents on social media is pretty normal. They are doing business with some of the biggest companies in the world. I think there must be protocols/SOPs/etc. to control the randomness of their samples.

@Mastadon

The “outcomes” metric in the WSJ/THE ranking is highly suspect because:

1. It includes the a reputation survey of people in academia - which is neither objective nor a true outcome. It mimics the controversial USNews “peer survey” and is one reason for the relatively strong correlation between the two rankings.

Let’s not worry about if reputation (10%) should be categorized as part of outcomes. But they are focusing on undergraduate’s teaching (like USNWR’s best undergraduate teaching)

This pillar also looks at the overall academic reputation of the college (10%), based on THE’s annual Academic Reputation Survey, a survey of leading scholars that helps us determine which institutions have the best reputation for excellence in teaching. We used the total teaching votes from our 2017 and 2018 reputation surveys.

@Mastadon

The “resources” metric in the WSJ/THE is highly suspect … Research papers per faculty (8%)

I agree with you that undergrad focused schools like LACs, Brown will be disadvantaged by this metric since their faculty members are not focusing on graduate research to publish papers. I also agree with you that the student faculty ratio is not a good proxy for undergraduate focused resources because professors at research focused universities usually don’t teach undergraduates so they can’t be counted as resources.

An interesting point to note is that the results of the survey of academics and the survey of students (i.e. the engagement survey) have a strong inverse correlation for top ranked schools and that the schools that @prezbucky thinks are “undergraduate focused” (Princeton and Caltech) have some of the lowest engagement scores.

I kind of think of the engagement index as “pressure cooker temperature” index. If undergrads are not the focus of the campus and being neglected (very large number of graduate students) or if the school is very rigorous (like UChicago, Princeton, Caltech), the schools are likely to have low engagement score.
Read http://talk.qa.collegeconfidential.com/princeton-university/1497933-the-daily-princetonian-survey-finds-almost-half-of-students-report-feeling-depressed.html
Why does any school want to have final exams after Christmas like Princeton? It ruins the whole holiday season.

It looks like that finally people start to read the methodologies of the rankings, which is a great thing.

[prezbucky]

@Mastadon

I don’t think Caltech is especially undergrad-focused, but I do think P’ton is – among highly selective private universities anyway.

I wonder why the engagement score for Princeton is relatively low, while their alumni giving rate is so high. Maybe some of them only really begin to feel the love after they leave.

Anyway, mine is not a ranking; it’s a multi-level parking garage. haha

[nrtlax33]

@prezbucky : So what are those colleges where students feel most engaged? Answer: In the WSJ/THE College Rankings, schools with a religious affiliation are leaders in being both inspirational and challenging according to WSJ’s own analysis.
Here are the top 10 by “Engagement”
1 Dordt College
2 Oklahoma Baptist University
3 Texas Christian University
3 Cedarville University
3 Harding University
6 Brown University
6 University of Southern California
6 Texas A&M University-College Station
6 Brigham Young University-Provo
6 Indiana Wesleyan University-Marion

One crucial aspect of a successful college experience is how students learn.
…
The engagement score in the Wall Street Journal/Times Higher Education College Rankings—measured mainly through roughly 189,000 survey responses over the past two years—monitors how challenged and inspired students feel inside and outside the classroom, as well as the breadth of courses their schools offer.
…
Eight of the top 11 schools for engagement have a religious affiliation. Those institutions often include in their missions the education of the entire student—body, mind and soul—and put a premium on group projects, mentoring and self-reflection.
…
**Brown University is one of the few schools that scored high in student engagement, tied at No. 6, as well as in the top ranks overall, tied for No. 7. Provost Richard Locke attributes the school’s high mark for engagement to the fact that students don’t have specific courses they’re required to take. “You only have students in your class who want to be there,” he says, which changes the tenor of classroom conversations.

Brown also encourages hands-on learning through its Engaged Scholars Program, with courses on the anthropology of homelessness bringing students to social-service agencies, for example, or urban-studies students learning about the tactics of community organizers.**
…
MIT Vice Chancellor Ian Waitz says the low score “puzzles me,” noting that internal surveys of the senior class are generally positive.

An MIT design class made recommendations last spring on how to revamp the first-year experience, giving students more flexibility in what courses they take, the opportunity to explore other disciplines and intensive advising to navigate it all.

The goal, Dr. Waitz says, is for students to “have a bit more opportunity to be inspired and retain that love of learning and solving problems” that they have at arrival and regain later in their academic careers.
…

Many elite colleges flunked when it came to challenging and engaging students. The Massachusetts Institute of Technology, No. 2 overall, landed at No. 304 on engagement. And the California Institute of Technology, No. 5 overall, ranked below 500 on that measure. I don’t want to talk about UChicago. If you are forced to walk 0.7 mile to food, I am sure you are not very engaged.
Go Bruno!

[Mastadon]

@Prezbucky -

Anyway, mine is not a ranking; it’s a multi-level parking garage.

The desirability of parking spaces in a parking garage is dependent more on ingress/egress points than levels, so your listing of schools by levels provides little to no value in picking the best place to park.

[prezbucky]

The “engagement” metric would be more valid if they actually just based it on the percentage of undergrads who’re engaged… to be married.

[Mastadon]

@prezbucky - As it it turns out, reporting on “marriage metrics” is not particularly valid either…

Did you know that 60 percent of Bates alums marry fellow alums? If you do know this “fact,” it’s time to forget it.

Untrue though it is, the 60 percent fact still circulates among Batesies and gets passed on from generation to generation.

The accurate figure, as of 2010, is approximately 12.5 percent. Of the 23,356 living alums in the Bates database, 2,914 have a spouse or partner who is also a Bates alum. (If you include alums whose Bates spouse or partner is deceased, the number creeps up a few points.)

(UPDATE: As of Feb. 13, 2015, of the 25,199 living alums in the Bates database in 2015, 2,956 have a spouse or partner who is also a Bates alum. That puts the marriage rate at 11.7 percent.)

In fact, I found that other colleges of our ilk also tend to have a mythical marriage statistic. At Colby, the myth is 50 percent and the reality 11 percent, while at Carleton it’s 60 and 15 percent, respectively. At Middlebury, the numbers are 60 and 13 percent.

http://www.bates.edu/news/2015/02/13/60-percent-marriage-myth/

[Publisher]

What about BYU ?

[prezbucky]

Laughing Out Loud

[nrtlax33]

Of the 23,356 living alums in the Bates database, 2,914 have a spouse or partner who is also a Bates alum.

Assuming that those who have not given money to school in five years have not being reclassified as dead.

https://washingtonmonthly.com/2000/09/01/playing-with-numbers/

[CCSavant]

The BYU statistics are misleading as many were married BEFORE they entered college…

[CCSavant]

I like PrezBucky’s rankings except I would push Cornell to Tier 4 and Emory to Tier 5. Certainly most of the schools in Tier 6 I would see in that category as well, but there are arguments for substituting other schools for U of Rochester, Lehigh and BU - or pushing them down a slight bit.