Cohort Default Rates or Graduation Performance Rank as College Quality Indicator? (Calling LA, MA, MO, NM, NC, WA residents especially!)

Having similar admission rates does not necessarily mean that the entering students at the different colleges have similar academic strength from high school or (for transfers) prior college.

1 Like

This is definitely a perspective that I hadn’t thought about and could conceivably be true. So I guess the question is whether the education is more effective because of greater student agency around costs & schedules, or whether it has to do with the supports and instruction a university provides.

If the former, then it raises questions about why flagships and public universities more popular with higher socioeconomic families don’t do more to provide for similar levels of student agency. If the latter, then I wonder what can be done to make a broader part of society realize the benefits of the undersung publics. Because if a residential college experience is more beneficial for the vast majority of students, then from an equity standpoint it would behoove states to figure out how to make dorm life a more affordable option. Because then the schools with higher commuter populations might then build more of the residential quality that can also provide the benefits that other studies have shown.

I recall my time living in a French dorm my junior year of college. It was a very basic place, with hall bathrooms (though the rooms did have a sink and bidet…the French!). There was no gym or other niceties that some US residential campuses now have. But each floor had a kitchen with 3 or 4 hot plates and a wall with refrigerator lockers. So students could control their food budgets, but there was still a social atmosphere because of cooking in the kitchen and everyone living in the same hall. And living in the dorm was far more affordable than sharing an apartment in the regular town. Implementing options like this seems as though it could make residential life more affordable for more students while still accruing the benefits of a residential college.

1 Like

I worked at a school with a large population of very low socioeconomic students. Public transportation in that urban area was very challenging. Households were often in turmoil, making studying difficult. The free breakfast and lunch safety net that sustained them through K-12 is gone, and food insecurity is real. It wasn’t a big surprise when many of these students dropped out. They then were saddled with the debt they had accumulated, with no degree. It was incredibly sad. The university has been working hard to try to put support systems in place, but … money.

2 Likes

Yes, this and all of this.

One of the dirty little secrets of educational statistics is that “graduation rate” is a nearly meaningless number, because not only doesn’t it measure what it claims to, it doesn’t even measure anything meaningful.

The definition of graduation rate for baccalaureate degrees is: Out of the total number of matriculating students at an institution for a given academic year, what percentage of them received a baccalaureate degree from that institution within 150% of the time normally allotted for that degree (i.e., generally 6 years), with some students removed from the calculation entirely for generally sensible reasons (e.g., students who died before the time period elapsed).

Notice what this does and doesn’t do, though.

First of all, it excludes a whole bunch of degree completions. I started at one 4-year college, and then transferred to another from which I graduated—and so I counted as a negative for the graduation rate of the first institution, and didn’t count as a positive for the one I graduated from.

In addition, it penalizes colleges that have a large number of part-time students—a student who takes, say, 9 credits per fall and spring semester at a college where it takes 120 credits to graduate will necessarily take longer than 6 years to graduate. (I work at a college that has one of the lowest 6-year graduation rates of any public 4-year college in the United States. But if you track our graduates through 8 or 10 years? We actually have a quite good completion rates at those time scales—we just have a whole bunch of students who attend part time, and lots who take semesters off along the way for financial or family reasons.)

There are more issues with the way it’s calculated, but that’s enough for now. Basically though, the TL;DR is that published graduation rates are stupid, stupid numbers that don’t relate to any sort of reality.

3 Likes

Appreciate all the feedback, which brings me to a couple of questions.

  1. On College Navigator, on the Retention & Graduation Rates section, part of it looks like this (this is New Mexico Highlands, source):

My reading is that 25% of students graduated within the designated time span and that 37% of the students transferred out and then that the remaining 38% of students either dropped out or are on a longer pathway toward graduation. Is my reading correct?

  1. Okay, the graduation rate has some issues, presumably, across nearly all schools, or at least across nearly all public schools. Thus, would looking at a graduation performance rate index (i.e. actual vs. projected) still be a fair comparison, as the numbers would be similarly flawed for everyone? Or would this be a case of garbage in, garbage out?

  2. Is there some kind of metric that is publicly available that people think would be useful at assessing a university’s effectiveness? I’ve read of colleges doing the Collegiate Learning Assessment (CSA) as a sort of pre-test/post-test for the first years and seniors, but schools generally don’t publish the results of those tests.

I guess I’m continuing on my my never-ending quest to find ways to assess the quality of a school, as I don’t think that major newspaper/magazine rankings have typically done a great job of capturing that. (For more on what my ideal ranking might look like, see post #2 over here: Create Your Dream College Ranking Methodology)

I think transfer out rate is one way to evaluate a school IF the student is interested in a “typical” traditional undergraduate experience. If more than a third transfer out, I would be interested in finding out why they leave. The transfer out rate itself is not necessarily an indication that the school isn’t “good,” but it’s an indication that the student may want to dig deeper.

Same with graduation rate. A low graduation rate may or may not indicate issues for a particular student. If I were going to school while working and raising a family, I would want a school that was amenable to my needs … and perhaps a lower graduation rate might be an indication that others like me attend the school. Every student has their own needs, and one size doesn’t fit all.

So how do you know if a school is any good? Like most things in life, you do your due diligence and make your best determination. I didn’t like my public school middle and high schools. My friends had no problem with them. My kids went elsewhere. All the kids are now doing just fine in life … the public schools didn’t work as well as I wanted for my kids, but they worked for my friends’ kids. Colleges are like that, too.

1 Like

NCer here so speaking for the NC list only.

No. That seems more a measure of how good the school is at predicting a graduation rate. I dont see where it has anything at all to do with students or quality of education. It’s just like The Price Is Right – which schools can guess closest to the actual graduation rate without going over.

No, but I try not to judge people based on which college they went to or if they went to college at all.

Except I do think Duke alums are often very self-centered and think a lot of themselves, so they have to overcome my prejudice against them. My spouse went to a semester of grad school there and left in part because of the other students’ attitudes. I also had a Duke alum I worked with daily say to me verbatim, “I don’t make mistakes” and mean in all sincerity. He was very egotistical. And I have professor friends there who have some things to say about the entitled students and the administration, but I do know a few nice people who have attended or are attending Duke, so I try not to write them all off.

Last one. No correlation at all. It’s The Price Is Right.

Doesn’t matter at all. I agree with dfbdfb that graduation rate is not a great measure, but I think it is more useful than this prediction gap which I really don’t see the point of measuring at all.

Doesn’t matter.

"Previously racially segregated colleges* is a really gross way to phrase that. They are Historically Black Colleges and Universities. Would you say that a women’s college was “previously gender segregated”?

The HBCUs in NC are sprinkled throughout the list, not clustered at the top or bottom. In fact, the flagship UNC- Chapel Hill is right between two HBCUs. I’ve italicized and bolded the HBCUs & Pembroke. The italicized only schools have a lot of lower income students also.

NORTH CAROLINA

Graduation Rate Performance Rank School State 8-year graduation rate Predicted Graduation Rate Category Difference Between Actual & Expected Graduation Rate
1 Winston-Salem State NC 63% 39% National 24%
17 East Carolina NC 67% 56% National 11%
56 UNC-Pembroke NC 46% 36% Master’s 10%
35 Elizabeth City State NC 44% 35% Bachelor’s 9%
73 Appalachian State NC 74% 65% Master’s 9%
87 North Carolina Central NC 49% 41% Master’s 8%
74 UNC-Chapel Hill NC 90% 84% National 6%
127 Fayetteville State NC 44% 38% Master’s 6%
111 Western Carolina NC 65% 60% National 5%
86 North Carolina A&T NC 51% 46% National 5%
102 UNC-Wilmington NC 72% 67% National 5%
122 UNC-Greensboro NC 58% 54% National 4%
146 North Carolina State NC 81% 78% National 3%
308 UNC-Charlotte NC 61% 63% National -2%
158 UNC-Asheville NC 63% 67% Liberal Arts -4%

But again I don’t think this gap shows anything but how well the school can predict graduation rates. You should be looking at the schools that get closest to 0 without going over if you want to measure something.

So NC State would be the winner by that metric and Winston-Salem State would be the big loser by underestimating so much. UNC-C and UNC-A would be losers too since they went over. I do think that is a reflection on UNC-A. It’s an interesting school but I had the impression a lot of students move there for the town rather than the school. And I’m not surprised that the actual graduation rate there is less than the school predicted.

Graduation rates are mostly correlated to the college’s incoming student academic strength (which is affected by admission selectivity) and the lack of financial barriers that students face (which can be affected by the college’s financial aid, but also whether students with or without lots of money attend).

It is no surprise that a college like Harvard, with generally high end incoming student academic strength, lots of students from families with money, and good financial aid for the rest of those who attend, has high graduation rates. But would Harvard have similar graduation rates if it traded students with UMass Boston?

1 Like

Precisely.

For all that we’d like to be able to do it, there are no (I repeat, louder: there are no) quantitative measures that provide a universal (or even nearly universal) measure of college quality, and the task is even more hopeless if you’re looking for something squishy like “fit”.

It appears that you misread what @ucbalumnus wrote. They were talking about state systems—so not HBCUs, but rather states that had imposed segregation on higher education. This means that they weren’t referring to, say, Alabama A&M and Alabama State and the other HBCUs in Alabama, but rather to the colleges throughout the entire state of Alabama, whether HBCUs or not.

Why is it gross to describe the previous state of racially segregated colleges as such? The legacy is that places that had racially segregated colleges have public HBCUs and HWCUs (W = White in this case). For example, NC A&T is a HBCU, while NCSU is a HWCU, based on their racial designations during the days of racial segregation in education. Private HBCUs also exist because they were founded due to lack of colleges for Black students during the days of racial segregation and exclusion.

These historical facts do not necessarily imply anything about the HBCUs’ and HWCUs’ current qualities, although perception among students often remains, resulting in continuing self-segregation (particularly in most non-Black students refusing to consider attending HBCUs).

Many of them were founded because many colleges were men only back in those days, so it would be accurate to see them as legacies of gender segregation in college education.

Aren’t the predicted graduation rates discussed here from a formula from a third party (Washington Monthly) rather than the colleges themselves?

1 Like

OK. I re-read what you wrote and realize that I interpreted it wrong. I’m sorry.

1 Like

Ah, I missed that. That’s what I get for speed reading. Read it more closely now. Even if so I don’t see the merit in this predicted rate. I get what they were trying to do now but I’m not sure how it helps with AustenNut’s quest for a better way to rank colleges. I think it could point to colleges that are better at lifting students up out of poverty though.

The idea is that a college is doing well if its students graduate at a higher rate than students of similar demographics graduate overall. Such things as better financial aid, student academic support services, and teaching quality could be helpful in this respect.

However, there could be problems with such a measure, since the expected graduation rate may not account for all relevant demographic factors (e.g. the Washington Monthly measure does not include incoming student academic stats) or distribution of majors. Colleges may also raise graduation rates in less desirable ways, such as reducing rigor to very low levels. Volume of general education requirements can also affect graduation rates, so a focus on graduation rates (either raw or relative to expected) can favor colleges with fewer general education requirements. As noted above, graduation rates may also be less accurate when there are more transfers out or in, or where there are part time students who take more than six years to graduate.

In other words, such measures are useful, but have limitations. They may reveal more of the treatment effect of a particular college, compared to how raw graduation rates mostly proxy incoming student characteristics affected by admission selection.

3 Likes