There are multiple dimensions to universities, which really makes rankings difficult if not impossible if there is not crystal clarity on what is being ranked. In this discussion, I can understand what both sides are saying. Michigan has higher ranked graduate programs and is a top research university compared to Notre Dame. These likely contribute strongly to “academic prestige”. Indeed, relatively few universities can go to-to-toe with Michigan in this regard. A number of the rankings are biased this way.
But I think the world would be a poorer place if we only allowed large research universities. Schools like Williams, which would obviously do poorly in graduate study (only two masters degree programs) and research. Williams focuses on undergraduate education, teaching, small classes, advising, and a residential experience. They are tough to match in this regard. USNews deals with this by having a category for liberal arts colleges.
But there are schools that fall somewhere on the ill-defined continuum in between these points. Notre Dame might fit somewhere in here along with schools like Wake Forest, William and Mary, etc.
The universe of schools that can function as a graduate/research powerhouse without sacrificing some undergraduate focus and intimacy is probably pretty small. Princeton comes to mind as a school that comes pretty close, so all it might take to do this is having a very high sticker price and a $22+ Billion endowment for only 8K students (even with no medical school). But even then, Princeton may not be able to cover all areas of graduate study and research as well as Berkeley, for instance.
You can certainly argue this either way, but I’m not sure what the point is. Both are fine schools and the OP should focus on what matters for them.