Anyone who has ever applied to grad school knows about how important it is to have good letters of recommendation for acceptance. Yet I sometimes wonder if they’re actually useful for predicting who is and isn’t a good candidate for graduate school. I only found [one article](http://neoacademic.com/2014/02/21/do-recommendation-letters-actually-tell-us-anything-useful/) that tries to analyze this question, and frankly I don’t think that it offers any meaningful insight into that question. I personally think that their ubiquity in the process must have been borne of some intrinsic value, yet they have many confounding factors associated with them that make me question their worth:
- Academia tends to be very mentor-based in terms of what you can accomplish. If you happen to have a professor who is willing to nurture you, to open doors for you and to give you the kind of experience that grad schools "like to see," you will be able to go far with only a moderate level of intelligence. If you fall into the wrong crowd in that sense, no matter what your intelligence is, your career advancement is hindered. Also tends to lead to undergrads from top schools with weaker achievements being valued over those from lower ranked schools.
- It is reputation based to a fault to the exclusion of high quality work done at lower ranked schools or for professors whose alma maters are lower ranked schools. Many admissions committees are looking for LORs from professors they work with personally because "we can trust them." That sounds extremely exclusionary and I would perhaps label it as a form of cronyism.
- It values pandering over practical experience, in that it often matters more how you pander to the desires of those who accept your LORs more than what you actually accomplished. Among those I've known, those whose research experience consisted more of reading articles and performing simple experiments for professors with more prestigious degrees tended to be valued over those who put in much more substantial commitments in less "academic" labs (e.g. those funded by private companies).
- Different professors offer different qualities of praise. Some will write amazing recommendations for rather weak students, while others won't even particularly praise their very best. I even know a case where one professor refused to write a recommendation for someone who did high quality work for many years in her lab because "I don't really know much about what you did here." Not as rare an occurrence as one would like.
Overall, the LOR system strikes me to a large extent as a system where success has more to do with which crowd you fell into than what you are actually capable of. This seems to lead to a self-propagating system whereby professors choose people who are just like them to have the opportunities that allow them to have success in academia, to the exclusion of people of different backgrounds (non-traditional students, women, those of a more practical rather than competitive nature, etc). Given the prevailing decline of academia over the past decades, it’s fair to wonder whether the core component of graduate admissions has some pretty glaring flaws.
Thoughts?