Among the issues are lack of uniformity of content, excessive use of superlatives, reliability—if the writer is not known to the recipient—and more.
Even if the writers are well-known academic surgeons, the degree of their personal knowledge of the applicants is not always clear.
During an extensive Twitter discussion over the weekend, someone mentioned that in an attempt to deal with this problem, emergency medicine had developed a standardized letter of recommendation.
A recent paper from the EM Standardized Letter of Recommendation Task Force shows that there is still work to be done. From the abstract:
For the question on "global assessment," students were scored in the top 10% in 234 of 583 of applications (40.1%), and 485 of 583 (83.2%) of the applicants were ranked above the level of their peers. Similarly, >95% of all applicants were ranked in the top third compared to peers, for all but one section under "qualifications for emergency medicine."
Ive written before that deans letters are more like public relations press releases than accurate assessments of a students performance. You will rarely find negative comments in them. But another recent paper by a group of psychiatrists found that The presence of any negative comments in the deans letter yielded significant correlations with future problems. Further, those applicants with future major problems had significantly more negative comments in the deans letter than did those with future minor problems. Other factors such as USMLE scores, failed courses, letters of recommendation, and interviewer ratings and comments did not predict future problems.
These problems are not new. A 1983 New England Journal opinion piece about recommendation letters entitled "Fantasy Land" is remarkable for its validity even today. Here are a few choice quotes.
Its a land where everyone is "a pleasure to work with," has "excellent initiative," is "enthusiastic and conscientious," and possesses and "above-average fund of knowledge."
No one is ever poor, fair, or average; they are all "very good" or "excellent."
The author, Dr. Richard B. Friedman, said letters of recommendation were useless and advocated doing away with them.
A brief JAMA essay by Dr. Henry Schneiderman in 1988 called for more openness in describing students but acknowledged that negative comments were often "the kiss of death."
He proposed a new system of categorizing medical student performance. Here are just a few examples.
@AmirGharferi suggested this:
"Dr.G, do you feel comfortable writing me a strong letter?"
"No."
"Ok, Ill find someone who is."
That works if the student is aware enough to ask, and the faculty member is honest enough to say no. In my experience, even the most marginal of students can find someone—in addition to the dean, of course—to write a good letter.
I am no longer involved in the process of selecting residents. I have no suggestions.
What is your solution to this problem?
