The Philosophers' Cocoon

As a candidate, so many things about the academic job-market seemed hopelessly opaque to me. What were search committees looking for? How did they read and evaluate dossiers? How did they decide who to interview, invite to campus, or hire? I've tried to shed some light on these things so far in this series, at least from my particular standpoint (having served on several search committees and having spoken to people at other institutions who have as well). I hope you are all finding the series helpful.

In any case, one of the things that seemed the most opaque to me as a job-candidate was the role that letters of recommendation play in the process. How are letters of recommendation read? What are search committee members looking for in them? What looks good? What doesn't? As I explain below, my own experience–which once again may be idiosyncratic–is that there is no simple answer to any of these questions. It's no wonder the role that letters play seems so opaque from the candidate's side of the process: it seems hardly less opaque to me on the hiring side (though I do think there may be a few areas of consistency that candidates may find helpful). Allow me to explain.

Recommendation letters & research jobs

Given that I've only served on the hiring side of things at a "teaching-oriented" university (albeit one that appears to be putting increased emphasis on research), and given that the people I've spoken to who have hired elsewhere work at similar universities, there is not much that I can say about the role that letters of recommendation play at research universities. However, as far as I can tell (though please correct me if I'm wrong, if you've hired at a research university), it appears to me that recommendation letters addressing your research can matter quite a bit. If, for instance, Derek Parfit (R.I.P.) or whomever recommends you as the Next Big Thing–and their recommendation is echoed by similar others–I have a hard time imagining that wouldn't make a difference. On the flip side, if one's letters are more tepid, or come from less-well-regarded figures, I have to imagine that might hurt (or at least, not help relative to candidates with the former type of letter).

So far, so intuitive. Is this wrong? Alternatively, are there particular things that stand out for research jobs I'm not thinking of? I happily encourage those who have hired at research departments to chime in – as again, I'd be the first to admit I'm mostly just speculating here. Why speculate at all, then? One reason I wanted to begin with research jobs is I think it may once again be useful to contrast these types of jobs with other types of jobs–"teaching-focused" jobs at liberal arts colleges, universities, and community colleges. For, as I will now explain, my sense is that letters can play quite a different role there.

Recommendation letters & teaching jobs

Let me begin this section with a frank admission: having hired at a teaching institution multiple times, I just haven't found research-related recommendation letters very helpful in evaluating candidates. I will explain why momentarily. However, before I get to that, a few quick points related to this.

Teaching-related letters

While I have gotten the sense that others at institutions like mine may attach more weight to research-related letters than I do, I have also gotten the sense that teaching-related letters play a significantly greater role for teaching jobs than research-related letters do. Why? For a couple of reasons.

First, whereas it is (in my experience) relatively easy to get a handle of what someone is like as a researcher by reading their research statement, writing sample, and other publications, my sense is that it is very hard to get a handle of what someone is actually like as a teacher from their teaching dossier. Sure, you can get the candidate's teaching philosophy – but it can be hard to tell to what extent their teaching methods in the classroom actually reflect that philosophy. For similar reasons, it can be hard to get a handle of how effective the person is as a teacher, as all you have to go on are the candidate's own materials and student evaluations (which yeah…I hope you know the problems with those). Consequently, teaching letters can be really helpful in deciding who to interview. A frank, detailed letter by a third party of what actually you do in the classroom–and frank assessment of what seems to work well and what you could improve at–can, given the rest of the teaching dossier, plausibly give the reader a much better sense of your strengths as a teacher than the teaching portfolio alone. Further, at least in my experience, a frank assessment of areas that could use improvement is not necessarily a bad thing. Indeed, I would much rather read a "balanced letter"–one that highlights a candidate's strengths and weaknesses as a teacher–than an overly effusive one that contains no discussion of areas of potential improvement. The 'balanced' letter is not only likely to strike me as probably more honest (who doesn't have areas they could improve in as a teacher?), but also helpfully informative: I'd like to know where the person thinks the candidate could improve, not necessarily so that I will 'rank them lower' as a candidate but in part to determine whether my own impressions of the candidate are accurate or inaccurate (as one often gets a feel for weaknesses when reading a portfolio anyway!).

That's one reason why teaching letters may play a bigger role than research letters for teaching jobs (though again, my experience may be idiosyncratic): teaching letters may simply be more informative than research letters, giving readers information about teaching that is harder to gain than similar information about research. A second reason, however, is more direct: teaching schools are hiring teachers. Yes, we care about research. Still, teaching comes first. At my school, we are expected to devote something like 45-60% of our time to teaching, 15-30% to research, and 15-30% to service. At some other schools, I get the sense the weight in favor or teaching is much higher, upwards of 75-80%. So, teaching letters are very important for a second reason: hiring an excellent teacher is often our highest hiring priority.

Research-related letters

What about research-related letters, then? In my experience, candidates usually have significantly more of these than they do teaching letters, even for teaching jobs (I had six research letters, and only one teaching one!). What role do these letters play when applying for a job at a teaching school? It may come as a surprise to candidates reading this that my honest answer is: I have no idea. I've gotten the sense from others I've spoken to that research-related letters are certainly read at teaching schools, and can make a difference in some nebulous sense. However, the role that they actually play in decisionmaking remains opaque to me. It remains opaque, in large part, because–having hired three times now–by and large I frankly did not find research-related letters very helpful. Why? For several reasons.

First, my sense is that because research-related letters tend to be overwhelmingly positive, it can be really hard to infer much from them for the sake of ranking candidates. The fact is, most letter-writers (at least in the US, though I have heard things may be different elsewhere) tend to talk up their candidates a great deal. I don't blame them, of course: they are writing in support of the candidate, nearly all of whom are pretty damn good philosophers! The problem is, when just about every letter is overwhelmingly positive, they tend to blur into each other, making it difficult to make heads or tails of which candidate really is the better researcher on the basis of the letter.

Second, my sense is that it isn't necessarily wise to attach greater weight to a letter based on the stature of the author (viz. their status as a Super-Famous Philosopher), as that runs the risk of introducing unhelpful bias into evaluating candidates (viz. is this candidate really better than another just because they wrote a dissertation under Super-Famous Person, who really likes them? Or, is the other candidate better even though they weren't supervised by Dr. Famous?).

Given these two problems, the most natural thing to do then is to read the candidate's materials (i.e. their research statement, writing sample, etc.) and then interpret the recommendation letter on the basis of one's own reading of the person's materials. However, my experience at this point is that one of two things can occur:

1. One can disagree with the letter-writers' evaluations (which in my experience can happen surprisingly often!)
2. One can agree with the letter-writers' evaluations (which in my experience also happens routinely).

The problem then is that if (1) is the case, the letter wasn't that useful (since you disagree with it), and if (2) is the case, the letter isn't that useful either (as you only 'lend weight' to the recommendation to the extent that you agree with it!). Of course, recommendation letters could in principle get you to reconsider or otherwise alter your perception of the candidate's work – buy my own sense, based in part on first-personal experience and partly by talking to others, is that all too many people seem liable to attach more weight to their own estimation of a candidate's work than anything else (particularly given recognition that letter-writers are not exactly disinterested parties).

Anyway, long story short (I know, TL/DR!), while teaching letters clearly play a significant role in hiring at teaching institutions, it is far less clear to me what role research letters play or how much they play it. But again, perhaps my experience is idiosyncratic. I would love to hear from others who have been on the hiring side of things to find out!