The Philosophers' Cocoon

In my previous post, I wrote about "referee-proofing" articles. As I explained there, my experience as an early-career philosopher has been that publishing in peer-reviewed journals is something of an art in itself. It is not merely a matter of having good arguments, but also a matter of presentation: a matter of preempting potential misunderstandings by referees, even misunderstandings that may seem from one's perspective to be silly, uncharitable, etc. In today's post, I want to discuss another part of the publishing process–"revise-and-resubmits"–as here too, at least in my experience, there seems to be something of an art to them; and, although it has been a number of years now since I finished graduate school, it can be an art that grad programs may not adequately train students in.

I received several revise-and-resubmits in graduate school. Unfortunately, none of them resulted in publications. Although these cases were a long time ago, and so it is difficult to remember details, I now have some broad idea of where I probably went wrong, and have since then developed some general strategies that have had much better results [although I have had many papers rejected outright since then, I haven't had an R&R rejected for over six years now]. What, then, are some good strategies for R&R's? Obviously, I can only report my own experience–and so I would encourage readers to take them with a grain of salt, and to share your own experiences [including experiences that may contradict mine]. Hopefully, if we share our experiences together, we can get a clearer idea of whether there are indeed any good general strategies for R&R's, and if so, what they are.

For those of you who have never had an R&R, when you resubmit your manuscript you normally not only submit a revised manuscript; you also submit a "reply to reviewers" document where you explain how you have [or haven't] responded to reviewers' initial critiques. Recently, in my social media feed an early-career philosopher asked what they should do if they disagree with a referee. Should they address the reviewer's critique in the paper, this person asked, even if they think the reviewer's critique is off-base; or, this person asked, should they merely explain their "reply to reviewers" why they think the reviewer's concern is mistaken? I think it is great that this person asked this question, as in my experience answering it correctly is absolutely critical to getting good results with R&R's. This person's thought, if I recall correctly, was that they should explain in their "reply to reviewers" why they disagreed with the reviewer's critique. My experience is that this is precisely the wrong way to go, and indeed, my first suggestion is this: 

Suggestion #1: With a few exceptions [and a caveat to be discussed momentarily], it is probably helpful to adopt the general attitude that "the reviewer is always right"–addressing whatever concerns they have in the paper itself, even if one thinks the concerns are mistaken.

Allow me to explain why I think this is the right way to go. If I recall correctly, in my earliest R&R's–that is, all of the failed ones–I adopted the strategy the above person thought they should adopt: if I thought a reviewer was right, I revised the paper and explained in my "reply to reviewers" how I revised the paper and why I thought they were right; but, if I thought they were not right, I merely explained in the reply to reviewers why I thought they were not right. Again, the results I experienced using this strategy were terrible: not a single publication. And, upon further reflection, it is not at all surprising to me why. Think, after all, about what reviewer comments are in the case of an R&R: [A] they are reviewers telling you explicitly what they think your paper needs to do to be publishable, and [B] the reviewers who think the things they are telling you are the very same people who will probably decide whether your paper is published. Unfortunately, as we all know, although our Platonic ideal of "the philosopher" might be "a person who responds correctly and dispassionately to the argument", reviewers are human beings–and it can be very difficult to get human beings to change their minds about things, especially when you just say that they are wrong [see e.g. any debate on social media about anything at all]. Accordingly, if you just say in your "reply to reviewers" that you think they are wrong about a critique–a critique that they think is correct, and which they think must be addressed for your paper to be publishable–you not addressing the critique in the paper itself is, essentially, you swimming against the proverbial current. It is, in essence, to ignore in your paper something the reviewer thinks is critical to address in the paper.

Now for the aforementioned caveat. I don't think you actually have to agree with reviewer's critiques in your revised manuscript. On the contrary, I routinely disagree with reviewers, even in the successful R&R's I've had. The critical thing though [in my experience] is to recognize and respond to the concerns in the paper–even if you think it is a misunderstanding or poor concern–rather than merely explaining in the reply to reviewers why you think it is so. To use an analogy taken from the [otherwise repulsive] recent film, The Wolf of Wall Street, if you are trying to sell someone a pen, it is probably not a good idea to tell them what you think they should want in a pen, telling them that their concerns about the pen you want to sell them are simply misguided and not worth addressing in the pen's design. Rather, if you actually want to sell the pen successfully, you should listen to what they tell you they want in a pen, and then give them a pen that actually–in addition to being a good pen–gives the person what they are looking for. This, in brief, is what I think one should shoot for in an R&R. When a reviewer raises a concern, they most likely want you to address it, not just in your reply to them, but in the paper itself [since, if you had addressed it in the first place, they might have never made the "mistake" you think they made in raising the concern]. Finally, I think one can do of this without in any way compromising the paper's philosophical integrity–for again, you don't have to say in the paper that it is a good concern [or whatever]. All you have to do is raise and address it as a possible concern readers might have–and, since your reviewer is at least one reader who had the concern, there are reasons to think it is a concern readers might have. So, I suggest, one should always try to raise and address every single reviewer concern somewhere in the paper, not just in the reply to reviewers–even the concerns you don't think are good ones.

There is, in my experience, another good reason for doing this: it actually makes for better philosophy papers. To build off my previous post, one of the more interesting trends I have noticed in my publishing career is that even if–in one's own view as an author–a particular reader concern or misunderstanding is misguided, it is quite likely that more than a few readers will have it. This has happened to me repeatedly, where I have sent a paper to multiple journals thinking a given reader concern is totally wrong, and yet, for all that, reader after reader has it. Accordingly, if you really want to write a good paper–a paper that people are likely to understand properly–addressing even the "mistaken" misunderstandings and concerns people raise is a very good thing to do: it can, once again, systematically preempt your paper being misunderstood by readers…which is a good thing! So, that's my first suggestion. Don't "fight" reviewers in your decisions of what to address in the paper. Address their concerns, even if you think their concerns are mistaken.

This brings me to my second suggestion:

Suggestion #2: Be as gracious to reviewers as possible, both in your reply to reviewers and in the paper itself [i.e. tone and gratitude matter]. 

I realize this may sound like "sucking up" to reviewers, but you know what? I have come to realize it is probably intrinsically right to thank reviewers for their feedback, even [again] feedback you think is mistaken–for, the more I have gone along in the publishing game, the more it seems to me that, meanspirited "feedback" aside, just about all feedback one receives from reviewers is helpful. Let me explain.

It is all too natural, I think, to get frustrated with reviewers who read your paper uncharitably, misinterpret key claims, or raise what you take to be poor concerns. But, in my experience, even these reviewers are doing you a favor. For again, as I discussed above, my experience is that if one reviewer is apt to read your paper uncharitably, misinterpret key claims, or raise poor concerns, other readers are likely to as well, including, yes, readers after the paper is published. So, really, the best thing one can do is actually get the uncharitable readings, misinterpretations, and poor concerns from reviewers…so that one can, through one's revisions, preempt the journal's readers from making the same mistakes.

Second, as someone who has reviewed a good number of papers, I think it is easy to underestimate how much effort reviewers may put into their comments, and how difficult reviewing can be. Even though I have been doing philosophy about half my life, I get asked to review papers that are difficult for me to interpret, follow, etc. Accordingly, even though I give it my best effort, I would not be too surprised whether I have been the proverbial "Reviewer #2" on occasion [the reviewer who seems to the author to totally misinterpret things, be unreasonable, etc.]. Now suppose, however, that I put together a review on paper X where I in fact misinterpret some of its key claims and raise misguided concerns…but the author merely explains in their "reply to reviewers" why I misinterpreted them and raised misguided concerns. Even if the author is right, mightn't this seem totally unsatisfactory to me? It is, at the very least, possible that I will not only be unconvinced by the author's argument–but also, even if I am convinced, it is possible that I will be disappointed that the author hasn't revised the paper to prevent other potential readers from making the same mistakes I made. If I am making a decision on whether to recommend a paper for publication or rejection, am I more likely to to recommend a paper for publication whose author [A] just explains to me why I was wrong, or who [B] thanks me for the concern and explains how they address it in the paper? It seems to me that [B] is not only more emotionally intelligent; it also once again seems to me philosophically intelligent. For, again, or so my experience is, even the badly mistaken reviewer really is probably doing you a philosophical favor–helping you see and address in the paper potential misinterpretations, bad concerns, etc., that other readers may well raise if/when the paper is published.

Consequently, what I do in my revised papers and reply to reviewer documents is respectfully thank reviewers for each and every one of their critiques, even the ones I think are mistaken, as in, "I thank Reviewer 1 for raising concern X. On pp. ZZ of the revised manuscript, I raise this concern and address it as follows…". I also tend to thank anonymous reviewers in footnotes in revised manuscripts, as I think it not only gives credit where credit is due, but also helps readers understand the intellectual history of the paper [viz. if concern X is attributed to a reviewer, it doesn't follow that it was my concern, but rather, a concern a reviewer had that other readers might have as well]. 

Which brings me to a final suggestion:

Suggestion #3: Put a great deal of time and effort into your reply-to-reviewers document, once again striking a gracious tone and clearly explaining, in nearly as much detail as in your revised paper, how you address each of the reviewers' concerns–and run it by at least one outside reader.

Again, I have a difficult time remembering the details of my early failed R&R's, but one thing I think I probably didn't do a good enough job with is the "reply to reviewers' concerns" document. I seem to recall that my early ones were relatively brief, saying things like, "I address concern C on p. Z by saying…". In recent years, my reply to reviewer responses have become much more detailed, essentially repeating [but seriously compressing] the entire relevant sets of passages in the revised paper. That is, I have found it helpful not to merely give the reviewer a brief overview of my revisions, along with page numbers of "where to look" for the revisions in the revised manuscript. Rather, with significant amounts of compression, I now more or less repeat the paper's revisions [in substantially compressed form] in the reply to reviewer document itself. My [admittedly anecdotal] impression is that this is probably really helpful, as it can help them focus entirely on how well you have addressed their concerns, rather than having to dive into the entire paper again to figure out whether your revisions are any good. Indeed, I halfway suspect that this approach might help avoid one of the dreaded pitfalls of R&R's: reviewers finding "new things wrong" with the paper they didn't focus on in the initial draft. After all, if you place the onus on the reader to figure out your revisions, merely giving a broad overview of your revisions in your "reply to reviewers document", they essentially have to read the entire paper carefully a second time to figure out what in the world you are saying. On the other hand, if your reply document explains in detail precisely how you reply to their concerns, and then points to particular pages where you give the arguments you have just summarized, you are focusing nearly 100% of your reviewer's attention on their initial set of concerns, showing them that you have systematically addressed them in detail, and pointing to precisely in the paper where you do so–which, offhand, seems likely to prevent them from finding new things to be concerned about. In any case, my "reply to reviewer" documents tend to be pretty darn long–6 or 7 single spaced pages in many cases. Although this may sound like "overkill", and I certainly do think one should try to compress one's reply as much as possible, it has, once again, had very good results.

Now for one final issue. What if, as sometimes happens, reviewers ask for opposing revisions [e.g. one reviewer thinks you should lengthen the paper, another thinks you should shorten it; one reviewer thinks you should say P, another thinks you should say not-P]? I have to admit that I haven't had too much experience with this, and so do not perhaps have the best advice. However, one tentative piece of advice I thought worth floating out there is that, since in these cases one probably does have to choose between following one reviewer's piece of advice or the others, once one makes whatever decision one makes, one should probably explain to the other reviewer why one made that choice, referring the one reviewer to the other [viz. "I thank Reviewer 1 for suggesting I do P. However, Reviewer 2 suggested I do not-P. Although I believe there is merit to both suggestions, I have chosen P in the manuscript for the following reasons…"]. But again, I am not sure about this – so I am curious what others' experiences have been. Are there any particularly good strategies for handling opposing reviewer concerns? 

In any case, these are just some of my suggestions for handling R&R's. I will be curious to see whether other people have similar experiences, or whether some of you have found different strategies to be effective. In any case, I think R&R's are an important issue to discuss–as, once again, at least in my experience, some early-career people surprisingly little guidance [e.g. in their grad programs] on how to do them well. Anyway, what are your experiences/suggestions with R&R's?