In our most recent "how can we help you?" thread, a reader asks:

How much weight should one give a mean review? One reviewer just tore into the paper and didn't have anything nice to say at all.

The second reviewer felt the opposite. While they weren't huge fans, they said they thought my thesis was ultimately convincing, and they said the reasoning was 'rigorous'. They provided quite a long review with lots of helpful feedback.

So, I'm not sure how much weight to give the mean review. I mean, the reviewer's comments are such that it's clear they considered my paper so bad that they were offended by it.

Unfortunately, I think this happens to virtually all of us. My spouse's dissertation advisor (in another academic area) is a seminal figure in their field, but told us that a couple of years ago they received a report saying that their paper must have been written by a grad student who has no idea how to do research. They laughed it off, saying that's just the way things go. Here's my story: a few years ago, I received two reports from a journal where one reviewer stated effusively that the paper stood to make a really important contribution to the area–with several detailed paragraphs explaining why–while the other reviewer wrote maybe five vague sentences that said that I don't have any idea what an argument is. Fortunately, the editor ignored them and rightly so. Other times, though, I've received mean reviews that convinced me that they were right: my paper was flawed and I needed to fix or abandon it.

So, how should one react to negative or mean reviews? In a similar context, noting just how many of his most influential papers were rejected, Jason Stanley once offered up some advice he'd heard from Robert Nozick: "He told me when he sent a paper out to a journal, he would first prepare a stack of envelopes, addressed to different journals. When the rejections came in, he would simply slip the paper into the next envelope." I think I heard something similar in grad school from another senior figure. But, while it's one way to go, I'm not sure it's the best answer. For, as noted above, at least in my experience, although it's sad that anonymized review can bring out the worst in people, sometimes you can learn important things from mean reviews. So, here's my answer to the OP: read the report and make up your own mind! If the mean review is more or less baseless, ignore it–but, if it makes good points, take them to heart, as difficult as it may be.

What does everyone else think?

Posted in ,

9 responses to “How to react to mean reviewer reports?”

  1. This basically echoes what Marcus said, but how much weight you should to give to mean reviews depends on how convincing their reasons are for trashing your work. If they’re highly unpleasant but don’t give good reasons for their criticisms, give it no weight.
    I really wish people wouldn’t be nasty in their reviews, but unfortunately anonymity promotes being a jerkwad, as internet comment sections continuously show us. You can be highly critical in the substance of your comments while still communicating in a respectful way.
    It’s easier to say this than to actually do it, but if somebody is highly unpleasant in their review, tell yourself that this says more about them than about you, because it’s possible to be critical without being nasty, but they chose to be nasty. And if the substance of their criticisms is accurate, well, that’s OK-it just means that your work has flaws. And everybody in this field is going to produce work that has flaws, sometimes fatal ones.

  2. cecil burrow

    I don’t think it is always a question of reviews ‘bringing out the worst in people’. When you are writing something to an editor, there is a virtue to a blunt and direct style. As an editor, I don’t want to have to read between lines in a reviewer’s report; I just want to know directly what they think. That’s of course not to deny that some people really are a-holes.
    I can’t fail to mention that Nozick’s ‘stack of envelopes’ advice is surely bad. Even the most dispiriting of referee reports often motivate me to revise the paper, even if just to ward off certain misunderstandings that I had previously thought went without saying. Those revisions it seems to me have always been improvements in the paper. The ‘stack of envelopes’ advice suggests that our judgments of our own work are indefeasible. That is surely false.

  3. Tim

    While this might not be the most popular advice, I recommend just ignoring mean and nasty reports. For me, dealing with a mean and nasty report can be emotionally upsetting for two reasons. First, in my experience, most mean reports involve a reviewer who didn’t read the paper carefully and is making lots of elementary mistakes. As a result, I get upset because now I’m reading something that’s not only insulting but unjustifiably insulting! Second, even if there are some OK points in the report, it’s still upsetting for them to be packaged in such a mean way.
    So I recommend saving the emotional turmoil, and skip reading the rest of a mean and nasty report. You might miss out on a few good criticisms (though that’s rare). But if there really are such obvious problems with the paper, another reviewer will point them out at another journal–hopefully with a little more grace.

  4. Journal editor

    If the negative review has nothing helpful to say, and gives no clear reasons for disliking the paper, I’d say that it should be given no weight. It sounds like the positive report has more constructive content to engage with.
    If the paper was rejected, it might just be best to ignore the report. If it got an R&R, then as far as possible, see whether there is anything constructive in the negative report that you can use to improve the paper. If you have a support network in philosophy, maybe show the report to a colleague or mentor in your network and see if they can make anything of it. They might be able to see something that you’re missing.

  5. grad student

    I have some very anecdotal evidence that might be reassuring. I’m towards the end of grad school, and I’ve had four main papers that have been sent multiple places (a few conferences each, and each has been sent to at least two journals.) So they’ve all received their fair share of referee reports. I’d say that two are work I’m really proud of, that my mentors have said are really good, that seem to me to be by far my best papers. The other two are fine — they each make a relatively quick, not-at-all groundbreaking point that hasn’t been made before, and I’ve put a lot less time into them. Both of the papers I’m proud of have received both glowing referee reports and super vicious, over the top-angry referee reports that read like personal insults. On the other hand, neither of the two medium papers has ever received a glowing referee report, but also never a report that was remotely mean. They’ve always been professional and to the point.
    Of course, it might just be random — I wonder if anyone else has had a similar experience. But it’s led me to think that maybe good papers, while more likely to get positive referee reports, are also more likely to get these bizarrely mean ones. I have both a charitable and a cynical theory as to why that might be. The charitable one is that referees are kinder in tone when they suspect that the paper was written by a grad student, and blunter when they suspect the paper was written by someone well-established. (“Why moderate my tone? It’s surely Professor XYZ, they can take it.”) So that would be why my more mediocre papers always have professional-sounding reports — it’s clear that they were written by someone at my career stage. The cynical one is that it’s easier to be kind in tone reviewing a paper that says your view is wrong when the paper is non-threatening. But if a paper says your view is wrong and gives very good reasons, and reads like it would be attractive to a wide swath of people… maybe that upsets you, and voila, you’re being extremely mean in the referee report. Maybe you’re even being more mean because you’re — consciously or subconsciously — worried the other referee report will be positive, and you want to balance it out.
    In any case, maybe take this as a point of encouragement: the frequency with which you get negative referee reports vs. positive probably is a sign about how strong your paper is. But the frequency with which you get really mean referee reports vs. polite ones, I think, is not — if anything, might be a sign that your paper is on to something.
    Last point. I just find these bizarrely mean referee reports totally baffling. You can be blunt or to the point without being mean — it’s honestly way easier. (“This paper should not be published in this journal. Here are the problems with it. [List of problems.] [End of report.]”) Some of the referee reports I get are just weird and unprofessional — I recently got one that tore me apart for something they misquoted me as saying. Another one tried to give a three line argument that any paper that argues for one of a class of views — a class of views that has been around for more than 100 years and is reasonably popular — is a nonstarter and should not be published, and then helpfully added that this would be clear to anyone who passed an intro logic class. A couple others read as if the reviewer is just angry that they’ve been asked to review a long paper. If this is you — and I get it, we’re all busy — just decline the invitation.

  6. Michel

    I set them aside. When I’m done grumbling to myself, I might revisit them if it seemed there might be anything useful in there. Otherwise, I just ignore them until I get an R&R. Sometimes I even ignore them after the R&R.
    In other words, you can file vicious comments in the garbage bin. (But try to remember not to make them yourself when refereeing!)

  7. Kapto

    Similar to Marcus’s advice: Don’t take it too hard. Nearly every major ground-breaking paper I know and love from the past century has been thought awful, or hopeless, by philosophers of comparable stature to the author (this is not a slogan — I can cite specific examples for every one that I have in mind). Had that peer been the reviewer, they’d have received the mean review you’re describing.
    But, do not squander the one worthy product these reviews provide: a chance to improve the paper, or at least its presentation, so as to make the next reader just a little less likely to get you wrong. There’s usually something, specific, that ticked off the reviewer, even unjustifiably and even if they were a-holes about how they put it. If it’s easy to fix or mitigate, why not?

  8. ChastenedAuthor

    OP here. Thanks to everyone for their replies. After a few days grumbling, I was able to look at the report a bit more clear-eyed.
    I was able to see the fundamental misunderstanding the reviewer had, and so that allowed me to make sure one of the distinctions in the paper is absolutely clear.
    The rest of the review is just too vague to do anything with. E.g. “Argument y on pg x is truly terrible.” Just not much I can do with that criticism.
    That said, I do think addressing what I could from the reviewer’s comments improved the paper.

  9. everwhat

    Have you seen that scene in Billy Madison…??

Leave a Reply

Discover more from The Philosophers' Cocoon

Subscribe now to keep reading and get access to the full archive.

Continue reading