In our March "how can we help you?" thread, a reader asks:

Has anyone had an experience where a Reviewer gave what seemed to be an inappropriate amount of comments? I know that, as an author, my only choice is basically to address them all, but I'm just curious if there are norms around this. I think the reviewer in question may be from an adjacent field, but their report contained at least 15 independent comments, many of which contained subcomments and extensive, detailed digressions into nearby literatures.

Has this happened to others? Is there a norm as to limiting comments, as many as one may have, to (I dunno) 5-8? Especially when they are very, very complex and wide-reaching?

Hmm, I don't know if this has happened to me, or whether there are norms here (though I suspect not). Even if there were a norm to only give 5-8 comments, what would prevent a referee from simply making each comment arbitrarily long, so as to fit in all of their feedback. In any case, I'm not sure how pervasive this issue is, or what might be done about it.

What do you all think?

Posted in

11 responses to “Too many reviewer comments?”

  1. Murali

    There are I think 2 different questions here.
    1. Have you gotten a large no of comments on an R&R, well in excess of 8?
    Yes. In fact quite often.
    2. Is the number/length of these comments in some way undesirable such as to qualify as too many or too long?
    Not really. More extensive comments make it clearer what the reviewer’s concerns are. That reduces the chance that it will be rejected ok resubmission.

  2. anon

    You absolutely have a choice in rejecting excessive feedback, as long as you’re able to make a strong case for why you’re doing so. I’ve done so.
    One sort of starting point that would be something like along the lines of: “Comment 8 makes an interesting suggestion for further research. But while it’s related to a question raised by the main argument of the paper, there’s enough work to be done in the paper in establishing that the argument is well-supported and that there are easy responses available to the objections raised in comments 1-7.”
    Mentioning word count limits can help, as can talking about how the additional comments are very, very useful, and that you just tragically lack space in the paper to give them the deep consideration they deserve. You can also try to get away with adding a footnote pointing to additional issues rather than entirely ignoring them.
    How much all this will succeed depends on some details unavailable to us (the specifics of your paper and the comments, what the editor thinks of this report, etc.).

  3. anon again

    I guess one might think of my suggested approach as still addressing a comment, insofar as the reply letter mentions the existence of the comment. But the idea is that you can decline to engage in as much detail as the referee did – much less – and also to revise the paper – in light of these comments.

  4. Looking back at reviews that I have received (in phil sci), 15 comments seems to be slightly above average, but I would not say that it is out of the norm. My most extreme was 74 in-line comments from a reviewer on the PDF of my manuscript, which I of course had to transcribe for the response to reviewers (on an R&R). It was a lot of work to reply to all of them (and the other two reviewers’ comments), but I take it as a normal part of the process. I would much rather get all the comments now rather than later on in the process!
    I think the norm for editors is to respect and hand over all comments from the reviewers, even if it seems to be extreme. I think the norm for reviewers is to justify their recommendation, and secondarily to help to improve the paper. Thus, you’ll get some reviewers who give one major objection and seven minor comments, and you’ll get some who recommend rejection and give several major objections without any minor comments.

  5. Michel

    Just to pick up on H’s comment above: you can condense the comments in your response. So, for example, I usually identify the three or so main concerns that referees bring up, then explain how I addressed them. I’ll follow up with shorter responses to a few less significant comments that nevertheless led to important changes, and leave the rest to be detected by referring to the red text in my revised manuscript (or whatever distinguishing feature you prefer).
    In other words, you can basically generate an abstract of the referee reports, and use that to guide how you respond.

  6. grad student

    Happens all the time to me. In fact, on various occasions, a single reviewer’s comments was longer than my actual manuscript (e.g. Reviewer’s response was 8k+ words to my 7k+ words manuscript). In such cases, it seems impossible to address the reviewer’s objections while still keeping within the acceptable word limit of the journal (if any).

  7. frequent submitter

    Isn’t it a good thing that the referee reports are long, unless they are unhelpful, irrelevant or poorly written? In my experience, the longer reports I have received were also the highest quality ones. I once heard in a funeral that the dead used to leave longer commments on students’ papers than the papers themselves. And this was meant as a high praise. Other commentators have provided practical advice to handle long comments which I agree with. But the lengthiness of a report would be the last thing I would complain about.

  8. Circe

    I see what frequent submitter is saying, but whenever I give an extensive number of comments I always think: well, if there are so many things to include, correct, etc. then should this be a rejection? My experience is that the more ready for publication I (and referees of my paper) think a paper is, the fewer comments I make (and the fewers I receive on my own papers). Isn’t that the natural way of things?
    A related issue is not so much an excessive number of points made by individual referees, but an excessive number of reports. I find even having three reports to deal with can be overloading. I was once told by an editor that a report I’d written was one of four sent to the author–I felt bad for them…

  9. Brad

    Circe
    David Hull describes a case in a biology journal where 17 referee reports were solicited for one manuscript! I also think that if the referee has too many comments then they should just recommend that the paper be rejected. Michel draws attention to something important. In responding to referees, a good strategy is to focus on the THREE key criticisms, and number them 1-3. Similarly, a good strategy for referees is to focus on the key concerns, and number them, 1-3.

  10. Circe

    Hi Brad. Ouch–I hope that it not normal!
    Personally, I would never do as Michel says. If it works for them, ok. But in every ‘reply to referee’ document I have seen and written (a good few…) the author goes point by point and either pushing back or saying what the changed in response. AFAIK, this is standard advice in the profession.

  11. Write Short Responses

    I and a coauthor of mine once received fifteen single-spaced pages of comments on a 6k-word draft. But though it took a lot of work to address them, they significantly improved the manuscript. And when we sent our response to the reviewer, we didn’t worry too much about explaining our changes in meticulous detail. (We did explain them, but we did so in two pages.) One way to help reviewers see that their comments have been addressed without transcribing them all is to put the altered parts of the revised manuscript in bold. Then, when they’re reading through it, they can see for themselves whether you’ve addressed their concerns.

Leave a Reply

Discover more from The Philosophers' Cocoon

Subscribe now to keep reading and get access to the full archive.

Continue reading