The Philosophers' Cocoon

A safe and supportive forum for philosophers.

Owned & Moderated by Marcus Arvan (University of Tampa)

Contact: marvan@ut.edu

Blog mission & moderation policy

IMPORTANT NOTE on how to ensure anonymity in commenting
Job-Market Reporting Thread (2026-27)
Job-Market Discussion Thread (2026-27)

recent posts

about

Dealing with incompetent referees?

In our newest "how can we help you?" thread, a reader asks:

Just got a bizarre referee report. I argued that a popular view — that phi-ing requires N — is false, because S is sufficient for phi-ing, and S doesn't entail N.

The referee wrote: "This is a straightforward necessity/sufficiency blunder: the claim is that [S] is sufficient for [phi-ing]. How is this in conflict with the [view that N is necessary for phi-ing]? It is just a sufficient condition." This was their main reason for rejection.

I am just astounded at this level of incompetence. I don't mind referees raising bad objections, but this is just basic logic stuff.

More practically, what do you do with something like this? Usually after a rejection I add clarification to the paper. But clarifying this particular point feels beyond silly.

I've heard anecdotally of at least one person complaining to an editor about a referee's silly mistake–but I've never done this and am not sure it's advisable. Also, while it may seem (or be!) silly, for my part I've always found it helpful to "referee-proof" papers by making every argumentative step as explicit as I can. Referees are human, they may read a paper more quickly than they should, and they may get tripped up by simple distinctions that shouldn't. Oftentimes, preventing silly mistakes by referees can just take a sentence to spell a relevant inference out.

What do readers think?

Posted in ,

18 responses to “Dealing with incompetent referees?”

  1. r1er

    I’ve successfully written to editors about stuff like this. I recommend doing the same. Anonymous refereeing ought not to mean that anything goes.

    Reply
  2. Investor

    The issue is that we have outsourced the labor of determining good philosophy to unpaid, anonymous referees while massively overproducing PhDs. There are two ways to fix the problem. Stop producing so many PhDs or start paying philosophers to referee and hold them accountable for bad work.
    I will not use the peer review system until it is fixed.

    Reply
  3. Hermias

    You must be new here! It sucks, especially if you’ve been waiting a while, but there are plenty more fish in the sea. Even if you are very obviously in the right, it just comes across as petulant/ego-wounded to go complaining about it. As Schope said, always bear in mind that the transition of words from the page to the reader’s mind is a hazardous voyage.

    Reply
  4. Daniel Weltman

    I submit the paper to another journal and hope I get luckier next time.

    Reply
  5. Rejected

    I’ve been lucky over the years to have received many excellent referee reports that have resulted in genuine improvements to my papers, but within the past year, I’ve received a few truly baffling reports that have resulted in rejection, including from highly-ranked specialist journals. These reports have included basic mistakes in reasoning akin to OP’s report and have otherwise displayed a lack of understanding of the topic of the paper and the literature surrounding it. In both cases where I have received them, they have been paired with a positive report recommending acceptance of the paper, so it has been extra frustrating knowing that the flawed report likely supplied the basis for the rejection.
    My response in both cases has simply been to move on, but it’s surprising that there is not a bit more editorial oversight in some of these cases. Is this just a function of editors being totally inundated with submissions and unable to scrutinize reports?

    Reply
  6. Circe

    I had one of these reports today. Just laugh and, like Rejected, move on.

    Reply
  7. Bill V.

    We think of refereeing as quality control, but we neither train people to do refereeing nor do we do quality control on those quality controls. We can do better.
    Refereeing is not a skill we teach in grad school or elsewhere–it is kind of just assumed that once you become expert in your field you will be able to do refereeing in that field. But giving good feedback (whether on student papers, in referee reports, on grant committees, on annual evaluations of fellow faculty, etc.) is neither natural nor easy for most people. Being expert in one thing does not mean we are expert in a related thing. We have had few to no models of good feedback before we start, so there’s nothing to imitate, even. I would love it if some editors got together and created an online training (even just a few YouTube videos) about how to do refereeing well.
    Giving feedback to referees would also help improve refereeing. We do it informally (in forums like this one, or when editors simply stop asking a particular person) but so indirectly that it is essentially useless.
    Feedback to referees could take several forms, including:
    (1) Editors saying, “Next time, (don’t/) do this.” (I have noticed that this seems to have worked in the case journals that have blanket “our referee reports will be civil” policies.)
    (2) Feedback channels from authors back to referees (“You made this mistake x”). Having just received a survey from my health insurance company about my anonymous rating of a recent doctor visit, this is fairly standard across our culture and shouldn’t be difficult to implement.
    (3) Sharing referee reports between referees or with an independent third person who could give feedback to the referees. One of the factors I suspect contributes to bad referee reports is the feeling that no one is watching, so it doesn’t matter if the report is uncareful, uncharitable, or biased. Seeing the other referee report (after the fact, of course) could allow someone to notice that they missed or misinterpreted something.
    This would all be more work, but this might be a case where the extra work would quickly pay off and greatly improve the system overall.

    Reply
  8. editor

    FWIW as an editor it is annoying when people complain about referee reports that they view as incompetent or misreading them when there are multiple available reasonable perspectives on whether that is true (e.g…. it is often the case that the person’s paper is far less clear/careful than they insist it is, I have read the paper, I agree with the referee report or at least think that the referee’s misreading is caused by issues with the paper and is not wholly on them). However if someone raised this sort of totally obvious, crystal clear incompetent reading/misunderstanding of basic informal logic concepts with me I would apologize and solicit a new report (actually, I’d like to think that given that I read each paper carefully before it goes out for review and then at least re-skim when I’m reading reports to decide what to do, that I would have noticed this myself and not used the report, but if someone pointed it out I would act). Your mileage may vary but I thought I’d mention this since it goes against what some of the comments say here.

    Reply
  9. Investor

    The problem is simple. There are too many submissions for free labor to the profession to handle the load. The minute I got an R&R at Synthese as a grad student I was asked to review for Synthese. That’s the system we have. 1 time in 10 I get a good referee report. Most of the time I get low quality reports. I get papers rejected either without explanation or due to obvious–I MEAN OBVIOUS–failures to read the text. Basically the impression I get is some really irritable person is being harassed for a report, so they just reject and make something up based on a quick glance. That’s the norm now.

    Reply
  10. R&R

    Not the OP situation, but just wanted to offer an experience with getting that kind of report later on in the review process. I had a referee request new revisions after one round of R&R, and they got upset my paper defended highly implausible claim about phenomenon x. The paper in fact defended that claim about the ‘good’ cases of x, and part of the point of the paper was to propose a disjunctivist view of x (already from the abstract), giving a different account of the ‘bad’ cases.
    I did make some of the other small requested changes, but started the response to reviewers – which I envisioned as addressed to the editor in the first instance – by explaining what the main complaint was and that I did not, in fact, defend the view. Paper accepted a few days later.

    Reply
  11. R&R

    Not the OP situation, but just wanted to offer an experience with getting that kind of report later on in the review process. I had a referee request new revisions after one round of R&R, and they got upset my paper defended highly implausible claim about phenomenon x. The paper in fact defended that claim about the ‘good’ cases of x, and part of the point of the paper was to propose a disjunctivist view of x (already from the abstract), giving a different account of the ‘bad’ cases.
    I did make some of the other small requested changes, but started the response to reviewers – which I envisioned as addressed to the editor in the first instance – by explaining what the main complaint was and that I did not, in fact, defend the view. Paper accepted a few days later.

    Reply
  12. R&R

    Not the OP situation, but just wanted to offer an experience with getting that kind of report later on in the review process. I had a referee request new revisions after one round of R&R, and they got upset my paper defended highly implausible claim about phenomenon x. The paper in fact defended that claim about the ‘good’ cases of x, and part of the point of the paper was to propose a disjunctivist view of x (already from the abstract), giving a different account of the ‘bad’ cases.
    I did make some of the other small requested changes, but started the response to reviewers – which I envisioned as addressed to the editor in the first instance – by explaining what the main complaint was and that I did not, in fact, defend the view. Paper accepted a few days later.

    Reply
  13. anon

    If the reviewer truly is getting basic logic wrong,and this error is the only sufficient reason the reviewer rejected the paper, I would appeal with as polite and short a message as you can that points out this error. But if the reviewer has other sufficient reasons for rejecting the paper that are not similarly egregious, you should move on.

    Reply
  14. ChastenedAuthor

    I understand this! I’ve gotten referee reports recently where the referee is arguing with the paper rather than evaluating the paper.
    I always want to write back, “Yeah, that would be a great discussion to have in the literature.”
    Another referee actually made an argument that proved my point and then included the line, “Author may believe this proves her point, but I disagree.”
    And one of these was a Leiter T-15 journal.
    My advice is, even if it feels silly, answer the objection explicitly. I usually like to put in something like, “One reviewer suggests” and then give the argument and the explicit counter-argument.
    Or, sometimes I’ll give a “thanks to an anonymous reviewer for pointing this out.”
    My hope is that’s a signal to other reviewers that I’m making this point, because other reviewers have already had to resort to pedantry to reject my paper.
    I don’t know if it gets interpreted that way. Maybe others can comment on if this is a good idea.

    Reply
  15. Michel

    I recently refereed a paper that went a few rounds. My initial verdict was R&R. When I got the paper back, there was no changelog, no reply to the referees, and no in-text indication of what had changed. As far as I could see, some typos had been fixed and some wording slightly altered, but that was all. Neither mine nor the other referees’ comments appeared to have been dealt with. So I sent it back for another round.
    The author’s reply made it clear they thought I was a moron (and they were also under the misapprehension that they had provided a reply to the referees–I checked twice with the editor, and they hadn’t).
    Don’t be that guy.

    Reply
  16. two hands

    I agree with the commentators who recommend writing to the editor about it.
    it is usually a bad idea, but not in every case. Editor has the responsibility to give a verdict based on a competent report.
    On the other hand, and this is unrelated to the above, I think one can try to give maximal charitability to the referee. maybe they are a grossly incompetent referee, but maybe they just are not as impressed by the claim that “S does not require N”, which lead to that complaint. Etc. In any case, try not to be worked up by this. Best luck

    Reply
  17. cecil burrow

    Regarding ChastenedAuthor’s comment, there are many basic questions that should not be relegated to the subsequent literature. If there is a basic question that many in the field would ask, that question needs to be decisively addressed in the original paper. I have definitely rejected many papers on those grounds, and see no reason to change that practice.

    Reply
  18. Mike Titelbaum

    I also think it’s fine to write to the editor in this case. Just be very polite about it. You could say something like, “I have never done this before, but I was very confused by this referee’s report, because the logic of my argument is…” and so forth. Could go well for you, could go badly—I’ve seen it both ways!

    Reply

Leave a Reply

Discover more from The Philosophers' Cocoon

Subscribe now to keep reading and get access to the full archive.

Continue reading