The Philosophers' Cocoon

6 responses to “The Problem of Unconceived Objections & Peer Review”

1. T. Parent

   January 2, 2014

   I think it depends on what is meant by ‘serious objection.’ Here is an attempt (not necessarily successful) at what ‘serious objection’ means when the objection sufficiently warrants rejection.
   Say that a “serious objection” is one that does not have a relatively simple fix, and aims to quell a relatively obvious objection.
   Say that an objection has a “relatively simple fix” iff: the reviewer can anticipate a defensible reply on the author’s behalf, or has good reason to believe that some such reply exists. (Charity is important here.)
   Say that a “relatively obvious” objection is one that most experts on the topic (admittedly a fuzzy set) would identify as an objection.
   Still, the big problem is that there can be good and bad assessments about what has a relatively simple fix, and about what most experts would identify as an objection. (Of course, human fallibility is bound to affect the process to some extent.)
   A helpful rule of thumb, I think, is to ask whether raising the objection to the paper would, itself, make for a worthwhile paper. (This point isn’t original to me, but I forget where I heard it.) If a worthwhile paper would plausibly result, then normally the fix isn’t relatively simple and the objection is relatively non-obvious.
   I am unsure whether the foregoing is altogether adequate. But hopefully it helps, if only to assist others in formulating their own thoughts on the matter.

   Loading…

   Reply
2. Marcus Arvan

   January 2, 2014

   Hi Moti: to follow up on T. Parent’s point, I don’t think many referees would say they accept or obey (PR). I think they would say they adopt something like the following:
   (PR)* If there is a serious objection (or serious objections) against the main thesis of the paper under review THAT THE PAPER (1) REALLY SHOULD HAVE DEALT WITH AND (2) COULD HAVE DEALT WITH ADEQUATELY, then recommend rejection.
   This principle, obviously, contains two contentious phrases (concerning what the paper should and could accomplish), but — or so I say — these are precisely the kinds of contextualized judgments that reviewers and editors should make when evaluating an article.
   If there is some obvious objection that the paper should have dealt with, but did not, then there are two options: (1) recommend reject, or (2) recommend R&R. I take it that what (PR)* also recommends in this case is a judgment on the reviewer’s behalf about whether this article could actually deal with the objection sufficiently well (in which case one should recommend an R&R), or whether some other, more remedial article is necessary to head off that objection before this article (the one under review) can really get off the ground.
   And that, I take it, is a matter of judgment. I’ve had reviewers reject papers when I felt like I could deal with their objection in the paper — and in those cases I did feel like the reviewer made the wrong recommendation (they should have given me an R&R). In other cases, though, I’ve been able to see that I really need to write another paper to deal with that issue, and in those cases I’ve accepted the rejection as justified.
   Thoughts?

   Loading…

   Reply
3. Wesley Buckwalter

   January 3, 2014

   One rough heuristic I’ve heard people use to help further qualify PR is to ask whether the possible serious objection would be interesting and valuable as its own paper in and of itself. If the answer is yes, that’s a sign that the original work makes an important contribution and a mark towards publishing. If the answer is no, that’s a mark toward rejection, that seems to prevent your puzzle about publishing above.

   Loading…

   Reply
4. Moti Mizrahi

   January 3, 2014

   Thanks for the comments, everyone.
   I certainly hope all of you are right that most referees do not subscribe to PR. Judging by my experience, FWIW, I have seen too many referee reports that go like this:
   Author argues that p. Here’s a problem for p. Therefore, reject!
   I like Ted’s rule of thumb and Wesley’s heuristic: roughly, raise an objection only if addressing it would make for a worthwhile standalone paper. As Marcus points out, however, judgments about what would make for a worthwhile paper will vary significantly among professional philosophers.

   Loading…

   Reply
5. Marcus Arvan

   January 4, 2014

   Moti: interesting paper in Argumentation, by the way. I just read it, and pretty sympathetic to its overall thrust. I’ve always found Alston-style arguments to be quite underwhelming for more or less the reasons you give. And don’t even get me started in the “partners in crime”-style arguments that are in so much vogue in recent defenses of moral realism (viz. “alternatives to moral realism face similar objections, so moral realism is no worse off than them” — as though that is actually a good reason to take any of the views that face those problems seriously). Anyway, cool paper – thanks for bestowing it upon us! 😉

   Loading…

   Reply
6. Moti Mizrahi

   January 4, 2014

   Thanks very much for reading my paper, Marcus. I’m glad to hear you agree with its main thesis. I agree with you about what you call “partners-in-crime” arguments. (I like the label.) They are quite lame. I suspect it might have something to do with failing to distinguish between motivating a view and supporting a view. But I’m not sure.

   Loading…

   Reply