In our most recent "how can we help you?" thread, a reader writes:
I'm at the point in my career when I'm just starting to receive fairly frequent requests to referee journal articles, and I'm finding myself really unsure of how many of these I should be doing. Would cocoon readers be willing to share their own approaches to this? How much refereeing do people do? And do people have good rules of thumb for deciding when to say yes and no to referee requests? (E.g., a certain number of reports per month, or a certain number of reports per reports one has received oneself, etc?) This is one of those mysterious parts of the profession where I just have no idea if I'm pulling my weight, doing way more than I should be doing, or roughly getting it right. Thank you, everyone!
Excellent questions! One reader submitted the following response:
At one point, I was refereeing 20 papers a year, and working as an editor for PLOSOne, handling about 6 papers a year (which meant getting referees and making a final decision on papers). Now I am a co-editor on a journal, and I no longer do PLOSOne. So I referee only about 12 papers a year, and only in a select set of journals (the best in my sub-field or very highly regarded general journals). I also review a few book proposals and manuscripts a years for various presses. Without knowing your circumstances, I think if you are doing 12 a year, then that is fine.
And another pointed out that we discussed this issue at length in another post a while back:
Your question was answered pretty extensively in this previous post: https://philosopherscocoon.typepad.com/blog/2020/09/how-often-should-one-review-for-journals.html I'm personally sympathetic to the view (expressed by several people in that discussion) that one should do at least as much refereeing work as one creates. So for everything one submits (including, of course, resubmissions of revised papers, and submitting a paper to a new journal after a rejection), one should referee two or more papers (since there will in most cases be two referees for each of your submissions).
I pretty much stick by to what I wrote in that earlier post. I try to referee at least as many papers as I submit, and think this is a good rule of thumb. But I'll add one further thing to the worries that I raised about refereeing 'too much.' The main worry that I raised in my earlier post is that if you're doing too much reviewing, that runs a particular moral hazard: namely, a relatively small number of referees such as yourself having too much power over what gets accepted or rejected at journals. I still believe that: we shouldn't want a small number of gatekeepers doing most of the refereeing. However, now that I review a lot more than I used to (I think I refereed about 20 papers last year), I have another worry: that if you review too much, you may do a worse job as a reviewer. As an author, I've sometimes received referee reports that came across to me as perfunctory and uncharitable. Other times, I received excellent reports: ones that, even if they recommend rejecting the paper, give detailed justifications as to why. As a reviewer, I think it's my duty to give the latter kinds of reports: ones that are charitable and fairly detailed, regardless of the editorial recommendation that I provide. But here's the thing: my sense is that the larger the number of referee assignments one takes on, the greater the temptation may be to do a worse job–to read papers more quickly, write reports more quickly and so on…making it more likely one may become a bad referee!
So, this is my suggestion: try to review at least as many papers as you send out, and review papers that you're well qualified to evaluate–but don't take on too many. If you do, you may end up doing authors, editors, and the profession more broadly a disservice. But these are just my thoughts. What are yours?
Leave a Reply