I have argued many times (see e.g. here and here) that our current anonymized peer-review system is outdated and problematic, and that we should reform our system in the direction of the public review process practiced in math and physics. Although there are some good things about our peer-review process (I, for one, have received some vital feedback from reviewers), I believe there are many reasons to move away from our current system. In this post, I suggest some additional reasons to the mix: psychological stress. I think our current peer-review model is unnecessarily stressful–especially for authors, but also for reviewers and journal editors. Before I explain how–and how I think the math/physics model would be less stressful–allow me to briefly recap some of the basic reasons why I think our model is outdated.
Anonymized peer-review made plenty of sense…in the 19th century. Back then–before air-travel, the internet, and so on–anonymous review was a feasible ideal. If you were a reviewer who received a manuscript in the mail, it would in most cases be difficult if not impossible to know who wrote it. In contrast, open public review–of the sort practiced in math and physics today–was not a feasible ideal, as there was no feasible way to openly share draft research for public vetting. Things are exactly the opposite today. In the digital age of the 21st century, ensuring anonymized review is prohibitively difficult: people routinely post drafts online, share them with other specialists by email, reviewers appear to "Google review" papers, and so on. Not only that: our current "anonymized" review process is increasingly unwieldy for everyone involved. The pace of research and journal submissions appears to have increased rapidly since the advent of the internet–in large part, I suspect, because of how digital media plausibly speed up all elements of the research process, from researching journal articles, to receiving peer-feedback, to submitting to journals and so on. Consequently, journals appear to be overwhelmed with increasing numbers of submissions, leading to difficulties soliciting reviewers, long wait-times for authors, and so on.
Our current model of "anonymized review" is especially problematic, I think, given that a very different model has–in my view–been demonstrated to work better. In math and physics, everyone uploads draft papers to the ArXiv, people read, comment on, and replies to papers on the ArXiv–and, after all of this public vetting, people submit their papers to journals. Although this alternative model essentially casts "anonymous" peer-review to the flames, it seems to work quite well: whether a paper is any good comes out in public discussion, as many people (as opposed to two randomly selected reviewers) publicly debate the merits of draft papers before they are even sent to journals, making arguments to authors and to each other on what is good about the paper, what is bad, and so on.
This non-anonymized/public peer-review process not only corrects for the "luck" associated with anonymized peer-review, where your paper's fate can depend on two randomly selected reviewers (instead making your paper available for many peers to publicly evaluate). It also increases how quickly research is disseminated, as people post their research online immediately rather than waiting months (or years!) for their paper to make their way through anonymized review. Not only that: it also improves scholarship, as once a paper is posted online, others can catch failures of scholarship (such as failure to cite sources adequately) before the paper is reviewed at a journal. Not only that: in improving scholarship and providing authors with public feedback prior to submission, the math and physics model makes things easier for referees and editors. Because the quality (or lack thereof) of a paper typically comes out publicly before submission, authors in the math and physics model may be less likely to clog up the journal review system with underdeveloped work. All of these are, in my view, very good things–things that should motivate us to move towards the math/physics model.
But these aren't the only things I think favor that model. I've been increasingly struck over the years–and especially lately–with how psychologically stressful our "anonymized" publishing model is on authors. It is well-known that grad school and academia more broadly are hazardous to mental health. While there may be many reasons why–and I cannot pretend to know that our publishing process plays a significant role–my experience, both first-personally and second/third-personally, is that some of the greatest stresses in academia are related to our current peer-review model. Indeed, our current "anonymized" peer-review model seems systematically designed to provoke a variety of different kinds of stressors.
First, our current system of "anonymized" peer-review plausibly can plausibly breed worry. First, it is very common in my experience for authors (not just in philosophy, but also in other fields that follow our model) to worry about being "scooped", or having someone else publish your research idea before you can (which can make months or years of your own work unpublishable – this has happened to me on several occasions myself!). This can be incredibly stressful. There you are, having drafted an article giving argument X ten months ago (or maybe two years ago)…and it sits under "anonymized" review at a journal while you see other people post drafts or forthcoming papers defending X. Second, it is–as we see here–all too common and natural to worry about whether anonymized peer-review is respected by reviewers. There you are, submitting your paper to a high-ranked journal, and in the days or weeks after you submit it, people mysteriously visit your website having searched for your paper title. This can be stressful too, making you worry that anonymized review was not respected.
Second, our current system of "anonymized" peer-review plausibly generates a variety of frustrations–with everything from the slow speed of the peer-review process, to incompetent or meanspirited reviews by referees, to the fact that, again, one must "hide" one's work for months or years when, as a researcher, one wishes one could share it in a more timely fashion. It can terribly frustrating as an author to have papers one is excited about–and which one would like to share publicly for discussion and feedback–and yet feel coerced, by current norms, to hide that work from public view for months or years as the philosophical conversation progresses. It can be especially frustrating to have to wait for the process to work when it involves waiting 6-9 months for one's paper to be reviewed, only to receive a 4-5 sentence "justification" for rejection by a reviewer (something which I know happens unfortunately often).
Finally, our current system plausibly generates a number of predictable disappointments. First, as an author, one can watch the same paper be rejected by a dozen journals before it is accepted somewhere–a series of disappointments over months or years capped off, if one is lucky, by a single success. Second, because the peer-review process is so private/anonymized, one can be often disappointed by failures of scholarship by other authors–for instance, authors who publish similar arguments to one's own without citing or engaging with them.
And these are just the stresses authors face! Recent online discussions at other blogs suggest journal editors are overwhelmed by submissions, reviewers sick of receiving underdeveloped work, and so on–all when there is a different peer-review model that substantially corrects for all of these problems: one that enables authors to post their work publicly immediately, give and receive feedback on publicly posted manuscripts, have their work reviewed publicly not just by one or two reviewers, have their work cited and engaged with prior to journal publication (which is standard in math and physics, as papers posted to the ArXiv are considered "published" even prior to submission to journals).
In summation, our current peer-review model is problematic. It's unnecessarily slow, stressful for everyone involved, and doesn't appear to preserve anonymity/control against bias anyway. There's a better publishing model, one that would be less stressful, improve scholarship, and improve the dissemination of research. What are we waiting for?
Leave a Reply