Gatekeeping has a bad rap. Still, though we loathe it, academics do it all the time. Our institutions depend on the well functioning of various gatekeeping mechanisms–mechanisms that help discern what should be funded or published, and what doesn't.
For instance, I am associate editor for journals Ergo and for Journal of Analytic Theology. In addition, I review for journals and for academic book presses (e.g., Cambridge, Oxford, MIT, Routledge, and Mohrbeck Sie, where I am in the editorial board). I have been, and am currently, a reviewer for several grant agencies including the National Endowment for the Humanities and the British Academy. I help decision small grants and prizes for the APA Public Philosophy committee. Also, I regularly review the research of academics who come up for tenure and promotion. Until last year, I was on a committee for grad admissions at SLU, and presently I am on two search committees for jobs searches.
So, a fair amount of my time is spent evaluating other people's work and trying to discern its quality. But how do we go about being gatekeepers? I think, inevitably, we end up being perhaps a little more conservative than we like, but we do have the power to shape the discipline. How the field advances depends on the quiet, often uncompensated labor of referees, editors, and other evaluators.
When I got my first tenure and promotion file to review, I wondered how to adjudicate these and how to communicate to the committee if (fortunately not the case for my first one) if I felt the file was a little thin, or if it plainly did not meet the criteria set out by the institution. I spoke about it at a conference over dinner to someone who has been an evaluator for many tenure and committee reviews.
"Some people," this professor said over wine and quiche, "Feel somewhat squeamish about giving a negative evaluation. They'll resort to saying something bland or they try to talk up what they believe is insufficient."
"So," I asked "How about you?"
"Well I am not squeamish," he replied "I am part of academia, and tenure and promotion is part of that institution. I value tenure and the academic freedom it affords, but tenure depends on the procedure of the tenure and promotion process. While in most cases, the body of research presents a clear case in favor of tenure, it sometimes doesn't, and the committee needs to know that. I need to be honest and diligent in my assessments."
I often think back of this reply and how it can be generalized. When people violate this implicit norm/social contract, problems might arise.
For instance, for one of the journals I am associate editor at, I received a referee report which read (in its entirety) something like this:
"This is an important, highly original paper. I have no further suggestions. Accept as-is".
Based on my own (non-specialist) reading I had some doubts and consulted a senior editor who said "Oh [name] always writes reports like this. You cannot trust this referee. Always just one line that is very positive without detail." Sure enough, a second reviewer whose report came in later identified many problems in the paper and we rejected it on that basis. I don't know what motivates that particular referee. Maybe he believes all research deserves a place for publication. But, working within the procedures and institutions, he delays the process and makes the decision more difficult. Hence, there is some expectation that you follow the disciplinary norms of evaluation.
However, there is a fine line between disciplinary norms of evaluation and the perpetuation of excluding some ideas as "less philosophical" (e.g., feminist philosophy, less commonly taught philosophical traditions), and even questioning whether something is philosophy at all. For this reason, I do think within the bounds of what's possible we should let our own preferences and judgments, subjective though they are, play a role in our evaluations. We're not beholden to how other people would do so.
To take but one example, I spoke to a philosopher a while ago who admitted a paper she refereed was so boring it was hard to reach the end. She found the conclusion–when finally reached–trivial and uninteresting. Boring and derivative philosophy has its place in advancing the discipline (as J Dmitri Gallow recently argued here), but surely if you find a paper so soporific you can hardly reach the end, that's a ground to reject it?
"No, I recommended conditional acceptance," she said, to my great surprise. "After all, it's exactly the sort of paper you see in top philosophy journals all the time. I think they're boring and I hardly read "general" philosophy journals anymore. But the discipline seems to think this stuff is great. It was also referee-proofed through and through, so nothing really objectionable. I didn't learn anything of interest, I didn't gain new insight, but well, at least it didn't seem obviously wrong."
I am curious what readers think, but my approach isn't like this. When I, as a reviewer, think a paper is trivial, even referee-proofed to the max, I'll recommend rejection. I am a bit more forgiving of papers that are bold and interesting, or in underrepresented traditions, though obviously they also need to meet standards, it's just that referee-proofing is not a feature that is prominent in my decision making. If everyone just replicates existing practices, then general philosophy journals will continue to churn out work I'm mostly uninterested in–since what has appeared has changed, and is changing, over time it's possible to tweak a bit at the margins.
How much you can tweak depends on the position you're in. As a referee you can perhaps help usher a paper through the process that otherwise would never have seen the light of day that you deem very promising and novel. As an editor in chief you can clearly signal that you want change and that you're open to new approaches, as Alexus McLeod recently did for the Philosophical Forum where he explicitly calls for more papers in less commonly taught traditions (one reason for why Johan and I decided to send our paper that combines genealogical approaches in continental philosophy with Indigenous philosophy to that journal). When an editor explicitly discusses policy and openness to newer and alternative approaches, one feels a little more confident that editor would seek out and do their best to find competent reviewers, not simply desk reject a paper in an underrepresented philosophical tradition.
We may question whether our institutions still work. For example, Marcus Arvan, Liam Bright and Remco Heessen have examined whether pooled/crowdsourced peer review might be better than our traditional system. And perhaps respecting the institution of tenure, given the decline in tenure track positions especially with respect to candidates on the job market, may feel like rearranging deck chairs on the Titanic. But my sense is that once you step into the gatekeeping mechanisms (as evaluator for grant agencies, referee, etc) you cannot suddenly change the system. We need to have collective efforts across the discipline to achieve this.
It's true that individual initiative can shift the field a bit. For example, a lot of my time evaluating other people's work and doing other service is tied up with open access initiatives: Ergo, Journal of Analytic Theology, and Faith and Philosophy (I am managing editor and oversee the publication process, and examine reprint permissions etc). As a result of this, de facto, I have a lot less time to devote to for-profit publishers such as Springer and Wiley. I still referee for these journals, and still submit to them. But because I've prioritized open access initiatives in service, there is inevitably less time to spend on these. As a referee and editor I encourage work I think is valuable and that I believe is underrepresented. This, for instance, led to two prize competitions I got grant funding for, in feminist and less commonly taught philosophy of religion.
Of course, not everyone might agree with me that open access (not for profit open access that requires high fees) and less commonly taught philosophy need to be promoted. But, it's inevitable that if you are in gatekeeping positions you have to make some calls about what you believe is valuable, and hopefully a diversity of perspectives leads to good outcomes. This can lead to slow shifts over time, and need to be supplemented by discipline-wide conversations if we want to make bigger changes.
Leave a Reply