The Philosophers' Cocoon

A number of philosophers in my social media feeds have raised questions recently about anonymized journal reviewing: specifically, about cases where they suspect the identify of the paper's author, given the author's manner of citations–specifically, the manner in which it appears that the author may be citing themselves throughout their manuscript. The standard case of this is something like the following:

One gets asked to review a paper. The paper repeatedly cites a not-very-well-known author X, placing that author's previous work at the center of the paper under submission, developing X's previous work in new directions. Since X is not very well-known, it seems likely to the reviewer/potential-reviewer that X is the author of the current paper. 

A number of different people in my social media feeds have recently raised cases like this, asking social media friends what they should do. Should they decline to review? Should they alert the editor that anonymized review has potentially been compromised, and let the editor decide? Etc. Although there have been disagreements in my feeds about what the right answer to these questions are, the most popular answers seem to be that reviewers in this kind of situation should either go ahead with the review if they are not certain who the author is, or alternatively, let the journal editor[s] know of the situation and let the editor decide. These both seem like reasonable answers to me, especially the latter–as reviewers are tasked with reviewing manuscripts by the journal editors [who therefore, it seems to me, should be consulted if there is any doubt].

However, I'd like to step back from these questions to some broader questions about how anonymized review can be compromised, and what ought to be done to "protect" anonymized review. As readers will see, I think there are some very serious puzzles to grapple with here that seem to me inadequately addressed at present–and indeed, I am not sure that anonymized review can be protected in a way that is either [A] effective, and/or [B] fair to authors. Let me explain.

I think perhaps the best way to introduce the host of puzzles I think arise is to reflect a bit more on the case I began this post with: the case where reviewers suspect they know an author's identity due to patterns of "apparent self-citation." To begin, it seems to me clearly legitimate for authors to write and attempt to publish papers further developing work they have previously published. If author X has published work defending an original argument or theory, they have convinced peer-reviewers in the past that the argument or theory in question is worth presenting to specialists for consideration. Further, even if that argument or theory has not had much "uptake" in the literature–for instance, if it has been largely ignored or perhaps cited only a few times in the literature–that does not seem to me sufficient reason to think the author should not be able to publish work further developing that argument or theory, as a large part of why they might do so might be to further demonstrate and develop its importance [in part due to demonstrating the argument/theory's fecundity, but also perhaps due to publishing the latter work in more highly-ranked journals than their earlier work]. To take a concrete case, even if Einstein's special theory of relativity didn't have much uptake in months or years after its 1905 publication in Annalen der Physik, it seems to me absurd to suppose that Einstein shouldn't be able to, say, submit his work defending the General Theory unless and until the former theory had serious uptake. The uptake, or lack thereof, of a person's work in a given area can have all kinds of explanations, including bias against the author or their ideas [as we all know, many controversial ideas in scientific and intellectual history were profoundly unpopular in the immediate aftermath of their publication–including, for example, Hume's Enquiry, which Hume said, "fell dead-born from the press"].

Now consider how an author in this position–an author attempting to build on their previous work–might attempt to "ensure" anonymized review. Here, there are roughly two options available:

1. Cite their previous work, but in the third person [for instance, in my case, Arvan XXXX].
2. Redact citations of their previous work, as in "[Author redacted to preserve anonymized review]".

On my social media feeds, some people have argued for the superiority of option [2], noting that some journals apparently require it in cases like these. But, is option [2] really superior? Does it "preserve anonymized review"? The answer, it seems to me, is plainly no, at least not in the kinds of cases described above: the case of a not-very-well-known author. To see how, suppose the author in question, Author X, is the only person to defend a given argument or theory in the literature. If the current paper under submission develops that argument or theory further, then "redacting" citations of the person's previous work actually accomplishes the opposite of redaction: it signals to readers precisely who the author is–as, by inference, the redacted person's identity could only be the person who previously published that argument or theory. In contrast, approach [1], while it may suggest that X is "probably" the author, at least leaves it an open possibility that they aren't the author [it could well be possible–if only unlikely–that someone else is systematically citing X's work].

Anyway, here's the problem: it doesn't seem like either [1] or [2] really solves the problem of "protecting" anonymized review. For, although some of us might like to assume the best about reviewers–namely, that they would not infer an author's identity from the above information–there is evidence that some reviewers, at least, aim to discern authors' identities when reviewing their papers, going so far as to "Google review" paper titles. Consequently, it seems as though there is really no very good way to "preserve anonymized" review in cases like these. If the reviewer reviews the paper, either on proposal [1] or [2], they can have a very good idea of who the paper's author likely is–and indeed, it is highly likely that any reviewer of the paper will be in a similar position, as in cases like [1] or [2] any reasonably intelligent person can infer who the author likely is. Conversely, if reviewers routinely decline to review papers in cases like these, authors are in effect "punished" for attempting to build on their previous work. Long story short: in cases like these, true "anonymized review" is nearly-impossible. Either no one reviews the paper, which is problematic, or someone does, which is also problematic, as pretty much anyone who reviews it will have a good idea of who the author probably is. 

Now, you might say, this is a unique case: most cases aren't like it, as the typical journal submission involves authors citing other people's work. However, this is far too quick, as there are countless other ways in which "anonymized" review is routinely jeopardized–among them:

1. People archiving paper drafts on their webpage[s] and/or announcing it on their social media feed: In my experience, this is quite common. Many people upload drafts of unpublished papers online. In some cases, people go so far as to announce it on social media, noting to large numbers of other people in the field, "I just uploaded paper X to my website-check it out! 
2. People presenting paper drafts at numerous conferences and department colloquia, including invite-only conferences with many of the most notable specialists in their field: These practices not only expose potential reviewers to papers directly, but can also do so indirectly, though relayed conversation [viz. "So-and-so has an interesting paper on that topic. I saw it presented at conference C"].
3. People [particularly job-candidates] referencing their research programs on their website: Even if one does not archive actual paper drafts on one's website, it is common practice for job-candidates to post research statements and CVs, both of which can give visitors [potentially, anyone who searches a key phrase in a paper] a pretty good idea of whether they are the author of an "anonymized" paper under review. 
4. PhD Dissertations are routinely archived online by Universities: Mine was, and anyone searching a key phrase from my early work [e.g. "nonideal original position"] could identify me as the likely author of many of my early papers.

In other words, in the modern internet-age, the ways in which anonymized review can be compromised are numerous, and increasingly easy to compromise. There are, of course, all kinds of ways in which some of these potential compromises might be "addressed", such as by journals prohibiting self-archiving drafts online prior to publication [on at least one reading, Phil Review's submission guidelines imply this]. But this at most "solves" one of the above problems–the problem of self-archiving drafts–and not, I think, in an adequate manner. First, there are perfectly legitimate scholarly reasons to self-archive online: to get feedback on unpublished work, etc. Second, in the history of intellectual thought, there have been many priority disputes–disputes over who arrived at an important scientific or philosophical result first–and self-archiving online is one way to establish priority. Although I know people who think priority disputes are tasteless–that it really shouldn't matter who "arrived at an idea first"–this is not only contentious [priority disputes do occur, and their being improperly resolved can be plausibly unfair to people whose work is not recognized, such as Rosalind Franklin]. Third, and most importantly, not allowing self-archiving not only does nothing to address [b]-[d]; it actually amplifies [b], as self-archiving online is one way that not very-well-placed people can even the playing field with people who are better placed [e.g. invited to more conferences, etc.].

Finally, I'm not sure what can really be done about [b]-[d]. What are we going to do: address [b] by telling people they cannot present at conferences and colloquia [for fear of compromising anonymized review]? Address [c] by tell people they shouldn't post research-statements or CVs online? Etc. In the modern internet age, these "solutions" seem increasingly absurd. The internet itself appears–to me at rate–to systematically compromise "anonymized" review. And so, it seems to me, we need to rethink "anonymized" review itself. As David Wallace points out in the comments section here and I have pointed out before here, academic mathematics and physics moved away from "anomymized" review long ago for more or less these reasons, and indeed, to improve academic communication [viz. the longer a paper is kept out of the public sphere, the longer its ideas are withheld from other researchers]. The alternative approach in math and physics today is to archive unpublished drafts on a central public site, the ArXiv, and see which ones have uptake in discussion. Paper drafts–both by "famous" and "non-famous" authors–are routinely posted, discussed, and critiqued prior to publication by experts in the field. Indeed, this public peer-review is actually arguably superior to preventing bias [including status quo and prestige bias], as online discussions are held both by famous and non-famous people–so, the strength and publishability [or lack thereof] of a given paper emerges publicly, as opposed to being decided by one, two, or three journal reviewers who may have kinds of all kinds of biases themselves. As David Wallace explains, and I concur, the workability–and advantages–of this kind of public vetting are pretty clear. It is not a perfect system, by any means [biases can of course still affect how new papers are received. But at least it is open and broadly "democratic"–enabling everyone to judge the quality of unpublished papers, as opposed to a couple of "anonymous" reviewers. And, as we have seen above–and, I think, we all know–"anonymized" review isn't perfect either, not by a long shot. "Anonymized" review is not only arguably routinely compromised; it also plausibly disadvantages non-well-placed scholars, plausibly generates priority disputes, and [as Wallace emphasizes] significantly hampers the efficient dissemination of research.

So, then, is it time to rethink "anonymous" review in philosophy? I think so–though, as always, I'm willing to rethink my view. What do you all think?