The following is a post submitted by Rögnvaldur ('Valdi') Ingthorsson (Lund University):
Academic Frustrations: Publications and Jobs
—Valdi Ingthorsson
I recently came across an old post by Jason Stanley in The Philosopher’s Cocoon about the frustration of getting papers through the peer-review process (Stanley on Peer-Review). The post put its finger on a couple of things that have frustrated me for a while. First, why is it so difficult to get philosophically interesting papers published in high-ranking journals? To my mind, these journals seem to be mostly filled with papers that don’t do much more than recirculate received views, albeit in a competent way. Second, why does quantity and ranking still rule over quality in the hiring process? I have come to think the two are closely related. As you can figure, I am looking at this from the perspective of someone who struggles to get published and to get a job. I very much expect that journal editors and tenured faculty in hiring committees will not recognise themselves in the description. But I think it is healthy for them to find out how the whole thing looks from the other side (even if it is all biased and exaggerated, a possibility I cannot rule out).
It is important for the credibility of this piece, to establish firmly that my difficulties in getting published are not due to my bumbling incompetence as a writer. My publications occur in seven SEP entries, my recent book on McTaggart’s Paradox was nominated for the APA Sanders Book Prize, and I have been awarded research grants to the combined sum of approximately $550.000. My book was actually described by a reviewer as “superbly written, exhibiting exceptional clarity, concision, and flow” (J. Mozersky, Notre Dame Philosophical Reviews). Indeed, the only consistent feedback I receive about my applications for jobs and funding is that my research is highly original and interesting. And I have received more than just a handful of quite detailed evaluations (we’re talking +50) because in Scandinavia applicants for university positions and research grants must be assessed by external reviewers (sometimes as many as 3), and their verdicts are by law public documentation. However, despite praise about originality and quality I am still consistently ranked low on the list of candidates because, the argument goes, I just haven’t published enough and not in the highest ranking journals.
As far as I can see, focusing on quality over quantity is the worst thing you can do as a young academic, despite everyone’s claim that it is quality that matters. My cynical advice to young scholars must be: cancel (for a while) your ambitions for true philosophical excellence and write some competent rehash of mainstream views (with some minute tweaks) for publication in the highest-ranking journals. Basically, write some stuff everyone can understand on first reading and therefore doesn’t require any effort on behalf of referees. I have heard many enough state out loud that it’s not their job as referees to make an effort to understand the papers they review; it is the author’s job to make it understandable from the first syllable. We are seeing here something comparable to the application of the ‘7 second rule’ of pitching; if your presentation doesn’t capture the critic in 7 seconds, they stop listening. Is philosophy dumbing down to that level? In a 7 second world, posterity would never have heard of Kant.
Don’t get me wrong. It is meant to be difficult to get papers published and I think we need the peer-review system. However, the system should be meant to weed out the incompetent and uninteresting but not also the interesting and innovative to favour mainly the competent and bland. My worry is that this is what the system actually does and I am not alone. Jason reports in his post that his best pieces meet with the most resistance and Tom Cochrane reveals in another recent post in the Philosopher’s Cocoon that his best paper went through 20 rejections. John Heil, in the inaugural editorial of the Journal of the American Philosophical Association, says that one of the main reasons for initiating the journal is the concern that the peer-review process, in its current shape, “can stifle unconventional papers, papers that take chances, papers that go out on a limb: interesting papers”. Now, others argue that this is all a good thing, because for each rejection the papers improve. That may be true, but don’t forget that it is the fact they were interesting from the beginning that they were rejected.
Going back to Jason’s post, ultimately he offers advice for frustrated people like myself: don’t get hung up on peer-reviewed publications, just get your papers ‘out there’ any which way you can. In his view, “whether your papers are accepted to leading peer reviewed journals doesn’t much matter, even for tenure, as long as they come out somewhere”. In his experience papers somehow “take on a life of their own” and find a way to their audience no matter where they are published.
Jason is right to say we shouldn’t underestimate other options, and perhaps he is right to think that peer-review publications are on the whole overrated, but I think he is wrong to think that publications in high-ranking journals really doesn’t matter that much for getting a job. I venture to say that Jason—who has had a straight run from a PhD from MIT, with Robert Stalnaker as advisor, to Oxford, Cornell, Michigan, Rutgers, and now Yale—lacks the perspective of the struggling young scholar. Getting tenure once you are in the living room is something altogether different from those trying to get a foot in the front door. And this is not to say Jason has had it easy or didn’t deserve it; all I am saying that he has not been in the position of applying for jobs in Philosophy when working as a temporary lecturer in Food and Nutrition at an unranked university.
Indeed, Marcus Arvan comments that Jason’s story is perhaps best understood as one more indication that the peer-review system needs tweaking; that we should not just accept it for the troubled beast that it is and find other ways to get our papers out there (well, we can do that anyway). And, to repeat, my concern is not with the fact that it is difficult to get published, but the fact that the system seems to make it extra hard to publish a certain kind of papers, those that argue for novel and interesting perspectives that challenge the mainstream.
So, as you may have guessed, I really struggle to get published. The record is 13 rejections for a paper that nevertheless (unbeknownst to me) became a central subject of a PhD thesis from Helsinki University in 2016 even before it came out in print. To put it in perspective, the thesis contrasts my take on truth with Crispin Wright’s alethic pluralism and the author argues in favour of my take on the issue. The author found the draft of my paper on my Academia website where it had been available since 2008, which actually proves Jason’s point that papers can take on a life of their own regardless of how they ‘come out’.
Also, like Jason, some of my best papers have not found their way into any journal, not even after 10 years in the reviewing process. The difference is that Jason finds it comforting to have a couple of papers up his sleeve in case of a dry spell, while for me they are potentially what stands between me and my first permanent job (after 16 years in academia).
My experience is that it is quantity and journal ranking that ultimately decides the outcome of job-applications, because those criteria rule supreme in the stage where it really matters, notably in the short-listing stage. Obviously, the rank of your grad-program will matter too, independently of the content of your thesis. And it does so in a way that basically excludes the possibility that someone with a few really good papers (good by content, not by where it is published) could ever best someone who only outdid them in quantity (except, of course, if some celebrity philosopher declared this someone a genius). I draw on my experience from more than 250 job applications in the last 15 years, so this is more than mere anecdotal evidence. I’ve estimated that I have spent a whole year of my career applying for jobs and grants. The grant writing has paid off, but that seems to have no implications for my job-seeking. Which is weird, considering that success in grant acquisition is always mentioned as an important criterion.
Outside of Scandinavia I have yet to come across a hiring committee that reads more than a writing sample during the shortlisting process (typically capped at 10.000 words), and in a vast majority of cases they read nothing at all. Ergo, quality only matters in the choice between those who have already been selected as quantitatively supreme on the basis of their CV.
As I said earlier, Scandinavian universities are required to appoint external reviewers to judge applicants. The reviewers are required to become acquainted with the applicants’ research (often capped to the 10 best works), and their verdicts are by law public documentation. In some countries it is only required that reviewers read the research of the shortlisted group (this is the case in Iceland and Norway), in which case shortlisting is performed on purely quantitative basis. Whenever I have received the verdict of such reviews, referees consistently stress the originality of my research, but without exception place me towards the lower end of the short list because I have too few publications and none in the highest-ranking journals. The only times quality beats quantity in Scandinavia too, is in comparison between quantitative equals.
I find it ironic (in a desperately depressing way) that while the philosophical community always reacts with horror and shock whenever anybody mentions the introduction of quantitative measures anywhere, then it seems to me that the entire tenured faculty across the globe is mainly selected on the basis of quantitative measures and rankings. Whatever qualitative assessment enters the process, it enters at such a late stage that it only serves to distinguish between those that excel quantitatively. In other words, when tenured faculty preaches quality over quantity, they are perhaps flattering themselves a bit too much. And of course this doesn’t mean that they aren’t doing excellent research, it just means that it is unlikely they got their job by outdoing everyone else on purely qualitative grounds (and don’t come back at me citing the exceptions).
Nothing of what I have said so far excludes other ways of beating the competition, ways that have nothing to do with your publications, i.e. nepotism and corruption. Let's not go down that sordid road.
The point is that if you are lucky enough to hit the ‘fast track’ early, then by all means focus on quality and just get your papers out there in any way you can, but you might have to consider another strategy if you still are looking to get a foot in the door. This latter group should not, and cannot afford to, be happy with the peer-review process as it works today.
But what can be done with the peer-review system? Let it be clear that (i) I am in favour of it, and rather think referees should be more selective than anything else, (ii) that I perfectly well understand that journals are swamped with an increasing number of submissions, and (iii) I know the refereeing side of the system, so I am not looking at this merely from the perspective of someone who doesn’t get published. I referee for 14 journals of which half count as high-ranking, and I review book manuscripts for major publishers. I definitely referee at least as many submissions as I submit (and resubmit) myself. During this academic year alone I have refereed 7 papers and one monograph.
I am in fact in two minds about whether anything can or should be done to the system. Maybe it is the best of all possible systems. Indeed, I suspect the main problem isn’t the peer-review system, but the philosophical community. Take for instance the increasing volume of submissions. One major cause for this increase is arguably the focus on quantity and ranking of journals in the admissions and hiring process at all stages; everything today hinges on publications, even getting into a PhD program may depend on how many publications you have. It used to be about getting the chapters of your thesis published after you finish, but now the aim seems to be to publish before you apply for a PhD program. Who administers this process? It is tenured members of the philosophical community. If what you wrote really matters, rather than how many you wrote, or where you published it, there would immediately be more focus on quality among writers and number of submissions would drop. But that presupposes that admissions and hiring committees start to rely on their own assessment rather than quantity and rankings. I suspect it is naïve to think that will ever happen, because everyone seems obsessed about quantity and ranking. I am still going to complain about it.
Obviously, any complaint about the peer-review system is automatically a complaint about the philosophical community that operates the system. Referees and editors are all part of the philosophical community. WE do it all and if anything is to change, WE need to look in our own bosoms. There is a need for a serious discussion in the philosophical community about the role of a referee, editor, etc. Clearly there is no consensus about it. Many seem to see it as a chore and a bore (perhaps to the same degree they see teaching as a chore and bore). For my part I regard refereeing as an integral part of being an academic, and doing it well is to do your duty to guarantee the quality of the outputs from our profession. It also serves to return a favour to all those referees that have done a good job refereeing my papers. Referees have given me invaluable advice and feedback in addition to some grief. My role as referee is also a part of making the work easier for journal editors around the globe, who in my experience are as a rule people of high ideals and ambition. Finally, I find refereeing enormously rewarding. Being forced to really dig into a paper and pick it apart nearly always ends up in some illuminating thought. Often of course the reward is simply in seeing how much better the paper could be if such and such changes were made. And I am happy to contribute with that because I wouldn’t want to make my effort go to waste. I do however also try to convey the message that I am just doing my best as a colleague; I am not taking the role of a superior authority. The authors will have to make the call on whether my judgement is valuable or not, but I am forced to take a stand on whether to recommend publication or rejection; that is the purpose of refereeing papers.
Having now put a lot of blame on the referees (i.e. us), I still think journals can very easily improve the refereeing process by making an effort to improve the instructions to referees. Provide clear criteria about how to judge the papers, and perhaps even provide a set of standardised sentences from which referees can pick and mix a verdict. There are papers that can be constructively rejected simply by formulations such as: ‘The paper is very poorly written with clear gaps in the argumentation’. ‘There are obvious problems with references’. ‘The Author does not relate to well-known works relevant for the subject’. ‘The paper is competently written but does not contribute with anything new to the issue’. Providing the option to pick standardised formulations does not prevent anyone from writing their own well-reasoned judgement if they want to. But most journals provide no guidance to referees or practically nothing at all. And none of those I know about really tell you how the editors are making the calls. I think it would be enormously beneficial if referees got feedback on the consequences of their verdicts.
The streamlining of the referee process is easiest for the category of papers that are obviously flawed and/or add nothing new, and arguably that is the largest category. The most difficult papers are the one’s trying to say something interesting and new, and which are therefore also as a rule more difficult to penetrate. Again, it would make a difference if the referees are instructed to look out for such papers, urged to make an extra effort with them and to opt to a greater extent for ‘revise and resubmit’ rather than ‘reject’ if the paper is interesting but problematic. But that is an editorial decision that depends on the will of the editors to deal with an increased number of R&Rs. And of course if they really want to promote interesting papers or rather cater for the mainstream.
Finally, I am entirely unsympathetic to desktop rejections without comment, or worse, with an absurd rationale for not providing comments. The best one yet is: “We cannot provide comments on all rejected papers. We focus rather on arriving at a well-informed judgment without undue delay”. How well-informed is a judgement that is difficult to write down? In my own case, the effort of judging a paper is equal to the effort of formulating reasons for acceptance/rejection. Hence I can write those reasons down and submit it with my review protocol on the webpage. I can’t see that the situation can be different for an editor that decides in favour of a desktop rejection without sending it to referees. Either they form a reason in their minds—and it can be about the fit of the paper to the goals of the journal rather than the quality of the paper—or they do not have a reason for rejecting it other than “I don’t like the look of this”. But in what world is that a well-informed judgement? Desktop rejection is the tools of people who think they can “feel” the quality of papers. I don’t think I have that ability and I doubt other really have it. I have too many times found out that papers I once thought were brilliant are actually rubbish, and vice versa.
If a journal cannot process their submissions properly because they are too many, then return a portion of them to sender with the honest declaration that they have been arbitrarily rejected because the journal cannot handle the volume of submissions at this time. Or, in case of a cursory scrutiny, then say that this is what you did. I have been rejected with the explanation that there simply were too many submissions at this time, or that submissions were closed until such and such a date due to the backlog. Rejections like that don’t leave you wondering, and you don’t waste any effort other than finding another journal to submit to.
To my mind, journals that allow their editors and/or referees to give a verdict without formulating a reason for the verdict have decided not to participate in the peer-review system. I will not send papers to such journals nor will I referee for them.
So, what does all this amount to in the end? Perhaps only to a plea to the philosophical community to make quality and innovation the main selection criteria when you are refereeing/hiring, and be more willing to engage in a R&R procedure with the papers that are ‘interesting’. At the same time I think we need to be more critical of the merely competent. How many competent papers arguing the same point do we need?
Leave a Reply