The Philosophers' Cocoon

A safe and supportive forum for philosophers.

Owned & Moderated by Marcus Arvan (University of Tampa)

Contact: marvan@ut.edu

Blog mission & moderation policy

IMPORTANT NOTE on how to ensure anonymity in commenting
Job-Market Reporting Thread (2025-26)
Job-Market Discussion Thread (2025-26)

recent posts

about

Schwenkler on an informal peer-review website

John Schwenkler wrote the following on facebook today and encouraged me to share it:

Idea: A website where philosophers can upload work in progress and receive feedback from other philosophers, in exchange for offering feedback of their own on others' work. (Think of it as an alternative to submitting unpolished work to journals in the hope of learning from referee reports.) It could be arranged so that you have to complete at least one report on another's paper for each report you receive on one of your own. And authors could rate the quality of these reports (insufficiently positive feedback means you'd need to try again), specify if they'd like reviewers with specific areas of expertise, etc.

Does such a thing already exist? If not, why not? And would anyone like to help try and make it happen?

I think this is a great idea. Academia.edu has something a bit similar: a new paper-workshop function where you can invite people to read and comment on your work. I've taken part in a few of them and they seem to work pretty well. However, it doesn't include author ratings of reviewers or a system requiring one to give reviewer reports to receive them. Personally, I think Schwenkler is describing a promising system of how refereeing might work at journals to speed up the process and quality of reviews–but we've already discussed that!

Anyway, what does everyone think of Schwenkler's idea? Would people take part if such a system existed? 

Posted in ,

10 responses to “Schwenkler on an informal peer-review website”

  1. Douglas W. Portmore

    I think that this is a great idea.

    Reply
  2. Pierre

    It’s a great idea, but I am not fully comfortable with the kind of reciprocity principle that is appealed to. Let me explain: as an unpublished young philosopher, I am more in need of feedback than I am able to provide it. Of course I can comment someone’s work, but compared to an older, more experienced author, it is likely that my comments would be of poor quality. I am less aware, as an author, of journals’ expectations and of what makes a work not only good, but also publishable, than someone with a good publication record is likely to be.
    An experienced author is more likely, say, to identify the kind of weaknesses that would lead to an outright rejection, or the kind of strength that turn a manuscript into a published article. In sum, the reciprocity principle is appealing (I would not deny that), but it seems to me that it remains unequal or asymmetric.

    Reply
  3. Richard Chappell

    Some friends and I put some thought into a similar idea many years back, but unfortunately never completed the project. But for those interested in the discussion (including various implementation proposals) see:
    http://philreview.pbworks.com/w/page/16449741/FrontPage
    It would be great to see something like this finally take off!

    Reply
  4. Elisa Freschi

    @John (and Marcus), the idea in part exists, in the sense that the “sessions” at Academia.edu (which are still in their beta-phase, but which at least I and Helen from the Cocooners here have already experimented with much profit) are something similar: you upload your paper and your followers can comment on it and usually give very constructive feedback. I am in touch with Richard Price of Academia.edu and he told me that they would like to add to the system the possibility of an actual overall rating of the paper. Perhaps working on that could be more profitable than starting a new website anew?
    @Pierre, I do not agree. Younger philosophers usually have more time and more enthusiasm and can in this sense provide more in-depth comments. In my experience (see above) with the sessions, some of my senior colleagues gave me very good pieces of advice, but several others just did not have the time and only corrected a couple of typos.

    Reply
  5. John Schwenkler

    Thanks for sharing this, Marcus.
    I’ve created a Google Document where people can brainstorm ideas and add their names to a list of volunteers: https://docs.google.com/document/d/1J19DbsiG2075CuslXM4AvYIKtGUe-dTdODBLr4m-bi8/edit?usp=sharing
    And thanks, Elisa, for the pointer to Sessions at academia.edu. I agree that this is useful, but suspect that lots of people would prefer anonymity in this sort of process, and also that some amount of reciprocity is more likely to make it work. I take Pierre’s point about the value of experienced reviewers, but of course it’s always open to people to write more reviews than they receive. It’s also worth noting, though, that in the present system more established philosophers are inundated with requests to referee for journals, whereas younger philosophers often complain that they don’t receive as many such requests as they would like. This is intended as a (partial) solution to that situation.

    Reply
  6. Pierre

    Elisa: Thanks for your reply 🙂 I very much hope my doubts are misguided, and your reply happily suggests they are at least partially so. Now that is not exactly where my discomfort stems from. Surely young philosophers are able to provide precious comments (there is no doubt about that), but I am not certain (as a matter of opinion) that comments from someone like me would be of great use as regards what makes a paper publishable, beyond its philosophical quality. So far I have little first-hand experience of what it is to successfully get through the publishing experience. I do have some experience of what it is to get through a selective procedure, but I still need, so to speak, to “furbish up my weapons” as a commenter/reviewer.
    But let us turn my (hopefully misguided) fears into a positive proposal: what about a “junior reviewer” status? That would help those who’d be willing to provide feedback, but still need to improve their abilities in that respect, to do so. Over time they would gain confidence as reviewers, learn the “tips and tricks” that make one a good reviewer, and so on.

    Reply
  7. Elisa Freschi

    @Pierre, that of the “junior reviewer” might be a good idea, as long as it helps junior scholars. In other words, I would not impose it on anyone and would rather encourage shy PhD students and the like to adopt it if they do not feel confident enough.

    Reply
  8. Joshua Mugg

    @Pierre,
    Even if you are young, after you get your first publication you will be asked to review for journals. I have reviewed for both journals and conferences while in grad school.

    Reply
  9. Eugene

    To help ensure reciprocity it might a good idea to have whatever system is eventually in place set up such that when X leaves comments on Y’s paper X’s comments are uploaded into a holding pen where they cannot be accessed by Y until Y reciprocates by uploading comments on X’s paper.

    Reply
  10. g

    Somewhat related: Rebecca Schuman from Slate has suggested a reciprocity system for peer review. The idea is that for every paper you submit to a journal, you have to have refereed a paper for that journal (I’m assuming that double-anonymity would still somehow be preserved). Not sure what I think about it, but I think it’s worth philosophers talking about. http://www.slate.com/articles/life/education/2014/07/the_easy_way_to_fix_peer_review_require_submitters_to_review_first.html

    Reply

Leave a Reply to Joshua MuggCancel reply

Discover more from The Philosophers' Cocoon

Subscribe now to keep reading and get access to the full archive.

Continue reading