The Philosophers' Cocoon

A safe and supportive forum for philosophers.

Owned & Moderated by Marcus Arvan (University of Tampa)

Contact: marvan@ut.edu

Blog mission & moderation policy

IMPORTANT NOTE on how to ensure anonymity in commenting
Job-Market Reporting Thread (2025-26)
Job-Market Discussion Thread (2025-26)

recent posts

about

A Philosophy Grad-Student Report?

There's a lot of discussion on the philosophy blogosphere today over whether the Gourmet Report should continue, and if so, how. I don't want to wade into that discussion, as there are plenty of other places for that. I want to explore something else.

Several commenters over at Daily Nous (including our own Helen De Cruz) have suggested that perhaps instead of program-reputation rankings, there should be (as the APA already has) reports of other things — for instance, program placement-rates, attrition rates, etc. I think these kinds of reports are good, as they give prospective students hard evidence of outcomes: whether people at particular programs ultimately get jobs, etc.

One thing that has not been discussed however (at least not to my knowledge) is whether a grad student survey of their own programs might be a worthwhile undertaking, providing prospective graduate students with data concerning how grad students perceive their own programs along various relevant dimensions. For instance, consider a survey distributed to grad students across a wide variety of programs asking them to log responses to items such as the following on a standard 1-5 Likert scale (1=strongly disagree, 5=strongly agree):

  1. My program has a strong job placement record.
  2. My program's attrition-rate (i.e. graduate students leaving the program) is too high.
  3. I worry that I might not finish my program.
  4. Faculty in my program are supportive.
  5. I have enjoyed my time in my graduate program.
  6. My program has a good "climate" overall.
  7. My program has a good climate for women.
  8. Etc.

It seems to me that such a report could be (A) quite valuable to prospective students (giving them a fuller picture of grad programs they might attend), as well as (B) a mechanism to hold programs better accountable for their performance. Both of which seem like good things to me.

Anyway, given that the Cocoon has sort of become a central place for early-career philosophers to congregate, and given that I believe I could spare the time and energy to construct, distribute, collect and analyze such data, I was wondering what everyone thinks. Would this be a good idea? If so, I'd be curious to see what types of survey items people would consider important to have on the survey. If you don't think it would be a good idea, I'd also be curious to hear why. I guess one possible problem is that programs could try to "game" the survey, either formally or informally, through expectations of "representing the program in a positive light." But, for all that, I'm not sure this possibility would be sufficient reason not to produce such a report. First, the report could be issued with caveats to that effect (noting to readers that they should still exercise their own due diligence prior to applying to or attending a program). Second, I guess I'm just not that cynical about grad students. I'd hazard a guess that, if the survey is collected in anonymity, unhappy grad students would give answers reflecting their genuine beliefs.

Anyway, what do you all think?

Posted in

18 responses to “A Philosophy Grad-Student Report?”

  1. Matt

    I think that this sort of survey would be extremely useful for prospective applicants. One thing that might be worth considering is whether or not to restrict participation to grads that have been at their programs for a certain length of time (a year, say, or maybe restrict it to ABDs). The idea being to eliminate overly-optimistic responses from students that are still excited from getting into a program or who have not yet met with the “endurance-based” difficulties of grad school yet.
    I would suggest that if you do run such a survey, it would be advisable to conduct one for terminal MA programs as well, given the increasing rate at which students are choosing to do an MA before applying to PhD programs.

    Reply
  2. Joe

    It’s a well-motivated proposal, but it has a basic problem. I know of a top 10 department full of graduate students who are very negative about their department. This in spite of the fact that it does comparatively very well in most of the metrics above, from placement to climate. I think grads are generally quite isolated from realities at other departments, yet it is the comparative data that is of interest to us.
    Also, there is always the threat of retaliation. If Faculty at a given department are already grouchy or distant or inaccessible, a published report by the grads calling them out is likely to exacerbate tensions. Grads have (or will perceive themselves to have) strong personal reasons to stay quiet about these things.
    I’m 100% in favor of programs being accountable for how they treat their grads, but there are severe epistemic and practical barriers to developing an objective ranking via questionnaires. This is why the Jennings Rankings are so valuable: two fairly ‘hard’ numbers are compared: prestige and job placement. Grads should, as a rule, avoid departments where it looks like a low Jennings ranking isn’t going to improve.

    Reply
  3. Alex Guerrero

    For some of the reasons mentioned above, there might be good reason to survey not just (or not even!) current grad students, but also recent program alums. This gets logistically tricky, of course. But those people are less likely to fear retaliation, will have a better sense of the field (having likely moved on to other places, with other climates, problems, etc.), and will be somewhat less invested in gaming the rankings either positively or negatively. That also helps increase the survey pool, so that there is greater anonymity (some programs are very small, particularly in terms of the numbers of current female and minority students).
    Another important element, I would think, is to include in the survey people who have left the program before getting a PhD. But again, that proves logistically tricky.
    And you’d want to talk to someone with serious surveying training, so as to do things in a methodologically appropriate (I’d worry a lot about response rate issues) and ethical way.
    I think this kind of thing could be useful, but I worry that the logistical difficulties might be considerable. I’m not convinced that bad data plus caveats is better than no data at all… If nothing else, it would be interesting to see how departments self-conceive…

    Reply
  4. Carolyn

    I suggested a graduate survey on my Ideal Rankings post (http://www.newappsblog.com/2014/07/an-ideal-ranking.html) and aim to try to implement one in the next year or two, with the help of others (http://www.newappsblog.com/2014/09/placement-data-news.html). If you have any insights on this, feel free to reach out to me by email.

    Reply
  5. Marcus Arvan

    Thanks for the comments on this, everyone.
    Matt: Thanks for the apt suggestions!
    Joe: I appreciate the concerns but I agree with you on the value of the Jennings rankings, and am now in touch with Carolyn (see below).
    Alex: I think data from recent grads is a good idea, but you’re right, the logistics are tough.
    Carolyn: Thanks for drawing my attention to your NewAPPs posts. I look forward to talking to you about these issues more!

    Reply
  6. jmugg

    I am concerned about conflict of interest in reporting. Isn’t in my interest for my grad school to do well? (It is for this reason that Leiter Rankers do not rank their current department nor the department where they earned their PhD).

    Reply
  7. Nick Byrd

    My only concern is that this lacks some kind of baseline. E.g., one and the same placement record my be rated differently by two different grad students. So it seems that one might also want to gather the actual placement data (and other easily measured metrics) alongside these self-reports.
    That being said, I heartily support the idea of gathering climate data.

    Reply
  8. Marcus Arvan

    jmugg: I appreciate the concern, but I’m not sure about it. Speaking as someone who has known people in bad situations, my experience has been that these people tend to want people to know about the problems in their program, so that the program faces pressure to improve.
    Suppose, for instance, your program had a poor placement rate. Would you want this to continue, or would you want your program to face public pressure to improve its practices? Or, consider a program with a bad climate that is making you miserable. Wouldn’t you again want your program to face pressure to improve?
    In any case, my experience has been that people in bad situations tend to want change, and change only tends to happen when there’s some kind of public pressure to do so — the kind of pressure this sort of grad-student survey might lead to.

    Reply
  9. Marcus Arvan

    Nick: Absolutely! It’s presumably best to collect objective and qualitative data. Objective data misses the qualitative elements, and qualitative data can come apart from objective facts. Both kinds of data together give a fuller picture.

    Reply
  10. anon

    We have implemented this at the University of Miami, after the McGinn scandal. I believe we only did a Grad Student Survey the year of his departure, not in the year since, but it may be scheduled again. What we found:
    1. Faculty retaliation.
    2. Censorship: Whether to share it with the Deans or not was a very heated discussion. Certainly we would not share it publicly, nor with prospective students.
    3. Some very clear, specific changes were implemented regarding the process of running the Grad Programme: The qualifying exam process was tweaked, the weekly talks were transformed into workshops, the Faculty received confirmation that McGinn’s replacement ought be a woman (it was). Logistically, regarding the operation of the department, these were small yet meaningful corrections.
    4. Regarding the climate of the department, it may have sent Faculty and Grad Students off to their own corners, Faculty licking their wounds. But in the wake of our scandal, we were already factioned.
    There needs to be a commitment to changing the environment. Graduate Students asked that Faculty to hire an outside mediator to come in and work on integrating the Survey results and reconnecting Faculty and Grad Students in a hospitable way, but that particular method has not happened, nor is it likely to. For now students still rely on getting a Dean to step-in when hostilities arise.

    Reply
  11. jmugg

    Marcus-
    I think you are right about ‘bad’ situations, but my worry is more so about the ‘in between’ places. That is, situations in where the department is not really bad, or really great, might get a bump.

    Reply
  12. Anonymous Assistant Professor

    Just want to point out a potential complication that arises when doing this kind of climate survey.
    When I was in graduate school the climate for women in my department was never that great, but it went through a few years that were pretty bad, then it got better because some new grad students came in that were more aware of climate issues in general. During the bad years, everybody would have said we had a great climate, because the grads were almost entirely made up of men who were responsible for the bad climate but they had no idea. During the good years, people would have had a more mixed response, because they were becoming aware of climate concerns they hadn’t been aware of before.
    Given the extreme demographics of so many departments this can’t be a unique thing. Just remember Michael Tooley’s claims that he had never witnessed any sexism at UC Boulder so he couldn’t believe it was happening. I suspect many grad students are similarly blissfully ignorant.

    Reply
  13. AC

    I agree that graduate student evaluations of their own departments would be useful. However, restricting survey participants to those students who have graduated or are ABD would undoubtedly lead to unrepresentative results. In particular, such a survey would likely omit most students who leave the program – regardless of their reason for leaving. For example, a student might leave because he or she (1) had a health issue, (2) had a problem with the faculty, staff, or another student, (3) found the department unsupportive, (4) couldn’t effectively study his or her desired area of expertise within that department, (5) ran out of or was never offered funding, (6) became overly discouraged by the field or its job prospects, (7) switched to another (presumably more favorable) program, or (8) had another personal or professional reason for departure. Gathering information about most of these issues could prove helpful to prospective students and should not be selected against based on participation prerequisites. I do agree, however, that there be a requirement that participants have been a member of the department for at least 1 or 2 semesters. This might be easily accomplished by conducting the study in January each year since most graduate philosophy programs (in the northern hemisphere anyway) only accept new students in the Fall.
    I think it is worth noting that, while this sort of survey is likely to produce a lot of unfavorable results, there will also be positive results. Furthermore, even if the results are overall negative, general negativity across all departments would put most departments on an even footing. It could also have the benefit of warning/preparing prospective grad students of the harsh reality that is grad school in philosophy.

    Reply
  14. anon

    the attrition question is especially important, and it’s great you’re including this.
    my phd program has a great placement record, for those who finish. but it also has substantial attrition which I suspect to be partly due to a very poor climate. i honestly can’t say whether the amount of attrition in this department is typical or below or above average. because I dont have that data for other departments

    Reply
  15. the ghost of philosophy past

    I was in graduate school in the 1990s. It was common then, in Canada and the USA, for Ph.D. programs to have a 50 % attrition rate. A friend who was at Cornell said the attrition rate was similar there in the 1990s. If all these people weigh in on your survey, then all programs will be damned.
    I think surveys are fraught with problems. Notoriously, people’s opinions about the value of things often run at odds with the true value of them. Course evaluations do not reliably track genuine learning (and, yes, I do have strong evaluations).

    Reply
  16. Anonymous Assistant Professor

    Let me just add that the WORST things that happened at my grad department with respect to climate I have never told anyone outside of my department. Other grad students and professors knew but did nothing, which was part of the problem. But the person affected by the worst things was my friend and I promised that friend I wouldn’t talk about it. This is not something I could even post about on what it’s like to be a woman in philosophy because the smallest details will potentially give away the people who I’m talking about. And for the same reason if I were taking any kind of climate survey I couldn’t talk about it either. I would only be able to mention it in the most oblique way, for example, something bad happened.
    That doesn’t mean you wouldn’t get useful information with a climate survey, just that we should always let prospective students know that there is no such instrument which will perfectly measure climate.

    Reply
  17. Marcus Arvan

    Anonymous Assistant Professor: Thanks for sharing your story. At this point, I think the plan is for the survey to be purely quantitative (i.e. responses to items on a 1-5 Likert scale), rather than qualitative (sharing experiences). This would prevent people from being identified.

    Reply
  18. Marcus Arvan

    ghost of philosophy past: Thanks for your comment. A few thoughts.
    First, it might be good for “all programs to be damned.” People entering PhD programs should know what they are getting into–especially the fact that they may only have a 50% chance of finishing (and a far lower chance of receiving a tenure-track job).
    Second, even if many departments fare badly in some respects (i.e. attrition rates), some may be much better than others–which again, would good information to know.
    Third, such negative information would put much needed pressure on programs to not let this sort of stuff happen. Sorry–there’s no excuse for 50% of your students failing to finish. There may be a long tradition of that, but it’s a bad tradition. Programs have an obligation to see to it that their students flourish. Not every student, of course (no program can achieve perfection)–but surely, many programs could do a whole lot better…and a Grad Report would put pressure on them to do so.
    Fourth, the Grad Report could make other important differences very clear (some departments have far better climates than others). This would not only help prospective students decide where to go. It would also, once again, put pressure on programs to improve.
    Finally, I think the analogy to student evaluations is poor. First, we will be collecting (as the APA does) objective information on placement rates. Such information can be used to judge how accurate grad-students perceptions are of their programs’ level of success. Second, we’re not talking about “measuring learning”–a notoriously difficult thing to measure. We’re measuring whether grad students are miserable, whether they feel supported by faculty, whether they have to go to school walking on eggshells everyday, etc.–in other words, things that deeply affect people’s lives, and which grad students are more than capable of judging in their own case.

    Reply

Leave a Reply

Discover more from The Philosophers' Cocoon

Subscribe now to keep reading and get access to the full archive.

Continue reading