At this point one may worry that such prizes are easily shaped by politics, and that our time will be spent adjudicating prizes rather than refereeing. On the former, I don’t see how politics can be removed from the equation entirely (as it is, the journal system has a major ‘trust us’ component recall). I do not deny that genuine masked review, when it happens, is a real potential equalizer. And notice that on my proposal, we don’t eliminate that. People who come from non prestigious intellectual backgrounds can still use journal publication to become noticed. (But the initial signal’s power may well be reduced.) And one can use masked review also in prize committees. Over time, prizes that show a good track record in their judgment will become more prestigious, and so complement journals in prestige generation and quality control.
Of course, it could be that my editorial experience is atypical–I’m just one scholar.
A version of this post first appeared at Professor Schliesser’s blog, Digressions & Impressions.
How to Fix the Referee Crisis in Professional Philosophy
by Eric Schliesser
In a recent post at The Philosophers’ Cocoon, Helen De Cruz (SLU) writes:
+This comment was added after initial publication of the post at Digressions & Impressions.
(i) an extreme scarcity of decent tenure track jobs relative to number of PhDs minted. This is a major source of anxiety throughout the profession
(ii) the unreasonably low acceptance rate of philosophy journals such that perfectly competent papers have to be refereed multiple times by a fixed number of referees
(iii) the perceived role that prestige and quality of publications play in hiring
and, perhaps,
(iv) the relative ease of free-riding in the current system. The real free riders are universities (who outsource quality control without rewarding it internally—my own university never even asks to list refereeing anywhere) rather than particular individuals.
In what follows, I take for granted that the low acceptance rates are functional, that is, maintained in order to produce prestige (recall this post). And the combination of (i-iii) has generated a veritable arms race in publication. In addition,
(v) the long delays in refereeing time, have made it prudent to have multiple papers under review at the same time, thus, exacerbating the underlying problem.
De Cruz’s piece has generated a huge discussion, not just among the Cocooners, but also here at Daily Nous. The situation she describes has been a long time coming (and for once I can claim I partially predicted it—recall here and here). There are a number of major interlinked reasons for the current status quo (extreme referee shortage):
It has come to the point that the most suitable referee(s) for a paper are almost never available. It takes us so long to find reviewers, sometimes a month, six weeks or more (this situation is also exacerbated by the fact that many people don’t respond to referee requests at all). This lengthens the span of time even more for the total review process. All sorts of small fixes don’t work anymore, e.g., shortening the time that people get to review, asking for alternatives (this is still very much appreciated, but unfortunately alternative referees are just as unavailable).
In what follows, I take for granted that the norm of regular publication has been good for professional philosophy. Reasonable people can disagree about this. But in my view it has allowed for more fine-grained discussion and for specializations to emerge that have improved quality. Yes, it risks generating hyperspecialized echo-chambers with lowest common denomination salami publications, but these, in turn, generate arbitrage opportunities for people who read widely or who practice what I call synthetic philosophy.
[Theo Van Doesburg – Contra Construction]
I also take for granted that (i) is entrenched in the political economy of modern universities. I am complicit in it; if you find a way to create a revolution, good for you. But what we can do, as a profession, is to cut the ties between prestige, journal publication, and hiring decisions and so reduce the need for referees. And we can do that if we destroy the role of pre-eminence of journals in manufacturing prestige, and restore them to the role of disseminators of knowledge and sites of discussion.
On the latter concern, about more time spent judging work for prizes, this is undeniably true. But I suspect it’s more fun to evaluate papers that have already been published—and so have undergone non-trivial quality and editorial control. Also, while writing a prize report is a non-trivial exercise, it’s only required for the winning paper(s). So, on balance I would expect it should reduce the load on those in the profession that are the back-bone of our current referee-system.**
One may worry that once journals can’t play the role of extreme prestige generators, then it entrenches the roles of PhD granting departments. I agree, and that would be bad. So this is the second step: another institution has to take its place. Luckily, such a practice already exists in neighboring disciplines and is present in incipient form in ours: the article (and book) prize. What we need is a proliferation of prizes in philosophy: conference prizes (which are pre-publication); professional association prizes (of the major and smaller societies); best article prizes by journals themselves; and we need more versions of The Philosopher’s Annual. In fact, as Don Ainslie reminded me, there should be more prizes on different grounds too: philosophy has far too few of these, which in turn impacts success in interdisciplinary competitions (e.g. national scholarly societies, grant competitions, etc.).+
**I thank Helen de Cruz and Neil Levy for discussion.
In the following guest post, Eric Schliesser (Amsterdam) provides a two-step solution to the referee crisis in philosophy.
This can be achieved in two steps: first, all the major philosophy journals should commit to publishing, say, 25% of articles received. That’s considerably higher than the status quo (Philosophical Quarterly publishes just 4% of over 850 papers received). This will have six obvious effects: it will reduce the load on referees; it will end the excessive role of chance in where your paper is published; it will reduce (but not eliminate) the steep prestige hierarchy of our current journal system; it will reduce the excessive delays in getting work into print caused by the difficulty of finding referees; it will turn most major journals into paper mills; it will increase citation rates within the profession.
But it is my strong suspicion that the peer review system is finally broken beyond reasonable repair. We’ve seen a slow worsening of the situation and the pandemic has finally broken the system. People are burnt out and overburdened, job candidates increasingly desperate.
*I also suspect my proposal solves an objection against those that think Condorcet jury theorem is not apt here, because prize awarding is standard setting not discovery of truth. But that’s for another time.
