Publons was acquired in 2017 by Clarivate (NYSE: CLVT), formed in 2016 when a private equity firm and an investment manager bought an intellectual property business. Valued at billion, Clarivate has a “bold entrepreneurial mission to help customers reduce the time from new ideas to life-changing innovation.” Whether the mission of philosophy is to discover the truth as traditionalists claim or to change the world as Marxists claim, a bold entrepreneurial mission is unlikely to remain aligned with it. By charging wealthy institutional customers large sums to access information while denying access to philosophers who can’t afford the fees, bold entrepreneurs can profit at the expense of philosophy. To avoid being overcharged for information they generated, philosophers should strike to keep it in the hands of public or philosophical entities like PhilPapers rather than private businesses. Nevertheless, the fact that so many journals have integrated their systems with Publons reveals the interest in publicizing refereeing data.
To publish good peer-reviewed philosophy, editors need good referees and good authors. But while authors enjoy the spotlight, referees work in darkness. Because authorship is rewarded while refereeing isn’t, there are too many manuscripts to properly referee. This would change if journal editors could publicly credit referees in a way that administrators could recognize. This could be achieved simply by making public how much refereeing everyone had done.
A Public Database of Referee Service
by Neil Sinhababu
Research universities seek to employ experts who improve their academic reputation. Being asked to referee is a sign of one’s expertise, and writing good reports is the sort of professional service that university promotion and tenure systems are designed to reward. With a public record of how many papers each philosopher has refereed, refereeing statistics might figure in evaluating candidates for senior research positions. Writers of external letters for senior research positions might find this information especially useful. The database will distinguish diligent and highly regarded scholars from those who refuse to referee or whom editors quietly recognize as unhelpful.
A few years ago, Neil Sinhababu, associate professor of philosophy at the National University of Singapore, wrote about the “publication crisis” in academic philosophy in a post entitled “2,000 Spaces for 10,000 Papers: Why Everything Gets Rejected & Referees Are Exhausted.” In this guest post*, he follows up with a proposal for how to help make things better.
Meaningful Benefit for Diligent Referees
Journal editors will likely have final say about which features the database has (and whether it comes into existence). To avoid requiring excessive additional labor, it will have to be integrated deeply and automatically into journal refereeing systems. This isn’t possible without the active participation of editors, who will therefore collectively determine whether and how a database is set up. This is a favorable alignment of interests. Editors need to solve their problem of having too few referees. By setting up the database, they can solve their problem together with everyone else’s problems.
The lack of any meaningful credit for refereeing threatens the system of blind review. Heavily rewarding successful authorship has generated an oversupply of papers relative to journal space. This oversupply of papers is mathematically expressed in high rejection rates, and the way authors respond to it makes it worse. Those who need fast professional credit for jobs or tenure must put even more papers into the system in hopes of getting something published. As the oversupply of papers overwhelms refereeing capacity, desk-rejection becomes increasingly common, and papers must be both written to appease referees and to avoid desk-rejection at first glance. Simply writing good papers for blind-reviewed journals becomes a less viable path for authors, and cultivating personal connections to those who control non-blinded venues becomes comparatively more viable.
[David Moreno – untitled (detail)]
Related: “The Publication Emergency” by J. David Velleman, “A Plea for More Short Journal Publications” by Avram Hiller, “Flipping the System: One Possible Solution to the Publishing Odyssey” by Felix Bender, “How to Write a Referee Report” by John Greco, “Reforming Refereeing” by Aaron Garrett, “The ‘Insanely Low Acceptance Rates’ of Philosophy Journals“, “The BJPS Referee Of The Year Award“, “How to Accelerate Refereeing“, “Citing (and Thanking) the Referees at the Journal that Rejected You, Part 2“, “Should Publishers Pay Referees and Authors?“, “Advice on Refereeing Papers“.
Settings and Features of the Database
For example, my colleague Zach Barnett suggests publicizing how quickly referees send reports. This would likely lead to faster refereeing of papers, with referees striving to distinguish themselves as especially prompt. While any incentives for refereeing would strengthen the hands of journal editors to insist that referees generally do their jobs well, publicizing how quickly people refereed would speed things up further. Other forms of information might be included to beneficial effect, and innovative thinkers should explore the possibilities.
The closest existing thing to such a database is Publons, which I signed up for years ago hoping that it might play the role in the profession envisioned here. But it hasn’t gained widespread acceptance and seems unlikely to do so. Its current owners have different priorities than philosophers.
Administrators from outside philosophy often have final say over whether philosophers are hired or promoted. As nonphilosophers, they can’t assess philosophical research on the merits. Quantitative measures of expertise and service keep them from making erratic decisions when philosophers’ careers are at stake. Serving on a committee that evaluates nonphilosophers for tenure has made it clear to me how much administrators want information of this form. By making it publicly known how many papers everyone has refereed, journal editors can direct administrators to reward those who do more than their fair share.
Protecting blind review requires excluding information that might connect individual referees with specific reports. Providing too much information might enable authors to identify referees who rejected their papers. There are many ways to prevent this by excluding or obscuring information. Refereeing statistics might be updated only once every few months or even annually, so that nobody can tell who rejected a paper at a specific time. The identities of the journals that someone refereed for might be stated only in some general way, or excluded entirely. Sensitive information might be restricted only to those with special authorizations or permissions.
Creating the Database
Integration with ORCID is a more promising possibility. After academic publishers established ORCID to better match author names to journal articles in 2009, they put it in the hands of an independent nonprofit group, which has controlled it since 2010. It’s not clear to me whether ORCID could be integrated with a database of this sort. But it has over 10 million users, so it’s accepted widely enough to be part of a solution.
Many different types of statistics might be recorded on the database. Publicizing helpful behavior might encourage more people to do helpful things.
Peer review of journal articles is a central but increasingly dysfunctional part of professional philosophy. Authors wait months for referee reports, often receiving rejections without comments. Editors implore referees to review manuscripts, often without success. Referees work hard to evaluate other philosophers’ manuscripts, almost invariably without reward.
Whether and how a refereeing database takes shape ultimately depends on journal editors. As the problem of too many manuscripts and too few referees becomes increasingly severe, editors’ need for a systemic solution increases. Setting up academic infrastructure to publicly credit referees for their service would rebalance the incentives and solve the problem. There are many details to work out, for example regarding how the database should treat differences in the journals’ refereeing procedures. But if editors need referees badly enough, they’ll have motivation to negotiate creative solutions when required.
Together, journal editors could solve systemic problems with peer review. They collectively have the power to reorder the system as they please. Professional philosophy dances to their tune, and if they change the tune, they’ll change the dance. Setting up a database that enables administrators to reward diligent referees would solve their problems, authors’ problems, administrators’ problems, and referees’ problems all at once.
Reward for refereeing would allow the drive towards professional advancement to express itself in refereeing and not just authorship. This helps to maintain the sustainability of blind review in both ways. First, it increases the supply of referees. Second, it offers those who need professional advancement an alternative to exacerbating the oversupply of papers. A public database of refereeing, consulted by university administrators, is the sort of academic infrastructure that would have these effects.
