How Often Are Philosophy Articles Actually Cited? Encouraging News (guest post)

Here’s a more specific breakdown, journal by journal. Of course with only 15 articles per journal, there will be lots of noise in the numbers when considered at this fine a grain.
Although this is not a large database, the results from this carefully designed search are, I think, striking.

How Often Are Philosophy Articles Actually Cited? Encouraging News
by Eric Schwitzgebel

(3) Articles in elite journals are cited only about three times as often as articles in unranked but reputable journals. To me, this was the most surprising result, and part of me wonders whether we’d see different results if we looked at different years or different unranked but reputable journals. Philosophical Review is much more prestigious in the eyes of the typical mainstream academic philosopher than is the typical unranked but reputable journal. I find this result encouraging, especially given the tiny acceptance rates at the elite journals (Philosophical Review reports accepting about twelve articles per year out of 600 submitted). Publishing in unranked journals is not shouting into the void. It is not defeat. The dynamic is less jackpot-or-nothing than I would have expected. You don’t have to get into a top-ten journal to have an impact. Philosophers do regularly read and cite articles from the less prestigious journals.
I draw the following general conclusions:
(A version of this post first appeared at his blog, The Splintered Mind.)
Related: “Philosophy Has High Rate of Uncited Publications“, “Citation Patterns Across Journals“, “Philosophers Don’t Read and Cite Enough“, “Citation Problems in Philosophy—and Some Fixes“, “Citation Rankings of Philosophers Based on Scopus Data“. See also, “Where Are They Now? The Healy 2100” by Jonathan Weisberg at his site.

Method

I selected six representative general philosophy journals for analysis: two “top ranked” journals (Philosophical Review and Noûs), two mid-ranked journals (Canadian Journal of Philosophy and Pacific Philosophical Quarterly), and two unranked but reputable journals (Philosophia and Southern Journal of Philosophy). I then downloaded the entire table of contents of these journals from the year 2012—giving a ten-year citation window—and excluded anything that wasn’t an ordinary full-length research article (e.g., book reviews, editors’ introductions, symposium proceedings). From each journal, I randomly selected 15 articles and noted their total number of citations in Google Scholar.

Results

Before you look at my analyses, any guesses?
Generally my reaction to analyses and comments of this sort is to think that they are considerably underestimating how often philosophy articles are actually cited. The “big data” interdisciplinary analyses often use methods that are a terrible fit for philosophy, such as looking at citations in the past two years. (Often in philosophy it takes two years or more to write and publish an article.) Also, I wonder what counts as an “article”. If we’re including two-page book reviews, it’s little wonder that they’d be little cited, and similarly if we’re including publications in predatory or obscure journals. What would be more interesting to know—and what I think most people in this discussion really care about—is how frequently cited are full-length research articles in “respectable” mainstream philosophy journals?

Histogram of total citations by article for articles published in 2012 from six representative general philosophy journals.

You might wonder if the articles that were cited just a few times were really just cases of self-citation (an author citing their own earlier work, rather than citation by another author); but this was not the case. Although there was naturally a bit of self-citation, most citations were by other authors even for the articles with just a few citations.

You might wonder whether the articles mostly gather citations in the first several years, with citation rates falling off substantially by year ten. I didn’t collect these data systematically, but a sampling of articles finds generally that about half of the citations are since 2018, with no substantial decline at the end. Presumably, then, these articles will continue to gather citations post-2022. I would estimate that these articles have collected only half of the citations that they will eventually collect, and perhaps they will collect many more if there is a long temporal tail into the future. If we double our citation estimates to account for this, the mean article will receive 62 citations over its lifetime and the median article will receive 40 citations.

Since Google Scholar doesn’t tend to include books as sources of citation (though it does include books as targets of citation), we should probably bump these numbers up even more. Ballpark guess: between monographs and articles in anthologies, about one-third of all philosophy citations appear in books. If that’s right, actual citation rates would be about 50% higher.

Conclusions

(4) Citation skew is much less extreme than it could be. Another respect in which philosophy citation practices are not jackpot-or-nothing is revealed by the smallish differences between the means and medians. Of course the means are higher: there’s right skew in the data, a tendency for high-end outliers to pull up the mean. But it’s not like most articles are cited 0-5 times and a few are a cited hundreds of times. Authors can reasonably expect that a decent article in a decent journal will have at least a moderate impact on their subfield.
(2) The majority of articles are cited often enough to be “in the conversation”. While an outsider to academia might not think that ten citations is much impact, I submit that a more appropriate perspective is this: If your article is cited at least ten times, then it is having an impact on other specialists in your subfield. Your article isn’t falling into the void. It is part of the conversation, and other scholars are reacting to it. A journal article needn’t have a huge influence outside its subarea to be successful. The beauty of academic journals is that they host technical pieces that often can only be appreciated by a few dozen specialists. If academia is worthwhile, then continuing those specialists’ conversations is worthwhile, and high citation rates for technical pieces should not be expected. (I intend this ten citation criterion as an approximately sufficient condition for impact, not a necessary condition.)
In the following guest post, Eric Schwitzgebel (UC Riverside) recounts what he found when, prompted by claims about how infrequently academic philosophy articles are cited, he looked at the citation rates of articles published in a few journals a decade ago.
You hear terrible things. You hear, for instance, that over 50% of philosophy articles are entirely uncited. You hear that the average philosophy article is cited less than 5 times. People will sometimes say things like the “median number of readers for a philosophy article is 1” or “there’s not much difference between publishing a paper and throwing it away“. Both of the last two comments appear in a recent Twitter thread launched by Helen De Cruz. It was reading this thread that inspired me to do the analyses I’ll share here.
Overall, the mean article was cited 31 times, and the median number of citations was 20. For the elite journals the mean was 50 and the median was 36; for the mid-ranked journals the mean was 25 and the median was 8; for the unranked journals, the mean was 17 and the median was 12. Only three of the 90 articles (3%) were cited zero times.

(1) Very few articles published in mainstream general philosophy journals remain entirely uncited. My estimate of 3% might be off, due to the smallish sample size and the particular journals selected; but it’s highly unlikely that non-citation rates of mainstream general philosophy articles are anything near 50%.