Philosophy Majors & High Standardized Test Scores: Not Just Correlation (guest post)

Contributions versus the ACT
I don’t have the money to perform any of the high-quality research I’ve outlined. Maybe the APA, at least, should invest some money running some versions of some of these studies. Given what I’ve presented here, there’s reason for some optimism about the results. From what I can tell, philosophy programs actually make their students smarter in measurable ways.

Moreover, of non-STEM fields, the most-common major-switchers are those who switch out of education. We know empirically (see table below) that intended education-majors are among the worst performers on the SAT and ACT, and so if people are switching out of an initially-declared education major into philosophy or religion, that would drive the performance of philosophy and religion majors down, not up.

Philosophy Majors & High Standardized Test Scores: Not Just Correlation
by Thomas Metcalf

Reply to Objection 3: Presumably those who are interested in other fields as SAT-takers or ACT-takers are similarly primed, then, to get the most out of their fields. Yet they don’t get as much of a benefit.
The second row is the average contribution across all ten of the previously listed majors. For example, on average, studying one of those majors makes you drop 2.64 percentage points from the average GMAT score versus what your ACT would have predicted. Maybe the average among the listed intended majors is to do 4% better than average across all test-takers on the ACT but 6.64% worse than the average across all test-takers on the GMAT, or something like that. (Why would the average among these very common majors be negative? My guess is that “undecided” intended majors, and intended majors in esoteric fields, are generally more academically skilled than the average student is.)
Objection 4: “But maybe philosophy majors respond to teaching better than the average student does.”

Complications and Limitations

Reply 2 to Objection 1: Yet suppose we assume that all students who intend to major in X are only 56% likely to become actual X-majors. Even then, we can appeal to an analogue of Condorcet’s Jury Theorem: as long as reliability is at-least-somewhat more than 50%, and the sample is large enough, random noise will partially cancel out in the verdict. About 1.5 million people took the GRE in the period surveyed. And as noted, people are about 56% likely in general to stick with their intended major, at least when they enroll.

Some sources lump together philosophy and religion, or philosophy and theology, etc., while others separate them. The same goes for other groups of majors. When I had to create an average for a combined program from two individual programs, I weighted it by population. For example, if there were 200 chemistry majors and 100 physics majors, and the chemistry majors average a score of 10 and the physics majors average a score of 8, I considered the “chemistry and physics” average to be (200 * 10 + 100 * 8) / (200 + 100), viz. 9.3.
The ACT, SAT, GRE, LSAT, and GMAT don’t try to measure exactly the same traits. The GRE, SAT, and ACT are fairly similar, at least. Where available and appropriate, I broke scores down by subject areas or subscores.
Some of the data come from different years. None of the ACT, SAT, GRE, and LSAT data are older than 2016. The ACT and SAT data are from 2016; the GRE data are from 2017-20; the LSAT and law-school acceptance rate data are from 2018-20. Thus, fortunately, the ACT and SAT data are a few years before the postgraduate tests, suggesting that the cohorts taking the pre-graduate tests are roughly around the same cohorts as those taking the post-graduate tests. The GMAT data are from 2001-05. The medical school acceptance-rate data are from 1998. The reason I have to use older data is that newer data cost money. For example, the American Association of Medical Colleges, which oversees the MCAT, wanted $500 to tell me the breakdown by major. So I just went with an older source about medical school acceptance rates by major. Go ahead and take that with a grain of salt. The source on the GMAT I found seems to cite reliable data, but I also don’t have the money to get up-to-date data directly from the GMAC.
In some cases, not all majors are available to measure. Fortunately, I was able to get data for most of the most-common majors, viz. arts (visual and performing); biological sciences; business; communications and journalism; education; engineering; English; English plus foreign languages; foreign languages; health sciences; legal studies; philosophy; philosophy plus religion and theology; physical sciences; biological plus physical sciences; social sciences; and social sciences plus law. Of course, I’ll label which are which below.
As noted, philosophy and religion are normally combined. Where we have separate data, philosophy usually slightly outperforms religion. So if philosophy-and-religion outperforms some average, then, a fortiori, philosophy probably outperforms that average. In any case, in what’s below, I’ll sometimes say “philosophy and religion” when I mean “philosophy and religion” or “philosophy and religious studies” or “philosophy and theology” or “philosophy, religious studies, and theology,” etc. It’s some combination of philosophy and theology or religious studies, unless I just say “philosophy” simpliciter, which excludes the other fields.
It’s worth adding that philosophy doesn’t greatly outperform religion on most of these. For example, in fine-grained major-lists, mainline “Philosophy” averages 160 Verbal, 154 Quantitative, and 4.4 Writing on the GRE. “Religion/Religious Studies” averages 158 Verbal, 151 Quantitative, and 4.3 Writing.
More generally, the results aren’t as probative with GMAT and med-school admissions, because certain majors are probably very over-represented among test-takers. Lots of different majors take the GRE, and a fair number take the LSAT, but presumably GMAT takers are dominated by business majors. So what looks like “How well do philosophy or religion majors do against the average GMAT takers?” is really more like “How well do philosophy or religion majors do on the GMAT against business majors?” Still, that sort of complication should be blunted because we’re still comparing philosophy and religion to a bunch of other non-business major fields of study. That is, if it were only philosophy and religion versus business, that would be one thing, but I also list the average contributions of several non-business majors too, and philosophy and religion still do very well versus the average test-taker (i.e., mostly business majors) than the other non-philosophy, non-religion, non-business majors do against the average test-taker (i.e., again, mostly business majors).
There are reasonable worries about bias in standardized tests. That’s a big issue, but at the very least, unless we think that some of these tests are more biased against some groups than others are, we can hope that the biases “cancel out.” That is, I’m not using standardized tests to predict how smart a student is; I’m using standardized tests to predict how good a student will be at (other, similar) standardized tests.
There are reasonable worries about the predictive power of these tests as well. This is an area of active research. The SAT and ACT, especially when combined with undergraduate GPA, show some predictive power about college success versus undergraduate GPA alone, and some of the allegedly debunking studies are arguably flawed. The GRE isn’t a great predictor of graduate school success overall, but sections of it do show some correlation with graduate-school success in the relevant subject area, especially when we set aside applied sciences–and applied sciences are a relatively small portion of what we’re trying to measure. The LSAT does provide some prediction of law-school success, though other factors provide better prediction. There is some positive relationship between GMAT scores and post-MBA pay, although it seems to be present at the holistic school level rather than among individual graduates. And, obviously, there is an association between being accepted to medical school and graduating from medical school. At the same time, it’s not obvious that people who tout the power of philosophy believe that philosophy’s main contributions are to graduate school success, law-school success, post-MBA pay, or medical school success.
We can also debate about whether the skills measured on these standardized tests are valuable in general.

Reply to Objection 4: If that’s itself a selection effect, then presumably we would have seen it reflected in ACT and SAT scores of intended philosophy majors, but we didn’t. If that’s a treatment effect, then philosophy is great: it produces students who respond better to teaching.

Results

Summary: Given what you’d expect from intended philosophy and religion majors’ SAT comprehensive scores, philosophy and religion majors do average on the LSAT, do well on law-school admission, and do extremely well on the GMAT and on med-school admission. (See Notes 3 and 4.)
I decided to collect standardized-test and admissions-rate data as best I could. But let me admit to some complications and limitations of my approach.
Really, what the data tell us is that it’s mathematics and natural-sciences students who switch majors (i.e. out of math and science) more than anyone else, and it’s STEM students who switch out because the courses were too difficult. But arguably, if that were inflating philosophy’s numbers, it would show up in the quantitative-reasoning numbers. As the data presented herein show, intended math-and-science majors are far better at the math section of the SAT than the intended philosophy-and-religion majors are. Yet a philosophy education only makes an average contribution to quantitative-reasoning scores.

Arts (visual and performing)
Business;
Communications;
Education;
Engineering;
English and foreign languages;
Health sciences;
Philosophy and religion;
Sciences (biological and physical); and
Social sciences and law;

Really, what the data tell us is that it’s mathematics and natural-sciences students who switch majors (i.e. out of math and science) more than anyone else, and it’s STEM students who switch out because the courses were too difficult. But arguably, if that were inflating philosophy’s numbers, it would show up in the quantitative-reasoning numbers. As the data presented herein show, intended math-and-science majors are far better at the math section of the SAT than the intended philosophy-and-religion majors are. Yet a philosophy education only makes an average contribution to quantitative-reasoning scores.

Arts (visual and performing)
Business;
Communications;
Education;
Engineering;
English;
Foreign languages;
Health sciences;
Philosophy, religion, and theology;
Sciences (biological);
Sciences (physical); and
Social sciences and law;

Arts (visual and performing)
Business;
Communications;
Education;
Engineering;
English and foreign languages;
Health sciences;
Philosophy and religion;
Sciences (biological and physical); and
Social sciences and law;

One common view says that the ACT is kinder to writing-oriented students and the SAT is kinder to math-oriented students. I don’t know whether that’s true, but if so, then the enormous gain to philosophy majors in the verbal and writing sections of the GRE versus the SAT is even more impressive.

Program	LSAT 2018-20 vs. SAT C	Law school 2018-20 vs SAT C	GMAT 2008-12 vs SAT C	Med school 1998 vs SAT C
P&R graduates vs. predictions from SAT scores	-2.41	1.74	+4.12	+28.20
Average contribution across 10 majors	-2.16	-2.38	-2.67	+14.18
P&R vs. avg contribution across 10 majors	-0.25	+4.11	+6.79	+14.02
P&R rank out of 10	5	3	1	1

Objection 5: “Okay, but in the end, do we really want to tell lots more people to major in philosophy, just so they can get these dubious benefits of high performance on standardized tests?”

Discussion So Far

Counter-Reply: “But wait, by claiming there’s diminishing marginal return to studying philosophy, are you really saying you learned a lot more in your first four years of philosophy (i.e., undergrad philosophy) than you learned in your next four (i.e., in grad school)?”
Most of us have seen the data: philosophy majors perform very well on the GRE, the LSAT, the MCAT, the GMAT, and on acceptance rates to postgraduate study. Daily Nous itself helpfully collects some of the data. But as critics such as Bryan Caplan, Jason Brennan, and Phillip Magness have recently observed, it’s not clear how much of this result is a treatment effect and how much is a selection effect. Perhaps earning a degree in philosophy actually makes you a lot smarter. But perhaps it was only the smart students who decided to major in philosophy in the first place.
Contributions versus SAT Comprehensive Scores
Some minor potential objections are already addressed in the “Complications and Limitations” section above. Here are some of the strongest objections.

Objections and Replies

In the set of major fields of study comprising:
philosophy and religion or philosophy, religion, and theology (“P&R”) contribute the following values of percentage-points, and ranks, versus the average scores across all ten majors:
Given what we would predict from SAT comprehensive and subscore results, philosophy and religion make an average contribution to GRE Verbal Reasoning and Quantitative Reasoning, a huge contribution to GRE Writing, a large contribution to law-school admissions, a huge contribution to GMAT scores, and a huge contribution to med-school acceptance rate. (See Complications and Limitations above, of course.)
Contributions versus SAT Subscores
In the set of major fields of study comprising:
As noted, to get better data, we would need to administer something like the GRE to a large cross-section of first-year students, and then again to those graduating seniors, and see which majors or which selections of courses made the largest contribution. Another useful project would be to construct an analogue of the GRE that attempts to measure the main skills that are normally included in “general-education requirements” and administer that to all students as first-years and as graduates, trying to derive a correlation between the raw number of philosophy courses taken (regardless of major) and performance on this test. It would also be nice to administer a test to first-years and that test again when they’re seniors or graduates, and track these students longitudinally, associating their improvement as graduates versus first-years with how many courses from each discipline they took during their career. I think this is something the Department of Education has reason and the resources to do; one would hope that they’d be interested in the effects of taking different sorts of college courses. (Someone might worry that the results would be used to cut certain departments’ budgets. I take that seriously, although I also think that if high-quality research shows that philosophy is useless with respect to some goal, then we should at least stop requiring or encouraging philosophy as a way of satisfying that goal.)
Granted, people tend to switch into humanities because they “enjoy” or are “interested in” the courses, while people tend to switch into non-humanities majors more often because they expect more pay or to contribute more to society. If enjoyment is a proxy for ease, then perhaps students who are very good at humanities are switching into humanities, while students who aren’t good at humanities are switching out. But that would be more likely to inflate the performance of the students in their undergraduate GPAs, not necessarily on the GRE and so on; the course content in humanities courses isn’t generally identical to GRE or LSAT content. Put another way, if students just enjoyed GRE content, then their switching into some major would inflate that major’s GRE scores. But students don’t report switching into humanities because those students enjoy GRE content.

Is there a way to identify whether, and if so to what extent, studying philosophy helps students do better on these measures of academic skill? Thomas Metcalf, associate professor of philosophy at Spring Hill College, thinks there is, and gave it a try. He explains his findings in the following guest post*.
Counter-Reply: “But maybe it was the intended math-and-science majors who were bad at math who switched out of those majors and into philosophy.”
The first row of data is the average (mean) contribution that actually getting a degree in philosophy or religion makes, in percentage points, to postgraduate tests, versus what we would expect from intended philosophy majors’ ACT scores. For example, studying philosophy and religion makes you score 19.09 percentage points higher versus the average. If intended philosophy-and-religion majors score 2% worse than the average ACT score, and they score 17.09% better than the average GRE Writing score, then the average contribution of studying philosophy is 19.09 percentage points.
We know that students enroll in humanities majors overwhelmingly because they enjoy those courses, and secondarily because they do well in those courses. In humanities relative to other majors, the latter is far more important. Thus, if we take humanities as a whole, this is further evidence that switching into humanities majors because the other major was too easy doesn’t occur, and switching out because the humanities major was too difficult doesn’t occur. Major switching mainly occurs because students enjoy the courses in their new major more, and partly also because they expect the new major to contribute to society. Humanities as a whole has the highest percentage of students switching in because they do well in those courses; if anything, this suggests that students who aren’t as academically skilled are switching into those courses.
The data we have suggest that a philosophy education makes an average contribution to verbal reasoning and to math skills, and a large contribution to analytic-writing skills and to the GMAT-relevant skills: chiefly writing and reasoning. The analytical-writing section of the GRE matches intended philosophy-education outcomes well. Interestingly, three of the GMAT’s four sections match the standardly advertised philosophy-education skills.
In the set of major fields of study comprising:
Given what we would predict from ACT comprehensive-score results, philosophy and religion make a much-better-than-average contribution to all these tests and admission rates, including a huge contribution to GRE Writing and to the GMAT.
Counter-reply: “Still, there’s such a low number of philosophy majors at most colleges that it’s not clear that a lot of good is being generated by having philosophy departments, at least in the area of general academic skills.”

Given the magnitude of some of these effects, and philosophy majors’ similar performance throughout the recent past (my GRE data are 2017-20 but similar data are available from e.g. 2015-18) I think “chance” is a poor explanation for philosophy majors’ overall performance on standardized tests and admissions. I think the two best explanations are “mostly a selection effect” (i.e., people who are already academically skilled choose to major in philosophy) and “mostly a treatment effect” (i.e., a philosophy education makes people a lot more academically skilled, at least in writing, law-school-relevant reasoning, GMAT-relevant reasoning, and maybe med-school-relevant qualifications). But, crucially, given that intended philosophy-and-religion majors aren’t that impressive relative to other intended majors, this is prima facie evidence that a substantial portion of the effect is treatment. In all likelihood, studying philosophy confers a non-negligible benefit to your GRE-Writing-related and your GMAT-related academic skills, and more of a benefit than most other majors would have conferred.
[Art: ceramic chain sculpture by Cecil Kemperink]
Counter-Reply: “But high-school courses don’t really teach the material on the SAT and ACT.”
Reply to Objection 5: Well, it’s not just standardized tests: it’s admission rates as well. But yes, the opportunity cost of generating lots of philosophy majors might be high. I’m not claiming here that the GRE etc. measure anything valuable or worthwhile in terms of, e.g., the productivity of society or the expected, total, net hedons generated per hour of philosophy instruction. That’s a much larger conversation. But my results should give administrators strong reason to provisionally maintain the existence of philosophy departments (and perhaps to include philosophy courses among general-education requirements or core curricula), at least until better data are available.
philosophy and religion or philosophy, religion, and theology (“P&R”) contribute the following values of percentage-points, and ranks, versus the average scores across all twelve majors:
You’ve probably heard that philosophy majors do well on standardized tests for admission to graduate and professional programs, such as the GRE, LSAT, MCATs, and GMATS. You’ve probably also heard the warning that correlation is not causation.
Reply to Counter-Reply: No, but I studied way more philosophy by hours expended in grad school than I did in undergrad, and I studied it among much-better philosophers as my peers (viz., other grad students) than my peers in undergrad were. I also probably learned more philosophy in grad school, while I probably learned more general academic skills in undergrad philosophy courses. (My teachers in both undergrad at UW Seattle and in grad school at CU Boulder were excellent, by the way.)
The fourth row is the rank for philosophy and religion out of those ten majors by the magnitude of its contribution. As you can see, philosophy and religion are the best out of the ten for the GRE Writing section, the GMAT, and the med-school admission rate. They’re very good everywhere else.
How to read this table and subsequent tables: The columns represent the contribution to the verbal GRE score, the quantitative-reasoning GRE score, the GRE writing score, the LSAT, the law-school admission rate, the GMAT and the med-school admission rates.
philosophy and religion or philosophy, religion, and theology (“P&R”) contribute the following values of percentage-points, and ranks, versus the average scores across all ten majors (see explanation below table on how to read it and subsequent tables; see the Appendix for a table comprising all those majors):

Provisional Conclusion

In this section, I’ll compare the “contributions” each major makes to each of the postgraduate examinations and admission rates. I’ll list the majors under consideration in the particular topic as well as philosophy-and-religion’s rank. For the raw data, see the Appendix.
Reply 1 to Objection 1: Yes, intended philosophy-and-religion majors are of lower than average likelihood of declaring the major that they claim to intend. Part of that effect may also be that the just-linked report included two-year colleges, and more generally, that many colleges don’t offer majors in philosophy and religion anyway. Thus the consistency between intending to major in philosophy or religion and actually majoring in philosophy or religion at a college that allows one to graduate with a degree in philosophy or religion is probably higher.
Reply to Counter-Reply: By presumably the same reasoning, undergraduate courses, then, don’t really teach the material on the GRE or LSAT.
Summary: Given what you’d expect from intended philosophy or religion majors’ ACT scores, actual philosophy or religion graduates do extremely well on the GRE Writing section, on the GMAT, and in medical-school admissions. (See Notes 1 and 2.)
NOTES

For GMAT data, replace “visual and performing arts” with “fine arts,” “communications” with “journalism,” “health sciences” with “medicine/nursing”; and “physical sciences” with “the average of biology, chemistry, and physics.” The latter is not weighted by population, since I didn’t have those data available. Still, since physics tends to outperform biology and chemistry in these sorts of tests, yet comprise far fewer majors, this helps “physical sciences” versus the average and correspondingly harms “philosophy and religion” in relative terms.
For med-school admission rate, replace “health sciences” with “the average of the scores of biology and biochemistry,” replace “physical sciences” with “the average of the scores of chemistry and physics,” replace “English and foreign languages” with “English,” replace “social sciences” with “history,” and replace “philosophy and religion” with “philosophy.” I grant that this may distort the results, so I don’t put a lot of weight on these data. Also, not all of these ten majors are represented in the med-school admission-rate data; see the full table in the Appendix. So it’s not a rank “out of 10” here.
For GMAT data, replace “visual and performing arts” with “fine arts,” “communications” with “journalism,” “health sciences” with “medicine/nursing”; and “physical sciences” with “the average of biology, chemistry, and physics.” The latter is not weighted by population, since I didn’t have those data available. Still, since physics tends to outperform biology and chemistry in these sorts of tests, yet comprise far fewer majors, this helps “physical sciences” versus the average and correspondingly harms “philosophy and religion” in relative terms.
For med-school admission rate, replace “health sciences” with “the average of the scores of biology and biochemistry,” replace “physical sciences” with “the average of the scores of chemistry and physics,” replace “English and foreign languages” with “English,” replace “social sciences” with “history,” and replace “philosophy and religion” with “philosophy.” I grant that this may distort the results, so I don’t put a lot of weight on these data.

Reply to Counter-Reply: My guess, although I don’t have data to back it up, is that training has diminishing marginal return. I would guess that you improve a lot more in the first 20 hours of studying the trombone than you do in the second 20 hours. So including philosophy among general-education requirements might do a lot of good. But if most of the improvement comes in later philosophy courses, maybe colleges should require lots of philosophy.