Science is getting harder
Like the rest of New Things Under the Sun, this article will be updated as the state of the academic literature evolves; you can read the latest version here.
Audio versions of this and other posts: Substack, Apple, Spotify, Google, Amazon, Stitcher.
One of the most famous recent papers in the economics of innovation is “Are Ideas Getting Harder to Find?” by Bloom, Jones, Van Reenen, and Webb. It showed that more and more R&D effort is necessary to sustain the present rates of technological progress, whether we are talking about Moore’s law, agricultural crop yields, healthcare, or other proxies for progress. Other papers that look into this issue have found similar results. While it is ambiguous whether the rate of technological progress is actually slowing down, it certainly seems to be getting harder and harder to keep up the pace.
What about in science?
A basket of indicators all seem to document a trend similar to what we see with technology. Even as the number of scientists and publications rises substantially, we do not appear to be seeing a concomitant rise in new discoveries that supplant older ones. Science is getting harder.
Before diving into these indicators, I want to head off one potential misunderstanding. My claim is that science is getting harder, in some sense, not that science is ending or that we are on the verge of running out of ideas. Instead, the claim is that discoveries of a given “size” are harder to bring about than in the past.
Share
Raw paper output
We’ll actually start with an indicator that shows no evidence of a slowdown though. Since scientists primarily communicate their discoveries via papers, the first place to look for evidence of increasing difficulty of making discoveries is in the number of papers scientists publish annually. The figure below, drawn from Dashun Wang and Albert-László Barabási’s (free!) book on the Science of Science compares publications to authors over the last century.
From Wang and Barabási (2021)
At left, we can see the number of papers and authors per year has increased basically in lockstep over the twentieth century. Note, the axis is a log-scale, so that a straight-line indicates exponential growth. Meanwhile, at right, the blue dashed line shows that the number of papers per author has hovered around 2 for a century and rather than falling, it is actually on the rise in recent decades. (As an aside, the solid red line at right is strong evidence for the rise of teams in science, discussed more here)
So absolutely no evidence that scientists are struggling to find stuff worth writing up. But that’s not definitive evidence, because scientists are strongly incentivized to publish and what constitutes a publishable discovery is whatever editors and peer reviewers think is publishable. If fewer big discoveries are made, scientists may just publish more papers on small discoveries. So let’s take a more critical look at the papers that get published and see if there are any indicators that they contain smaller discoveries than in the past.
Nobel Prizes
Let’s start by looking at some discoveries whose importance is universally acknowledged. The Nobel prize for discoveries in physics, chemistry, and medicine is one of the most prestigious scientific prizes and has a history long enough for us to see any long-run trends. Using a publicly available database on Nobel laureates by Li et al. (2019), we can identify the papers describing research that is eventually awarded a Nobel prize, and the year these papers were published. Note several papers might be associated with any given award. For each award year, we can then ask, what share of the papers related to the discovery were published in the preceding twenty years. The results of that are presented below, though I smooth the data by taking the ten-year moving average.
Share of papers describing Nobel-prize winning work, published in the preceding 20 years. 10-year moving average.Author calculations, based on data from Li et al. (2019).
Prior to the 1970s, on average 90% of the time, awards went to papers published in the last twenty years. But by 2015, the ten-year moving average was closer to 50%.
So recent discoveries seem to have a harder time getting recognized as Nobel-worthy, relative to a few decades ago. We can also compare the importance of different discoveries that won Nobel prizes. In 2018, Patrick Collison and Michael Nielsen asked physicists, chemists, and life scientists to pick the more important discovery (in their field) from sets of two Nobel prize winning discoveries. For example, they might ask a physicist to say which is more important, the discovery of Giant Magnetoresistance (awarded the Nobel in 2007) or the discovery of the Compton effect (awarded in 1927). For each decade, they look at the probability a randomly selected discovery made in that decade would be picked by their survey respondents over a randomly selected discovery made in another decade. The results are below:1
Probability a discovery in a given decade is rated more important than discovery in another decadeFrom Collison and Nielsen (2018)
A few points are notable from this exercise. First, physicists seem to think the quantum revolution of the 1910s-1930s was the best era for physics and it’s been broadly downhill since then. That’s certainly consistent with discoveries today being in a sense smaller than the ones of the past, at least for physics.
In contrast, for chemistry and physiology/medicine, the second half of the twentieth century has outperformed the first half. In the Nobel prize data, within the second half of the century, there is no obvious trend up or down for chemistry and medicine. While that’s better than physics, it remains consistent with the notion that science might be getting harder. As we can see in the first figure here, the number of papers and scientists rose substantially between 1950 and 1980, which naively implies that the number of candidates for Nobel-prize winning discoveries should also have risen substantially. If we are selecting the most important discovery from a bigger pool of candidates, we should expect that discovery to be judged more important than discoveries picked from smaller pools. But that doesn’t seem to be the case.
So Nobel prize data is also consistent with the idea that discoveries today aren’t what they used to be. Whereas it used to be quite common for work published in the preceding twenty years to be recognized for a Nobel, that doesn’t happen nearly so much today. That said, an alternative explanation is that the Nobel committee is just trying to work through an enormous backlog of Nobel-worthy work which they want to recognize before the discoverers die. In this explanation, we’ll eventually see just as many awards for the work of today.
But it’s not clear to me this is how the committee is actually thinking: recent work is awarded half the time still if the committee thinks the discovery is sufficiently important. For example, Jennifer Doudna and Emmanuelle Charpentier were awarded a Nobel for their work on CRISP-R in 2020, less than a decade after the main discoveries. And when you look specifically at the work performed in the 1980s, it doesn’t seem particularly notable, relative to work in the 40s, 50s, 60s, and 70s, despite the fact that many more papers were published in that decade.
Top Cited Papers
Still, perhaps the Nobel prize is simply too idiosyncratic for us to learn much from. Next, let’s look at another indicator of big discoveries, one which shouldn’t be biased by the sort of factors peculiar to the Nobel. This is the most top-cited papers in a given field. For example, if we look at the top 0.1% most highly cited papers of all time in a particular field, we could ask how easy is it for a new paper to join their ranks. If that has fallen over time, then that’s further evidence that today’s papers aren’t making the same contributions as yesterday’s.
On the other hand though, we might think it should get harder and harder to climb to the top 0.1%, even if discoveries are not getting smaller. After all, if discoveries are of constant size, earlier works have more time to get citations; it may not be possible for later papers to catch up, even if they are just as good. But there are also some factors that lean in the opposite direction. First, if work is only cited when relevant, then newer work should have an easier time being relevant to newer papers. Since the number of new papers grows over time, that gives one advantage to the new; they can be tailored to a bigger audience, in some sense. Second, the most esteemed papers of all time may actually stop being cited at high rates, because their contributions become part of common knowledge: it is no longer necessary to cite Newton when talking about gravity, or even Watson and Crick when asserting DNA has a double-helix shape.
So let’s proceed with seeing if there has been any change in how easy or hard it is to become a top cited paper, noting that won’t be the last piece of evidence we look at.
The closest paper I know of that looks into this is Chu and Evans (2021), which looks at the probability of a new paper ever becoming one of the top 0.1% most cited, even for just one year. But this paper does not plot this probability against time, like the previous charts: instead, it plots this probability against the size of a field, measured by the number of papers published per year. In the scatterplot below, each point corresponds to a field in a year. On the horizontal axis is the number of papers published in the fields in that year and on the vertical axis the probability a paper in that field and year is ever among the top 0.1% most cited. The colored lines are trends for each of these ten fields. Note this figure only includes papers published in the year 2000 or earlier. Since the analysis is conducted with data from 2014, every paper has more than a deca...