Sam Bowman (and others) have made an excellent radio documentary on why the pace of scientific innovation has slowed, and what we can do to fix it. The whole thing is well worth listening to; these are some notes I’ve made on the bits I want to make sure I remember.
Ideas are becoming harder to find, research suggests; Nicholas Bloom’s 2020 paper on this found that “research productivity is declining sharply”. Moore’s Law proves to be a good example of this: “the number of researchers required to double chip density today is more than 18 times larger than the number required in the early 1970s”. Not good!
“Eroom’s Law” is a clever term for the problem plaguing drug development; Jack Scannell coined the term to refer to his finding that “the number of new drugs approved per billion US dollars spent on R&D has halved roughly every 9 years since 1950”. (“Eroom” is “Moore” backwards.) In the show, Scannell blames this in part on older drugs going off-patent and becoming generics: “You’ve got an ever improving back catalog of almost free medicine against which new medicines have to compete.”
Corporate research isn’t very good at developing truly revolutionary ideas, as those are often hard to monetise (the Internet is a good example). And so it’s concerning that government R&D, which can be quite good at funding those sorts of ideas, is falling as a share of overall R&D.
Scientific journals aren’t well-suited to encouraging revolutionary ideas, either. Margit Osterloh says that peer reviewers “agree more on scientific work which is garbage”, with more unorthodox (and revolutionary) ideas garnering more disagreement. As papers need multiple peer reviewers’ support to be published, that’s a problem. Osterloh thinks that introducing random selection could help solve the problem. She says that in the 18th Century, the University of Basel increased its reputation by randomly selecting professors. Today, some funding bodies (including New Zealand’s health research council and Switzerland’s national science foundation) are experimenting with similar methods. (As a big believer in the benefits of randomisation I find this very exciting!).
Again, the whole programme is worth listening to, you can find it here.