Show Notes
In this episode, we will discuss multiverse analysis–what it is, why we use it, and how we should use it. We also discuss many analyst projects, and Daniel's new preprint on multiverse analysis.
Shownotes
Auspurg, K. (2025). Robustness is better assessed with a few thoughtful models than with billions of regressions. Proceedings of the National Academy of Sciences, 122(43), e2521917122. Chamberlin, T. C. (1890). The method of multiple working hypotheses. Science, (366), 92–96. Dochtermann, N. A., & Jenkins, S. H. (2011). Developing multiple hypotheses in behavioral ecology. Behavioral Ecology and Sociobiology, 65(1), 37–45. Dreber, A., & Johannesson, M. (2025). The credibility gap: Evaluating and improving empirical research in the social sciences. Routledge. Gelman, A., & Loken, E. (2013). The garden of forking paths: Why multiple comparisons can be a problem, even when there is no “fishing expedition” or “p-hacking” and the research hypothesis was posited ahead of time. Department of Statistics, Columbia University, 348(1-17), 3. Knief, U., & Forstmeier, W. (2025). Are our 95% CIs only worth 45% confidence? bioRxiv, 2025-08. Lakens, D., Rasti, S., & Tunç, M. N. (2026). There is only one correct analysis. (Preprint) Platt, J. R. (1964). Strong Inference: Certain systematic methods of scientific thinking may produce much more rapid progress than others. Science, 146(3642), 347–353. Simonsohn, U., Simmons, J. P., & Nelson, L. D. (2020). Specification curve analysis. Nature Human Behaviour, 4(11), 1208–1214. Steegen, S., Tuerlinckx, F., Gelman, A., & Vanpaemel, W. (2016). Increasing transparency through a multiverse analysis. Perspectives on Psychological Science, 11(5), 702–712. Wimsatt, W. C. (2007). Re-engineering philosophy for limited beings: Piecewise approximations to reality. Harvard University Press.
More from Nullius in Verba
In today’s episode, we discuss the classic 1959 Rede Lecture by scientist and novelist C. P. Snow. We reflect on whether there is still a divide between the humanities and the sciences, how important it is to be educated in both, whether the humanities and the sciences care just as much about the truth, whether scientists need to know Shakespeare, and what the second law of thermodynamics is about. Enjoy!
In this two-part episode, we examine scientific codes of conduct, using the Netherlands Code of Conduct for Research Integrity as our primary example. We discuss why such codes are necessary, where their guiding principles come from, and how (and whether) they are enforced. We also discuss many of the standards of good research practice derived from said principles.
In this two-part episode, we examine scientific codes of conduct, using the Netherlands Code of Conduct for Research Integrity as our primary example. We discuss why such codes are necessary, where their guiding principles come from, and how (and whether) they are enforced.
In this episode, we will reflect on how governments should support science, from the 1945 policy report Science: The Endless Frontier by Vannevar Bush, and then move on to the recent policy document by the Trump Administration 'Science: A New Golden Age'. We reflect on what we like about the policy report and what we like less. Is the linear model from basic research to application indeed outdated? Are metascience units a good idea? And will AI accelerate and radically transform how we do science? Enjoy. Shownotes:
After a short break due to busy calendars, we are back to discuss the role automation and artificial intelligence can play in science. Is AI evil, or does it just amplify the problems that were already present in science? Can automated AI driven checks help scientists to improve the way they work? If so, where are these tools helpful, and when? Enjoy. The Babbage quote is from Babbage in November 1839, recalling events in 1821; quoted in Harry Wilmot Buxton and Anthony Hyman (1988), Memoir of the Life and Labours of the Late Charles Babbage.