CAMBRIDGE, MA–December 16, 2020–In 2019, the National Academies of Science, Engineering, and Medicine (NASEM) published a consensus report for the US Congress–Reproducibility and Replicability in Science–which addressed a major methodological crisis in the sciences: The fact that many experiments and results are difficult or impossible to reproduce. The conversation about this report and this vital topic continues in a special, twelve-article feature in issue 2:4 of the Harvard Data Science Review (HDSR), publishing today.
Growing awareness of the replication crisis has rocked the fields of medicine and psychology, in particular, where famous experiments and influential findings have been cast into doubt. But these issues affect researchers in a wide range of disciplines–from economics to particle physics to climate science–and addressing them requires an interdisciplinary approach.
“The overall aim of reproducibility and replicability is to ensure that our research findings are reliable,” states HDSR Editor-in-Chief Xiao-li Meng in his editorial. “Reliability does not imply absolute truth–which is an epistemologically debatable notion to start with–but it does require that our findings are reasonably robust to the relevant data or methods we employ.”
“Designing sound replication studies requires a host of data science skills, from statistical designs to causal inference to signal-noise separation, that are simultaneously tailored by and aimed at substantive understanding,” Meng continues.
Guest edited by Victoria Stodden (University of Illinois, Urbana-Champaign), the special theme collection presents research and commentary from an interdisciplinary group of scholars and professionals. Articles include:
- Interview with Reproducibility and Replicability in Science Committee Chair Harvey V. Fineberg, President of the Gordon and Betty Moore Foundation and HDSR guest editor, Victoria Stodden, committee member by HDSR Editor-in-Chief Xiao-Li Meng
- “Self-Correction by Design“ by Marcia McNutt, President of NAS
- “Leveraging the National Academies ‘Reproducibility and Replication in Science’ Report to Advance Reproducibility in Publishing“ by Manish Parasha, Assistant Director for Strategic Computing at the White House Office of Science and Technology Policy, and Director of the Office of Advanced Cyberinfrastructure at the National Science Foundation
- “Toward Reproducible and Extensible Research: from Values to Action“ by Aleksandrina Goeva (Broad Institute), Sara Stoudt (Smith College), Ana Trisovic (Harvard University)
- “Reproducibility and Replicability in Economics” by Lars Vilhuber (Cornell University)
- “Reproducibility and Replicability in Science, A Metrology Perspective“ by Anne L. Plant (National Institute of Standards and Technology) and Robert J. Hanisch (National Institute of Standards and Technology)
- “Perspectives on Data Reproducibility and Replicability in Paleoclimate and Climate Science“ by Rosemary T. Bush (Northwestern University), Andrea Dutton (University of Wisconsin, Madison), Michael N. Evans (University of Maryland, College Park), Rich Loft (National Center for Atmospheric Research), and Gavin A. Schmidt (National Aeronautics and Space Administration)
- “Science Communication in the Context of Reproducibility and Replicability: How Non-Scientists Navigate Scientific Uncertainty“ by Emily Howell (University of Wisconsin-Madison)
- “Learning Lessons on Reproducibility and Replicability in Large Scale Genome-Wide Association Studies“ by Xihong Lin (Harvard University)
- “Selective Inference: The Silent Killer of Replicability“ by Yoav Benjamini (Tel Aviv University)
- “Trust but Verify: How to Leverage Policies, Workflows, and Infrastructure to Ensure Computational Reproducibility in Publication“ by Craig Willis (University of Illinois at Urbana-Champaign) and Victoria Stodden
- “Reproducibility and Replication of Experimental Particle Physics Results“ by Thomas R. Junk (Fermi National Accelerator Laboratory) and Louis Lyons (University of Oxford, Emeritus)
The editors hope to take advantage of the collaborative features available on the open-source publishing platform, PubPub, where HDSR is hosted. Readers around the world can freely read, annotate, and comment on the essays–continuing this important conversation.
###
The Harvard Data Science Initiative, launched in 2017, is a cross-University initiative working at the nexus of statistics, computer science, and related disciplines to gain insights from complex data in nearly every research domain. Those insights can be deployed to address issues ranging from global economics and inequality to targeted medical treatments, privacy and security, health and the environment, scientific discovery, education, and many more. While the collection and analysis of data has long held an important role in academic research, the Harvard Data Science Initiative strengthens, deepens, and expands this work by advancing methodologies, enabling breakthroughs, promoting new research collaborations, and enhancing Harvard’s educational mission. All of these efforts are rooted in an urgent desire to improve our world: how can we best use data for the common good?
The Harvard Data Science Review is published for the Harvard Data Science Initiative by the MIT Press. Established in 1962, the MIT Press (Cambridge, MA and London) is one of the largest and most distinguished university presses in the world and a leading publisher of books and journals at the intersection of science, technology, art, social science, and design. MIT Press books and journals are known for their intellectual daring, scholarly standards, interdisciplinary focus, and distinctive design. For almost 50 years the MIT Press journals division has been publishing journals that are at the leading edge of their field and launching new journals that have nurtured burgeoning areas of scholarship.
PubPub is an open-source publishing platform from the Knowledge Futures Group for collaboratively editing and publishing journals, monographs, and other open access scholarly content. The Knowledge Futures Group, a nonprofit originally founded as a partnership between the MIT Press and MIT Media Lab, builds and sustains technology for the production, curation, and preservation of knowledge in service of the public good.
Contact:
Amy Harris
Senior Marketing Manager
The MIT Press
[email protected]
(857) 331-4265