How to assemble science

Every day, humanity produces over 19,000 new scientific articles, totaling around 7 million per year in 2025. This vast output includes rigorous research, contradictory findings, and even flawed studies. The sheer volume makes it difficult to determine what is reliable or significant. For example, conflicting news headlines about health topics like coffee or red wine often stem from isolated studies rather than comprehensive reviews of all available evidence. This problem is compounded by systemic issues: scientists are incentivized to publish frequently, journalists prioritize sensational findings, and few efforts are made to synthesize existing knowledge. The result is a fragmented landscape where important insights can be overlooked or misrepresented, leaving the public confused about what to believe.

A single study, even if flawed, can create widespread confusion or fear when reported in isolation. For instance, in September 2025, the US Food and Drug Administration (FDA) announced a warning label for the painkiller acetaminophen (also known as paracetamol), suggesting it increased autism risk during pregnancy. This claim made global headlines, but subsequent research revealed it was based on cherry-picked data. A major review published months later concluded that existing evidence did not support the link. By then, many women had unnecessarily avoided a medication widely considered safe for pregnant women. This example highlights how isolated studies, when amplified by media, can cause real-world harm by distorting public understanding of scientific consensus.

In 1976, statistician Gene Glass introduced a groundbreaking method called meta-analysis to synthesize findings from multiple studies. Glass developed this technique after noticing flaws in how research was reviewed, such as vote-counting, where studies were tallied without considering their size or quality. For example, a study with 85 out of 100 patients improving would be counted the same as one with 55 out of 100, ignoring the strength of the effect. Glass and his wife, psychologist Mary Lee Smith, analyzed over 370 studies on psychotherapy and converted their varied results into a standard measure called effect size. This allowed them to average the findings, proving psychotherapy had a beneficial impact. The method was so influential that it earned its own field of study.

A systematic review is a rigorous process for gathering and evaluating all available evidence on a specific question. Researchers search global databases for published and unpublished studies, then filter thousands down to the most relevant ones. They assess the quality of each study, extract data, and often combine results using meta-analysis to estimate overall effects. This approach emerged alongside meta-analysis in the 1970s and 1980s. For example, in 1991, Iain Chalmers used systematic reviews to show that many standard medical practices in pregnancy and childbirth lacked evidence and could even be harmful. His work laid the foundation for the Cochrane Collaboration, an organization dedicated to producing high-quality systematic reviews in healthcare.

Founded in 1993 by Iain Chalmers and colleagues, the Cochrane Collaboration became a cornerstone of evidence-based medicine. By 2010, researchers published 11 systematic reviews on healthcare every day, totaling around 2,000 reviews within a decade. These reviews inform clinical guidelines that doctors use daily, making them an invisible but essential part of modern medicine. For instance, systematic reviews helped debunk harmful practices like shaving women’s pubic hair during labor and performing routine episiotomies. Without such syntheses, outdated or dangerous advice might persist unchallenged, as seen in the tragic case of babies sleeping on their stomachs, which contributed to thousands of sudden infant death syndrome (SIDS) cases before evidence was synthesized in 2005.

Systematic reviews are not limited to medicine. Other fields have adopted similar approaches to synthesize evidence and guide decisions. For example, the International Initiative for Impact Evaluation holds over 1,700 systematic reviews on international development topics like sanitation and malnutrition. The Education Endowment Foundation in London created a teaching and learning toolkit using systematic reviews, which is now used by 70% of school leaders in England to make spending decisions. This toolkit shows that teaching metacognition—the ability to learn how to learn—can add eight months of learning progress per year, while repeating a school year may set students back by two months. Similarly, environmental organizations like CEEDER host over 2,100 evidence reviews on conservation strategies.

Climate scientists have long used evidence synthesis, with the Intergovernmental Panel on Climate Change (IPCC) publishing six major assessments since 1988. These reports helped establish that humans are causing climate change by synthesizing vast amounts of research. However, they have focused less on solutions. In 2024, a group called What Works Climate Solutions launched an effort to create a Cochrane-like bank of evidence syntheses on effective climate policies. This project aims to provide governments with data on which strategies—such as cutting emissions or adapting to warming—work best. The findings will feed into the next IPCC assessment, due in late 2029, potentially guiding global policy decisions.