Systematic reviews and meta-analyses form the basis for healthcare and health policy, but access to the data on which the results are based is often lacking. This is shown in a new policy forum article from Karolinska Institutet published in the journal PLOS Medicine.
Systematic reviews and meta-analyses summarize the results of many previous studies and are used, for example, when developing healthcare guidelines. In a new policy forum article, researchers analyze how a lack of access to underlying data can affect the reliability of these compilations.
The researchers describe how meta-analyses are often forced to rely on incomplete summaries of results rather than raw data. This can lead to assumptions and calculations that are difficult to verify retrospectively, and which in some cases can influence the conclusions. The problem is common despite the fact that many journals and research funders require data to be shared. Previous studies show that only a small proportion of researchers who promise to share their data actually do so.
”When we don’t have access to the data on which the analyses are based, it becomes difficult to review, replicate, and fully understand the results. This can ultimately influence decisions in both healthcare and health policy,” says lead author Saul Martin Rodriguez, a researcher at the Department of Laboratory Medicine at Karolinska Institutet.
The article also highlights historical examples where a lack of access to data contributed to misleading conclusions, including regarding hormone therapy during menopause. Only when detailed patient data became available could the risks of the treatment be clarified, leading to revised guidelines.
The researchers argue that clearer and more controlled requirements for data sharing are needed, adapted to legal and ethical conditions. They emphasize that transparency is not just a technical issue but also concerns research culture and responsibility.
“Increased transparency can strengthen trust in research and reduce the risk of erroneous conclusions having a major impact,” says Saul Martin Rodriguez.
The article was developed in collaboration with Professor David Moher, director of the Centre for Journalology, Canada. See the article for research funders and potential conflicts of interest.