REPEAT (Reproducible Evidence: Practices to Enhance and Achieve Transparency)

“Replication is a cornerstone of the scientific method. Historically, public confidence in the validity of healthcare database research has been low. Drug regulators, patients, clinicians, and payers have been hesitant to trust evidence from databases due to high profile controversies with overturned and conflicting results. This has resulted in underuse of a potentially valuable source of real-world evidence.?…

Division of Phamacoepidemiology & Pharmacoeconomics [DoPE]

Brigham & Women’s Hospital and Harvard Medical School.”

Symposium: A critical analysis of the scientific reform movement

“As the science reform movement has gathered momentum to change research culture and behavior relating to openness, rigor, and reproducibility, so has the critical analysis of the reform efforts. This symposium includes five perspectives examining distinct aspects of the reform movement to illuminate and challenge underlying assumptions about the value and impact of changing practices, to identify potential unintended or counterproductive consequences, and to provide a meta perspective of metascience and open science. It’s meta, all the way up.

Each presenter will provide a 15-minute perspective followed by a concluding discussion among the panelists and a time to address audience questions. Visit cos.io/meta-meta to view session abstracts and speaker info.”

PsychOpen CAMA

“PsychOpen CAMA enables accessing meta-analytic datasets, reproducing meta-analyses and dynamically updating evidence from new primary studies collaboratively….

A CAMA (Community Augmented Meta Analysis) is an open repository for meta-analytic data, that provides meta-analytic analysis tools….

PsychOpen CAMA enables easy access and automated reproducibility of meta-analyses in psychology and related fields. This has several benefits for the research community:

Evidence can be kept updated by adding new studies published after the meta-analysis.
Researchers with special research questions can use subsets of the data or rerun meta-analyses using different moderators.
Flexible analyses with the datasets enable the application of new statistical procedures or different graphical displays.
The cumulated evidence in the CAMA can be used to get a quick overview of existing research gaps. This may give an idea of which study designs or moderators may be especially interesting for future studies to use limited resources for research in a way to enhance evidence.
Given existing meta-analytic evidence, the necessary sample size of future studies to detect an effect of a reasonable size can be estimated. Moreover, the effect of possible future studies on the results of the existing meta-analytic evidence can be simulated.
PsychOpen CAMA offers tutorials to better understand the reasoning behind meta-analyses and to learn the basic steps of conducting a meta-analysis to empower other researchers to contribute to our project for the benefit of the research community….”

 

 

Principles of open, transparent and reproducible science in author guidelines of sleep research and chronobiology journals

Abstract:  Background: “Open science” is an umbrella term describing various aspects of transparent and open science practices. The adoption of practices at different levels of the scientific process (e.g., individual researchers, laboratories, institutions) has been rapidly changing the scientific research landscape in the past years, but their uptake differs from discipline to discipline. Here, we asked to what extent journals in the field of sleep research and chronobiology encourage or even require following transparent and open science principles in their author guidelines.

Methods: We scored the author guidelines of a comprehensive set of 27 sleep and chronobiology journals, including the major outlets in the field, using the standardised Transparency and Openness (TOP) Factor. The TOP Factor is a quantitative summary of the extent to which journals encourage or require following various aspects of open science, including data citation, data transparency, analysis code transparency, materials transparency, design and analysis guidelines, study pre-registration, analysis plan pre-registration, replication, registered reports, and the use of open science badges.

Results: Across the 27 journals, we find low values on the TOP Factor (median [25 th, 75 th percentile] 3 [1, 3], min. 0, max. 9, out of a total possible score of 29) in sleep research and chronobiology journals.

Conclusions: Our findings suggest an opportunity for sleep research and chronobiology journals to further support recent developments in transparent and open science by implementing transparency and openness principles in their author guidelines.

Why they shared: recovering early arguments for sharing social scientific data | Science in Context | Cambridge Core

Abstract:  Most social scientists today think of data sharing as an ethical imperative essential to making social science more transparent, verifiable, and replicable. But what moved the architects of some of the U.S.’s first university-based social scientific research institutions, the University of Michigan’s Institute for Social Research (ISR), and its spin-off, the Inter-university Consortium for Political and Social Research (ICPSR), to share their data? Relying primarily on archived records, unpublished personal papers, and oral histories, I show that Angus Campbell, Warren Miller, Philip Converse, and others understood sharing data not as an ethical imperative intrinsic to social science but as a useful means to the diverse ends of financial stability, scholarly and institutional autonomy, and epistemological reproduction. I conclude that data sharing must be evaluated not only on the basis of the scientific ideals its supporters affirm, but also on the professional objectives it serves.

 

An interview with protocols.io: CEO Lenny Teytelman on partnering with PLOS – The Official PLOS Blog

“Our ongoing partnership with protocols.io led to a new and exciting PLOS ONE article type, Lab Protocols, which offers a new avenue to share research in-line with the principles of Open Science. This two-part article type gives authors the best of both platforms: protocols.io hosts the step-by-step methodology details while PLOS ONE publishes a companion article that contextualizes the study and orchestrates peer review of the material. The end result is transparent and reproducible research that can help accelerate scientific discovery.

Read our interview with protocols.io CEO and co-founder Dr. Lenny Teytelman where he discusses what triggered the concept for the company, how researchers can benefit from this collaboration and how his team settled on such an adorable racoon logo….”

An interview with protocols.io: CEO Lenny Teytelman on partnering with PLOS – The Official PLOS Blog

“Our ongoing partnership with protocols.io led to a new and exciting PLOS ONE article type, Lab Protocols, which offers a new avenue to share research in-line with the principles of Open Science. This two-part article type gives authors the best of both platforms: protocols.io hosts the step-by-step methodology details while PLOS ONE publishes a companion article that contextualizes the study and orchestrates peer review of the material. The end result is transparent and reproducible research that can help accelerate scientific discovery.

Read our interview with protocols.io CEO and co-founder Dr. Lenny Teytelman where he discusses what triggered the concept for the company, how researchers can benefit from this collaboration and how his team settled on such an adorable racoon logo….”

Improving Social Science: Lessons from the Open Science Movement | PS: Political Science & Politics | Cambridge Core

“Recent years have been times of turmoil for psychological science. Depending on whom you ask, the field underwent a “replication crisis” (Shrout and Rodgers 2018) or a “credibility revolution” (Vazire 2018) that might even climax in “psychology’s renaissance” (Nelson, Simmons, and Simonsohn 2018). This article asks what social scientists can learn from this story. Our take-home message is that although differences in research practices make it difficult to prescribe cures across disciplines, much still can be learned from interdisciplinary exchange. We provide nine lessons but first summarize psychology’s experience and what sets it apart from neighboring disciplines….”