Supporting rigor through reproducibility | JAMIA Open | Oxford Academic

“Community abstracts are now mandatory for accepted submissions to JAMIA Open. Community abstracts support a key goal in dissemination of published work to the stakeholders that have the most to gain– the patients. I acknowledge the challenge for many of us who spend our time writing and communicating to research audiences to write Community Abstracts. However, for our field to have the most impact, we must convey our work to the greater community. Community understanding for our findings and innovations in leveraging informatics approaches to improve health and health care is a crucial first step toward building a foundation for reproducibility. That is, if a patient understands work presented in a publication, they should expect that it can be reproduced for themselves.

Reproducibility is on equal footing with rigor in terms of importance in the pursuit of knowledge. Many funding agencies now require explicit description for how proposed work will not only be rigorous but also reproducible. JAMIA Open has strongly encouraged the availability of any associated data (eg, through Dryad) to support reproducibility. Data that are made available through readily accessible, public repositories supports not only verification studies, but can also form the basis for new studies. Data sets can also be enhanced and curated to provide common “benchmark” datasets for algorithm evaluation.

JAMIA Open was established as a Level 1 Data Availability journal, meaning that authors were encouraged to share their data publicly. We are now shifting to be a Level 2 Data Availability journal, meaning that, in addition to sharing data publicly, each publication must include a Data Availability Statement. Of course, the release of data should only be done where ethically possible and in accordance to relevant laws. A description of the Oxford University Press Data Availability policies can be found here: https://academic.oup.com/journals/pages/authors/preparing_your_manuscript/research-data-policy….”

Full article: Promoting scientific integrity through open science in health psychology: results of the Synergy Expert Meeting of the European health psychology society

Abstract:  The article describes a position statement and recommendations for actions that need to be taken to develop best practices for promoting scientific integrity through open science in health psychology endorsed at a Synergy Expert Group Meeting. Sixteen Synergy Meeting participants developed a set of recommendations for researchers, gatekeepers, and research end-users. The group process followed a nominal group technique and voting system to elicit and decide on the most relevant and topical issues. Seventeen priority areas were listed and voted on, 15 of them were recommended by the group. Specifically, the following priority actions for health psychology were endorsed: (1) for researchers: advancing when and how to make data open and accessible at various research stages and understanding researchers’ beliefs and attitudes regarding open data; (2) for educators: integrating open science in research curricula, e.g., through online open science training modules, promoting preregistration, transparent reporting, open data and applying open science as a learning tool; (3) for journal editors: providing an open science statement, and open data policies, including a minimal requirements submission checklist. Health psychology societies and journal editors should collaborate in order to develop a coordinated plan for research integrity and open science promotion across behavioural disciplines.

 

Data sharing policies in scholarly publications: interdisciplinary comparisons on JSTOR

Abstract:  Digital sharing of research data is becoming an important research integrity norm. Data sharing is promoted in different avenues, one being the scholarly publication process: journals serve as gatekeepers, recommending or mandating data sharing as a condition for publication. While there is now a sizeable corpus of research assessing the pervasiveness and efficacy of journal data sharing policies in various disciplines, available research is largely piecemeal and mitigates against meaningful comparisons across disciplines. A major contribution of the present research is that it makes direct across-discipline comparisons employing a common methodology. The paper opens with a discussion of the arguments aired in favour and against data sharing (with an emphasis on ethical issues, which stand behind these policies). The websites of 150 journals, drawn from 15 disciplines, were examined for information on data sharing. The results consolidate the notion of the primacy of biomedical sciences in the implementation of data sharing norms and the lagging implementation in the arts and humanities. More surprisingly, they attest to similar levels of norms adoption in the physical and social sciences. The results point to the overlooked status of the formal sciences, which demonstrate low levels of data sharing implementation. The study also examines the policies of the major journal publishers. The paper concludes with a presentation of the current preferences for different data sharing solutions in different fields, in specialized repositories, general repositories, or publishers’ hosting area.

 

The necessity of data transparency to publish

“Even though The Journal of Social Psychology was one of the first psychology journals to adopt open science badges (J. E. Grahe, 2014), and the first to require Research Materials Transparency (J. Grahe, 2018), we have resisted requiring Data Transparency. The reasons for this have varied across the years, but most recently we paused for two reasons which I will present momentarily. However, our reasons were generally concerned that early adoption would drive away too many authors and we needed to wait. In the early spring of 2020, the editors once again discussed adopting Data Transparency as a requirement for publication, but again demurred. Though our other concerns were again discussed, the onset of the CV-19 pandemic was our primary caution. In short, we recognized that this decision will require a transition as authors grapple with a new reality of sharing their data as a condition of publication, and we were waiting until the time was right to implement the new rules. Well, the time has come, and this editorial is the announcement that Data Transparency will now be required for publication in The Journal of Social Psychology. Along with a short explanation of the timing, this editorial also describes what is required versus recommended in our new data sharing policy.”

Data sharing policies of journals in life, health, and physical sciences indexed in Journal Citation Reports [PeerJ]

Abstract:  Many scholarly journals have established their own data-related policies, which specify their enforcement of data sharing, the types of data to be submitted, and their procedures for making data available. However, except for the journal impact factor and the subject area, the factors associated with the overall strength of the data sharing policies of scholarly journals remain unknown. This study examines how factors, including impact factor, subject area, type of journal publisher, and geographical location of the publisher are related to the strength of the data sharing policy.

Methods

From each of the 178 categories of the Web of Science’s 2017 edition of Journal Citation Reports, the top journals in each quartile (Q1, Q2, Q3, and Q4) were selected in December 2018. Of the resulting 709 journals (5%), 700 in the fields of life, health, and physical sciences were selected for analysis. Four of the authors independently reviewed the results of the journal website searches, categorized the journals’ data sharing policies, and extracted the characteristics of individual journals. Univariable multinomial logistic regression analyses were initially conducted to determine whether there was a relationship between each factor and the strength of the data sharing policy. Based on the univariable analyses, a multivariable model was performed to further investigate the factors related to the presence and/or strength of the policy.

Results

Of the 700 journals, 308 (44.0%) had no data sharing policy, 125 (17.9%) had a weak policy, and 267 (38.1%) had a strong policy (expecting or mandating data sharing). The impact factor quartile was positively associated with the strength of the data sharing policies. Physical science journals were less likely to have a strong policy relative to a weak policy than Life science journals (relative risk ratio [RRR], 0.36; 95% CI [0.17–0.78]). Life science journals had a greater probability of having a weak policy relative to no policy than health science journals (RRR, 2.73; 95% CI [1.05–7.14]). Commercial publishers were more likely to have a weak policy relative to no policy than non-commercial publishers (RRR, 7.87; 95% CI, [3.98–15.57]). Journals by publishers in Europe, including the majority of those located in the United Kingdom and the Netherlands, were more likely to have a strong data sharing policy than a weak policy (RRR, 2.99; 95% CI [1.85–4.81]).

Conclusions

These findings may account for the increase in commercial publishers’ engagement in data sharing and indicate that European national initiatives that encourage and mandate data sharing may influence the presence of a strong policy in the associated journals. Future research needs to explore the factors associated with varied degrees in the strength of a data sharing policy as well as more diverse characteristics of journals related to the policy strength.

 

Research transparency promotion by surgical journals publishing randomised controlled trials: a survey | Trials | Full Text

Abstract:  

Objective

To describe surgical journals’ position statements on data-sharing policies (primary objective) and to describe key features of their research transparency promotion.

Methods

Only “SURGICAL” journals with an impact factor higher than 2 (Web of Science) were eligible for the study. They were included, if there were explicit instructions for clinical trial publication in the official instructions for authors (OIA) or if they had published randomised controlled trial (RCT) between 1 January 2016 and 31 December 2018. The primary outcome was the existence of a data-sharing policy included in the instructions for authors. Data-sharing policies were grouped into 3 categories, inclusion of data-sharing policy mandatory, optional, or not available. Details on research transparency promotion were also collected, namely the existence of a “prospective registration of clinical trials requirement policy”, a conflict of interests (COIs) disclosure requirement, and a specific reference to reporting guidelines, such as CONSORT for RCT.

Results

Among the 87 surgical journals identified, 82 were included in the study: 67 (82%) had explicit instructions for RCT and the remaining 15 (18%) had published at least one RCT. The median impact factor was 2.98 [IQR?=?2.48–3.77], and in 2016 and 2017, the journals published a median of 11.5 RCT [IQR?=?5–20.75].

The OIA of four journals (5%) stated that the inclusion of a data-sharing statement was mandatory, optional in 45% (n?=?37), and not included in 50% (n?=?41).

No association was found between journal characteristics and the existence of data-sharing policies (mandatory or optional). A “prospective registration of clinical trials requirement” was associated with International Committee of Medical Journal Editors (ICMJE) allusion or affiliation and higher impact factors. Journals with specific RCT instructions in their OIA and journals referenced on the ICMJE website more frequently mandated the use of CONSORT guidelines.

Conclusion

Research transparency promotion is still limited in surgical journals. Standardisation of journal requirements according to ICMJE guidelines could be a first step forward for research transparency promotion in surgery.

Will COVID-19 mark the end of scientific publishing as we know it?

“Under the pressure of a global health crisis, the argument for open access has sunk in. Following calls from the World Health Organization and government leaders, over 150 publishers, companies, and research institutions have agreed to temporarily make all content related to COVID-19 free to read, ensuring efforts to understand the virus can go forth undeterred….

Is this the catalyst that breaks up the bonds of an old publishing model once and for all? …”

Data Sharing & Citation | Wiley

“At Wiley, we support the growing movement to make research more open, because this leads to a fairer, more efficient and accountable research landscape, which will ultimately drive a more effective and faster pace of discovery. We are committed to improving openness, transparency, and reproducibility of research. Fundamental to enabling reproducible research is the easy access to and ready discovery of its supporting data, made possible through a robust and universal framework that allows research data to be cited through standard reference lists. This will ensure that data is treated as a first-class research object, easily accessible as part of the scholarly literature, and that researchers are credited for their work.

Select an option below to learn more about Wiley’s data sharing and citation policies and service….”

Journal research data sharing policies

Abstract:  The practices for if and how scholarly journals instruct research data for published research to be shared is an area where a lot of changes have been happening as science policy moves towards facilitating open science, and subject-specific repositories and practices are established. This study provides an analysis of the research data sharing policies of highly-cited journals in the fields of neuroscience, physics, and operations research as of May 2019. For these 120 journals, 40 journals per subject category, a unified policy coding framework was developed to capture the most central elements of each policy, i.e. what, when, and where research data is instructed to be shared. The results affirm that considerable differences between research fields remain when it comes to policy existence, strength, and specificity. The findings revealed that one of the most important factors influencing the dimensions of what, where and when of research data policies was whether the journal’s scope included specific data types related to life sciences which have established methods of sharing through community-endorsed public repositories. The findings surface the future research potential of approaching policy analysis on the publisher-level as well as on the journal-level. The collected data and coding framework is provided as open data to facilitate future research and journal policy monitoring.

 

Data-sharing recommendations in biomedical journals and randomised controlled trials: an audit of journals following the ICMJE recommendations | BMJ Open

Abstract:  Objective To explore the implementation of the International Committee of Medical Journal Editors (ICMJE) data-sharing policy which came into force on 1 July 2018 by ICMJE-member journals and by ICMJE-affiliated journals declaring they follow the ICMJE recommendations.

Design A cross-sectional survey of data-sharing policies in 2018 on journal websites and in data-sharing statements in randomised controlled trials (RCTs).

Setting ICMJE website; PubMed/Medline.

Eligibility criteria ICMJE-member journals and 489 ICMJE-affiliated journals that published an RCT in 2018, had an accessible online website and were not considered as predatory journals according to Beall’s list. One hundred RCTs for member journals and 100 RCTs for affiliated journals with a data-sharing policy, submitted after 1 July 2018.

Main outcome measures The primary outcome for the policies was the existence of a data-sharing policy (explicit data-sharing policy, no data-sharing policy, policy merely referring to ICMJE recommendations) as reported on the journal website, especially in the instructions for authors. For RCTs, our primary outcome was the intention to share individual participant data set out in the data-sharing statement.

Results Eight (out of 14; 57%) member journals had an explicit data-sharing policy on their website (three were more stringent than the ICMJE requirements, one was less demanding and four were compliant), five (35%) additional journals stated that they followed the ICMJE requirements, and one (8%) had no policy online. In RCTs published in these journals, there were data-sharing statements in 98 out of 100, with expressed intention to share individual patient data reaching 77 out of 100 (77%; 95% CI 67% to 85%). One hundred and forty-five (out of 489) ICMJE-affiliated journals (30%; 26% to 34%) had an explicit data-sharing policy on their website (11 were more stringent than the ICMJE requirements, 85 were less demanding and 49 were compliant) and 276 (56%; 52% to 61%) merely referred to the ICMJE requirements. In RCTs published in affiliated journals with an explicit data-sharing policy, data-sharing statements were rare (25%), and expressed intentions to share data were found in 22% (15% to 32%).

Conclusion The implementation of ICMJE data-sharing requirements in online journal policies was suboptimal for ICMJE-member journals and poor for ICMJE-affiliated journals. The implementation of the policy was good in member journals and of concern for affiliated journals. We suggest the conduct of continuous audits of medical journal data-sharing policies in the future.