Assessment of transparency indicators across the biomedical literature: How open is open?

Abstract:  Recent concerns about the reproducibility of science have led to several calls for more open and transparent research practices and for the monitoring of potential improvements over time. However, with tens of thousands of new biomedical articles published per week, manually mapping and monitoring changes in transparency is unrealistic. We present an open-source, automated approach to identify 5 indicators of transparency (data sharing, code sharing, conflicts of interest disclosures, funding disclosures, and protocol registration) and apply it across the entire open access biomedical literature of 2.75 million articles on PubMed Central (PMC). Our results indicate remarkable improvements in some (e.g., conflict of interest [COI] disclosures and funding disclosures), but not other (e.g., protocol registration and code sharing) areas of transparency over time, and map transparency across fields of science, countries, journals, and publishers. This work has enabled the creation of a large, integrated, and openly available database to expedite further efforts to monitor, understand, and promote transparency and reproducibility in science.



Mandating access: assessing the NIH’s public access policy | Economic Policy | Oxford Academic

Abstract:  In April 2008, the National Institutes of Health (NIH) implemented the Public Access Policy (PAP), which mandated that the full text of NIH-supported articles be made freely available on PubMed Central – the NIH’s repository of biomedical research. This paper uses 600,000 NIH articles and a matched comparison sample to examine how the PAP impacted researcher access to the biomedical literature and publishing patterns in biomedicine. Though some estimates allow for large citation increases after the PAP, the most credible estimates suggest that the PAP had a relatively modest effect on citations, which is consistent with most researchers having widespread access to the biomedical literature prior to the PAP, leaving little room to increase access. I also find that NIH articles are more likely to be published in traditional subscription-based journals (as opposed to ‘open access’ journals) after the PAP. This indicates that any discrimination the PAP induced, by subscription-based journals against NIH articles, was offset by other factors – possibly the decisions of editors and submission behaviour of authors.


Systematic examination of preprint platforms for use in the medical and biomedical sciences setting | BMJ Open

Abstract:  Objectives The objective of this review is to identify all preprint platforms with biomedical and medical scope and to compare and contrast the key characteristics and policies of these platforms.

Study design and setting Preprint platforms that were launched up to 25 June 2019 and have a biomedical and medical scope according to MEDLINE’s journal selection criteria were identified using existing lists, web-based searches and the expertise of both academic and non-academic publication scientists. A data extraction form was developed, pilot tested and used to collect data from each preprint platform’s webpage(s).

Results A total of 44 preprint platforms were identified as having biomedical and medical scope, 17 (39%) were hosted by the Open Science Framework preprint infrastructure, 6 (14%) were provided by F1000 Research (the Open Research Central infrastructure) and 21 (48%) were other independent preprint platforms. Preprint platforms were either owned by non-profit academic groups, scientific societies or funding organisations (n=28; 64%), owned/partly owned by for-profit publishers or companies (n=14; 32%) or owned by individuals/small communities (n=2; 5%). Twenty-four (55%) preprint platforms accepted content from all scientific fields although some of these had restrictions relating to funding source, geographical region or an affiliated journal’s remit. Thirty-three (75%) preprint platforms provided details about article screening (basic checks) and 14 (32%) of these actively involved researchers with context expertise in the screening process. Almost all preprint platforms allow submission to any peer-reviewed journal following publication, have a preservation plan for read access and most have a policy regarding reasons for retraction and the sustainability of the service.

Conclusion A large number of preprint platforms exist for use in biomedical and medical sciences, all of which offer researchers an opportunity to rapidly disseminate their research findings onto an open-access public server, subject to scope and eligibility.

Comparing quality of reporting between preprints and peer-reviewed articles in the biomedical literature | Research Integrity and Peer Review | Full Text

Abstract:  Background

Preprint usage is growing rapidly in the life sciences; however, questions remain on the relative quality of preprints when compared to published articles. An objective dimension of quality that is readily measurable is completeness of reporting, as transparency can improve the reader’s ability to independently interpret data and reproduce findings.


In this observational study, we initially compared independent samples of articles published in bioRxiv and in PubMed-indexed journals in 2016 using a quality of reporting questionnaire. After that, we performed paired comparisons between preprints from bioRxiv to their own peer-reviewed versions in journals.


Peer-reviewed articles had, on average, higher quality of reporting than preprints, although the difference was small, with absolute differences of 5.0% [95% CI 1.4, 8.6] and 4.7% [95% CI 2.4, 7.0] of reported items in the independent samples and paired sample comparison, respectively. There were larger differences favoring peer-reviewed articles in subjective ratings of how clearly titles and abstracts presented the main findings and how easy it was to locate relevant reporting information. Changes in reporting from preprints to peer-reviewed versions did not correlate with the impact factor of the publication venue or with the time lag from bioRxiv to journal publication.


Our results suggest that, on average, publication in a peer-reviewed journal is associated with improvement in quality of reporting. They also show that quality of reporting in preprints in the life sciences is within a similar range as that of peer-reviewed articles, albeit slightly lower on average, supporting the idea that preprints should be considered valid scientific contributions.

Biomedical Data Sharing Among Researchers: A Study from Jordan | JMDH

Abstract:  Background: Data sharing is an encouraged practice to support research in all fields. For that purpose, it is important to examine perceptions and concerns of researchers about biomedical data sharing, which was investigated in the current study.

Methods: This is a cross-sectional survey study that was distributed among biomedical researchers in Jordan, as an example of developing countries. The study survey consisted of questions about demographics and about respondent’s attitudes toward sharing of biomedical data.
Results: Among study participants, 46.9% (n=82) were positive regarding making their research data available to the public, whereas 53.1% refused the idea. The reasons for refusing to publicly share their data included “lack of regulations” (33.5%), “access to research data should be limited to the research team” (29.5%), “no place to deposit the data” (6.5%), and “lack of funding for data deposition” (6.0%). Agreement with the idea of making data available was associated with academic rank (P=0.003). Moreover, gender (P-value=0.043) and number of publications (P-value=0.005) were associated with a time frame for data sharing (ie, agreeing to share data before vs after publication).
Conclusion: About half of the respondents reported a positive attitude toward biomedical data sharing. Proper regulations and facilitation data deposition can enhance data sharing in Jordan.

Towards a unified open access dataset of molecular interactions | Nature Communications

The International Molecular Exchange (IMEx) Consortium provides scientists with a single body of experimentally verified protein interactions curated in rich contextual detail to an internationally agreed standard. In this update to the work of the IMEx Consortium, we discuss how this initiative has been working in practice, how it has ensured database sustainability, and how it is meeting emerging annotation challenges through the introduction of new interactor types and data formats. Additionally, we provide examples of how IMEx data are being used by biomedical researchers and integrated in other bioinformatic tools and resources.


A detailed open access model of the PubMed literature | Scientific Data

Abstract:  Portfolio analysis is a fundamental practice of organizational leadership and is a necessary precursor of strategic planning. Successful application requires a highly detailed model of research options. We have constructed a model, the first of its kind, that accurately characterizes these options for the biomedical literature. The model comprises over 18 million PubMed documents from 1996–2019. Document relatedness was measured using a hybrid citation analysis?+?text similarity approach. The resulting 606.6 million document-to-document links were used to create 28,743 document clusters and an associated visual map. Clusters are characterized using metadata (e.g., phrases, MeSH) and over 20 indicators (e.g., funding, patent activity). The map and cluster-level data are embedded in Tableau to provide an interactive model enabling in-depth exploration of a research portfolio. Two example usage cases are provided, one to identify specific research opportunities related to coronavirus, and the second to identify research strengths of a large cohort of African American and Native American researchers at the University of Michigan Medical School.