Two major study retractions in one month have left researchers wondering if the peer review process is broken.
“Recognizing the growing interest in preprints, NLM is today launching the first phase of the NIH Preprint Pilot, which will test the viability of making preprints searchable in PubMed Central (PMC) and, by extension, discoverable in PubMed, starting with COVID-19 preprints reporting NIH-supported research.
To be clear, NLM is not building a preprint server for NIH investigators, nor are we developing a comprehensive preprint discovery resource. Rather, through this pilot, we plan to add a curated collection of preprints from eligible preprint servers to our established literature resources. In doing so, our goal is to improve scholarly communications by accelerating and expanding the findability of NIH research results.
With the encouragement of NIH leadership, NLM has been exploring ways to leverage its literature databases to help accelerate the discoverability and maximize the impact of NIH-supported research via preprints. The planned pilot builds on guidance released by NIH in March 2017, which encouraged NIH investigators to use preprints and other interim research products to speed the dissemination of research and enhance the rigor of their work through public comments and new scientific collaborations….”
“The global outbreak of coronavirus disease 2019 (COVID-19) has seen a deluge of clinical studies, with hundreds registered on clinicaltrials.gov. But a palpable sense of urgency and a lingering concern that “in critical situations, large randomized controlled trials are not always feasible or ethical” (1) perpetuate the perception that, when it comes to the rigors of science, crisis situations demand exceptions to high standards for quality. Early phase studies have been launched before completion of investigations that would normally be required to warrant further development of the intervention (2), and treatment trials have used research strategies that are easy to implement but unlikely to yield unbiased effect estimates. Numerous trials investigating similar hypotheses risk duplication of effort, and droves of research papers have been rushed to preprint servers, essentially outsourcing peer review to practicing physicians and journalists. Although crises present major logistical and practical challenges, the moral mission of research remains the same: to reduce uncertainty and enable caregivers, health systems, and policy-makers to better address individual and public health. Rather than generating permission to carry out low-quality investigations, the urgency and scarcity of pandemics heighten the responsibility of key actors in the research enterprise to coordinate their activities to uphold the standards necessary to advance this mission….”
“The pace of scientific publishing has accelerated dramatically in response to the COVID pandemic. Journals have sped up time from submission to publication, and scientists have uploaded thousands of papers to open-access preprint servers without first going through the normal peer-review process. As the volume and speed of scientific publishing has increased, it’s perhaps inevitable that mistakes will slip through — mistakes that can have serious stakes and consequential outcomes in the context of a highly politicized pandemic….”
“The Covid-19 pandemic has changed the world as we know it, and research is no exception. Globally, scientists are working together at unprecedented speed, in a race against time to understand the virus and its treatment, sharing data and results as fast as they can. Journal editors are cooperating and becoming more flexible. Embargoes are lifted, paywalls abolished and preprint servers like MedrXiv and bioRxiv have accelerated research evaluation and discussion. Suddenly the demand for instant access to the relevant research literature has become self-evident. How could the argument for full and immediate Open Access still be ignored?…”
“There are many dimensions to an emergent pandemic such as COVID?19. Leaders all over the world are grappling with the complexity of this disease as they consider the multifaceted burden it places on communities, healthcare delivery systems, financial security, and the role of government. Effective policy must address each and every dimension of this pandemic and do so early and decisively while also earning public confidence in order to ensure cooperation and compliance. While some policy makers are adept at the formation, implementation, and assessment of public policy, many rely on a myriad of resources for support through complex problems. Among these resources are the support of experts representing various dimensions of the problem situation, historical data from previous pandemics, observational data from other impacted regions, and the use of systems dynamics tools that predict the spread and impact of disease over time. System dynamics describes the process of representing a complex system with interrelated parts that interact in a nonlinear and unpredictable method within a system and predicting those interactions and outcomes. The process of mapping a system is described as system thinking, and this step alone can force its users to easily understand and predict outcomes in a complex system using mental modeling. The next step applies the map into a mathematical model through a system of algorithms to calculate the interactions and make predictions regarding future interactions and outcomes over time. System dynamics has many applications for both routine and unexpected problem situations and represents an important decision support tool that helps make determinations about the effectiveness of potential policy interventions prior to implementation. Modeling as a resource not only has many implications for the current pandemic, but also may play an important role in the dynamic field of medical physics….”
“Since the start of the COVID?19 pandemic, hundreds of scientific papers that are related to the disease in one way or another have been uploaded to pre?print servers for anyone to read and comment on. This has been a boon of early insights into SARS?CoV?2 for the research community, but there are also concerns that some of these attract more credence than they deserve. For some, pre?print servers represent a windfall in rapid information at a crucial time for society, while for others, they are a murky meandering away from scientific rigour….”
“The advent of coronavirus disease 2019 (COVID?19) has generated an unparalleled level of interest from the medical and non?medical community. As clinician?scientists, we watch in astonishment at the exponential growth of academic publications in journals. In January 2020, PubMed saw a sharp rise in the number of publications related to COVID?19, which continues to grow….
The urgent nature of this situation means a number of preliminary studies and publications on COVID?19 are fast?tracked through the peer review process — or not at all — in the hope of rapidly publicising important findings, opinions and experiences. However, hastily penned observations may mislead and do more harm than good….”
“The report key findings include:
As of 1 June 2020, there have been upwards of 42,700 scholarly articles on COVID-19 published, 3,100 clinical trials, 420 datasets, 270 patents, 750 policy documents, and 150 grants.
Preprints have rapidly established as a mainstream research output and a key part of COVID-19 research efforts. They started at relatively low levels in early January 2020 and accounted for around one quarter of research output by the beginning of May 2020.
To date, more than 8,300 organisations have been involved in supporting COVID-19 research, with over 71,800 individual researchers identified as working on COVID-19 research.
The highest intensity of research into COVID-19 began in China and gradually migrated west mirroring the movement of the virus itself.
While the US and EU have both now published more than China in journals such as The Lancet, New England Journal of Medicine and JAMA, China continues to benefit from an early mover advantage and continues to enjoy the lionshare of the citations. While research in the field is clearly moving quickly, it currently remains anchored to China’s early publications.
A density map of global COVID-19 paper production shows there are three to four major centres of research: an extended area in China composed of several cities—Wuhan, where the virus is alleged to have started, Beijing and Shanghai; Europe, specifically Italy and the UK, two of the harder hit countries; the US’s east coast research corridor including Boston and New York; and finally, a lighter focus from the Californian institutions on the West coast.
The top producing institution of COVID-19 research (since the beginning of 2020) is in China, Huazhong University of Science and Technology, followed by Harvard University and the University of Oxford.
The top healthcare producers of COVID-19 research (since the beginning of 2020) are Zhongnan Hospital of Wuhan University, then Renmin Hospital of Wuhan University, and Massachusetts General Hospital.
While the proportion of internationally co-authored work is steady, the vast majority of research on COVID to date has been unusually authored within countries.
At the time of writing, 156 grants totalling at least 20.8m USD have been awarded to COVID-themed researchers in public institutions.
Much of the clinical trial initiation activity in January and February is sponsored by China and this then begins to fall off in March, April and May. We see a similar wave for Europe and the US, but shifted back by two months, beginning in March….”
“The research world has moved faster than many would have suspected possible in response to the COVID-19 pandemic. In five months, a volume of work has been generated that even the most intensive of emergent fields have taken years to create.
In our new report, How COVID-19 is Changing Research Culture, we investigate the research landscape trends and cultural changes in response to COVID-19. The report includes analysis of publication trends, geographic focal points of research, and collaboration patterns….”