News & Views: Preprints and COVID-19: Findings from our PRW Survey – Delta Think

“In anticipation of Peer Review Week 2020, and in consideration of the theme ‘Trust in Peer Review’, Delta Think surveyed broadly across a two-week period in August to determine whether COVID-19 has impacted perceptions of preprints. The survey was open to everyone – from Publishers to Librarians to Researchers to the Lay Public with an interest in scientific output.

We entered into the survey with open minds, though a few recurring themes circulating in news outlets were on our minds including: the idea that both traditional journal submissions and articles posted to preprint servers have spiked in the last six months, owing to COVID-19; and the hunch that perhaps increased traffic indicated a correlated spike in trust, or conversely, that challenged findings making their way into mainstream news might have reduced trust. Was the reputation associated with preprints in jeopardy? Or could they play a critical role in speeding up science at a time so critical for the global health community? With these early loose hypotheses and questions, we launched the survey.

We explore the survey results below. In the spirit of transparency, we will make the data collected available within the next few weeks. It’s also important to note that while this was a quick survey, and is not meant to include a representative sample from the participating audiences, it provides interesting top-level findings and points to areas that may be ripe for a further investigation or a deeper dive in future….”

Viral Science: Masks, Speed Bumps, and Guard Rails: Patterns

“With the world fixated on COVID-19, the WHO has warned that the pandemic response has also been accompanied by an infodemic: overabundance of information, ranging from demonstrably false to accurate. Alas, the infodemic phenomenon has extended to articles in scientific journals, including prestigious medical outlets such as The Lancet and NEJM. The rapid reviews and publication speed for COVID-19 papers has surprised many, including practicing physicians, for whom the guidance is intended….

The Allen Institute for AI (AI2) and Semantic Scholar launched the COVID-19 Open Research Dataset (CORD-19), a growing corpus of papers (currently 130,000 abstracts plus full-text papers being used by multiple research groups) that are related to past and present coronaviruses.

Using this data, AI2, working with the University of Washington, released a tool called SciSight, an AI-powered graph visualization tool enabling quick and intuitive exploration
6

 of associations between biomedical entities such as proteins, genes, cells, drugs, diseases, and patient characteristics as well as between different research groups working in the field. It helps foster collaborations and discovery as well as reduce redundancy….

The research community and scientific publishers working together need to develop and make accessible open-source software tools to permit the dual-track submission discussed above. Repositories such as Github are a start….”

Open and Reproducible Research Group (ORRG)

“Open Science is better science. Research benefits from sharing data and scientific knowledge which is publicly available, making open science essential. The Open and Reproducible Research Group (ORRG) uses evidence-based and computational approaches to make research cultures more open, transparent and participatory through new practices and technologies. In our interdisciplinary team we combine competences in philosophy, sociology, and information science with computer science and life science. We research services, policies, and tools to investigate and foster the uptake and evaluation of Open Science practices in the following areas: …”

 

Accelerating open science in physics | Research Information

“Some argue there is enough money in the system to afford a transition to open access. Whether this is the case or not, there is no current solution or global plan in place to adjust the allocation and flow of funding so it resides at the levels exactly commensurate to where research is produced. These are not intractable challenges. But they require global consensus on the goal of open science, coordinated action to build the infrastructure, and incentives to create lasting change. This will take time….

Although Covid-19 might have reinforced the value of open science, its benefits are well understood by many in the physics community, and we are a long-standing proponent. But there is still much work for all involved if we are to transition to a fully and sustainably open landscape in physics and beyond.

To that end, we will maintain an open dialogue with the physical scientists and scientific organisations we serve and continue to seek more insight into what’s specifically important to them. Over the coming year we will engage in a series of projects to speak to the global physical science community so we can contribute to more open science – more ‘open physics’ – and we look forward to reporting back.”

Accelerating open science in physics | Research Information

“Some argue there is enough money in the system to afford a transition to open access. Whether this is the case or not, there is no current solution or global plan in place to adjust the allocation and flow of funding so it resides at the levels exactly commensurate to where research is produced. These are not intractable challenges. But they require global consensus on the goal of open science, coordinated action to build the infrastructure, and incentives to create lasting change. This will take time….

Although Covid-19 might have reinforced the value of open science, its benefits are well understood by many in the physics community, and we are a long-standing proponent. But there is still much work for all involved if we are to transition to a fully and sustainably open landscape in physics and beyond.

To that end, we will maintain an open dialogue with the physical scientists and scientific organisations we serve and continue to seek more insight into what’s specifically important to them. Over the coming year we will engage in a series of projects to speak to the global physical science community so we can contribute to more open science – more ‘open physics’ – and we look forward to reporting back.”

Publishing at Any Cost? The Need for the Improvement of the Quality of Scholarly Publications

Abstract:  At a time of great dynamism among publishers of scientific publications, with the inevitability of Open Access and  the ease of publishing online at low cost, it is possible to find publications with different levels of scientific respectability. In this context, the improvement of the quality of scholarly publications emerges as a critical element for publishers, authors and academic institutions, as well as for society in general. This opinion piece discusses Open Access journals with different levels of quality,  focusing on the following quality-promoting measures: blacklists, author’s preparation, and institutional prevention. The analysis allows concluding that the open review will be one of the key elements in the process of clarification and promotion of the level of quality and consequent scientific respectability of each of the Journals, of the thousands currently existing, a number that is likely to increase. 

‘At-risk articles’: the imperative to recover lost science

“As it stands currently, there is no way for authors to retract an article, acknowledge its prior ‘publication’ and submit it for open peer review on a preprint server. However, the limitation is social and political, not technical, and so I urge disciplinary communities and publication stakeholders to think about the question of what to do with the at-risk research in their field. Rather than dismissing it or ignoring it, researchers must reframe how they think of this potentially valuable and at-risk research and consider it as an issue the disciplinary community must try to solve.”

Open up: a survey on open and non-anonymized peer reviewing | Research Integrity and Peer Review | Full Text

Abstract:  Background

Our aim is to highlight the benefits and limitations of open and non-anonymized peer review. Our argument is based on the literature and on responses to a survey on the reviewing process of alt.chi, a more or less open review track within the so-called Computer Human Interaction (CHI) conference, the predominant conference in the field of human-computer interaction. This track currently is the only implementation of an open peer review process in the field of human-computer interaction while, with the recent increase in interest in open scientific practices, open review is now being considered and used in other fields.

Methods

We ran an online survey with 30 responses from alt.chi authors and reviewers, collecting quantitative data using multiple-choice questions and Likert scales. Qualitative data were collected using open questions.

Results

Our main quantitative result is that respondents are more positive to open and non-anonymous reviewing for alt.chi than for other parts of the CHI conference. The qualitative data specifically highlight the benefits of open and transparent academic discussions. The data and scripts are available on https://osf.io/vuw7h/, and the figures and follow-up work on http://tiny.cc/OpenReviews.

Conclusion

While the benefits are quite clear and the system is generally well-liked by alt.chi participants, they remain reluctant to see it used in other venues. This concurs with a number of recent studies that suggest a divergence between support for a more open review process and its practical implementation.

Open peer review: promoting transparency in open science | SpringerLink

“Open peer review (OPR), where review reports and reviewers’ identities are published alongside the articles, represents one of the last aspects of the open science movement to be widely embraced, although its adoption has been growing since the turn of the century. This study provides the first comprehensive investigation of OPR adoption, its early adopters and the implementation approaches used. Current bibliographic databases do not systematically index OPR journals, nor do the OPR journals clearly state their policies on open identities and open reports. Using various methods, we identified 617 OPR journals that published at least one article with open identities or open reports as of 2019 and analyzed their wide-ranging implementations to derive emerging OPR practices. The findings suggest that: (1) there has been a steady growth in OPR adoption since 2001, when 38 journals initially adopted OPR, with more rapid growth since 2017; (2) OPR adoption is most prevalent in medical and scientific disciplines (79.9%); (3) five publishers are responsible for 81% of the identified OPR journals; (4) early adopter publishers have implemented OPR in different ways, resulting in different levels of transparency. Across the variations in OPR implementations, two important factors define the degree of transparency: open identities and open reports. Open identities may include reviewer names and affiliation as well as credentials; open reports may include timestamped review histories consisting of referee reports and author rebuttals or a letter from the editor integrating reviewers’ comments. When and where open reports can be accessed are also important factors indicating the OPR transparency level. Publishers of optional OPR journals should add metric data in their annual status reports.

 

How is your discipline tracking in open peer review? | Nature Index

“Medical and health sciences journals are leading the way in the adoption of open peer review, a new study has found.

Open peer review, which involves publishing peer review reports and reviewer identities alongside journal articles, has been gaining traction over the last two decades as a way of tackling recognized problems in the traditional peer review process, such as lack of transparency and accountability.

In an analysis of 617 open peer review journals – defined as those that have published peer review reports and reviewer identities alongside at least one article – medical and health sciences journals accounted for almost half (42%).

Natural sciences journals followed, accounting for 38%….”