Self-correction of science: a comparative study of negative citations and post-publication peer review

Abstract:  This study investigates whether negative citations in articles and comments posted on post-publication peer review platforms are both equally contributing to the correction of science. These 2 types of written evidence of disputes are compared by analyzing their occurrence in relation to articles that have already been retracted or corrected. We identi-fied retracted or corrected articles in a corpus of 72,069 articles coming from the Engineer-ing field, from 3 journals (Science, Tumor Biology, Cancer Research) and from 3 authors with many retractions to their credit (Sarkar, Schön, Voinnet). We used Scite to retrieve contradicting citations and PubPeer to retrieve the number of comments for each article, and then we considered them as traces left by scientists to contest published results. Our study shows that contradicting citations are very uncommon and that retracted or corrected articles are not more contradicted in scholarly articles than those that are neither retracted nor corrected but they do generate more comments on Pubpeer, presumably because of the possibility for contributors to remain anonymous. Moreover, post-publication peer review platforms, although external to the scientific publication process contribute more to the correction of science than negative citations. Consequently, post-publication peer review venues, and more specifically the comments found on it, although not contributing to the scientific literature, are a mechanism for correcting science. Lastly, we introduced the idea of strengthening the role of contradicting citations to rehabilitate the clear expression of judgment in scientific papers.

scite awarded $1.5 million Fast Track SBIR Grant from the National Institutes of Health to expand the use of Smart Citations

“scite, a platform for the discovery and evaluation of scientific articles, today announced it has been awarded $1.5 million from Phase II of its Fast-Track Small Business Innovation Research (SBIR) grant by the National Institute on Drug Abuse (NIDA) of the National Institutes of Health (NIH)….”

All Citations Aren’t Created Equal

“Indeed, when we as researchers first look at a paper, we look at where it was published, who are the authors and where are they from, and some metrics like downloads, reads, and of course, citations. Most of this information is superficial, contributing no real useful information to understanding the research. Even citations, as used today, are mostly used as a number to make a quick assessment of the work, where the higher the number of citations an article has, the better.

However, citations represent a wealth of information. Behind each of the 41 articles that cite my work are years of directly related research and many thousands, if not millions, of dollars of research funding. But if I want to learn what these articles say about my work, I would need to read each of them. This is so impractical that it is effectively never fully done.

We’re changing that at scite, a new platform that uses deep learning to show how an article has been cited and, specifically, if it has been supported or contradicted, where the citations appear in the citing paper, and if it is a self-cite or a citation from a review or article. In short, we want to make citations smart–citations that not merely tell how many times an article is cited, but also provide the context for each citation and the citation meaning, such as whether it provides supporting or contradicting evidence for the cited claim. …”

Europe PMC Integrates Smart Citations from scite – scite – Medium

“scite, an award-winning citation analysis platform, and Europe PMC, an open science discovery tool that provides access to a worldwide collection of life science publications, have partnered to display what scite calls smart citations on the Europe PMC platform.

Smart citations advance regular citations by providing more contextual information beyond the information that one study references another. Specifically, smart citations provide the excerpt of text surrounding the citation, the section of the article in which the reference is mentioned, and indicate whether the citing study provides supporting or contradicting evidence. As a result, one can evaluate a study of interest much faster….”

scite awarded NIH SBIR Fast-Track grant – scite – Medium

“Scite, Inc., a Brooklyn-based startup focused on making research more reliable, has been awarded a Small Business Innovation Research (SBIR) Fast-Track grant by the National Institute on Drug Abuse (NIDA) of the National Institutes of Health (NIH) to conduct further research required for developing and commercializing a platform that identifies studies supporting or contradicting prior research. The award is issued in two phases with a Phase I limit of $225,000 and a Phase II limit of $1,500,000 with the second phase contingent upon meeting specific milestones….”

The National Science Foundation Awards scite Competitive R&D Grant to Build Tool to Identify and…

“scite, Inc. has been awarded a National Science Foundation (NSF) Small Business Innovation Research (SBIR)grant for $224,559 to conduct research and development (R&D) work ondeveloping a deep learning platform that can evaluate the reliability of scientific claims by citation analysis….”

scite: Making Science More Reliable – scite – Medium

Today, we are introducing scite to make it easier to tell what is fact and what is not.

scite is a platform that allows anyone to see if a scientific report has been supported or contradicted by subsequent work. We do this by using deep learning and a network of experts to analyze hundreds of millions of citation statements, classifying them as supporting, contradicting, or just mentioning, and presenting the results in an easy to understand interface. Thus, anyone can check if a scientific paper has been supported or contradicted with just a few clicks….”