[1608.07878] TrueReview: A Platform for Post-Publication Peer Review

Abstract:  In post-publication peer review, scientific contributions are first published in open-access forums, such as arXiv or other digital libraries, and are subsequently reviewed and possibly ranked and/or evaluated. Compared to the classical process of scientific publishing, in which review precedes publication, post-publication peer review leads to faster dissemination of ideas, and publicly-available reviews. The chief concern in post-publication reviewing consists in eliciting high-quality, insightful reviews from participants. 

We describe the mathematical foundations and structure of TrueReview, an open-source tool we propose to build in support of post-publication review. In TrueReview, the motivation to review is provided via an incentive system that promotes reviews and evaluations that are both truthful (they turn out to be correct in the long run) and informative (they provide significant new information). TrueReview organizes papers in venues, allowing different scientific communities to set their own submission and review policies. These venues can be manually set-up, or they can correspond to categories in well-known repositories such as arXiv. The review incentives can be used to form a reviewer ranking that can be prominently displayed alongside papers in the various disciplines, thus offering a concrete benefit to reviewers. The paper evaluations, in turn, reward the authors of the most significant papers, both via an explicit paper ranking, and via increased visibility in search.

Predecessors of preprint servers

Abstract:  Although there was an early experiment in the 1960s with the central distribution of paper preprints in the biomedical sciences, these sciences have not been early adopters of electronic preprint servers. Some barriers to the development of a ‘preprint culture’ in the biomedical sciences are described. Multiple factors that, from the 1960s, fostered the transition from a paper-based preprint culture in high energy physics to an electronic one are also described. A new revolution in scientific publishing, in which journals come to be regarded as an overlay on electronic preprint databases, will probably overtake some areas of research much more quickly than others.

Predecessors of preprint servers

Abstract:  Although there was an early experiment in the 1960s with the central distribution of paper preprints in the biomedical sciences, these sciences have not been early adopters of electronic preprint servers. Some barriers to the development of a ‘preprint culture’ in the biomedical sciences are described. Multiple factors that, from the 1960s, fostered the transition from a paper-based preprint culture in high energy physics to an electronic one are also described. A new revolution in scientific publishing, in which journals come to be regarded as an overlay on electronic preprint databases, will probably overtake some areas of research much more quickly than others.

SciPost:

From the “about” page:

  • Two-way open access
    Publicly-funded science should be openly accessible to scientists and the general public, perpetually, worldwide. Conversely, scientists should not have to pay publishing charges to disseminate the fruits of their research efforts.
  • Non-profit
    Academics do not perform research for profit, and by extension the publication of their scientific results should not involve commercial profit-making.
  • By Professionals
    Scientists should carry the final responsibility for all stages in the scientific publishing process.
  • Peer-witnessed refereeing
    Scientific publications should undergo the strictest possible peer refereeing process, witnessed by the community instead of hidden behind closed doors.
  • Accountable and credited refereeing
    Peer refereeing should be accountable, and should be incentivized by being credited.
  • Post-publication evaluation
    Peer evaluation does not stop at the moment of publication.

Quantum – the open journal for quantum science

“We propose the launch of an arXiv overlay journal for quant-ph. Quantum is a free and open access peer-reviewed journal that provides high visibility for quality research on quantum science and related fields. It is an effort by researchers and for researchers to make science more open and publishing more transparent and efficient….”

Épijournal de Géométrie Algébrique – Homepage

“Épijournal de Géométrie Algébrique is a peer-reviewed mathematical journal founded in 2015. The selection of articles follows a usual editorial process: the submissions are evaluated by the editorial board after reception of the referee reports. Articles are published in english or in french (with an abstract in english)….Épijournal de Géométrie Algébrique also aims at promoting the Épisciences platform among the mathematical community. This platform (a new tool devoted to scientific edition) enables to publish peer-reviewed research articles submitted from an open repository. Developed by the CCSD (a common service of CNRS, INRIA and University of Lyon), it consists of a high-quality technical interface and it benefits from a long-term institutional support. Its evolution (development of the interface and creation of new tools) is carried out in close colaboration with the scientific community….”

I am thinking of starting an…arxiv-based overlay journal in logic

“I am thinking of starting an effort to mount an arxiv-based overlay journal in logic, in the style of Discrete Analysis (https://gowers.wordpress.com/2015/09/10/discrete-analysis-an-arxiv-overlay-journal/). 

My idea would be a completely open-access journal focussing on topics in mathematical and philosophical logic, on the overlay journal concept.

Please let me know below if there is interest in supporting such a venture. Would you submit your research articles to such a journal? Would you serve as referee? As editor? To what extent would the community support such a venture? 

Please vote-up or share this post if this is a venture that you would support.  If there is strong support for such an effort, I will take it more seriously to make it happen….”

ArXiv, comments, and “quality control” | The Accidental Mathematician

“Those of you who browsed the arXiv recently may have seen a link to a user survey on top of the page (as of now, apparently no longer online). I ignored it a few times, until a friend brought this particular bit to my attention … Sure enough, I took the survey. As it turned out, the arXiv was also asking for feedback on what it calls ‘quality control’: actions such as rejecting ‘papers that don’t have much scientific value,’ flagging papers that have ‘too much text re-use from an author’s earlier papers’ (self-plagiarism) or from papers by other authors (plagiarism), or moderating pingbacks (such as links from blogs or articles) before they appear on the arXiv … Now, here’s what all this might mean for the future of the arXiv. Allow me a little bit of speculation here.  The arXiv has become the universally accepted default repository for mathematicians, not only because it provides a service we need, but also because, in not attempting to do more than that, it gives us no reason to not use it. We don’t have to worry that the paper might not ‘qualify,’ that it’s too long or too short, or too expository, or not sufficiently tailored for the ‘right’ audience. We simply post what we think is right. We expect and welcome feedback (I often post papers on the arXiv prior to journal submission, specifically for that purpose), but the site does not allow public abuse or internet flame wars, so no need to worry about that. The bare-bones structure is not a bug, it’s a feature that has been essential for the arXiv’s success …”

The Open Journal by arfon

“Along with a few friends, we’ve written code to support peer review as a service. It assumes that some group of people (think editors or academic community) want to make comments on a thing (think paper). All you need is a URL for the thing – we think that GitHub repositories offer lots of nice advantages but it could be a dropbox link, or an arXiv paper or anything. Comments are just a list of issues on the paper or on specific parts of it; they are opened by a reviewer and then resolved by the reviewer after dialogue with the author, authors or editors. Once (enough) issues are resolved, your paper is ‘accepted’ and the service will proclaim the news to anything listening.1

That’s all you actually need to start your own publishing revolution. To give you an idea of how this might work, imagine building a new journal. Let’s call it the Open Journal of Astrophysics….”